Type-mismatched writes on `date`/`time`/`json`/`csv` fields aren't validated before insert
Describe the Bug
Wrong-typed values on date/dateTime/timestamp/time/json fields skip validation and reach the DB driver raw, surfacing as a 500 with the raw SQL query instead of a clean 400 INVALID_PAYLOAD (see //5 below for the string case, which is validated correctly). csv fields have the opposite problem: bad input is silently corrupted instead of throwing INVALID_PAYLOAD (//1).
[!NOTE] I originally only tested with Postgres. Now that I've tested with SQL lite there are more database writes that shouldn't happen and it's less of a "oops, we aren't catching the malformed payload at the right layer" and more of a "we have some payloads that are risky here."
To Reproduce
# Initial testing with Postgres...
curl -X POST localhost:8055/collections -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" -d '{
"collection": "type_mismatch_repro",
"schema": {},
"fields": [
{ "field": "endvisit_at", "type": "dateTime", "meta": {} },
{ "field": "time_field", "type": "time", "meta": {} },
{ "field": "properties", "type": "json", "meta": { "special": ["cast-json"] } },
{ "field": "tags", "type": "csv", "meta": { "special": ["cast-csv"] } }
]
}'
# 200 OK -> {
# "data": {
# "collection": "type_mismatch_repro",
# "schema": {
# "schema": "public",
# "name": "type_mismatch_repro",
# "comment": null
# },
# "meta": null
# }
# }
# //1 bad csv (object) is accepted, silently corrupted instead of a 400 error
curl -X POST localhost:8055/items/type_mismatch_repro -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \
-d '{"tags": {"wrong": "input"}}'
# 200 OK -> {
# "data": {
# "id": 1,
# "endvisit_at": null,
# "time_field": null,
# "properties": null,
# "tags": [
# "{\"wrong\":\"input\"}"
# ]
# }
# }
# //2 bad dateTime (object)
curl -X POST localhost:8055/items/type_mismatch_repro -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \
-d '{"endvisit_at": {"wrong": "input"}}'
# 500 Internal Server Error -> {
# "errors": [
# {
# "message": "insert into \"type_mismatch_repro\" (\"endvisit_at\") values ($1) returning \"id\" - invalid input syntax for type timestamp: \"{\"wrong\":\"input\"}\"",
# "extensions": {
# "code": "INTERNAL_SERVER_ERROR",
# "stack": "..."
# }
# }
# ]
# }
# //3 bad time (object)... `time` has no write-time validation at all
curl -X POST localhost:8055/items/type_mismatch_repro -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \
-d '{"time_field": {"wrong": "input"}}'
# 500 Internal Server Error -> {
# "errors": [
# {
# "message": "insert into \"type_mismatch_repro\" (\"time_field\") values ($1) returning \"id\" - invalid input syntax for type time: \"{\"wrong\":\"input\"}\"",
# "extensions": {
# "code": "INTERNAL_SERVER_ERROR",
# "stack": ".."
# }
# }
# ]
# }
# //4 bad json (plain string)
curl -X POST localhost:8055/items/type_mismatch_repro -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \
-d '{"properties": "not a json field"}'
# 500 Internal Server Error -> {
# "errors": [
# {
# "message": "insert into \"type_mismatch_repro\" (\"properties\") values ($1) returning \"id\" - invalid input syntax for type json",
# "extensions": {
# "code": "INTERNAL_SERVER_ERROR",
# "stack": "...""
# }
# }
# ]
# }
# //5 control: bad dateTime STRING is validated correctly
curl -X POST localhost:8055/items/type_mismatch_repro -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \
-d '{"endvisit_at": "not-a-date"}'
# 400 Bad Request -> {
# "errors": [
# {
# "message": "Invalid payload. Invalid DateTime format in field \"endvisit_at\".",
# "extensions": {
# "reason": "Invalid DateTime format in field \"endvisit_at\"",
# "stack": "..."
# "code": "INVALID_PAYLOAD"
# }
# }
# ]
# }
# only //1 ever reached the DB; //2,3,4,5 were rejected before insert
curl localhost:8055/items/type_mismatch_repro -H "Authorization: Bearer $ADMIN_TOKEN"
# 200 OK -> {
# "data": [
# {
# "id": 1,
# "endvisit_at": null,
# "time_field": null,
# "properties": null,
# "tags": [
# "{\"wrong\":\"input\"}"
# ]
# }
# ]
# }Results
| Bad Input | Postgres | SQLite |
|---|---|---|
| //1 csv <- object | 200, silently corrupted to tags: ["{\"wrong\":\"input\"}"] |
200, same corruption |
| //2 dateTime <- object | 500, invalid input syntax for type timestamp: "{\"wrong\":\"input\"}" |
500, a TypeError: value.getFullYear is not a function |
| //3 time <- object | 500, invalid input syntax for type time: "{\"wrong\":\"input\"}" |
200, silently corrupted to time_field: "{\"wrong\":\"input\"}" |
| //4 json <- plain string | 500, invalid input syntax for type json |
200, silently corrupted to properties: "not a json field" |
| //5 dateTime <- bad string (control) | 400 INVALID_PAYLOAD, "Invalid DateTime format in field "endvisit_at"." |
400 INVALID_PAYLOAD, same, validated correctly |
| final read, rows persisted | only //1 (id: 1) |
//1, //3, //4 (id: 1,2,3), three corrupted rows |
Directus Version
v12.0.3
Hosting Strategy
Self-Hosted (Custom)
Database
PostgreSQL & SQLite (probably all drivers impacted somehow but only tested with these two)
Found when troubleshooting... https://github.com/directus/directus/pull/28228#pullrequestreview-5186889676
Source: directus/directus