max_by/min_by with N corrupt ROW fields after NULL-to-non-NULL replacement (476, 483)

Trino version

476 and 483

Please describe the bug

max_by(x, y, n) and min_by(x, y, n) can corrupt fields of a ROW value when a top-N entry containing a null field is replaced by a row with a non-null value in that field. Subsequent VARCHAR fields contain bytes from the wrong fields, and valid input strings can become invalid UTF-8.

Reproduced on unmodified official Trino 476 and 483 servers using only SQL literals. No external catalog, source table, storage service, or ORC write is needed.

Minimal reproduction

Run on either server version:

WITH input(k, imageurl, body, url) AS (
    VALUES
        (0, CAST(NULL AS varchar), 'АААААААААА', 'https://example.test/post/1913'),
        (1, 'aaaaaaaaaaaaa', 'АААААААААА', 'https://example.test/post/1913')
), rows AS (
    SELECT k, CAST(ROW(imageurl, body, url)
        AS ROW(imageurl varchar, text varchar, url varchar)) AS post
    FROM input
)
SELECT max_by(post, k, 1) FROM rows;

The image string contains 13 ASCII a characters. The text contains ten Cyrillic А characters (U+0410, UTF-8 bytes D0 90). All input values are valid UTF-8.

Expected result

A one-element array containing the row with k = 1, with its fields unchanged:

imageurl = aaaaaaaaaaaaa
text     = АААААААААА
url      = https://example.test/post/1913

Actual result

imageurl is unexpectedly NULL. Its bytes are included in text, and text bytes spill into url. Both latter fields contain malformed UTF-8, and the server returns these bytes in the HTTP result.

To inspect the corruption without requiring the client to decode those VARCHAR values, replace the final SELECT above with:

SELECT transform(max_by(post, k, 1), p -> ARRAY[
    to_hex(to_utf8(p.imageurl)),
    to_hex(to_utf8(p.text)),
    to_hex(to_utf8(p.url))
]) AS actual_hex
FROM rows;

On both versions, the selected row's fields are:

imageurl: NULL
text: 61616161616161616161616161D090D090D090D0
url: 90D090D090D090D090D090D09068747470733A2F2F6578616D706C652E74

This SQL-level hex result shows that the values are already corrupt before JSON serialization. The text ends with an incomplete UTF-8 sequence (D0). In the raw JSON response, its next byte is the closing quote (22), causing Invalid UTF-8 middle byte 0x22.

Reproduction results and environment

Each case ran as 25 separate queries on each server: 100/100 failing queries per version, 200/200 overall. The N=10 cases use additional input rows to fill the heap before replacement; their exact SQL is included in the reproduction archive. This rate describes these local tests, not every possible execution plan or input order.

• Official images trinodb/trino:476 and trinodb/trino:483, with no patched classes or source changes.
• Linux ARM64 containers, 2 CPUs and a 2 GiB memory limit per server, one local server per version.
• Server versions verified with both SELECT version() and /v1/info.
• Results fetched directly through the HTTP statement protocol and checked with strict UTF-8 decoding in Python, independently of the CLI.
• One unmodified failing HTTP response saved for each of the eight case/version combinations. All eight also produce the exact 0x22 error when parsed with the official Trino 483 client codec.
• Image digests, query IDs, per-run checks and raw responses are included in the reproduction archive.

The issue was noticed after upgrading a production cluster to 483, but the independent 476 reproduction means this is not established as a regression introduced in 483. The first affected version and behavior on later releases/master have not been established. The two-argument forms of these functions were not tested.

Controls

The following controls passed on both versions:

1. Return the input rows without aggregation: all fields remain correct and valid UTF-8.
2. Keep both input rows with N=2: no heap replacement, and all fields remain correct.
3. Make imageurl non-null in both input rows: the NULL-to-non-NULL transition is avoided and the selected row remains correct.
4. Use ordered array_agg followed by slice: the selected row remains correct.

Suspected cause

Source inspection points to stale null flags when reusing a flat row record:

• TypedKeyValueHeap.add() / clear() / set() in 483 replace a full-heap entry without clearing all of the old record's null flags. The corresponding 476 implementation is identical.
• RowType flat writing and reading in 483 set the marker for a null field but do not reset it when writing a non-null field. Reading a field marked null does not advance the variable-width offset.

This explains the observed null image field and displacement of bytes into subsequent fields. The malformed UTF-8 is a consequence of the field corruption; the underlying symptom is an incorrect query result.

Workaround

We rewrote the production-shaped SELECT to rank rows with row_number() for each ordering and use ordered array_agg with a filter selecting at most ten rows per output array. This avoids the affected ROW top-N aggregation path. Synthetic validation passed 20/20 queries on each version, with exact comparisons of all nested fields, nullable values, Unicode text, ties and shuffled input. These checks covered the SELECT, not a production CTAS/ORC round trip.

Client stack trace

The following trace was produced by parsing the captured 483/max_by-n1.response.bin with the official Trino 483 client codec. No bytes were reconstructed or injected. The Java helper only reads the saved response and prints the parser exception.

Captured response: outputs/local-repro/483/max_by-n1.response.bin
com.fasterxml.jackson.databind.JsonMappingException: Invalid UTF-8 middle byte 0x22
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1079] (through reference chain: io.trino.client.QueryResults["data"])
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:400)
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:359)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1969)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:610)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:446)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1502)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:340)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectReader._bind(ObjectReader.java:2117)
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1252)
	at io.trino.client.TrinoJsonCodec.fromJson(TrinoJsonCodec.java:123)
	at CheckWireJsonStackTrace.main(CheckWireJsonStackTrace.java:13)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:565)
	at jdk.compiler/com.sun.tools.javac.launcher.SourceLauncher.execute(SourceLauncher.java:264)
	at jdk.compiler/com.sun.tools.javac.launcher.SourceLauncher.run(SourceLauncher.java:138)
	at jdk.compiler/com.sun.tools.javac.launcher.SourceLauncher.main(SourceLauncher.java:76)
Caused by: com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 middle byte 0x22
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1079]
	at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2599)
	at com.fasterxml.jackson.core.JsonParser._constructReadException(JsonParser.java:2625)
	at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:837)
	at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidOther(UTF8StreamJsonParser.java:3789)
	at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidOther(UTF8StreamJsonParser.java:3796)
	at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._decodeUtf8_2(UTF8StreamJsonParser.java:3554)
	at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2664)
	at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishAndReturnString(UTF8StreamJsonParser.java:2614)
	at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:294)
	at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:626)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:103)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:24)
	at com.fasterxml.jackson.databind.DeserializationContext.readTree(DeserializationContext.java:1084)
	at io.trino.client.QueryDataJacksonModule$Deserializer.deserialize(QueryDataJacksonModule.java:68)
	at io.trino.client.QueryDataJacksonModule$Deserializer.deserialize(QueryDataJacksonModule.java:54)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:562)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:608)
	... 14 more

Reproduction artifacts

The accompanying trino-476-483-issue-evidence.zip contains:

• Exact SQL for all aggregation cases and controls.
• verify_servers.py, a Python standard-library HTTP test runner, and Docker startup/rerun instructions.
• Per-version results.json files, including query IDs and invalid-byte checks.
• Eight raw .response.bin files and the SQL hex proof.
• Image digests and the client-codec verification output.
• The full stack trace and the small Java helper used to produce it.

All datasets in these artifacts are synthetic. Can you confirm the null-flag handling in this aggregation path and whether a fix is already tracked?

trino-476-483-issue-evidence.zip