[BUG][Sandbox] Distributed sort+head on multi_value keyword intermittently returns an empty LIST for a fetched document
Describe the bug
On a two-shard composite parquet index with tags: {type: keyword, multi_value: true}, the query-then-fetch shape source = idx | sort latency | head 2 | fields id, tags intermittently returns an empty LIST [] for one of the top-N documents whose stored value is ["red","green"]. The id column is correct; only the late-materialized LIST payload is lost.
Observed response (doc 2 should be [red, green]):
datarows=[[1, [blue, blue, red]], [2, []]]Reproduction is placement-dependent: with auto-generated _ids the failure occurred in 1 of 6 runs, when _cat/shards showed a 2/4 document split (shard0: 2 docs, shard1: 4 docs); runs with other splits passed. The same query on the one-shard index and the descending variant (sort - latency | head 3) pass consistently. Composite indices reject custom _id/routing (index.append_only.enabled, _routing needs FULL_TEXT_SEARCH), so the fixture cannot pin placement; use repeated runs.
To Reproduce
MultiValueProjectionIT.testSortAscendingHeadThenListProjection in sandbox/qa/analytics-engine-rest (currently @AwaitsFix on this issue). Run the class ~6 times.
Expected behavior The fetched LIST cell matches the stored value regardless of which shard the document lives on.
Additional context Related: #22902 (search foundation / QTF LIST fetch), #23040 (distributed stream schema conformance).
Source: opensearch-project/OpenSearch