[FEAT] Step Functions Map ItemReader should honor MaxItemsPath and a JSONata MaxItems expression
Service
Step Function
API Action / Feature
ReaderConfig.MaxItemsPath in JSONPath state machines, and ReaderConfig.MaxItems written as a {% %} expression in JSONata state machines. Both readers are affected: arn:aws:states:::s3:getObject and arn:aws:states:::s3:listObjectsV2.
AWS Documentation
Why is this needed?
CreateStateMachine accepts both forms, but Floci only reads a literal MaxItems at run time.
A MaxItemsPath, or a JSONata expression in MaxItems, resolves to 0, which means no limit.
The Map then starts an iteration for every item instead of the bounded set the definition asks for.
Nothing fails, so a workflow that limits a fan-out through the input looks correct locally and behaves differently on AWS.
Minimal example (JSONPath):
"ItemReader": {
"Resource": "arn:aws:states:::s3:listObjectsV2",
"ReaderConfig": { "MaxItemsPath": "$.limit" },
"Parameters": { "Bucket": "my-bucket", "Prefix": "input/" }
}With input {"limit": 2} AWS runs 2 children. The emulator runs one per object.
This is not new. The s3:getObject reader from #1587 has read only MaxItems since it was added.
#3460 moved that lookup into one maxItems(itemReader) helper shared by both readers and documents MaxItemsPath as not supported. Copilot flagged the gap in the #3460 review.
Suggested fix
- Resolve the limit the way
resolveMapMaxConcurrencyalready resolvesMaxConcurrencyPathand a JSONataMaxConcurrencyexpression (path against the Map input, expression through the JSONata evaluator, then a non-negative integer check). - Pass the result to the S3 listing for listObjectsV2 and to
applyMaxItemsfor getObject. - The error name and message for a value that is not a non-negative integer should be checked against real AWS first.
Until this lands, failing with States.ItemReaderFailed and the existing "not yet implemented by the emulator" wording would be better than the silent 0.
Are you willing to contribute a PR?
- Yes
- No
Source: floci-io/floci