Iceberg output: the `row_operation` example uses a non-existent bloblang ternary
The third published example for the iceberg output's row_operation field cannot
be used as written. It appears in the field's Examples block on the docs site and
in the in-repo generated docs:
row_operation: ${! this.op == "d" ? "delete" : "upsert" }1. Bloblang has no ternary operator. Even correctly quoted, the mapping does not parse:
$ echo '{"op":"d"}' | rpk connect blobl 'root = this.op == "d" ? "delete" : "upsert"'
failed to parse mapping: line 1 char 23: expected line break
|
1 | root = this.op == "d" ? "delete" : "upsert"
| ^---Inside a config, as an interpolated string, the same expression is rejected at lint time:
$ rpk connect lint repro.yaml
repro.yaml(8,40) required: expected end of expression, got: ? "de2. As published, unquoted, it is not valid YAML. The : inside the value makes
the parser read it as a nested mapping, so the config fails before Bloblang is even
reached:
$ rpk connect lint repro.yaml
repro.yaml(1,1) yaml: line 8: mapping values are not allowed in this contextReproduction
repro.yaml, using the example verbatim:
input:
generate: { count: 1, mapping: 'root.op = "d"', interval: "" }
output:
iceberg:
catalog: { url: "http://127.0.0.1:8181" }
namespace: n
table: t
row_operation: ${! this.op == "d" ? "delete" : "upsert" }
identifier_fields: [ id ]
max_in_flight: 1
storage: { aws_s3: { bucket: b, region: us-east-1 } }rpk connect lint repro.yaml fails with the YAML error above. Quoting the value
changes it to the Bloblang parse error above. Neither form runs.
Reproduced on Redpanda Connect 4.102.0 and 4.105.0.
Where it lives
- Source of truth:
internal/impl/iceberg/config.go, therow_operationfield's thirdExample(...)call (line ~322 onmainat time of writing). - Generated doc in-repo:
docs/modules/components/pages/outputs/iceberg.adoc. - Published pages showing it:
- https://docs.redpanda.com/connect/components/outputs/iceberg/#row_operation
- the Redpanda Cloud mirror of the same component page
The other two examples on that field, insert and ${! metadata("op") }, are both
correct, as is the row_operation line in the change-data-capture example further
down the page.
Suggested fix
Replace the ternary example with either of these, both of which lint and run:
row_operation: '${! match this.op { "d" => "delete", _ => "upsert" } }'
row_operation: '${! if this.op == "d" { "delete" } else { "upsert" } }'i.e. in config.go:
Example(`${! match this.op { "d" => "delete", _ => "upsert" } }`).then regenerate the in-repo docs with make docs (cmd/tools/docs_gen).
Quoting the value in the rendered example is worth considering as a general habit
for interpolated fields, since any expression containing a : hits the same YAML
ambiguity even when the Bloblang itself is valid.
Source: redpanda-data/connect