[Go Functions] Close the configuration parity gaps with the Java and Python runtimes (master issue)

Search before reporting

• I searched in the issues and found nothing similar.

This is a master issue. The individual gaps are tracked as the sub-issues listed below; this one exists to hold the comparison that produced them and to say what they have in common.

#26412 is the counterpart for the Python runtime, from the same audit.

Motivation

The Go function runtime honours a much smaller part of FunctionDetails than the Java and Python runtimes do. Most of the difference is silent: the configuration is accepted by pulsar-admin, stored by the worker, reported back faithfully by functions get, carried into the instance in the protobuf, and then never read.

I audited the three runtimes field by field against Function.proto, reading the consuming code rather than grepping for field names — the generated Function_pb2.py and pulsar-function-go/pb contain every field name, so a naive search reports full support for all three.

Producer maxPendingMessages and maxPendingMessagesAcrossPartitions are absent from both runtimes on master but are already addressed by the open batching PRs (#26392, #26393), so they are not tracked here. useThreadLocalProducers has no equivalent in either client.

The pattern worth naming. EFFECTIVELY_ONCE is the one the Go runtime gets right:

// pulsar-function-go/pf/instanceConf.go:137
if instanceConf.funcDetails.ProcessingGuarantees == pb.ProcessingGuarantees_EFFECTIVELY_ONCE {
    panic("Go instance current not support EFFECTIVELY_ONCE processing guarantees.")
}

Unsupported, and it says so. Every other row in that table fails the other way: accepted and ignored. An operator has no signal short of observing the behaviour in production and inferring backwards. Where a gap is not going to be closed soon, refusing it explicitly is strictly better than dropping it silently, and that would be a reasonable interim resolution for several of these.

Solution

Progress. Two of the five below are fixed on master, along with the separately-filed batching gaps; the table above reflects the current state rather than the state at filing.

Close the gaps, tracked individually:

• retainOrdering / retainKeyOrdering ignored when selecting the subscription type — #26405 (fixed by #26414 and #26421)
• retryDetails populated but never applied, so no dead letter policy is created — #26406
• most of ConsumerSpec unapplied: cryptoSpec, schemaProperties, consumerProperties, poolMessages, messagePayloadProcessorSpec — #26407
• forwardSourceMessageProperty not implemented — #26408
• negativeAckRedeliveryDelayMs not applied to the consumer — #26409 (fixed by #26415)

Related, filed separately because they are a different runtime and a different fix:

• Python: poolMessages / consumerProperties / messagePayloadProcessorSpec unapplied — #26410
• Python: negativeAckRedeliveryDelayMs unapplied — #26411 (fixed by #26413)

Filed separately and not duplicated here: #26390 / #26391 (batching — both closed by #26392 / #26393), #26397 (Python dead letter, PR #26400 open), #26403 (Go custom metric collectors, PR #26458 open).

Each sub-issue is independently fixable and none blocks another. Ordering by user impact, retainKeyOrdering first: it is the only one in the list whose absence silently breaks a guarantee the user explicitly asked for.

Alternatives

Fixing these as one change was considered and rejected: they touch different parts of instance.go, some need a design decision (whether an unimplementable field should refuse rather than ignore), and a single large PR would be hard to review and hard to revert selectively.

Anything else?

The audit was done against origin/master. Java is treated as the reference implementation throughout — I spot-checked the Java side for the rows that mattered (PulsarSource for retryDetails, BatchingUtils for batchingSpec, JavaInstanceRunnable for the guards) rather than auditing it exhaustively.

Are you willing to submit a PR?

• I'm willing to submit a PR!