Baike.dev
All toolsAI codingTrendingOpen sourceNewsSubmit
Log in
Back to tool/Back to issues
#40159·beam

[Bug]: Go SDK row coder encodes negative INT32 schema fields as 10 byte varints that Java and Python reject

Author: EliaaazzzCreated Sep 17, 2026Updated Sep 17, 2026
LabelsgoP2

What happened?

The Go SDK reflection row coder encodes int32 struct fields with EncodeVarInt(rv.Int()), the signed 64 bit varint, so a negative value takes 10 bytes: -1 becomes ff ff ff ff ff ff ff ff ff 01 (sdks/go/pkg/beam/core/graph/coder/row_encoder.go, the reflect.Int32 case shared with the other int kinds).

The other SDKs encode the INT32 schema type as the varint of the unsigned 32 bit form of the value, 5 bytes at most:

  • Java VarIntCoder calls VarInt.encode(int), which masks with 0xFFFFFFFFL, so -1 is ff ff ff ff 0f. VarInt.decodeInt throws varint overflow for the Go bytes because they decode to a negative long.
  • Python VarInt32Coder writes v & 0xFFFFFFFF (write_var_int32 in slow_stream.py) and read_var_int32 packs the decoded value with struct.pack('<I', v), which raises for the negative value produced by the Go bytes.

So a Go row with a negative int32 field cannot be read by the Java or Python SDK. Positive values are unaffected, and Go can already read the 5 byte form because reflect.Value.SetInt truncates to 32 bits.

This is the INT32 counterpart of #40151 (INT16). Found while working on #39684.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Prism Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Source: apache/beam

View original on GitHubView discussion on GitHub