Parquet: reading variant columns fails with NoSuchMethodError when commons-lang3 < 3.13 is on the classpath

Apache Iceberg version

1.11.0 (latest release) — also reproduced on the 1.12.0 RC0 staging artifacts and present on main

Query engine

Other (plain iceberg-core + iceberg-parquet + iceberg-data; any engine that ships commons-lang3 < 3.13 is affected)

Please describe the bug

Reading a Parquet file that contains a variant column fails with

java.lang.NoSuchMethodError: 'java.util.stream.Stream org.apache.commons.lang3.stream.Streams.of(java.lang.Iterable)'
    at org.apache.iceberg.parquet.ParquetVariantReaders.children(ParquetVariantReaders.java:488)
    at org.apache.iceberg.parquet.ParquetVariantReaders.children(ParquetVariantReaders.java:481)
    at org.apache.iceberg.parquet.ParquetVariantReaders$ShreddedVariantReader.<init>(ParquetVariantReaders.java:237)
    at org.apache.iceberg.parquet.ParquetVariantReaders.shredded(ParquetVariantReaders.java:78)
    at org.apache.iceberg.parquet.VariantReaderBuilder.value(VariantReaderBuilder.java:140)

whenever commons-lang3 older than 3.13.0 is on the classpath.

Root cause

ParquetVariantReaders.children(Iterable) calls org.apache.commons.lang3.stream.Streams.of(Iterable), which was added in commons-lang3 3.13.0:

https://github.com/apache/iceberg/blob/apache-iceberg-1.11.0/parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java#L484-L490

iceberg-parquet does not declare commons-lang3 as a dependency (it is not in gradle/libs.versions.toml nor in the iceberg-parquet block of build.gradle), so at runtime the class resolves against whatever version an engine or another transitive dependency happens to provide. In the Iceberg Gradle build the test classpath resolves to a recent version, so the unit tests pass, but many consumers get 3.12.0:

• Hadoop 3.3.x and 3.4.x (hadoop-client-runtime) ship commons-lang3 3.12.0
• orc-core 1.9.x (pulled by iceberg-data) depends on commons-lang3 3.12.0
• Spark 3.5 and Flink 1.20 ship commons-lang3 3.12.0

The usage was introduced by #12139 (Parquet variant readers, 1.9.0), so this affects 1.9.0 through 1.11.0 and the 1.12.0 RC. It is not a regression in 1.12.0.

Reproducer

Minimal Maven project with a JUnit test: (attach iceberg-variant-lang3-repro.zip, or paste the test below). The test creates a v3 table with a variant column, writes one Parquet file with GenericParquetWriter and reads it back with IcebergGenerics. The variant values are null on purpose: the failure happens while building the reader, before any value is decoded.

mvn -q test                          # fails (commons-lang3 3.12.0)
mvn -q test -Dlang3.version=3.18.0   # passes

Suggested fix

Drop the commons-lang3 usage in ParquetVariantReaders.children; the bundled Guava already covers it:

private static List<TripleIterator<?>> children(Iterable<ParquetValueReader<?>> readers) {
  return ImmutableList.copyOf(
      Iterables.concat(
          Iterables.transform(
              Iterables.filter(readers, Objects::nonNull), ParquetValueReader::columns)));
}

Alternatively declare commons-lang3 explicitly in iceberg-parquet, but that adds a new runtime dependency and still leaves engines that pin 3.12.0 in the classpath with a conflict, so removing the usage seems preferable. I can open a PR for either.

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time