Querying newly registered Iceberg table fails with "ClassNotFoundException: PrestoS3FileSystem"

Environment

Presto native cluster (Prestissimo) — Java coordinator with plugin.dir=/opt/presto-server/native-plugin/, C++/Velox workers.

How to reproduce

1. Register a new Iceberg table pointing at an S3 location:
   sql复制

   CALL iceberg.system.register_table(
       schema => 'myschema',
       table_name => 'my_table',
       metadata_location => 's3://my-bucket/my-table/metadata'
   );

2. With iceberg.pushdown-filter-enabled=true (the default), run any query:
   sql复制

   SELECT COUNT(*) FROM my_table;

3. Query fails immediately with:
   复制

   java.lang.ClassNotFoundException: Class com.facebook.presto.hive.s3.PrestoS3FileSystem not found

Other Iceberg tables that have been successfully queried before do not fail. The failure is specific to tables queried for the first time after registration.

Root cause

Each Presto plugin runs in its own PluginClassLoader. PrestoS3FileSystem is only visible to the hive-hadoop2/iceberg plugin ClassLoaders — not to the JVM System ClassLoader.

When iceberg.pushdown-filter-enabled=true, IcebergFilterPushdown.getConnectorPushdownFilterResult() runs during query planning on the main Presto planning thread. That thread's context ClassLoader is the System ClassLoader, not the plugin ClassLoader.

The full call chain:

IcebergFilterPushdown.getConnectorPushdownFilterResult()
  → IcebergUtil.getPartitionKeyColumnHandles(tableHandle, icebergTable, typeManager)
    → table.snapshot(snapshotId).allManifests(table.io())   // reads S3 manifest list
      → new Path(s3Uri).getFileSystem(conf)                 // creates S3 FileSystem
        → HiveCachingHdfsConfiguration.getConfiguration(context, uri)
          → defaultConfig = hiveHdfsConfiguration.getConfiguration(context, uri)
            // defaultConfig.classLoader = Thread.currentThread().getContextClassLoader()
            //                           = System ClassLoader   ← BUG
          → return new CachingJobConf(lambda, defaultConfig)
        → PrestoFileSystemCache.get(uri, cachingJobConf)
          → cachingJobConf.createFileSystem(uri)            // FileSystemFactory path, no cache
            → conf.getClassByName("com.facebook.presto.hive.s3.PrestoS3FileSystem")
              // uses defaultConfig.classLoader = System ClassLoader
              // System ClassLoader cannot find PrestoS3FileSystem
              → ClassNotFoundException  ✗

The underlying cause is that HiveHdfsConfiguration creates a Configuration per calling thread via a ThreadLocal, inheriting Thread.currentThread().getContextClassLoader() as the Configuration's classLoader field. When the calling thread is the planning thread, this classLoader is the System ClassLoader.

HiveCachingHdfsConfiguration.getConfiguration() wraps this Configuration in a CachingJobConf. PrestoFileSystemCache detects CachingJobConf instanceof FileSystemFactory and calls createFileSystem() directly, bypassing the FileSystem cache entirely. The lambda then calls conf.getClassByName("PrestoS3FileSystem"), which uses the thread-inherited System ClassLoader — and fails.

Why only newly registered (or newly invalidated) tables?

If an Iceberg Table object was previously loaded by an executor thread (which has the plugin ClassLoader as its context ClassLoader), that object can be served from cache on subsequent planning-time accesses. The cached table.io() was set up with the correct ClassLoader.

Newly registered tables have never been loaded from an executor thread. With hive.metastore.cache.disabled-caches=TABLE (common on Prestissimo), tables that have been modified also bypass the metastore cache, forcing a fresh load on every query from the planning thread. Both cases always hit this bug.

Secondary observation

With iceberg.pushdown-filter-enabled=true, SELECT COUNT(*) on an Iceberg table produces a TableScan operator — workers physically scan data files. With pushdown-filter-enabled=false, it produces a Values operator — the count is resolved entirely from Iceberg manifest metadata at planning time with no data file access. This indicates that IcebergFilterPushdown interferes with the aggregate pushdown optimizer (IcebergMetadataOptimizer) that would otherwise answer COUNT(*) from metadata statistics. This is a separate performance regression worth fixing independently.

Proposed fix

In HiveCachingHdfsConfiguration.getConfiguration(), pin the plugin ClassLoader on the Configuration immediately after obtaining it:

Configuration defaultConfig = hiveHdfsConfiguration.getConfiguration(context, uri);
// Pin the plugin ClassLoader so that conf.getClassByName() resolves plugin classes
// (e.g. PrestoS3FileSystem) regardless of which thread calls getConfiguration().
defaultConfig.setClassLoader(getClass().getClassLoader());

This ensures class resolution always uses the plugin ClassLoader, regardless of which thread (planning or executor) calls getConfiguration().

Workaround

Set the session property before querying:

SET SESSION iceberg.pushdown_filter_enabled = false;

Or set iceberg.pushdown-filter-enabled=false in the iceberg catalog properties. Note that this workaround causes SELECT COUNT(*) to use a TableScan instead of a metadata Values node, which is significantly slower for large tables.