deltaLakeAzure throws NOT_IMPLEMENTED (Code 48) on schema-evolved tables

Description

Reading a DeltaLake table via deltaLakeAzure table function (or DeltaLakeAzure engine) fails with Code: 48. NOT_IMPLEMENTED when the table has undergone schema evolution (a column was added after initial creation).

Root cause

For Azure, isDeltaKernelEnabled returns false (only S3 and Local are supported), so the legacy DeltaLakeMetadataImpl reader is used. In DeltaLakeMetadata.cpp:299–305, processMetadataFile compares the schema from each metaData entry in the _delta_log/*.json transaction log against the first schema seen. When a column was added, a second metaData entry exists with the evolved schema — the != check fires and throws unconditionally, instead of taking the latest schema.

The S3 path works correctly because DeltaKernel handles schema evolution transparently.

How to reproduce

Step 1 — create a schema-evolved DeltaLake table on Azure using deltalake (delta-rs):

import pyarrow as pa
from deltalake import write_deltalake

storage_options = {
    "AZURE_STORAGE_ACCOUNT_NAME": "<account>",
    "AZURE_STORAGE_ACCOUNT_KEY": "<key>",
}
uri = "az://<container>/repro/schema_evo_test"

# Write initial table: 2 columns
write_deltalake(uri, pa.table({
    "id": pa.array([1, 2], type=pa.int64()),
    "value": pa.array(["a", "b"], type=pa.string()),
}), mode="overwrite", storage_options=storage_options)

# Append with a new column (schema evolution)
write_deltalake(uri, pa.table({
    "id": pa.array([3, 4], type=pa.int64()),
    "value": pa.array(["c", "d"], type=pa.string()),
    "extra": pa.array([100, 200], type=pa.int64()),
}), mode="append", schema_mode="merge", storage_options=storage_options)

Step 2 — query from ClickHouse:

SELECT * FROM deltaLakeAzure(
    'https://<account>.blob.core.windows.net',
    '<container>',
    'repro/schema_evo_test',
    '<account>',
    '<key>'
);

Error message

Code: 48. DB::Exception: Reading from files with different schema is not possible
(columns format version: 1
2 columns:
`id` Nullable(Int64)
`value` Nullable(String)
 is different from columns format version: 1
3 columns:
`id` Nullable(Int64)
`value` Nullable(String)
`extra` Nullable(Int64)
). (NOT_IMPLEMENTED)

Stack: DeltaLakeMetadataImpl::processMetadataFile → DeltaLakeMetadataImpl::processMetadataFiles → DeltaLakeMetadata::create

Expected behavior

ClickHouse reads the table using the latest schema, returning NULL for extra in rows written before the schema change — the same behaviour as S3 via DeltaKernel.

Affected surface

• deltaLakeAzure table function ❌
• DeltaLakeAzure table engine ❌
• deltaLake / DeltaLake on S3 ✅ (works via DeltaKernel)

Related

• #64291 — general DeltaLake schema evolution tracking issue