#19803·datahub

Iceberg source does not mark partition columns with isPartitioningKey

Author: yjyspreeCreated Sep 16, 2026Updated Sep 16, 2026

Describe the feature gap

The iceberg source does not populate SchemaField.isPartitioningKey, so partitioned Iceberg tables never show the "Partition Key" badge on their columns in the UI. Partitioning is only exposed as a partition-spec JSON string in the dataset's custom properties.

isPartitioningKey already exists in the metadata model and is populated by other sources (BigQuery, Unity Catalog, Hive, S3/GCS data lake), and the UI renders it (PartitioningKeyLabel in the schema table and field drawer). The Iceberg source is the outlier, and its Capabilities table on the docs page still says "Partition Support: Currently not supported."

The comment in _get_partition_aspect ("Since Datahub does not have a place in its model to store this information") predates the field and is no longer accurate for the column-level flag; it still holds for partition transforms (day, bucket[16], ...), which have no place in the schema model.

To Reproduce

  1. Create a partitioned Iceberg table, e.g. PARTITIONED BY (days(ts), region)
  2. Ingest it with the iceberg source
  3. Open the dataset's Schema tab: neither ts nor region carries the Partition Key badge
  4. The Properties tab shows partition-spec: [{"name": "ts_day", "transform": "day", "source": "ts", ...}, ...]

Expected behavior

The source columns of the table's partition spec (ts, region above) are flagged with isPartitioningKey=true, consistent with the other sources, so the UI shows the Partition Key badge. The partition-spec property should stay as-is since it carries the transform information the column flag cannot express.

Version

Reproduced on master (metadata-ingestion/src/datahub/ingestion/source/iceberg/iceberg.py, _get_schema_fields_for_schema only runs the Avro conversion) and on v1.6.0.

Additional context

I have a PR ready for this: it resolves each partition field's source_id via Schema.find_column_name() and sets the flag on the matching schema field (nested struct columns included), updates the Capabilities entry, and adds unit tests plus the integration golden files.