[Feature] Support pluggable external catalog SPI for Apache Gravitino integration
Motivation
Apache Gravitino already provides engine connectors for Spark, Trino, Flink, and Daft, allowing engines to use Gravitino as a unified metadata catalog.
For Spark, this integration works because Spark provides a pluggable Catalog Plugin interface. A third-party connector can be packaged as a JAR, configured through spark.sql.catalog.*, and loaded without modifying Spark core.
For StarRocks, I would like to explore a similar integration:
CREATE EXTERNAL CATALOG gravitino_catalog
PROPERTIES (
"type" = "gravitino",
"uri" = "http://gravitino:8090",
"metalake" = "demo"
);The goal is for StarRocks to discover and query catalogs managed by Gravitino, such as Iceberg, Hive, JDBC, and other metadata sources.
Current Situation
StarRocks already supports multiple external catalog types, including Hive, Iceberg, Hudi, Delta Lake, JDBC, Paimon, Elasticsearch, and Unified Catalog.
StarRocks can also connect to Gravitino's Iceberg REST endpoint for Iceberg tables.
However, this only covers Iceberg semantics. It does not provide a general Gravitino engine connector that exposes Gravitino's unified multi-catalog model to StarRocks.
Question
Does StarRocks currently provide, or plan to provide, a stable SPI for adding third-party external catalog implementations?
If not, would the community prefer one of the following approaches?
- Add a pluggable external catalog SPI so projects like Apache Gravitino can provide connectors independently.
- Add a built-in
gravitinoexternal catalog type inside StarRocks. - Continue relying on existing catalog types such as Iceberg REST and JDBC, without a general Gravitino integration.
References
- Apache Gravitino Spark connector: https://gravitino.apache.org/docs/1.3.0/spark-connector/spark-connector/
- Apache Gravitino Iceberg REST with StarRocks: https://gravitino.apache.org/docs/1.3.0/iceberg-rest-engine/starrocks/
- Gravitino StarRocks catalog: https://gravitino.apache.org/docs/1.3.0/jdbc-starrocks-catalog/
- Related Gravitino issue: https://github.com/apache/gravitino/issues/7822
Source: StarRocks/starrocks