#5780·kedro

Add a deploy-databricks skill

Author: SajidAlamQBCreated Sep 18, 2026Updated Sep 18, 2026
LabelsIssue: Feature Request

Description

Add a skill that guides an AI assistant through deploying a Kedro project to Databricks, following the same pattern as catalog-config: check what is installed, read current docs, and never rely on training data for plugin flags or platform details.

Context

Databricks is the most common deployment target. A review of #kedro-client-queries found the same questions recurring from 2024 to 2026 across three docs rewrites: which of the Databricks paths to use, how to package the project, and how to get configuration onto the platform.

Parent: kedro-org/kedro#2058.

Scope

  • Deploy the PySpark spaceflights starter to Databricks Free Edition by hand using kedro-databricks, and record every step and failure. This is the source material for the skill.
  • Write skills/deploy-databricks/SKILL.md covering: choosing between Git folders, Databricks Connect and asset bundles; packaging; environment-specific conf/ and --conf-source; the conf/local requirement; Unity Catalog volumes rather than DBFS; secrets from Databricks secret scopes; choosing task granularity with the plugin's resource generator; running databricks bundle validate before deploying.
  • State clearly what is outside Kedro (cluster configuration, permissions, Terraform errors) and point to Databricks documentation for it.
  • Registry entry with path globs for databricks.yml, Databricks conf files and CI workflows.
  • Packaged-skill tests, and a docs row in "Vibe coding with skills".

Acceptance With the skill installed, an assistant gives correct answers to the recurring client questions, using the April 2026 thread (asset bundle with wheel vs plugin vs Git folders, nodes to jobs, cluster config) as the reference case: it answers the first two and defers the third to Databricks.

Follow-up Once the pre-flight grouping validation lands in core, the skill should instruct the assistant to run it before generating a bundle, https://github.com/kedro-org/kedro/issues/5778.