pathspec < 1.1 upper bound blocks install on Apache Airflow 3.2+ / 2.11.1+
Is this a new bug in dbt-core?
- I believe this is a new bug in dbt-core
- I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
dbt-core>=1.12.0 still caps pathspec at <1.1 (raised from <0.13 in #12714, resolving #12385). Apache Airflow itself has since moved past that ceiling in its own official constraints files:
- Airflow 2.11.1 / 2.11.2 pin pathspec==1.0.4
- Airflow 3.2.0 pins pathspec==1.0.4
- Airflow 3.3.1 (current latest) pins pathspec==1.1.1
1.1.1 falls outside dbt-core's <1.1 allowance, so any package that depends on both dbt-core and apache-airflow (e.g. airflow-dbt-python, which many people run as an Airflow provider on Amazon MWAA) cannot be installed on any Airflow release from 2.11.1 onward, including every currently-supported MWAA version.
Expected Behavior
pathspec's own changelog doesn't show anything relevant to dbt between 1.0 and 1.1.1 that would justify the cap. It would be good to raise the upper bound again (or drop it, per the original ask in #9372) so dbt-core stays installable alongside current Airflow releases.
Steps To Reproduce
uv pip install "apache-airflow==3.3.1" "airflow-dbt-python[snowflake]==3.6.0"
--constraint https://raw.githubusercontent.com/apache/airflow/constraints-3.3.1/constraints-3.11.txt
Relevant log output
╰─▶ Because dbt-core>=1.12.0 depends on pathspec>=0.9,<1.1 and
pathspec>=0.9,<0.13, we can conclude that all of:
...
And because airflow-dbt-python==3.6.0 depends on dbt-core>=1.8.0 and you require airflow-dbt-python[snowflake]==3.6.0, we can conclude that your requirements are unsatisfiable.Environment
- OS: NA
- Python: 3.11
- dbt: 1.12.5 (also reproduces on 1.11.x, 1.10.x)Which database adapter are you using with dbt?
snowflake
Additional Context
This is what's currently blocking upgrading airflow-dbt-python past 3.0.1 for anyone running dbt on MWAA/Airflow — the previous blocker (a missing airflow.providers.common.compat.sdk module on older Airflow) is fixed on Airflow 2.11.1+, but this pathspec ceiling is now the only thing standing in the way.
Source: dbt-labs/dbt-core