Cannot add numpy when requires-python = ">=3.8,<3.13"
Describe the bug
I cannot use pdm to create a working environment with numpy that supports python back to 3.8 but which works with python 3.12. It seems like the locking mechanism is using -S direct_minimal_versions even if it is disabled, resulting in numpy 1.24.4, which is incompatible with python 3.12.
One potential workaround seems to be to lock specific versions of python.
pdm lock --python "==3.8.*"
pdm lock --python "==3.9.*" --append
pdm lock --python 3.10 --appendNow pdm install works, but this fails if the lock file does not exist. What I am supposed to include in pyproject.toml for my package to work in the wild?
Note: pip install seems to do the right thing, so maybe the latter concern is not an issue, but I was hoping to use PDM for everything (i.e. installing code for tests).
To reproduce
pdm init --python 3.12 -n
sed -i '' "s/==3\.12\.\*/>=3.8,<3.13/g" pyproject.toml # requires-python = ">=3.8,<3.13"
pdm add numpyWorkaround
This works
rm pdm.lock
pdm lock --python "==3.8.*"
pdm lock --python "==3.9.*" --append
pdm lock --python 3.10 --append
pdm installAbridged Output
...
⠴ 0:00:00 Resolve for environment (>=3.8,<3.13) 2 resolved, 0 to resolveINFO: Use `-q/--quiet` to suppress these warnings, or ignore them per-package with `ignore_package_warnings` config in [tool.pdm] table.
0:00:00 Lock successful.
Changes are written to pyproject.toml.
Synchronizing working set with resolved packages: 1 to add, 0 to update, 0 to remove
✖ Install numpy 1.24.4 failed
✖ Install numpy 1.24.4 failed
ERRORS:
add numpy failed:
Traceback (most recent call last):
...
File ".../pdm/lib/python3.11/site-packages/pyproject_hooks/_impl.py", line 402, in _call_hook
raise BackendUnavailable(
pyproject_hooks._impl.BackendUnavailable: Cannot import 'setuptools.build_meta'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
...
File ".../pdm/lib/python3.11/site-packages/pdm/builders/base.py", line 86, in wrapper
raise BuildError(str(e)) from e
pdm.exceptions.BuildError: Cannot import 'setuptools.build_meta'
0:00:02 ✖ Some package operations failed. 1/1
See /Users/mforbes/Library/Logs/pdm/pdm-install-146cih8y.log for detailed debug log.
[InstallationError]: Some package operations failed.
WARNING: Add '-v' to see the detailed tracebackExpected Behavior
PDM to install a version of numpy that works with python 3.12.
Environment Information
PDM version:
2.23.1
Python Interpreter:
.../tmp/pdm/bug_numpy/.venv/bin/python (3.12)
Project Root:
.../tmp/pdm/bug_numpy
Local Packages:
{
"implementation_name": "cpython",
"implementation_version": "3.11.9",
"os_name": "posix",
"platform_machine": "arm64",
"platform_release": "23.6.0",
"platform_system": "Darwin",
"platform_version": "Darwin Kernel Version 23.6.0: Thu Dec 19 20:44:10 PST 2024; root:xnu-10063.141.1.703.2~1/RELEASE_ARM64_T6000",
"python_full_version": "3.12.9",
"platform_python_implementation": "CPython",
"python_version": "3.12",
"sys_platform": "darwin"
}Verbose Command Output
After removing the lock file and with "numpy>=1.24.4":
$ pdm install -v
WARNING: Lockfile does not exist
Updating the lock file...
pdm.termui: ======== Start resolving requirements ========
pdm.termui: numpy>=1.24.4
pdm.termui: Adding requirement python<3.13,>=3.8
pdm.termui: Adding requirement numpy>=1.24.4
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.10 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.10" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.10 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.10" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.10 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.10" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.10 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.10" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.10 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.10" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.10 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.10" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.10 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.10" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.10 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.10" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.10 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.10" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python<3.13,>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python<3.13,>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/resolver/providers.py:190: PackageWarning: Skipping [email protected] because it requires Python>=3.9 but the lock targets to work with Python<3.13,>=3.8. Instead, another version of numpy that supports Python<3.13,>=3.8 will be used.
If you want to install [email protected], narrow down the `requires-python` range to include this version. For example, "<3.13,>=3.9" should work.
found = self.repository.find_candidates(
pdm.termui: ======== Starting round 0 ========
pdm.termui: Adding new pin: python None
pdm.termui: ======== Starting round 1 ========
pdm.termui: Adding new pin: numpy 1.24.4
pdm.termui: ======== Starting round 2 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: python None
pdm.termui: numpy 1.24.4
INFO: Use `-q/--quiet` to suppress these warnings, or ignore them per-package with `ignore_package_warnings` config in [tool.pdm] table.
pdm.termui: Fetching hashes for [email protected]
Changes are written to pdm.lock.
STATUS: Resolving packages from lockfile...
Synchronizing working set with resolved packages: 0 to add, 1 to update, 0 to remove
pdm.termui: Overwriting distribution numpy
pdm.termui: Using cached response for https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz
unearth.preparer: Downloading <Link https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz (from https://pypi.org/simple/numpy/)> (10.9 MB)
pdm.termui: Preparing environment(Isolated mode) for PEP 517 build...
pdm.termui: Saving wheel to cache: <Link https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz (from https://pypi.org/simple/numpy/)>
pdm.termui: Running PEP 517 backend to build a wheel for <Link https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz (from https://pypi.org/simple/numpy/)>
pdm.termui: ======== Start resolving requirements ========
pdm.termui: Adding requirement python==3.12.9
pdm.termui: Adding requirement Cython<3.0,>=0.29.30
pdm.termui: Adding requirement wheel==0.37.0
pdm.termui: Adding requirement setuptools==59.2.0
pdm.termui: ======== Resolution Result ========
pdm.termui: python None
pdm.termui: setuptools 59.2.0
pdm.termui: wheel 0.37.0
pdm.termui: cython 0.29.37
pdm.termui: Fetching hashes for [email protected]
pdm.termui: Fetching hashes for [email protected]
pdm.termui: Fetching hashes for [email protected]
pdm.termui: Installing [email protected]...
pdm.termui: Using cached response for https://files.pythonhosted.org/packages/7e/26/9d8de10005fedb1eceabe713348d43bae1dbab1786042ca0751a2e2b0f8c/Cython-0.29.37-py2.py3-none-any.whl
unearth.preparer: Downloading <Link https://files.pythonhosted.org/packages/7e/26/9d8de10005fedb1eceabe713348d43bae1dbab1786042ca0751a2e2b0f8c/Cython-0.29.37-py2.py3-none-any.whl (from https://pypi.org/simple/cython/)> (989 kB)
pdm.termui: Installing [email protected]...
pdm.termui: Using cached response for https://files.pythonhosted.org/packages/18/ad/ec41343a49a0371ea40daf37b1ba2c11333cdd121cb378161635d14b9750/setuptools-59.2.0-py3-none-any.whl
unearth.preparer: Downloading <Link https://files.pythonhosted.org/packages/18/ad/ec41343a49a0371ea40daf37b1ba2c11333cdd121cb378161635d14b9750/setuptools-59.2.0-py3-none-any.whl (from https://pypi.org/simple/setuptools/)> (952 kB)
pdm.termui: Installing [email protected]...
pdm.termui: Using cached response for https://files.pythonhosted.org/packages/04/80/cad93b40262f5d09f6de82adbee452fd43cdff60830b56a74c5930f7e277/wheel-0.37.0-py2.py3-none-any.whl
unearth.preparer: Downloading <Link https://files.pythonhosted.org/packages/04/80/cad93b40262f5d09f6de82adbee452fd43cdff60830b56a74c5930f7e277/wheel-0.37.0-py2.py3-none-any.whl (from https://pypi.org/simple/wheel/)> (35 kB)
pdm.termui: Synchronization complete.
✖ Update numpy 2.2.4 -> 1.24.4 failed
pdm.termui: Error occurs updating numpy:
Traceback (most recent call last):
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/builders/base.py", line 84, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/builders/wheel.py", line 24, in build
requires = self._hook.get_requires_for_build_wheel(self.config_settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pyproject_hooks/_impl.py", line 196, in get_requires_for_build_wheel
return self._call_hook(
^^^^^^^^^^^^^^^^
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pyproject_hooks/_impl.py", line 402, in _call_hook
raise BackendUnavailable(
pyproject_hooks._impl.BackendUnavailable: Cannot import 'setuptools.build_meta'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Volumes/Data/apps/conda_arm64/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/installers/synchronizers.py", line 50, in update_candidate
self.manager.overwrite(dist, can)
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/installers/manager.py", line 63, in overwrite
installed = self.install(candidate)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/installers/manager.py", line 33, in install
prepared.build(),
^^^^^^^^^^^^^^^^
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/models/candidates.py", line 417, in build
self._cached = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/builders/base.py", line 86, in wrapper
raise BuildError(str(e)) from e
pdm.exceptions.BuildError: Cannot import 'setuptools.build_meta'
Retry failed jobs(2/2)
pdm.termui: Overwriting distribution numpy
pdm.termui: Preparing environment(Isolated mode) for PEP 517 build...
pdm.termui: Saving wheel to cache: <Link https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz (from https://pypi.org/simple/numpy/)>
pdm.termui: Running PEP 517 backend to build a wheel for <Link https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz (from https://pypi.org/simple/numpy/)>
✖ Update numpy 2.2.4 -> 1.24.4 failed
pdm.termui: Error occurs updating numpy:
Traceback (most recent call last):
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packages/pdm/builders/base.py", line 84, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Volumes/Data/apps/pipx/venvs/pdm/lib/python3.11/site-packagesSource: pdm-project/pdm