how to share large packages between projects/environments?
Author: fancyeriiCreated Feb 2, 2024Updated Feb 9, 2024
LabelsType: Question :grey_question:
I have many projects with shared large packages such as pytorch(4G+). Every time I create a new project I need to install it. It wastes time and disk space. Is there any way to share these large packages like maven(java) or docker.
Java's maven is project based-- it manage each projects indenpendently. But all the dependents(jars) are stored centrally in .m2. So If project A and project B share exactly the same version of a lib, There is only one copy of this jar file.
I tried and searched many python package manager tools: pip,conda,pipenv,poetry,PDM. No of them has this features. A similar issue can be found https://github.com/pypa/packaging-problems/issues/328. But I can't find solution there.
Source: pypa/pipenv