Be able to specify which version of setuptools has to be used

[rvanlaar]

• [x ] I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Feature Request

I just ran into this bug: tikitu/jsmin#34
Setuptools >=58 removed functionality jsmin relies on.
The result is that jsmin can't be installed and our CI/CD pipeline fails.

My solution was to run the following before a build in out CI: poetry run pip install --upgrade setuptools==57.5.0
To counter this, it would be great if the setuptools version can be specified in the pyproject.toml.

[intgr]

I had the same issue with demjson package. Was quite surprised that poetry ignores setuptools version pinning.

% poetry init -n
% poetry shell
% poetry add setuptools==57.2.0
% poetry install
Installing dependencies from lock file
% pip freeze --all |grep setuptools
setuptools==58.0.4
% grep setuptools pyproject.toml
setuptools = "57.2.0"
% poetry add demjson
  • Installing demjson (2.2.4): Failed
[...]
      Complete output (1 lines):
      error in demjson setup command: use_2to3 is invalid.
[...]

[StevenMMortimer]

Big thanks @rvanlaar! Your suggestion works well in a Docker container. I've implemented by adding the poetry pip install of setuptools before running poetry install

RUN pip install --no-cache-dir 'poetry==1.1.5'
RUN poetry run pip install 'setuptools==57.5.0'
RUN poetry install --no-root

[finswimmer]

Hello,

for poetry <1.2 we have marked setuptools as "unsafe" to remove or change because poetry relies on it. We removed the limitation in the current preview release. You can now add setuptools to your pyproject.toml like any other dependency.

fin swimmer

[rvanlaar]

Hi,

Thank you for your response and confirmation that the issue can be handled in poetry 1.2.

I'm wondering, would it be possible to specify the setuptools version to use in the [build-sytem] part of the configuration?
In this case, the setuptools pin is only needed during builds.

Roland

[lerouxrgd]

Still failing on 1.2.0:

Because xxxx depends on setuptools (==57.5.0) which doesn't match any versions, version solving failed.

[intgr]

Works for me with Poetry 1.2.0

% poetry init -n
% poetry shell
% poetry --version
Poetry (version 1.2.0)
% poetry add setuptools==57.2.0
[...]
Package operations: 0 installs, 1 update, 0 removals

  • Updating setuptools (65.3.0 -> 57.2.0)
% pip freeze --all |grep setuptools
setuptools==57.2.0

[heimalne]

In Poetry 1.4.1 we apparently can't override the setuptools version. Running poetry install on an older gym==0.21.0 version should work with an older setuptools < 66.0.0 version but fails on

# file: pyproject.toml
[tool.poetry]
name = "dummy_projectgym"
version = "0.0.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = ">=3.9,<3.11"
poetry = "1.4.1"
setuptools = "65.6.3"         # <-------------- doesn't seem to be used
gym = "0.21"                       # <--------------

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

with an error

error in gym setup command: 'extras_require' must be a dictionary whose values are strings or lists of strings containing valid project/version requirement specifiers.

Adding setuptools to the build-system section didn't help either.

[build-system]
requires = ["poetry-core>=1.0.0", "setuptools==65.6.3"]

The only hacky workaround i found was installing the problematic package through pip within poetry beforehand:

$ poetry run pip install gym==0.21.0
$ poetry install

Somehow that pip instance uses an older setuptools version (probably the default 58 one) but no idea why.

[clintonroy]

The other bit of the error is

Note: This error originates from the build backend, and is likely not a problem with poetry but with gym (0.21.0) not supporting PEP 517 builds. You can verify this by running 'pip wheel --use-pep517 "gym (==0.21.0)"'.

Which fails in the same way. i.e. this is not a poetry bug, but an issue with that old version of gym, newer versions of gym (e.g. 0.26.2) are fine; use the most recent version of gym.

[heimalne]

this is not a poetry bug, but an issue with that old version of gym, newer versions of gym (e.g. 0.26.2) are fine; use the most recent version of gym.

If i could, i would ;). But in a corporate environment you're often stuck with older versions for a while. The issue with Poetry is that it doesn't use the mentioned setuptools package (with pip we can). Correct me if i'm wrong in that this is a bug in Poetry.

[dimbleby]

you are wrong, this is not a bug in poetry.

(A setuptools dependency in pyproject.toml describes the package that should eventually be installed in the project environment. This has nothing to do with any temporary environment used to build packages such as gym.)

[heimalne]

(A setuptools dependency in pyproject.toml describes the package that should eventually be installed in the project environment. This has nothing to do with any temporary environment used to build packages such as gym.)

Thanks for the clarification. Is it possible then to specify the setuptools version for that temporary build environment? At least pip is able to use the final project environment setuptools.

[dimbleby]

No. It would be the responsibility of gym to declare its build requirements if for some reason it wanted to rely on an old version of setuptools.

(poetry always uses PEP517 builds with isolated build environments. You could reproduce this in pip, if you wanted to, by adding --use-pep517 to your command line)

[geiszla]

No. It would be the responsibility of gym to declare its build requirements if for some reason it wanted to rely on an old version of setuptools.

(poetry always uses PEP517 builds with isolated build environments. You could reproduce this in pip, if you wanted to, by adding --use-pep517 to your command line)

Not being able to specify the setuptools version will break all older repos eventually and will hurt reproduceability.

Currently it is impossible to install stable-baselines3 using Poetry because of its gym==0.21 dependency that requires setuptools@<=65.5.0 (and possibly many other un/less maintained repos as well). Waiting for the maintainers to fix it also won't help, because many of these repos are not maintained anymore (or development is slow). Correct me if I'm wrong, but as I see this will mean that Poetry won't be useable for older projects (or will require significant manual setup efforts) and it won't be guaranteed that one can install the exact required dependencies using Poetry (the whole point of poetry.lock).

I feel like Poetry shouldn't be the one protecting developers from messing with their setuptools versions. They must know the risks of doing so if it's necessary for some reason (like the one mentioned above).

[dtch1997]

Being unable to install Stable Baselines3 with Poetry is very annoying.

The only hacky workaround i found was installing the problematic package through pip within poetry beforehand:

$ poetry run pip install gym==0.21.0
$ poetry install

This worked for me, but it's unpleasant to have to do this.

[geiszla]

In the meanwhile we switched to a conda-based environment, where stables-baselines3 is installed through conda, while most of the other dependencies are installed through poetry. It's a bit messy setup, but it seems to work (also, this way we can install some other packages, which are not on PyPI). There are some nice instructions on how to do this in this SO answer.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.