Hang when trying to resolve dependencies for faust-large-message-serializer = "^2.0.1"

[mathieu-lemay]

• Poetry version: 1.4.2
• Python version: 3.11.3
• OS version and name: Arch Linux
• pyproject.toml: See below

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Poetry hangs when trying to resolve dependencies for faust-large-message-serializer = "^2.0.1". The issue can be reproduced with this pyproject.toml file, by running poetry lock.

[tool.poetry]
name = "faust-bug"
version = "0.1.0"
description = ""
authors = ["Mathieu Lemay <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"
faust-large-message-serializer = "^2.0.1"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

It seems to be related to urllib3 now being at version 2.0.2 (versions 2.0.0 and 2.0.1 have been yanked). If I add the following dependency, everything works as expected: urllib3 = "<2".

[Jeremiah-England]

I can also reproduce with this:

[tool.poetry]
name = "poetry-test"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"
packages = [{include = "poetry_test"}]

[tool.poetry.dependencies]
python = "^3.10"
requests = "^2.30.0"
boto3 = "^1.9"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

To me the problem appears to be that:

1. requests supports urllib3 >=1.21.1,<3 2.0
2. requests dependencies are resolved first. The latest urllib3 does not conflict with any of boto3's direct dependencies. So it is installed selected.
3. Then boto3 starts to lock its dependencies but one of its grandchildren dependencies required urllib3 < 2.0.
4. The grandchild keeps getting its version number decremented trying to find something that works for the urllib3 2.x that we have already installed selected.

Here are the terminated -vvv logs:

Using virtualenv: /home/jje/.cache/pypoetry/virtualenvs/poetry-test-HVesM_gM-py3.10
   1: fact: poetry-test is 0.1.0
   1: derived: poetry-test
   1: fact: poetry-test depends on requests (^2.30.0)
   1: fact: poetry-test depends on boto3 (^1.9)
   1: selecting poetry-test (0.1.0)
   1: derived: boto3 (>=1.9,<2.0)
   1: derived: requests (>=2.30.0,<3.0.0)
Source (PyPI): 1096 packages found for boto3 >=1.9,<2.0
Source (PyPI): 1 packages found for requests >=2.30.0,<3.0.0
   1: fact: requests (2.30.0) depends on charset-normalizer (>=2,<4)
   1: fact: requests (2.30.0) depends on idna (>=2.5,<4)
   1: fact: requests (2.30.0) depends on urllib3 (>=1.21.1,<3)
   1: fact: requests (2.30.0) depends on certifi (>=2017.4.17)
   1: selecting requests (2.30.0)
   1: derived: certifi (>=2017.4.17)
   1: derived: urllib3 (>=1.21.1,<3)
   1: derived: idna (>=2.5,<4)
   1: derived: charset-normalizer (>=2,<4)
Source (PyPI): 29 packages found for certifi >=2017.4.17
Source (PyPI): 34 packages found for urllib3 >=1.21.1,<3
Source (PyPI): 11 packages found for idna >=2.5,<4
Source (PyPI): 18 packages found for charset-normalizer >=2,<4
   1: selecting idna (3.4)
   1: selecting charset-normalizer (3.1.0)
   1: selecting certifi (2022.12.7)
   1: selecting urllib3 (2.0.2)
   1: fact: boto3 (1.26.126) depends on botocore (>=1.29.126,<1.30.0)
   1: fact: boto3 (1.26.126) depends on jmespath (>=0.7.1,<2.0.0)
   1: fact: boto3 (1.26.126) depends on s3transfer (>=0.6.0,<0.7.0)
   1: selecting boto3 (1.26.126)
   1: derived: s3transfer (>=0.6.0,<0.7.0)
   1: derived: jmespath (>=0.7.1,<2.0.0)
   1: derived: botocore (>=1.29.126,<1.30.0)
Source (PyPI): 1 packages found for s3transfer >=0.6.0,<0.7.0
Source (PyPI): 11 packages found for jmespath >=0.7.1,<2.0.0
Source (PyPI): 1 packages found for botocore >=1.29.126,<1.30.0
   1: fact: s3transfer (0.6.0) depends on botocore (>=1.12.36,<2.0a.0)
   1: selecting s3transfer (0.6.0)
   1: fact: botocore (1.29.126) depends on jmespath (>=0.7.1,<2.0.0)
   1: fact: botocore (1.29.126) depends on python-dateutil (>=2.1,<3.0.0)
   1: fact: botocore (1.29.126) depends on urllib3 (>=1.25.4,<1.27)
   1: derived: not botocore (==1.29.126)
   1: fact: no versions of botocore match >1.29.126,<1.30.0
   1: conflict: no versions of botocore match >1.29.126,<1.30.0
   1: derived: not botocore (>1.29.126,<1.30.0)
   1: conflict: botocore (1.29.126) depends on urllib3 (>=1.25.4,<1.27)
   1: ! botocore (==1.29.126) is partially satisfied by not botocore (>1.29.126,<1.30.0)
   1: ! which is caused by "no versions of botocore match >1.29.126,<1.30.0"
   1: ! thus: botocore (>=1.29.126,<1.30.0) requires urllib3 (>=1.25.4,<1.27)
   1: fact: botocore (>=1.29.126,<1.30.0) requires urllib3 (>=1.25.4,<1.27)
   1: derived: not botocore (>=1.29.126,<1.30.0)
   1: derived: not boto3 (==1.26.126)
Source (PyPI): 1095 packages found for boto3 >=1.9,<1.26.126 || >1.26.126,<2.0
   1: fact: boto3 (1.26.125) depends on botocore (>=1.29.125,<1.30.0)
   1: fact: boto3 (1.26.125) depends on jmespath (>=0.7.1,<2.0.0)
   1: fact: boto3 (1.26.125) depends on s3transfer (>=0.6.0,<0.7.0)
   2: selecting boto3 (1.26.125)
   2: derived: s3transfer (>=0.6.0,<0.7.0)
   2: derived: jmespath (>=0.7.1,<2.0.0)
   2: derived: botocore (>=1.29.125,<1.30.0)
Source (PyPI): 1 packages found for botocore >=1.29.125,<1.29.126
   2: fact: s3transfer (0.6.0) depends on botocore (>=1.12.36,<2.0a.0)
   2: selecting s3transfer (0.6.0)
   2: fact: botocore (1.29.125) depends on jmespath (>=0.7.1,<2.0.0)
   2: fact: botocore (1.29.125) depends on python-dateutil (>=2.1,<3.0.0)
   2: fact: botocore (1.29.125) depends on urllib3 (>=1.25.4,<1.27)
   2: derived: not botocore (==1.29.125)
   2: fact: no versions of botocore match >1.29.125,<1.29.126
   2: conflict: no versions of botocore match >1.29.125,<1.29.126
   2: derived: not botocore (>1.29.125,<1.29.126)
   2: conflict: botocore (1.29.125) depends on urllib3 (>=1.25.4,<1.27)
   2: ! botocore (==1.29.125) is partially satisfied by not botocore (>1.29.125,<1.29.126)
   2: ! which is caused by "no versions of botocore match >1.29.125,<1.29.126"
   2: ! thus: botocore (>=1.29.125,<1.29.126) requires urllib3 (>=1.25.4,<1.27)
   2: fact: botocore (>=1.29.125,<1.29.126) requires urllib3 (>=1.25.4,<1.27)
   2: derived: not botocore (>=1.29.125,<1.29.126)
   2: derived: not boto3 (==1.26.125)

[dimbleby]

Not a bug, see also #7858 ,#7856. Please close.

[Jeremiah-England]

Oh, I see. So it would have eventually resolved. It's just that there are tons of botocore packages it tries first.

In my case, restricting boto3 to the latest version solves the resolution time issue without needing to add an explicit urllib3 = "<2" dependency. Thanks!

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.