Poetry unable to resolve dependency using asterisk wildcard (*)

[matthewarmand]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Arch Linux

• Poetry version: 1.0.9

• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/matthewarmand/a281ea411867253662008becfe38aeab

Issue

I'm noticing an issue with using a wildcard (*) for a version specification for a dependency in pyproject.toml. In this new project, I'm adding a dev dependency of black for code formatting. Generally I prefer to allow any version for this, so I'm attempting to use the wildcard syntax (docs). The docs seem to indicate that my example gist above should work, but poetry is having trouble resolving the dependency.

In searching for other issues, this seems similar to #1281, though that's an older issue. I tried the mitigation steps at the end of that issue (deleting poetry.lock, etc) but it does not seem to help this case. I also noticed #2626, and I'm wondering if this is somehow related to that issue, perhaps a side effect. Opened this one separately though in case its a different issue.

Sample commands and output

When I try to do this via manual modification of pyproject.toml and using poetry install -vvv, I see the following output:

Using virtualenv: /home/matt/.cache/pypoetry/virtualenvs/example-k3bGbc8x-py3.8
Updating dependencies
Resolving dependencies...
   1: fact: example is 0.1.0
   1: derived: example
   1: fact: example depends on django (~2.2)
   1: fact: example depends on black (*)
   1: fact: example depends on black (*)
   1: selecting example (0.1.0)
   1: derived: black (*)
   1: derived: django (~2.2)
   1: fact: no versions of black match *
   1: conflict: no versions of black match *
   1: ! black (*) is satisfied by black (*)
   1: ! which is caused by "example depends on black (*)"
   1: ! thus: version solving failed
   1: Version solving took 0.017 seconds.
   1: Tried 1 solutions.

[SolverProblemError]
Because example depends on black (*) which doesn't match any versions, version solving failed.

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/usr/lib/python3.8/site-packages/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/usr/lib/python3.8/site-packages/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/usr/lib/python3.8/site-packages/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/usr/lib/python3.8/site-packages/poetry/console/commands/install.py", line 69, in handle
    return_code = installer.run()
  File "/usr/lib/python3.8/site-packages/poetry/installation/installer.py", line 74, in run
    self._do_install(local_repo)
  File "/usr/lib/python3.8/site-packages/poetry/installation/installer.py", line 161, in _do_install
    ops = solver.solve(use_latest=self._whitelist)
  File "/usr/lib/python3.8/site-packages/poetry/puzzle/solver.py", line 36, in solve
    packages, depths = self._solve(use_latest=use_latest)
  File "/usr/lib/python3.8/site-packages/poetry/puzzle/solver.py", line 190, in _solve
    raise SolverProblemError(e)

Alternatively, I also attempted to let poetry add the dependency using poetry add --dev -vvv black="*". The PyPI lines below seem particularly suspect, so maybe this is related to poetry's PyPI interaction in the wildcard case. In this case, I see the following output:

Using virtualenv: /home/matt/.cache/pypoetry/virtualenvs/example-k3bGbc8x-py3.8
PyPI: 20 packages found for black *

Updating dependencies
Resolving dependencies...
   1: fact: example is 0.1.0
   1: derived: example
   1: fact: example depends on django (~2.2)
   1: fact: example depends on black (*)
   1: fact: example depends on black (*)
   1: selecting example (0.1.0)
   1: derived: black (*)
   1: derived: django (~2.2)
PyPI: No release information found for django-1.0.1, skipping
PyPI: No release information found for django-1.0.2, skipping
PyPI: No release information found for django-1.0.3, skipping
PyPI: No release information found for django-1.0.4, skipping
PyPI: No release information found for django-1.1, skipping
PyPI: No release information found for django-1.1.1, skipping
PyPI: No release information found for django-1.1.2, skipping
PyPI: 15 packages found for django >=2.2,<2.3
PyPI: 0 packages found for black *
   1: fact: no versions of black match *
   1: conflict: no versions of black match *
   1: ! black (*) is satisfied by black (*)
   1: ! which is caused by "example depends on black (*)"
   1: ! thus: version solving failed
   1: Version solving took 0.018 seconds.
   1: Tried 1 solutions.

[SolverProblemError]
Because example depends on black (*) which doesn't match any versions, version solving failed.

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/usr/lib/python3.8/site-packages/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/usr/lib/python3.8/site-packages/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/usr/lib/python3.8/site-packages/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/usr/lib/python3.8/site-packages/poetry/console/commands/add.py", line 149, in handle
    status = installer.run()
  File "/usr/lib/python3.8/site-packages/poetry/installation/installer.py", line 74, in run
    self._do_install(local_repo)
  File "/usr/lib/python3.8/site-packages/poetry/installation/installer.py", line 161, in _do_install
    ops = solver.solve(use_latest=self._whitelist)
  File "/usr/lib/python3.8/site-packages/poetry/puzzle/solver.py", line 36, in solve
    packages, depths = self._solve(use_latest=use_latest)
  File "/usr/lib/python3.8/site-packages/poetry/puzzle/solver.py", line 190, in _solve
    raise SolverProblemError(e)

[matthewarmand]

I also did notice this advice at #743 (comment), but nothing else in this project depends on black (at least according to poetry.lock when I remove black) so it seems unlikely that there's some other hidden dependency further constraining the version.

[jaklan]

Exactly the same issue, also with black.

[peterdeme]

Yep, asterisk doesn't work with black on 1.0.10.

[wasdee]

This also affects the version specification in packages with setup.cfg .
akaihola/darker#52

[mdengler]

This issue still exists with poetry 1.0.10 and a dependency on black = "*" in pyproject.toml. A workaround, and observations:

Workaround

Use:

[tool.poetry.dependencies]
black = ">=20.8b1"

...instead of black = "*" or any other version specifier.

Observations

Is this related to the b1 version suffix black currently has?

$ python -m poetry search black | head -4

black (20.8b1)
 The uncompromising code formatter.

After rm -rf poetry.lock .venv:

1. If I use black = ">=20", poetry install fails
2. If I use black = ">=19.8b1", poetry install works (?!)
3. If I remove the dependency on black, poetry install works (with my pyproject.toml).

Environment / versions to reproduce

$ head -2 /etc/os-release
NAME=Fedora
VERSION="33 (Workstation Edition)"

$ python -m poetry --version
Poetry version 1.0.10

$ python -V
Python 3.9.2


$ rm poetry.lock
rm: cannot remove 'poetry.lock': No such file or directory

$ python -m poetry install
Updating dependencies
Resolving dependencies... (0.1s)

[SolverProblemError]
Because PACKAGENAME depends on black (*) which doesn't match any versions, version solving failed.


$ grep -C 3 black pyproject.toml


[tool.poetry.dev-dependencies]
black = "*"
flake8 = "*"
flake8-bugbear = "*"
ipython = "*"
--
[...]


[tool.black]
target-version = ['py38']  # no py39 yet
exclude = '''
(
--
'''

[tool.isort]
profile = "black"
multi_line_output = 3
extra_standard_library = ['pandas', 'numpy', 'dateutil', 'pytz', 'psycopg2', 'sqlalchemy']
color_output = true

$ python -m poetry install
Updating dependencies
Resolving dependencies... (0.1s)

[SolverProblemError]
Because PACKAGENAME depends on black (*) which doesn't match any versions, version solving failed.
$

[matthewarmand]

Fwiw @mdengler I'm no longer noticing this problem on poetry versions 1.1.x, is there a reason you're stuck on 1.0.x and can't update?

[mdengler]

Looks like I was just on an old version (thought I had updated to the latest but clearly not). So this bug can be closed?

[matthewarmand]

Probably yeah. I'd just left it open until a maintainer popped in but that hasn't happened, and since it seems to have been incidentally fixed as of 1.1.x, seems it can be closed.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.