Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle constraints on indirect dependencies #1522

Closed
raylu opened this issue Feb 16, 2024 · 6 comments · Fixed by #1796
Closed

handle constraints on indirect dependencies #1522

raylu opened this issue Feb 16, 2024 · 6 comments · Fixed by #1796
Assignees
Labels
bug Something isn't working great writeup A wonderful example of a quality contribution 💜 resolver Related to the package resolver

Comments

@raylu
Copy link

raylu commented Feb 16, 2024

if I have this requirements.in

virtualenv

and this requirements.txt

distlib==0.3.8
    # via virtualenv
filelock==3.8.0
    # via virtualenv
platformdirs==2.6.2
    # via virtualenv
virtualenv==20.16.5

then pip-compile requirements.in and uv pip compile requirements.in -o requirements.txt do the same thing (no functional changes to the txt file) so everything is great

if I add this constraints.txt

filelock==3.8.0

and rm requirements.txt, then pip-compile requirements.in -c constraints.txt produces

distlib==0.3.8
    # via virtualenv
filelock==3.8.0
    # via
    #   -c constraints.txt
    #   virtualenv
platformdirs==3.11.0
    # via virtualenv
virtualenv==20.21.1

pip-compile picked the last version of virtualenv that supported the old filelock (20.22.0 requires filelock>=3.11)

however,

$ uv pip compile requirements.in -c constraints.txt -v
[...]
        0.008705s   3ms DEBUG uv_resolver::resolver Searching for a compatible version of virtualenv (*)
        0.008872s   4ms DEBUG uv_resolver::resolver Selecting: virtualenv==20.25.0 (virtualenv-20.25.0-py3-none-any.whl)
   uv_resolver::resolver::get_dependencies package=virtualenv, version=20.25.0
     uv_resolver::resolver::distributions_wait package_id=virtualenv-20.25.0
              0.009378s   0ms DEBUG uv_client::cached_client Found fresh response for: https://files.pythonhosted.org/packages/83/22/54b1180756d2d6194bcafb7425d437c3034c4bff92129c3e1e633079e2c4/virtualenv-20.25.0-py3-none-any.whl
error: There are conflicting versions for `filelock`: `filelock>=3.12.2, <4` does not intersect with `filelock==3.8.0`
@zanieb
Copy link
Member

zanieb commented Feb 16, 2024

Interesting, thank you for the clear report!

@zanieb zanieb added the bug Something isn't working label Feb 16, 2024
@charliermarsh
Copy link
Member

Ah yeah, ok, I see the issue here.

@charliermarsh charliermarsh self-assigned this Feb 18, 2024
@charliermarsh
Copy link
Member

Have a fix but needs a few more test cases.

@zanieb zanieb added great writeup A wonderful example of a quality contribution 💜 resolver Related to the package resolver labels Feb 18, 2024
@edwardpeek-crown-public

Hit this as well in my experimentation if you want another real-world case. A one-liner repro:

# Doesn't work
uv pip compile <(printf '%s\n' 'psycopg[c]>=3') -c <(printf '%s\n' 'psycopg==3.1.13' 'psycopg-c==3.1.13')

# Works
uv pip compile <(printf '%s\n' 'psycopg[c]>=3') -o <(printf '%s\n' 'psycopg==3.1.13' 'psycopg-c==3.1.13')

@charliermarsh
Copy link
Member

Thank you! I'm hoping to fix it today.

charliermarsh added a commit that referenced this issue Feb 22, 2024
## Summary

This revives a PR from long ago
(#383 and
astral-sh/pubgrub#24) that modifies how we deal
with dependencies that are declared multiple times within a single
package.

To quote from the originating PR:

> Uses an experimental pubgrub branch (#370) that allows us to handle
multiple version ranges for a single dependency to the solver which
results in better error messages because the derivation tree contains
all of the relevant versions. Previously, the version ranges were merged
(by us) in the resolver before handing them to pubgrub since only one
range could be provided per package. Since we don't merge the versions
anymore, we no longer give the solver an empty range for conflicting
requirements; instead the solver comes to that conclusion from the
provided versions. You can see the improved error message for direct
dependencies in [this
snapshot](https://github.com/astral-sh/puffin/pull/383/files#diff-a0437f2c20cde5e2f15199a3bf81a102b92580063268417847ec9c793a115bd0).

The main issue with that PR was around its handling of URL dependencies,
so this PR _also_ refactors how we handle those. Previously, we stored
URL dependencies on `PubGrubPackage`, but they were omitted from the
hash and equality implementations of `PubGrubPackage`. This led to some
really careful codepaths wherein we had to ensure that we always visited
URLs before non-URL packages, so that the URL-inclusive versions were
included in any hashmaps, etc. I considered preserving this approach,
but it would require us to rely on lots of internal details of PubGrub
(since we'd now be relying on PubGrub to merge those packages in the
"right" order).

So, instead, we now _always_ set the URL on a given package, whenever
that package was _given_ a URL upfront. I think this is easier to reason
about: if the user provided a URL for `flask`, then we should just
always add the URL for `flask`. If we see some other URL for `flask`, we
error, like before. If we see some unknown URL for `flask`, we error,
like before.

Closes #1522.

Closes #1821.

Closes #1615.
@charliermarsh
Copy link
Member

Should be fixed in v0.1.7 (out now).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working great writeup A wonderful example of a quality contribution 💜 resolver Related to the package resolver
Projects
None yet
4 participants