Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate entries with platform constraints in requirements.txt regression #2446

Closed
keith opened this issue Nov 26, 2024 · 1 comment · Fixed by #2447
Closed

Duplicate entries with platform constraints in requirements.txt regression #2446

keith opened this issue Nov 26, 2024 · 1 comment · Fixed by #2447
Assignees
Milestone

Comments

@keith
Copy link
Member

keith commented Nov 26, 2024

If I have a requirements.in like this:

--extra-index-url https://download.pytorch.org/whl/cpu
torch==2.4.1; platform_machine != "x86_64"
torch==2.4.1+cpu; platform_machine == "x86_64"

Which I use to generate a universal requirements.txt with uv using: uv pip compile --universal requirements.in -o requirements_lock.txt --emit-index-url --no-strip-markers, I end up with multiple entries for torch in that lockfile:

SNIP
torch==2.4.1+cpu ; platform_machine == 'x86_64'
    # via -r requirements.in
torch==2.4.1 ; platform_machine != 'x86_64'
    # via -r requirements.in
SNIP

When I attempt to pull those in with:

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
    hub_name = "pip",
    python_version = "3.11",
    requirements_lock = "//:requirements_lock.txt",
)
use_repo(pip, "pip")

and use them with:

load("@pip//:requirements.bzl", "requirement")

alias(
    name = "foo",
    actual = requirement("torch"),
)

It fails with:

ERROR: Traceback (most recent call last):
	File "/private/var/tmp/_bazel_ksmiley/edd497f7df052e6d5ffcf384da842af2/external/rules_python~~pip~pip/torch/BUILD.bazel", line 9, column 35, in <toplevel>
		"//_config:is_python_3.11": "pip_311_torch_linux_x86_64_osx_x86_64_windows_x86_64",
Error: dictionary expression has duplicate key: "//_config:is_python_3.11"
ERROR: /private/var/tmp/_bazel_ksmiley/edd497f7df052e6d5ffcf384da842af2/external/rules_python~~pip~pip/torch/BUILD.bazel: no such target '@@rules_python~~pip~pip//torch:pkg': target 'pkg' not declared in package 'torch' defined by /private/var/tmp/_bazel_ksmiley/edd497f7df052e6d5ffcf384da842af2/external/rules_python~~pip~pip/torch/BUILD.bazel
ERROR: /Users/ksmiley/Downloads/pyrepro/BUILD.bazel:3:6: no such target '@@rules_python~~pip~pip//torch:pkg': target 'pkg' not declared in package 'torch' defined by /private/var/tmp/_bazel_ksmiley/edd497f7df052e6d5ffcf384da842af2/external/rules_python~~pip~pip/torch/BUILD.bazel and referenced by '//:foo'
ERROR: Analysis of target '//:foo' failed; build aborted: Analysis failed

This setup temporarily worked after this landed #2377 but then regressed in 2abca35

🔬 Minimal Reproduction

Build in this project: pyrepro.zip

@aignas
Copy link
Collaborator

aignas commented Nov 27, 2024

Thanks for the repro

aignas added a commit to aignas/rules_python that referenced this issue Nov 27, 2024
…ms set

It seems that during bazelbuild#2424 I broke the rendering of aliases for the
cases when the target platform is set. This means that the feature for
multiplatform whls when `experimental_index_url` has never worked even
though it was advertised. This ensures that the rendering is happening
correctly and adds extra missing tests.

Whilst at it:
- add an extra test for `pip.parse` handling of env markers that I added
  to ensure that the error is not in the module extension.
- Cleanup unused code - error message constant and the repo arg in
  `whl_config_setting`.

Fixes bazelbuild#2446
@aignas aignas added this to the v1.0.0 milestone Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants