Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preserve directory-level standalone build symlinks #9723

Merged
merged 2 commits into from
Dec 10, 2024
Merged

Conversation

charliermarsh
Copy link
Member

@charliermarsh charliermarsh commented Dec 8, 2024

Summary

This PR improves our "don't fully resolve symlinks" behavior for python-build-standalone builds based on learnings from astral-sh/python-build-standalone#380 (comment).

Specifically, we can now robustly detect whether a target executable will lead to a valid prefix or not, and iteratively resolve symlinks until we find a valid target executable.

Test Plan

Direct symlink to python

Correctly resolves to the symlink target, rather than the symlink itself.

❯ ln -s /Users/crmarsh/.local/share/uv/python/cpython-3.12.6-macos-aarch64-none/bin/python foo
❯ cargo run venv --python ./foo
❯ cat .venv/pyvenv.cfg
home = /Users/crmarsh/.local/share/uv/python/cpython-3.12.6-macos-aarch64-none/bin
implementation = CPython
uv = 0.5.7
version_info = 3.12.6
include-system-site-packages = false
prompt = uv
❯ .venv/bin/python -c "import sys"

Symlink to the Python installation

Correctly does not resolve the symlink.

❯ ln -s /Users/crmarsh/.local/share/uv/python/cpython-3.12.6-macos-aarch64-none bar
❯ cargo run venv --python ./bar
❯ cat .venv/pyvenv.cfg
home = /Users/crmarsh/workspace/uv/bar/bin
implementation = CPython
uv = 0.5.7
version_info = 3.12.6
include-system-site-packages = false
prompt = uv
❯ .venv/bin/python -c "import sys"

Direct symlink to python in a symlinked Python installation

Correctly resolves the direct symlink, but not the symlink of the Python installation.

❯ ln -s bar/bin/python baz
❯ cargo run venv --python ./baz
❯ cat .venv/pyvenv.cfg
home = /Users/crmarsh/workspace/uv/bar/bin
implementation = CPython
uv = 0.5.7
version_info = 3.12.6
include-system-site-packages = false
prompt = uv
❯ .venv/bin/python -c "import sys"

@charliermarsh charliermarsh added the enhancement New feature or improvement to existing functionality label Dec 8, 2024
@charliermarsh
Copy link
Member Author

This is pretty handy because... symlinking the full directory is much more common, and useful?

@charliermarsh charliermarsh marked this pull request as draft December 8, 2024 16:13
@charliermarsh charliermarsh force-pushed the charlie/sym branch 2 times, most recently from 6f5c49e to 0e1a70e Compare December 8, 2024 17:29
@charliermarsh charliermarsh marked this pull request as ready for review December 8, 2024 17:36
@charliermarsh
Copy link
Member Author

charliermarsh commented Dec 9, 2024

Based on what I've learned here, I think the right thing to do would be: check if we can find STDLIB_LANDMARKS in any parent directory of the executable. If not, then we should iteratively resolve it until we find the first resolved symlink that does have STDLIB_LANDMARKS. This would make it a non-issue!

@charliermarsh
Copy link
Member Author

(I think I'll try rewriting to use the logic I mentioned above, since it will also work in more nuanced cases, like a direct symlink to a Python executable that's a symlink to a Python installation.)

@@ -58,22 +59,39 @@ pub(crate) fn create(
// considered the "base" for the virtual environment. This is typically the Python executable
// from the [`Interpreter`]; however, if the interpreter is a virtual environment itself, then
// the base Python executable is the Python executable of the interpreter's base interpreter.
let base_executable = interpreter
.sys_base_executable()
.unwrap_or(interpreter.sys_executable());
let base_python = if cfg!(unix) && interpreter.is_standalone() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it still correct to only gate this to standalone interpreters?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible that this would be equally applicable to other interpreters, but I'm a little more hesitant... The else behavior matches the standard library, and we're just less familiar with how those "other interpreters" behave.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. Fine with me, this logic seems very "correct" though.

@charliermarsh charliermarsh merged commit 6772cf8 into main Dec 10, 2024
64 checks passed
@charliermarsh charliermarsh deleted the charlie/sym branch December 10, 2024 20:41
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Dec 12, 2024
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.5.7` -> `0.5.8` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>astral-sh/uv (astral-sh/uv)</summary>

### [`v0.5.8`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#058)

[Compare Source](astral-sh/uv@0.5.7...0.5.8)

**This release does not include the `powerpc64le-unknown-linux-musl` target due to a build issue. See [#&#8203;9793](astral-sh/uv#9793) for details. If this change affects you, please file an issue with your use-case.**

##### Enhancements

-   Omit empty resolution markers in lockfile  ([#&#8203;9738](astral-sh/uv#9738))
-   Add `--install-dir` to to `uv python install` and `uninstall` commands ([#&#8203;7920](astral-sh/uv#7920))
-   Add `--show-urls` and `--only-downloads` to `uv python list` ([#&#8203;8062](astral-sh/uv#8062))
-   Add `uv python list --all-arches` ([#&#8203;9782](astral-sh/uv#9782))
-   Add `uv run --gui-script` flag for running Python scripts with `pythonw.exe` ([#&#8203;9152](astral-sh/uv#9152))
-   Allow `--gui-script` on Unix ([#&#8203;9787](astral-sh/uv#9787))
-   Allow download of Python distribution variants optimized for newer x86\_64 microarchitectures ([#&#8203;9781](astral-sh/uv#9781))
-   Allow execution of `pyw` files on Unix ([#&#8203;9759](astral-sh/uv#9759))
-   Allow users to specify URLs in `project.dependencies` and `tool.uv.sources` ([#&#8203;9718](astral-sh/uv#9718))
-   Encode mutually-incompatible pairs of markers ([#&#8203;9444](astral-sh/uv#9444))
-   Improve the error message when a Python install request is not valid ([#&#8203;9783](astral-sh/uv#9783))
-   Preserve directory-level standalone build symlinks ([#&#8203;9723](astral-sh/uv#9723))
-   Add support for `uv publish --index <name>` ([#&#8203;9694](astral-sh/uv#9694))
-   Reframe `--locked` and `--frozen` as `--check` operations for `uv lock` ([#&#8203;9662](astral-sh/uv#9662))
-   Rename Python install scratch directory from `.cache` -> `.temp` ([#&#8203;9756](astral-sh/uv#9756))
-   Enable `uv tool uninstall uv` on Windows ([#&#8203;8963](astral-sh/uv#8963))
-   Improve self-dependency hint to make shadowing clear ([#&#8203;9716](astral-sh/uv#9716))
-   Refactor unavailable metadata to shrink the resolver ([#&#8203;9769](astral-sh/uv#9769))
-   Show 'depends on itself' for proxy packages ([#&#8203;9717](astral-sh/uv#9717))
-   Show a dedicated error for missing subdirectories ([#&#8203;9761](astral-sh/uv#9761))
-   Show a dedicated hint for missing `git+` prefixes ([#&#8203;9789](astral-sh/uv#9789))

##### Performance

-   Eagerly error when parsing `pyproject.toml` requirements ([#&#8203;9704](astral-sh/uv#9704))
-   Use copy-on-write when normalizing paths ([#&#8203;9710](astral-sh/uv#9710))

##### Bug fixes

-   Avoid enforcing non-conflicts in `uv export` ([#&#8203;9751](astral-sh/uv#9751))
-   Don't drop comments between items in TOML tables ([#&#8203;9784](astral-sh/uv#9784))
-   Don't fail with `--no-build` when static metadata is available ([#&#8203;9785](astral-sh/uv#9785))
-   Don't filter non-patch registry version ([#&#8203;9736](astral-sh/uv#9736))
-   Don't read metadata from stale `.egg-info` files ([#&#8203;9760](astral-sh/uv#9760))
-   Enforce correctness of self-dependencies ([#&#8203;9705](astral-sh/uv#9705))
-   Fix projects's typo in resolver error messages ([#&#8203;9708](astral-sh/uv#9708))
-   Ignore `.` prefixed directories during managed Python installation discovery ([#&#8203;9786](astral-sh/uv#9786))
-   Improve handling of invalid virtual environments during interpreter discovery ([#&#8203;8086](astral-sh/uv#8086))
-   Normalize relative paths when `--project` is specified ([#&#8203;9709](astral-sh/uv#9709))
-   Respect self-constraints on recursive extras ([#&#8203;9714](astral-sh/uv#9714))
-   Respect user settings for tracing coloring ([#&#8203;9733](astral-sh/uv#9733))
-   Retry on tar extraction errors ([#&#8203;9753](astral-sh/uv#9753))
-   Add conflict markers to the lock file ([#&#8203;9370](astral-sh/uv#9370))
-   De-duplicate resolution markers ([#&#8203;9780](astral-sh/uv#9780))
-   Avoid 403 error hint for PyTorch URLs ([#&#8203;9750](astral-sh/uv#9750))
-   Avoid treating non-existent `--find-links` as relative URLs ([#&#8203;9720](astral-sh/uv#9720))
-   Omit Windows Store `python3.13.exe` et al ([#&#8203;9679](astral-sh/uv#9679))
-   Replace executables with broken symlinks during `uv python install` ([#&#8203;9706](astral-sh/uv#9706))

##### Documentation

-   Fix build failure links ([#&#8203;9740](astral-sh/uv#9740))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or improvement to existing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants