Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Entry in .git_archival.txt makes repository artifacts non-reproducible #1394

Open
dvzrv opened this issue Apr 21, 2024 · 4 comments
Open

Entry in .git_archival.txt makes repository artifacts non-reproducible #1394

dvzrv opened this issue Apr 21, 2024 · 4 comments
Assignees
Labels
Bug Bug report in proxy server

Comments

@dvzrv
Copy link

dvzrv commented Apr 21, 2024

Describe the bug
Hi 👋

We are currently rebuilding all relevant packages against Python 3.12 on Arch Linux.

While revisiting the proxy.py package I noticed that it uses setuptools_scm and a .git_archival.txt setup which makes the artifacts of this repository non-reproducible.

As we would like to be able to rely on transparent sources (those not created by a custom process) for our packages, it would be great if you could revise/ remove the .git_archival.txt file from this repository.

To Reproduce
Steps to reproduce the behavior:

  1. Download an auto-generated tarball of a tag/ tag commit right after its creation
  2. Download an auto-generated tarball of a tag/ tag commit after an additional commit has been done on the default branch
  3. The checksums do not match

Expected behavior
The artifacts of a repository (the auto-generated tarballs) do not change due to arbitrary changes to the repository done after a tag.

Version information

  • OS: all
  • Browser: n/a
  • Device: n/a
  • proxy.py Version: all with the .git_archival.txt

Additional context
The problem has been discussed upstream in pypa/setuptools-scm#806 and led to them revising (removing) the problematic content in question in their documentation (pypa/setuptools-scm#1033).

Screenshots
n/a

@dvzrv dvzrv added the Bug Bug report in proxy server label Apr 21, 2024
@abhinavsingh
Copy link
Owner

Including @webknjaz in the discussion. We already changed a few things within .git_archival.txt last month (see #1345). I also understand the concern that @dvzrv is raising (regarding non-reproducible artifacts). Can you provide some guidance here from your experience. Thank you

@webknjaz
Copy link
Contributor

webknjaz commented Apr 24, 2024

The guidance is the same as in many other repos where @dvzrv opened the same issue — use sdists on PyPI. They are immutable and are the source for the downstreams. Using regular Git instead of the exported archive should also work.
Proxy.py can also use https://github.com/re-actors/checkout-python-sdist to guarantee that all the necessary sources are actually available in sdists long-term.

@dvzrv
Copy link
Author

dvzrv commented Apr 24, 2024

The guidance is the same as in many other repos [..] — use sdists on PyPI.

No, the guidance should instead be to follow the revised upstream documentation of setuptools_scm (https://github.com/pypa/setuptools_scm/pull/1033/files) and remove known problematic settings from that file :)

Using regular Git instead of the exported archive should also work.

It doesn't (as one can not lock the file contents at a given tag due to the problematic settings in .git_archival.txt).

@abhinavsingh AFAIK this entire topic is basically down to two things (for you as project maintainer):

  • If you have to rely on building from auto-generated source tarballs and can not use the available overrides, use a .git_archival.txt without the problematic settings.
  • If you don't have to rely on building from auto-generated source tarballs at all, don't use a .git_archival.txt (as overrides exist)

@webknjaz
Copy link
Contributor

Ah, so one line can be removed and I that'll work. I'll need to adjust it in my projects too, then.

Git archive is something that is useful for non-tagged commits so I wouldn't remove the use of this feature. It's much more frictionless, than having to instruct the users to set env vars.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Bug report in proxy server
Projects
None yet
Development

No branches or pull requests

3 participants