Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export ZERO_AR_DATE for macos linker invocations #71931

Merged
merged 1 commit into from
May 26, 2020

Conversation

alexcrichton
Copy link
Member

This commit attempts to improve reproducibility of builds on macOS by
exporting the ZERO_AR_DATE=1 environment variable for all invocations
of the linker. While it looks like this env var is targeted at just the
ar command (which does actually read this) it appears that recent-ish
versions of the linker also read this environment variable. This
env var forces the linker to set a deterministic zero value for the
mtime in the N_OSO field of the object file.

Currently it's believe that older versions of the linker will simply
ignore this env var, while newer versions will read it and produce a
deterministic output for compilations with debuginfo.

Closes #47086
Closes #66568

@rust-highfive
Copy link
Collaborator

r? @eddyb

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 5, 2020
@joelpalmer joelpalmer added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2020
@crlf0710 crlf0710 added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 22, 2020
@eddyb
Copy link
Member

eddyb commented May 25, 2020

I'm not a macOS person but the explanation looks reasonable. Sorry for the delay!

@bors r+

@bors
Copy link
Contributor

bors commented May 25, 2020

📌 Commit d60ef36c876284a6eedd90df394ec79b63b6a9e2 has been approved by eddyb

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 25, 2020
@bors
Copy link
Contributor

bors commented May 26, 2020

⌛ Testing commit d60ef36c876284a6eedd90df394ec79b63b6a9e2 with merge 7bec456cd7c73c68787f244d5f1df6084ae20bf7...

@bors
Copy link
Contributor

bors commented May 26, 2020

💔 Test failed - checks-azure

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 26, 2020
This commit attempts to improve reproducibility of builds on macOS by
exporting the `ZERO_AR_DATE=1` environment variable for all invocations
of the linker. While it looks like this env var is targeted at just the
`ar` command (which does actually read this) it appears that recent-ish
versions of the linker *also* read this environment variable. This
env var forces the linker to set a deterministic zero value for the
mtime in the N_OSO field of the object file.

Currently it's believe that older versions of the linker will simply
ignore this env var, while newer versions will read it and produce a
deterministic output for compilations with debuginfo.

Closes rust-lang#47086
Closes rust-lang#66568
@alexcrichton
Copy link
Member Author

Looks like cp -r doesn't follow symlinks on Linux but does on macOS, causing a failure. Looks like cp -R doesn't follow symlinks on both!

@bors: r=eddyb

@bors
Copy link
Contributor

bors commented May 26, 2020

📌 Commit afd88f2 has been approved by eddyb

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 26, 2020
@bors
Copy link
Contributor

bors commented May 26, 2020

⌛ Testing commit afd88f2 with merge 5239f5c...

@bors
Copy link
Contributor

bors commented May 26, 2020

☀️ Test successful - checks-azure
Approved by: eddyb
Pushing 5239f5c to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label May 26, 2020
@bors bors merged commit 5239f5c into rust-lang:master May 26, 2020
@alexcrichton alexcrichton deleted the reproducible-macos branch May 26, 2020 21:01
@pnkfelix
Copy link
Member

Hey @alexcrichton , why did you restrict this to just Mac OS?

I am seeing the reproducible-build-2 test fail once again on a build of rustc with debug=true (just like #66568 and I think #47086), but this time on a Linux host.

So I am wondering: What's the downside of exporting this environment variable on all platforms? Or, all unix platforms at least?

@eddyb
Copy link
Member

eddyb commented Oct 15, 2021

@pnkfelix I can't find anything from a quick googling that suggests this applies to anything other than Apple's own ld64 linker (or lld's Mach-O mode, which emulates it).

@pnkfelix
Copy link
Member

Hmm. So either Linux machines do not run into this embedding of an unstable mtime value (and my local test is failing for some other reason), or Linux machines do run into it, but the workaround deployed here does not apply.

Does that sound correct?

@eddyb
Copy link
Member

eddyb commented Oct 15, 2021

Does that sound correct?

Yes, that matches my understanding. I remember recently running into something with tar, where the defaults are atrocious1, but I thought ar wasn't subject to the same issues.

1. --sort=name --mtime=1970-01-01Z --numeric-owner --owner=0 --group=0 are the flags needed by tar - note the Z, without it, it depends on your timezone, that was "fun" to debug. and --mtime=0 means "start of today" or some useless thing like that, so you have to put in the UNIX epoch by hand

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

reproducible-build-2 fails if debuginfo-level > 0 OSX compilation with debuginfo isn't deterministic
7 participants