Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert using MCP510 in bootstrap #118810

Merged
merged 1 commit into from
Dec 11, 2023
Merged

Conversation

Kobzol
Copy link
Contributor

@Kobzol Kobzol commented Dec 10, 2023

This reverts commit 40c3d35. The commit started dogfooding MCP510 to enable lld when building the compiler , but it broke tests, because we don't pass -Zunstable-options on enough places. This PR hotfixes that, and temporarily makes the "self-contained" option.. not very self-contained. I'll send a proper fix later, but I want to unblock rustc developres that use lld locally.

r? @nnethercote (who discovered the problem)

This reverts commit 40c3d35. The option was dogfooded for using lld with MCP510 , but it broke testing with LLD, because we don't pass `-Zunstable-options` on enough places.
@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Dec 10, 2023
@nnethercote
Copy link
Contributor

It fixes the failures for me.

@bors r+ p=1

Priority bump because this breaks tests for compiler devs who set use-lld = true, which should be just about everyone...

@bors
Copy link
Contributor

bors commented Dec 10, 2023

📌 Commit 3157f21 has been approved by nnethercote

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 10, 2023
@bors
Copy link
Contributor

bors commented Dec 10, 2023

⌛ Testing commit 3157f21 with merge f0227fe...

bors added a commit to rust-lang-ci/rust that referenced this pull request Dec 10, 2023
…nethercote

Revert using MCP510 in bootstrap

This reverts commit 40c3d35. The commit started dogfooding MCP510 to enable `lld` when building the compiler , but it broke tests, because we don't pass `-Zunstable-options` on enough places. This PR hotfixes that, and temporarily makes the "self-contained" option.. not very self-contained. I'll send a proper fix later, but I want to unblock rustc developres that use `lld` locally.

r? `@nnethercote` (who discovered the problem)
@rust-log-analyzer
Copy link
Collaborator

The job aarch64-gnu failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
[RUSTC-TIMING] syn test:false 16.972
   Compiling zerofrom-derive v0.1.3
[RUSTC-TIMING] sharded_slab test:false 2.092
   Compiling yoke-derive v0.7.2
##[error]The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

  local time: Mon Dec 11 00:02:00 UTC 2023
  network time: Mon, 11 Dec 2023 00:02:00 GMT
##[endgroup]
##[endgroup]
Session terminated, killing shell...[RUSTC-TIMING] derive_more test:false 8.642
[RUSTC-TIMING] ruzstd test:false 6.411
   Compiling displaydoc v0.2.4
 ...killed.
##[error]The operation was canceled.

@bors
Copy link
Contributor

bors commented Dec 11, 2023

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Dec 11, 2023
@nnethercote
Copy link
Contributor

@bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 11, 2023
@bors
Copy link
Contributor

bors commented Dec 11, 2023

⌛ Testing commit 3157f21 with merge 1cb200c...

@bors
Copy link
Contributor

bors commented Dec 11, 2023

☀️ Test successful - checks-actions
Approved by: nnethercote
Pushing 1cb200c to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Dec 11, 2023
@bors bors merged commit 1cb200c into rust-lang:master Dec 11, 2023
11 checks passed
@rustbot rustbot added this to the 1.76.0 milestone Dec 11, 2023
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (1cb200c): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-6.0% [-6.0%, -6.0%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -6.0% [-6.0%, -6.0%] 1

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.4% [1.4%, 1.4%] 1
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 1.4% [1.4%, 1.4%] 1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 669.249s -> 668.219s (-0.15%)
Artifact size: 314.13 MiB -> 314.10 MiB (-0.01%)

@Kobzol Kobzol deleted the revert-mcp-510-bootstrap branch December 11, 2023 07:11
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jan 4, 2025
…zkan

Allow using self-contained LLD in bootstrap

In rust-lang#116278, I added a `"self-contained"` mode to the `rust.use-lld` bootstrap option, which was designed for using the built-in LLD for linking compiler artifacts. However, this was later reverted in rust-lang#118810.

This PR brings the old logic back, which switches LLD in bootstrap from `-fuse-ld=lld` to [MCP510](rust-lang/compiler-team#510 way of passing linker flags to enable LLD (both external and self-contained). So this does two changes:
1) Goes from `-fuse-ld=lld` to MCP510
2) Actually makes it possible to use the self-contained LLD to compile compiler artifacts

Regarding the second commit: Since rust-lang#86113, we have been passing `-fuse-ld=lld` as a target flag to all tests when `use-lld = true` is enabled. This kind of worked for all tests, since it was just a linker argument, which has bypassed any compiler checks, and probably resulted only in some warning if the given target linker didn't actually support LLD. However, after the first commit, some tests actually start failing with this approach:
```
error: linker flavor `gnu-lld-cc` is incompatible with the current target
   |
   = note: compatible flavors are: llbc, ptx
```
So the second commit removes the passing of LLD flags as target flags to tests. I don't think that it's a good idea to pass specific compiler flags to all tests unconditionally, tbh. The doctest command from rust-lang#86113 doesn't go through compiletest anymore, and doctests should be quite a lot faster since rust-lang#126245 in general.

CC `@the8472`

If someone has a beefy machine, it would be nice to test whether this doesn't regress test execution speed. How to do that:
1) Enable `rust.use-lld = true` and `rust.lld = true` in `config.toml`
2) Benchmark `./x test tests/ui --force-rerun` between `master` and this PR

Once this is tested in the wild, I would like to make the self-contained LLD the default in CI, hopefully to make CI builds faster.

r? `@onur-ozkan`
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jan 4, 2025
Rollup merge of rust-lang#135001 - Kobzol:bootstrap-mcp-510, r=onur-ozkan

Allow using self-contained LLD in bootstrap

In rust-lang#116278, I added a `"self-contained"` mode to the `rust.use-lld` bootstrap option, which was designed for using the built-in LLD for linking compiler artifacts. However, this was later reverted in rust-lang#118810.

This PR brings the old logic back, which switches LLD in bootstrap from `-fuse-ld=lld` to [MCP510](rust-lang/compiler-team#510 way of passing linker flags to enable LLD (both external and self-contained). So this does two changes:
1) Goes from `-fuse-ld=lld` to MCP510
2) Actually makes it possible to use the self-contained LLD to compile compiler artifacts

Regarding the second commit: Since rust-lang#86113, we have been passing `-fuse-ld=lld` as a target flag to all tests when `use-lld = true` is enabled. This kind of worked for all tests, since it was just a linker argument, which has bypassed any compiler checks, and probably resulted only in some warning if the given target linker didn't actually support LLD. However, after the first commit, some tests actually start failing with this approach:
```
error: linker flavor `gnu-lld-cc` is incompatible with the current target
   |
   = note: compatible flavors are: llbc, ptx
```
So the second commit removes the passing of LLD flags as target flags to tests. I don't think that it's a good idea to pass specific compiler flags to all tests unconditionally, tbh. The doctest command from rust-lang#86113 doesn't go through compiletest anymore, and doctests should be quite a lot faster since rust-lang#126245 in general.

CC `@the8472`

If someone has a beefy machine, it would be nice to test whether this doesn't regress test execution speed. How to do that:
1) Enable `rust.use-lld = true` and `rust.lld = true` in `config.toml`
2) Benchmark `./x test tests/ui --force-rerun` between `master` and this PR

Once this is tested in the wild, I would like to make the self-contained LLD the default in CI, hopefully to make CI builds faster.

r? `@onur-ozkan`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants