Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[do not merge] test #129714 revert on bootstrap times #129851

Closed
wants to merge 2 commits into from

Conversation

lqd
Copy link
Member

@lqd lqd commented Sep 1, 2024

It seems we're currently experiencing some instability with the benchmarking results. #129714 had weird results, somehow failing to measure bootstrap and other benchmarks. Since then bootstrap measurements have also increased. Let's see if a revert does anything to bootstrap times at the very least, to see if the PR is maybe involved.

r? ghost

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Sep 1, 2024
@lqd
Copy link
Member Author

lqd commented Sep 1, 2024

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Sep 1, 2024
@bors
Copy link
Contributor

bors commented Sep 1, 2024

⌛ Trying commit 7841fc6 with merge 60a3a34...

bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 1, 2024
[do not merge] test rust-lang#129714 revert on bootstrap times

It seems we're currently experiencing some instability with the benchmarking results. rust-lang#129714 had weird results, somehow failing to measure bootstrap and other benchmarks. Since then bootstrap measurements [have also increased](https://perf.rust-lang.org/bootstrap.html). Let's see if a revert does anything to bootstrap times at the very least, to see if the PR is maybe involved.

r? ghost
@rust-log-analyzer

This comment has been minimized.

… r=compiler-errors"

This reverts commit 6b9ed71, reversing
changes made to 8c7a7e3.
@Kobzol
Copy link
Contributor

Kobzol commented Sep 1, 2024

@rust-timer build 60a3a34

@rust-timer

This comment has been minimized.

@lqd
Copy link
Member Author

lqd commented Sep 1, 2024

The more the merrier. Now with a #127537 revert, since it seems to be more of the culprit for bootstrap times.

@bors try

@bors
Copy link
Contributor

bors commented Sep 1, 2024

⌛ Trying commit 892c95c with merge b0dec66...

bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 1, 2024
[do not merge] test rust-lang#129714 revert on bootstrap times

It seems we're currently experiencing some instability with the benchmarking results. rust-lang#129714 had weird results, somehow failing to measure bootstrap and other benchmarks. Since then bootstrap measurements [have also increased](https://perf.rust-lang.org/bootstrap.html). Let's see if a revert does anything to bootstrap times at the very least, to see if the PR is maybe involved.

r? ghost
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (60a3a34): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.7% [0.5%, 0.8%] 2
Regressions ❌
(secondary)
0.4% [0.4%, 0.4%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-0.7% [-1.2%, -0.4%] 3
All ❌✅ (primary) 0.7% [0.5%, 0.8%] 2

Max RSS (memory usage)

Results (primary 0.0%, secondary 0.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.6% [0.4%, 1.7%] 33
Regressions ❌
(secondary)
0.8% [0.4%, 1.7%] 50
Improvements ✅
(primary)
-1.0% [-2.3%, -0.4%] 20
Improvements ✅
(secondary)
-1.1% [-3.0%, -0.4%] 25
All ❌✅ (primary) 0.0% [-2.3%, 1.7%] 53

Cycles

Results (primary 0.1%, secondary -0.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.8% [0.4%, 3.5%] 30
Regressions ❌
(secondary)
1.0% [0.4%, 5.2%] 56
Improvements ✅
(primary)
-0.8% [-2.9%, -0.4%] 21
Improvements ✅
(secondary)
-1.1% [-6.5%, -0.4%] 51
All ❌✅ (primary) 0.1% [-2.9%, 3.5%] 51

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 790.965s -> 791.894s (0.12%)
Artifact size: 338.44 MiB -> 338.53 MiB (0.03%)

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Sep 1, 2024
@lqd
Copy link
Member Author

lqd commented Sep 1, 2024

@bors
Copy link
Contributor

bors commented Sep 1, 2024

☀️ Try build successful - checks-actions
Build commit: b0dec66 (b0dec66dd4ba3f647b067a903fe73f1283153720)

@lqd
Copy link
Member Author

lqd commented Sep 1, 2024

@rust-timer build b0dec66

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (b0dec66): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.6% [0.5%, 0.7%] 2
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.5% [-1.0%, -0.2%] 60
Improvements ✅
(secondary)
-0.7% [-2.2%, -0.3%] 19
All ❌✅ (primary) -0.4% [-1.0%, 0.7%] 62

Max RSS (memory usage)

Results (primary -0.6%, secondary -0.7%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.7% [0.4%, 1.4%] 15
Regressions ❌
(secondary)
0.7% [0.4%, 1.3%] 17
Improvements ✅
(primary)
-0.8% [-2.9%, -0.4%] 75
Improvements ✅
(secondary)
-1.0% [-2.7%, -0.4%] 103
All ❌✅ (primary) -0.6% [-2.9%, 1.4%] 90

Cycles

Results (primary -0.4%, secondary -0.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.1% [0.4%, 3.1%] 15
Regressions ❌
(secondary)
0.9% [0.4%, 3.6%] 52
Improvements ✅
(primary)
-0.7% [-3.5%, -0.4%] 61
Improvements ✅
(secondary)
-1.2% [-3.4%, -0.4%] 76
All ❌✅ (primary) -0.4% [-3.5%, 3.1%] 76

Binary size

Results (primary -0.7%, secondary -1.4%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.7% [-1.6%, -0.0%] 82
Improvements ✅
(secondary)
-1.4% [-2.3%, -0.6%] 24
All ❌✅ (primary) -0.7% [-1.6%, -0.0%] 82

Bootstrap: 790.965s -> 751.017s (-5.05%)
Artifact size: 338.44 MiB -> 338.43 MiB (-0.00%)

@rustbot rustbot removed the perf-regression Performance regression. label Sep 1, 2024
@lqd lqd force-pushed the test-revert-129714 branch from 892c95c to 9af460b Compare September 1, 2024 20:26
@lqd
Copy link
Member Author

lqd commented Sep 1, 2024

ready in case #129854 doesn't work out
r? kobzol

@rustbot
Copy link
Collaborator

rustbot commented Sep 1, 2024

Could not assign reviewer from: kobzol.
User(s) kobzol are either the PR author, already assigned, or on vacation, and there are no other candidates.
Use r? to specify someone else to assign.

@lqd
Copy link
Member Author

lqd commented Sep 2, 2024

Issue looks to be in the collector DB, no need to merge this.

@lqd lqd closed this Sep 2, 2024
@lqd lqd deleted the test-revert-129714 branch September 2, 2024 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants