Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: update rustc-perf version used in CI and also the corresponding PGO benchmarks #95724

Merged
merged 1 commit into from
Apr 9, 2022
Merged

CI: update rustc-perf version used in CI and also the corresponding PGO benchmarks #95724

merged 1 commit into from
Apr 9, 2022

Conversation

Kobzol
Copy link
Contributor

@Kobzol Kobzol commented Apr 6, 2022

The old version was from May 2021. The rustc-perf benchmarks have seen a significant overhaul recently, so let's see if the new benchmarks can improve PGO performance.

@rust-highfive
Copy link
Collaborator

r? @kennytm

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 6, 2022
@Kobzol
Copy link
Contributor Author

Kobzol commented Apr 6, 2022

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Apr 6, 2022
@bors
Copy link
Contributor

bors commented Apr 6, 2022

⌛ Trying commit 4a31a7ddcea8fe9805c3e5ec109cfa7dd98dc49c with merge 4e588bd7921c625cbd174d36019b756c3f4a89e7...

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@Kobzol Kobzol force-pushed the ci-update-rustc-perf branch from 4a31a7d to 7865285 Compare April 6, 2022 07:45
@Kobzol
Copy link
Contributor Author

Kobzol commented Apr 6, 2022

@bors try

@bors
Copy link
Contributor

bors commented Apr 6, 2022

⌛ Trying commit 78652850eee351a42819ebb16408e0e23189186d with merge eeb04f75b8aca0f9a59de245b827af8601417357...

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@bors
Copy link
Contributor

bors commented Apr 6, 2022

💔 Test failed - checks-actions

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 6, 2022
@Kobzol Kobzol force-pushed the ci-update-rustc-perf branch from 7865285 to 87b417e Compare April 6, 2022 10:23
@Kobzol
Copy link
Contributor Author

Kobzol commented Apr 6, 2022

@bors try

@bors
Copy link
Contributor

bors commented Apr 6, 2022

⌛ Trying commit 87b417e9b5009aa34a5a7880c99f79bdae125981 with merge acb059f589ddb9e518c7a0afe86f34b8b5fbff6b...

@bors
Copy link
Contributor

bors commented Apr 6, 2022

☀️ Try build successful - checks-actions
Build commit: acb059f589ddb9e518c7a0afe86f34b8b5fbff6b (acb059f589ddb9e518c7a0afe86f34b8b5fbff6b)

@rust-timer
Copy link
Collaborator

Queued acb059f589ddb9e518c7a0afe86f34b8b5fbff6b with parent ce1a813, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (acb059f589ddb9e518c7a0afe86f34b8b5fbff6b): comparison url.

Summary:

  • Primary benchmarks: 🎉 relevant improvements found
  • Secondary benchmarks: mixed results
Regressions 😿
(primary)		 Regressions 😿
(secondary)		 Improvements 🎉
(primary)		 Improvements 🎉
(secondary)		 All 😿 🎉
(primary)
count1 1 19 90 96 91
mean2 0.2% 1.1% -0.7% -0.8% -0.7%
max 0.2% 1.8% -2.1% -2.3% -2.1%

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR led to changes in compiler perf.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf -perf-regression

Footnotes

  1. number of relevant changes ↩

  2. the arithmetic mean of the percent change ↩

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Apr 6, 2022
@Kobzol
Copy link
Contributor Author

Kobzol commented Apr 6, 2022

Results look rather good, slight upgrades in most benchmarks. Since we now want to optimize for the benchmarks in rustc-perf, I think that this is worth merging.

r? @nnethercote

@rust-highfive rust-highfive assigned nnethercote and unassigned kennytm Apr 6, 2022
@Kobzol Kobzol force-pushed the ci-update-rustc-perf branch from 99bebd1 to 49efa23 Compare April 7, 2022 07:42
@nnethercote
Copy link
Contributor

@bors r+

@bors
Copy link
Contributor

bors commented Apr 7, 2022

📌 Commit 49efa23 has been approved by nnethercote

@bors
Copy link
Contributor

bors commented Apr 7, 2022

⌛ Testing commit 49efa23 with merge 051ddcc0cb2111e36d52105f0c3d60dc7a63b5c1...

@bors
Copy link
Contributor

bors commented Apr 7, 2022

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Apr 7, 2022
@rust-log-analyzer
Copy link
Collaborator

The job dist-apple-various failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

@lqd
Copy link
Member

lqd commented Apr 7, 2022

@bors retry weird failure on dist-apple-various, no apparent error

retrying once to see if it's a fluke

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 7, 2022
@bors
Copy link
Contributor

bors commented Apr 8, 2022

⌛ Testing commit 49efa23 with merge 4aa34e17b4eae914cfe2b717b97388515fb8681d...

@bors
Copy link
Contributor

bors commented Apr 8, 2022

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Apr 8, 2022
@Kobzol
Copy link
Contributor Author

Kobzol commented Apr 8, 2022

That.. looks spurious again?

@rust-log-analyzer
Copy link
Collaborator

The job i686-msvc-1 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot) 
Some tests failed in compiletest suite=ui-fulldeps mode=ui host=i686-pc-windows-msvc target=i686-pc-windows-msvc

error: test run failed!
status: exit code: 101
command: PATH="D:\a\rust\rust\build\i686-pc-windows-msvc\stage2\lib\rustlib\i686-pc-windows-msvc\lib;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x86;D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-bootstrap-tools\i686-pc-windows-msvc\release\deps;D:\a\rust\rust\build\i686-pc-windows-msvc\stage0\bin;D:\a\rust\rust\ninja;D:\a\rust\rust\msys2\mingw32\bin;C:\hostedtoolcache\windows\Python\3.10.4\x64\Scripts;C:\hostedtoolcache\windows\Python\3.10.4\x64;C:\msys64\usr\bin;D:\a\rust\rust\sccache;C:\Program Files\MongoDB\Server\5.0\bin;C:\aliyun-cli;C:\vcpkg;C:\cf-cli;C:\Program Files (x86)\NSIS;C:\tools\zstd;C:\Program Files\Mercurial;C:\hostedtoolcache\windows\stack\2.7.5\x64;C:\cabal\bin;C:\ghcup\bin;C:\tools\ghc-9.2.2\bin;C:\Program Files\dotnet;C:\mysql\bin;C:\Program Files\R\R-4.1.3\bin\x64;C:\SeleniumWebDrivers\GeckoDriver;C:\Program Files (x86)\sbt\bin;C:\Program Files (x86)\GitHub CLI;C:\Program Files\Git\bin;C:\Program Files (x86)\pipx_bin;C:\hostedtoolcache\windows\go\1.15.15\x64\bin;C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts;C:\hostedtoolcache\windows\Python\3.7.9\x64;C:\hostedtoolcache\windows\Ruby\2.5.9\x64\bin;C:\tools\kotlinc\bin;C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.322-6\x64\bin;C:\npm\prefix;C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;C:\ProgramData\kind;C:\Program Files\Eclipse Foundation\jdk-8.0.302.8-hotspot\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\ProgramData\Chocolatey\bin;C:\Program Files\Docker;C:\Program Files\PowerShell\7;C:\Program Files\Microsoft\Web Platform Installer;C:\Program Files\dotnet;C:\Program Files\Microsoft SQL Server\130\Tools\Binn;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn;C:\Program Files\nodejs;C:\Program Files\OpenSSL\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;C:\Program Files\TortoiseSVN\bin;C:\Program Files\CMake\bin;C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.5\bin;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Program Files\GitHub CLI;C:\tools\php;C:\Program Files (x86)\sbt\bin;C:\SeleniumWebDrivers\ChromeDriver;C:\SeleniumWebDrivers\EdgeDriver;C:\Program Files\Amazon\AWSCLIV2;C:\Program Files\Amazon\SessionManagerPlugin\bin;C:\Program Files\Amazon\AWSSAMCLI\bin;C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin;C:\Program Files (x86)\Microsoft BizTalk Server;C:\Program Files\LLVM\bin;C:\Users\runneradmin\.dotnet\tools;C:\Users\runneradmin\.cargo\bin;C:\Users\runneradmin\AppData\Local\Microsoft\WindowsApps" "D:\\a\\rust\\rust\\build\\i686-pc-windows-msvc\\test\\ui-fulldeps\\stdio-from\\a.exe"
stdout: none
--- stderr -------------------------------
I/O error: operation failed to complete synchronously
thread 'main' panicked at 'assertion failed: child2.wait()?.success()', D:\a\rust\rust\src/test\ui-fulldeps\stdio-from.rs:50:5
------------------------------------------




failures:
    [ui] src/test\ui-fulldeps\stdio-from.rs

test result: FAILED. 65 passed; 1 failed; 1 ignored; 0 measured; 0 filtered out; finished in 26.49s

Build completed unsuccessfully in 0:36:54
make: *** [Makefile:72: ci-subset-1] Error 1

@lqd
Copy link
Member

lqd commented Apr 8, 2022

@bors retry #95759

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 8, 2022
@bors
Copy link
Contributor

bors commented Apr 9, 2022

⌛ Testing commit 49efa23 with merge 399dd80...

@bors
Copy link
Contributor

bors commented Apr 9, 2022

☀️ Test successful - checks-actions
Approved by: nnethercote
Pushing 399dd80 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Apr 9, 2022
@bors bors merged commit 399dd80 into rust-lang:master Apr 9, 2022
@rustbot rustbot added this to the 1.62.0 milestone Apr 9, 2022
@Kobzol Kobzol deleted the ci-update-rustc-perf branch April 9, 2022 10:50
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (399dd80): comparison url.

Summary:

  • Primary benchmarks: 🎉 relevant improvements found
  • Secondary benchmarks: mixed results
Regressions 😿
(primary)		 Regressions 😿
(secondary)		 Improvements 🎉
(primary)		 Improvements 🎉
(secondary)		 All 😿 🎉
(primary)
count1 0 13 102 101 102
mean2 N/A 1.0% -0.7% -1.0% -0.7%
max N/A 1.3% -2.2% -2.7% -2.2%

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

Footnotes

  1. number of relevant changes ↩

  2. the arithmetic mean of the percent change ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rylev rylev rylev left review comments

@nnethercote nnethercote nnethercote left review comments

Assignees

@nnethercote nnethercote

Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Milestone
1.62.0
Development

Successfully merging this pull request may close these issues.
10 participants
@Kobzol @rust-highfive @rust-timer @bors @rust-log-analyzer @nnethercote @lqd @rylev @kennytm @rustbot