nondeterministic ordering in rmeta section #113584
Labels
A-metadata
Area: Crate metadata
A-reproducibility
Area: Reproducible / deterministic builds
C-bug
Category: This is a bug.
T-compiler
Relevant to the compiler team, which will review and decide on the PR/issue.
Comments
rustbot
added
the
needs-triage
|
The zip file contains two rlibs that were produced by consecutive runs of the command (and renamed in the archive): Running --- /dev/fd/63 2023-07-11 18:50:47.875147356 +0000
+++ /dev/fd/62 2023-07-11 18:50:47.875147356 +0000
@@ -1,5 +1,5 @@
-libotf1.rlib(lib.rmeta): file format elf64-x86-64
+libotf2.rlib(lib.rmeta): file format elf64-x86-64
Contents of section .note.gnu.property:
0000 04000000 10000000 05000000 474e5500 ............GNU.
0010 020000c0 04000000 03000000 00000000 ................
@@ -44829,20 +44829,20 @@
af160 00d60101 070002d1 0d00d601 01070002 ................
af170 d10d00d6 01050700 02d30d00 e3010107 ................
af180 00024322 51586191 01ee01f4 014fa602 ..C"QXa......O..
- af190 85012f54 b202bd02 12000f6f 70656e74 ../T.......opent
- af1a0 68726561 645f7275 7374c101 00018059 hread_rust.....Y
- af1b0 01000013 6f70656e 74687265 61645f72 ....openthread_r
- af1c0 7573743a 3a6f74c1 00000011 506c6174 ust::ot.....Plat
- af1d0 666f726d 3a3a6275 696c6465 72c10000 form::builder...
- af1e0 019ae32b 00000180 59020000 226f7065 ...+....Y..."ope
- af1f0 6e746872 6561645f 72757374 3a3a6f74 nthread_rust::ot
- af200 3a3a496e 7374616e 63653a3a 6e6577c1 ::Instance::new.
- af210 010001ec e32b0200 01b3e32b 010001cb .....+.....+....
- af220 e32b0200 001d6f70 656e7468 72656164 .+....openthread
- af230 5f727573 743a3a6f 743a3a49 6e737461 _rust::ot::Insta
- af240 6e6365c1 010001a5 e42b0200 01b3e32b nce......+.....+
- af250 0200019a e32b0200 01805900 01000100 .....+....Y.....
- af260 ac0201ec e32b0000 01cbe32b 010001a5 .....+.....+....
+ af190 85012f54 b202bd02 12001d6f 70656e74 ../T.......opent
+ af1a0 68726561 645f7275 73743a3a 6f743a3a hread_rust::ot::
+ af1b0 496e7374 616e6365 c1010001 80590100 Instance.....Y..
+ af1c0 00226f70 656e7468 72656164 5f727573 ."openthread_rus
+ af1d0 743a3a6f 743a3a49 6e737461 6e63653a t::ot::Instance:
+ af1e0 3a6e6577 c1000000 11506c61 74666f72 :new.....Platfor
+ af1f0 6d3a3a62 75696c64 6572c101 0001e8e3 m::builder......
+ af200 2b020001 80590200 019ae32b 0000000f +....Y.....+....
+ af210 6f70656e 74687265 61645f72 757374c1 openthread_rust.
+ af220 0100018f e42b0200 01c1e32b 02000013 .....+.....+....
+ af230 6f70656e 74687265 61645f72 7573743a openthread_rust:
+ af240 3a6f74c1 010001af e42b0200 01e8e32b :ot......+.....+
+ af250 00000180 59000100 0100ac02 019ae32b ....Y..........+
+ af260 0200018f e42b0000 01c1e32b 010001af .....+.....+....
af270 e42b0000 154402c9 6015201a 0915611e .+...D..`. ...a.
af280 1b1e111e 191e1416 a20316c7 0416c125 ...............%
af290 16b51416 fb1a16b3 2d16902d 16ae2d16 ........-..-..-.
@@ -68050,7 +68050,7 @@
0020 74727461 62002e73 796d7461 62002e72 trtab..symtab..r
0030 6d657461 00 meta.
-libotf1.rlib(libopenthread_fuchsia.openthread_fuchsia.6f211ea536ce9cdf-cgu.0.rcgu.o): file format elf64-x86-64
+libotf2.rlib(libopenthread_fuchsia.openthread_fuchsia.6f211ea536ce9cdf-cgu.0.rcgu.o): file format elf64-x86-64
Contents of section .strtab:
0000 006d656d 63707900 5f524e76 4d73635f .memcpy._RNvMsc_
0010 4e744373 31736a36 4e397447 6438725f NtCs1sj6N9tGd8r_truncated diff of "assembly" just to show that the section of interest is rmeta: --- /dev/fd/63 2023-07-11 19:12:15.178842024 +0000
+++ /dev/fd/62 2023-07-11 19:12:15.182842348 +0000
@@ -1,5 +1,5 @@
-libotf1.rlib(lib.rmeta): file format elf64-x86-64
+libotf2.rlib(lib.rmeta): file format elf64-x86-64
Disassembly of section .note.gnu.property:
@@ -274986,7 +274986,7 @@
af18a: 01 f4 addl %esi, %esp
af18c: 01 4f a6 addl %ecx, -0x5a(%rdi)
af18f: 02 85 01 2f 54 b2 addb -0x4dabd0ff(%rbp), %al
- af195: 02 bd 02 12 00 0f addb 0xf001202(%rbp), %bh
+ af195: 02 bd 02 12 00 1d addb 0x1d001202(%rbp), %bh
af19b: 6f outsl (%rsi), %dx
af19c: 70 65 jo 0xaf203 <.rmeta+0xaf203>
af19e: 6e outsb (%rsi), %dx
diff of --- /dev/fd/63 2023-07-11 12:13:01.497722323 -0700
+++ /dev/fd/62 2023-07-11 12:13:01.501722285 -0700
@@ -4010,11 +4010,11 @@
__formatter
( ze
C"QXa
+openthread_rust::ot::Instance
+"openthread_rust::ot::Instance::new
+Platform::builder
openthread_rust
openthread_rust::ot
-Platform::builder
-"openthread_rust::ot::Instance::new
-openthread_rust::ot::Instance
p1,L
%t-1
!ovH[It appears that something in the rmeta section is ordered nondeterminstically. |
|
Version info: |
saethlin
added
the
T-compiler
workingjubilee
added
the
A-reproducibility
|
btw rlibs are archive files, so you can just |
|
After extracting to $ sha256sum libotf1.lib.rmeta libotf2.lib.rmeta
1a32d69f37fe98ae269a69edd1d4b0090cb369dc4bd1c14b473dd8952ce0c112 libotf1.lib.rmeta
3f38f6563e4c034ccd31fb0a9218f8aa4225d7aa25ff8d9eecdada4dccec449e libotf2.lib.rmetaI still need a way to look inside, human-readably. |
|
Can we identify the code that emits the difference parts of the rmeta? |
|
I suspect working this down to a smaller reproducer will be a better first step. If minimizing it can deduce what aspect of the input program is tripping the nondet, that would probably make fixing this much easier. Perhaps https://github.com/Nilstrieb/cargo-minimize can help? |
saethlin
removed
the
needs-triage
Auto-reducing doesn't work well with nondeterministic behavior. If we had some clue about what the rmeta differences are, we might be able to attempt to manually reduce or exercise the compiler in ways that increase the likelihood of reproducing. |
|
Can anyone look inside the rmeta files attached in the above comment and describe the difference? |
|
We had the same problem, using version 1.72.0 of rustc. After verification using current version(1.75.0), we find this problem has been solved. Which PR fixed it? |
|
There is a fair chance that this wasn't intentionally fixed, in which case nobody currently knows which PR it fixed. You could bisect between the nightly corresponding to 1.72 and the one corrwsponding to 1.75 if you want to know, but it will take quite a bit of time. You will probably want to pass |
|
FWIW, our continuous builder also saw this issue disappear when we bumped to 1.76. |
#119372 (comment) you can see this issue |
Excellent, thank you. I'd be happy to call this issue closed. |
81 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We are running determinism checks on
rustcoutputs in our build infrastructure, basically running arustcaction twice in a row (backing up outputs from the first run) and comparing outputs. We have found that in one of our build environments (not directly accessible), this determinism check fails rather reliably on the same targets, while the exact same compilation appears deterministic in another local environment, and is thus difficult to manually reproduce.We have been able to save and retrieve difference-pairs of rlibs when the determinism check fails, and need help determining the nature of the differences. So far we have determined that the differences lie in the
rmetasection of the rlibs, but we do not understand the meaning or origin of these differences. This is where we need help. Identifying these content difference may help us create more reliably reproducible test cases to follow up with.I will attach one pair of such rlibs shortly, with some more details.
The text was updated successfully, but these errors were encountered: