Update rand in the stdlib tests, and remove the getrandom feature from it.

[thomcc]

The main goal is actually removing getrandom, so that eventually we can allow running the stdlib test suite on tier3 targets which don't have getrandom support. Currently those targets can only run the subset of stdlib tests that exist in uitests, and (generally speaking), we prefer not to test libstd functionality in uitests, which came up recently in #104095 and #104185. Additionally, the fact that we can't update rand/getrandom means we're stuck with the old set of tier3 targets, so can't test new ones.

Anyway, I haven't checked that this actually does allow use on tier3 targets (I think it does not, as some work is needed in stdlib submodules) but it moves us slightly closer to this, and seems to allow at least finally updating our rand dep, which definitely improves the status quo. Checked and works now.

For the most part, our tests and benchmarks are fine using hard-coded seeds. A couple tests seem to fail with this (stuff manipulating the environment expecting no collisions, for example), or become pointless (all inputs to a function become equivalent). In these cases I've done a (gross) dance (ab)using RandomState and Location::caller() for some extra "entropy".

Trying to share that code seems way more painful than it's worth given that the duplication is a 7-line function, even if the lines are quite gross. (Keeping in mind that sharing it would require adding rand as a non-dev dep to std, and exposing a type from it publicly, all of which sounds truly awful, even if done behind a perma-unstable feature).

See also some previous attempts:

• Update rand deps #86963 (in particular Update rand deps #86963 (comment) which explains why this is non-trivial)
• Update rand to 0.8 in library/ and compiler/ #89131
• Avoid using rand::thread_rng in the stdlib benchmarks. #96626 (comment) (I tried in that PR at the same time, but settled for just removing the usage of thread_rng() from the benchmarks, since that was the main goal).
• Upgrade rand to v0.8 in libstd #104185
• Probably more. It's very tempting of a thing to "just update".

r? @Mark-Simulacrum

[rustbot]

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

• Stabilizing library features
• Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
• Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
• Changing public documentation in ways that create new stability guarantees
• Changing observable runtime behavior of library APIs

[thomcc]

Is the SmallRng seed's size really target-dependent? That's awful.

[thomcc]

I'll look into that and finish this up tomorrow.

@rustbot author

[thomcc]

This seems to pass on all the targets I care to test on that would normally fail (most notably wasm32), which implies it really has excised getrandom from the stdlib's dev-dep tree (this seems to be confirmed by cargo tree, but that can be misleading).

So I don't think anything else is needed here, and it's ready for review.

@rustbot ready

[klensy]

If all goes well, maybe bump getrandom (https://github.com/rust-random/getrandom/blob/master/CHANGELOG.md#028---2022-10-20) too?

[thomcc]

I'll look into bumping getrandom in a later PR when I can examine the changelog more closely (which is something we try to do for non-test-only deps, even well-vetted ones like getrandom). Part of my short-term motivation is unblocking my testing of #103503, which does require rust-random/getrandom#317, and thus a newish getrandom (although I'm unsure if that's been in a release yet).

[the8472]

For the most part, our tests and benchmarks are fine using hard-coded seeds. A couple tests seem to fail with this (stuff manipulating the environment expecting no collisions, for example), or become pointless (all inputs to a function become equivalent). In these cases I've done a (gross) dance (ab)using RandomState and Location::caller() for some extra "entropy".

It feels iffy to run randomized tests with fixed seeds. Can we conditionally import and use the getrandom feature on some tier-1 targets?

[thomcc]

It feels iffy to run randomized tests with fixed seeds. Can we conditionally import and use the getrandom feature on some tier-1 targets?

Doing this conditionally would be more painful since I'd have to duplicate a body of a #[cfg(...)] and a [target.'cfg(...)'.dev-dependencies] in more places. It also would add more conditional compliation that only fails on obscure targets, which is annoying for maintenance. I'm not sure it's really worth it given that most of those cases that actually benefit from randomized testing, I've tried to use a RNG method that isn't a fixed seed (e.g. the hacky test_rng() thing should genuinely give a different seed at least each test run).

I guess I'll take a look at what it looks like though.

[thomcc]

I guess I'll take a look at what it looks like though.

As per #104658 (comment) I didn't end up going this route, but I have made it so that the places that were using non-determinism to improve testing are still using it. Avoiding copying the cursed function twice required moving a few of the tests inside liballoc's unit tests as a result.

[bjorn3]

Note to self: update https://github.com/bjorn3/rustc_codegen_cranelift/blob/master/patches/0022-sysroot-Disable-not-compiling-tests.patch

[thomcc]

This was discussed in the most recent t-libs meeting. The consensus was that this implementation is acceptable.

I've rebased and squashed (to simplify the rebase) so this should be good to go.

@rustbot ready

[Mark-Simulacrum]

@bors r+ rollup=iffy

[bors]

📌 Commit a4bf36e has been approved by Mark-Simulacrum

It is now in the queue for this repository.

[bors]

⌛ Testing commit a4bf36e with merge 2afe585...

[bors]

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 2afe585 to master...

[rust-timer]

Finished benchmarking commit (2afe585): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.7%	[0.5%, 0.8%]	7
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.1%	[-0.1%, -0.1%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

This benchmark run did not return any relevant results for this metric.