rwlock: avoid Stacked Borrows violation

[RalfJung]

Fixes #121626. Changes code recently added in #110211.

It seems like RwLock assumes that code like this is legal:

use std::cell::UnsafeCell;

fn main() {
    let mut c = UnsafeCell::new(42);
    let ptr = c.get();
    c = UnsafeCell::new(13);
    unsafe { ptr.read(); }
}

However, Stacked Borrows makes this illegal. That may be overeager, and indeed Tree Borrows allows this code, but Tree Borrows might also allow too much code, so until we are more sure about the future of the Rust aliasing model we should avoid code like this.

For the RwLock, this means not mixing direct write access to the mutable local variable node, with access through (interior mutable) shared references.

r? @m-ou-se @joboet

[RalfJung]

I'm not actually sure why we know that they can't race. After all, as the Miri error shows, an alias can clearly still exist and be used again in the future when these writes happen. This assumption is not new; we've been using non-atomic writes here already before this PR. Previously this was safe code only because rustc did not realize that the NonNull::from(&node) creates a borrow that could last into the next loop iteration.

[RalfJung]

I should probably either make this a relaxed store, or make the function unsafe.

[joboet]

The weird thing is that the code actually doesn't make the assumption you mention; the shared reference that supposedly is invalidated is only created after all mutable accesses have occurred. Or so I thought...

[RalfJung]

It seems Miri found an interleaving where an old shared reference gets used after the next loop iteration did some mutable accesses to the same node again.

If that is itself a bug then this PR is the wrong fix. In that case the SB error in Miri would just mask a data race error.

[joboet]

That is in itself a bug, yes, because the loop is only repeated when the node is not in a queue, otherwise you could get use-after-frees and things like that.

[m-ou-se]

r? @joboet

[RalfJung]

This is the wrong fix, we need to figure out where the problem lies in the concurrency logic.