Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix slice::ChunksMut aliasing #94247

Merged
merged 3 commits into from
Jul 27, 2022
Merged

Conversation

saethlin
Copy link
Member

@saethlin saethlin commented Feb 22, 2022

Fixes #94231, details in that issue.
cc @RalfJung

This isn't done just yet, all the safety comments are placeholders. But otherwise, it seems to work.

I don't really like this approach though. There's a lot of unsafe code where there wasn't before, but as far as I can tell the only other way to uphold the aliasing requirement imposed by __iterator_get_unchecked is to use raw slices, which I think require the same amount of unsafe code. All that would do is tie the len and ptr fields together.

Oh I just looked and I'm pretty sure that ChunksExactMut, RChunksMut, and RChunksExactMut also need to be patched. Even more reason to put up a draft.

@rust-highfive
Copy link
Collaborator

r? @joshtriplett

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 22, 2022
@the8472
Copy link
Member

the8472 commented Feb 22, 2022

but as far as I can tell the only other way to uphold the aliasing requirement imposed by __iterator_get_unchecked is to use raw slices, which I think require the same amount of unsafe code

But that could be improved by implementing slicing and split_at on them.

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 3, 2022
@saethlin saethlin force-pushed the chunksmut-aliasing branch 2 times, most recently from 9643dd8 to be3f695 Compare March 25, 2022 04:04
@the8472 the8472 assigned the8472 and unassigned joshtriplett Mar 30, 2022
@the8472
Copy link
Member

the8472 commented Mar 30, 2022

We discussed this in the libs team meeting. The general approach looks acceptable, but we'd like the new slice pointer APIs to be split out into a separate PR so they could be reviewed on their own. This PR can then be rebased once that's accepted.

@saethlin
Copy link
Member Author

I think that sounds awesome. If there is someone else willing to do the new slice pointer APIs that would be ideal, my plate is pretty full already so it might take me some time to get through that whole process.

@the8472
Copy link
Member

the8472 commented Apr 2, 2022

If there is someone else willing to do the new slice pointer APIs that would be ideal

#95594

the8472 pushed a commit to the8472/rust that referenced this pull request Apr 2, 2022
Split out from rust-lang#94247

This adds the following methods to raw slices that already exist on regular slices

* `*mut [T]::is_empty`
* `*mut [T]::split_at_mut`
* `*mut [T]::split_at_unchecked`

These methods reduce the amount of unsafe code needed to migrate ChunksMut and related iterators
to raw slices (rust-lang#94247)

Co-authored-by:: The 8472 <[email protected]>
@saethlin saethlin force-pushed the chunksmut-aliasing branch from 61d9d94 to ecb728f Compare May 16, 2022 04:41
@rustbot rustbot added the T-libs Relevant to the library team, which will review and decide on the PR/issue. label May 16, 2022
@rust-log-analyzer

This comment has been minimized.

@saethlin
Copy link
Member Author

saethlin commented May 18, 2022

Blocked on #95594
@rustbot label: +S-blocked

@rustbot rustbot added the S-blocked Status: Blocked on something else such as an RFC or other implementation work. label May 18, 2022
JohnTitor added a commit to JohnTitor/rust that referenced this pull request Jun 1, 2022
Additional `*mut [T]` methods

Split out from rust-lang#94247

This adds the following methods to raw slices that already exist on regular slices

* `*mut [T]::is_empty`
* `*mut [T]::split_at_mut`
* `*mut [T]::split_at_mut_unchecked`

These methods reduce the amount of unsafe code needed to migrate `ChunksMut` and related iterators
to raw slices (rust-lang#94247)

r? `@m-ou-se`
@bors
Copy link
Contributor

bors commented Jun 2, 2022

☔ The latest upstream changes (presumably #97632) made this pull request unmergeable. Please resolve the merge conflicts.

@saethlin
Copy link
Member Author

saethlin commented Jun 2, 2022

@rustbot label: -S-blocked

@rustbot rustbot removed the S-blocked Status: Blocked on something else such as an RFC or other implementation work. label Jun 2, 2022
@saethlin saethlin force-pushed the chunksmut-aliasing branch from ecb728f to 6627931 Compare July 3, 2022 04:07
@rustbot
Copy link
Collaborator

rustbot commented Jul 3, 2022

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

  • Stabilizing library features
  • Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
  • Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
  • Changing public documentation in ways that create new stability guarantees
  • Changing observable runtime behavior of library APIs

@saethlin
Copy link
Member Author

saethlin commented Jul 3, 2022

I am stuck. The code works, but I cannot figure out how to write safety comments in most of these cases, because what makes the unsafe code valid is highly nonlocal. Basically I imagine these all as "this is safe because it is an invariant of the type" but that doesn't seem remotely helpful as a safety comment. Help?

Here are where I'm definitely missing safety comments (those that exist may need improving too):

tidy error: /home/ben/rust/library/core/src/slice/iter.rs:1762: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:2032: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:2075: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:2076: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:2728: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:2729: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:2788: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:2789: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:3035: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:3061: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:3087: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:3106: undocumented unsafe
tidy error: /home/ben/rust/library/core/src/slice/iter.rs:3107: undocumented unsafe

@saethlin saethlin marked this pull request as ready for review July 3, 2022 04:10
@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 3, 2022
@rust-log-analyzer

This comment has been minimized.

@saethlin saethlin force-pushed the chunksmut-aliasing branch from 6627931 to 7919e42 Compare July 3, 2022 04:15
@rust-log-analyzer

This comment has been minimized.

@the8472
Copy link
Member

the8472 commented Jul 3, 2022

I am stuck. The code works, but I cannot figure out how to write safety comments in most of these cases, because what makes the unsafe code valid is highly nonlocal. Basically I imagine these all as "this is safe because it is an invariant of the type" but that doesn't seem remotely helpful as a safety comment. Help?

It's mostly about the new *mut [T]::split_at_mut calls, right?

I think this can be addressed in several steps

  • put a general comment on the struct field that gives an overview
    • constructed from a valid &mut [T]
    • splitting always in-bounds, producing exclusively-owned subslices that can be converted back to &mut [T] and that keeps * mut [T]::len valid, which in turn makes the bounds checking in split_at_mut valid.
    • special-case: see __iterator_get_unchecked
  • perhaps the split_at_mut docs can be improved? Currently state the general safety requirements, but perhaps they could have a better explanation in which specific circumstances those requirements are automatically fulfilled.

Or perhaps even simpler: We could explain that all uses should be treated as-if it were a &mut [T], which make the whole thing safe except in the case where __iterator_get_unchecked is called, but then the TrustedRandomAccess contract takes over requiring only-once access to each subslice.

library/core/src/slice/iter.rs Outdated Show resolved Hide resolved
library/core/src/slice/iter.rs Outdated Show resolved Hide resolved
library/core/src/slice/iter.rs Outdated Show resolved Hide resolved
@rust-log-analyzer

This comment has been minimized.

@saethlin saethlin force-pushed the chunksmut-aliasing branch from 05dc00a to 746afe8 Compare July 27, 2022 01:26
@the8472
Copy link
Member

the8472 commented Jul 27, 2022

@bors r+

@bors
Copy link
Contributor

bors commented Jul 27, 2022

📌 Commit 746afe8 has been approved by the8472

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 27, 2022
bors added a commit to rust-lang-ci/rust that referenced this pull request Jul 27, 2022
…laumeGomez

Rollup of 7 pull requests

Successful merges:

 - rust-lang#94247 (Fix slice::ChunksMut aliasing)
 - rust-lang#99358 (Allow `ValTree::try_to_raw_bytes` on `u8` array)
 - rust-lang#99651 (Deeply deny fn and raw ptrs in const generics)
 - rust-lang#99710 (lint: add bad opt access internal lint)
 - rust-lang#99717 (Add some comments to the docs issue template to clarify)
 - rust-lang#99728 (Clean up HIR-based lifetime resolution)
 - rust-lang#99812 (Fix headings colors)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit ef81fca into rust-lang:master Jul 27, 2022
@rustbot rustbot added this to the 1.64.0 milestone Jul 27, 2022
@RalfJung
Copy link
Member

Is the PR description outdated? With that description (which will now live forever in our git history) it doesn't sound like this was ready to land...

@saethlin
Copy link
Member Author

🤦 it's outdated

saethlin added a commit to saethlin/rust that referenced this pull request Aug 1, 2022
These were accidentally removed in rust-lang#94247 because the representation was
changed from &mut [T] to *mut T, which has !Send + !Sync.
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 3, 2022
Add back Send and Sync impls on ChunksMut iterators

Fixes rust-lang#100014

These were accidentally removed in rust-lang#94247 because the representation was changed from `&mut [T]` to `*mut T`, which has `!Send + !Sync`.
@saethlin saethlin deleted the chunksmut-aliasing branch September 3, 2022 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stacked borrows fails on {ChunksMut,ChunksExactMut}::__iterator_get_unchecked()
9 participants