Specify bit validity and padding of some types

[joshlf]

Specify the bit validity and padding of the primitive numeric types, bool, char, and pointer and reference types.

Closes #1291

[RalfJung]

LGTM! I'm confident this is covered by the FCP in rust-lang/unsafe-code-guidelines#439.

[joshlf]

Thanks for all of the discussion on this, @RalfJung!

[joshlf]

Is merging this blocked on anything?

[RalfJung]

I don't think so, but review capacity on reference PRs is very low.

[joshlf]

Oh does this require more than just your LGTM?

[RalfJung]

With the reference I am never sure.^^ @ehuss is our main editor here.

[ehuss]

Can you say why this is placed in the type-layout chapter? In general, that chapter is focused on the size and alignment of types, and not what constitutes a valid type. Valid bit patterns for types are generally documented in their corresponding type pages (like bool specifies that only 0 and 1 are valid bytes).

[ehuss]

This "may" seems confusing to me. When is it valid or not valid?

[RalfJung]

Those details are undecided as of now.

[ehuss]

Oh, I see, thanks!

[joshlf]

Can you say why this is placed in the type-layout chapter? In general, that chapter is focused on the size and alignment of types, and not what constitutes and valid type. Valid bit patterns for types are generally documented in their corresponding type pages (like bool specifies that only 0 and 1 are valid bytes).

I can do that, it'd just require somewhat more text since the second paragraph (about going from T to [u8; size_of::<T>()]) would need to be duplicated on the pages for bools, chars, and numeric types. If that's your preference, I'm happy to do that.

[ehuss]

Yea, I'm fine with duplicating it. For the bool case, that could also be worded a little different since "every byte" is a little confusing since there is only one byte. Thanks!

[joshlf]

Where do you think this text should go? On the page for pointer types?

A byte at any offset in a reference or pointer type may not be a valid u8 (the semantics of transmuting a reference or pointer to a non-pointer type is currently undecided).

Given that it would be lacking the context of the paragraph it's currently in, maybe we could rewrite it to something like the following, in its own section:

Bit validity

Despite pointers and references being similar to usizes in the machine code emitted on most platforms, the semantics of transmuting a reference or pointer type to a non-pointer type is currently undecided. Thus, it may not be valid to transmute a pointer or reference type, P, to a [u8; size_of::<P>()].

[joshlf]

Okay, I updated to this text in pointer.md, so the PR is ready for review. Happy to edit this as requested.

Bit validity

Despite pointers and references being similar to usizes in the machine code emitted on most platforms, the semantics of transmuting a reference or pointer type to a non-pointer type is currently undecided. Thus, it may not be valid to transmute a pointer or reference type, P, to a [u8; size_of::<P>()].

[joshlf]

One follow-up thought inspired by google/zerocopy#294: @RalfJung and @ehuss, would you be on board with me also adding the following to pointer.md? If this is at all controversial, I'll just follow up in a separate PR to avoid holding this one up.

It is always sound to produce a thin null pointer (i.e., for T: Sized and P = *const T or P = *mut T, transmute::<_, P>([0u8; size_of::<P>()])). Note that this can be done without unsafe code using the ptr::null or ptr::null_mut functions.

[RalfJung]

Yeah this is fine for me, it just will affect people working with extern types (but those are still unstable anyway).

[ehuss]

Thanks!

[joshlf]

@RalfJung For my own edification, how does this interact with pointer-to-integer as casts? I can imagine a few possibilities (not mutually exclusive):

• as casts are sound, but transmutations are not necessarily sound (even though you can emulate a transmutation by doing an as cast followed by a transmutation from usize)
• the first bullet is true and pointer-to-integer as casts probably shouldn't have been allowed in safe Rust, but it's too late

[RalfJung]

The current working model is that as casts are special:

• ptr as usize is like https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.expose_addr
• usize as ptr is like https://doc.rust-lang.org/nightly/std/ptr/fn.from_exposed_addr.html

In contrast:

• "transmute ptr to usize" is like https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.addr
• "transmute usize to ptr" is like https://doc.rust-lang.org/nightly/std/ptr/fn.invalid.html

Here, "transmute" refers to transmute but also any other way to re-interpret bytes at a different type, such as type punning through union field accesses and raw pointer casts.

All of these operations are safe and sound, but as their docs show, they behave quite differently and that can affect the soundness of later code that uses the values they produce.

Note that this is a prototype model, which has some unresolved problems and which is very much not stabilized.

[joshlf]

Okay that all makes sense, thanks!

Note that this is a prototype model, which has some unresolved problems and which is very much not stabilized.

I always assume that of everything in this space unless documented otherwise :)