Add support for bit data

[zwarich]

See #327 for a previous RFC proposing this.

[engstad]

@zwarich When it is marked postponed, is it an indefinite postponement, or is it limited to version 1.0?

[engstad]

@zwarich Also, in your comment for #327, you mention that it is not defined exactly with respect to memory layout -- but on the contrary it is exactly defined. When I first got feedback of the first version I received comments to that exact nature, so I worked on it to make it exactly defined. Perhaps the RFC isn't clear enough? If so, I'm happy to clarify it.

[zwarich]

@engstad It means that it is postponed until after 1.0, but only that it will be considered again, not that it will definitely be accepted.

The RFC didn't mention any padding or alignment, but maybe the implied answer there was to align whole bitdata structs the same as a 32-bit integer and have no internal alignment / padding?

[engstad]

@zwarich That's exactly right. For bit-data, there is no internal alignment or padding. If that's needed, you simply add unnamed bit-fields. There's an example in the RFC itself.

    ...
    Leaf  { tag  = 3 : u2, _: u2, tri0 : u20, tri1 : u20, tri2 : u20 }

Here, the "_ : u2" is internal padding. Also notice that the compiler will have to check that the bit-field size is the same for each variant, which would help the developer in making sure bit-sizes are exact.

As far as embedding bitdata in memory, this is guided by the "carrier type". If a bitdata type is defined to be of a u32 carrier type, for instance, it will inherit the same all alignment restrictions etc. within a struct as the u32 type.

[ghost]

@alexcrichton pointed me here when I asked if we should move rust-lang/rust#5346 to rfcs. So, assuming this is the right place, here are links to Erlang's bit syntax docs for inspiration and reference:
http://www.erlang.org/doc/programming_examples/bit_syntax.html
http://www.erlang.org/doc/reference_manual/expressions.html#id78511
http://www.erlang.org/doc/reference_manual/expressions.html#id79906
http://www.erlang.org/doc/reference_manual/data_types.html#id65994

If this is the wrong rfc, please let me know and I'll open a new one.

[tomgco]

Seeming that 1.0 has now been released and 1.2 is about to be released, I was wondering if this could be considered again? Having this in rust would definitely make implementing bit based protocols much more concise.

Thanks for all of the hard work all!

[DigiTester]

FWIW I'm in the same position - I'd like to interact with embedded devices that use a very compact packet and the shift-and-mask approach is error prone. Mapping to enumerated types is somewhat annoying as well since some have unused values in the range. We have a partial Java implementation but it's taken a ridiculous amount of work to correctly consume and generate a small subset of the available packet types. I'd like to use Rust on the server side (and probably will), but being able to map an incoming byte array onto a struct would make the code simpler and more robust. In our case the exact bit layout is known and the C implementations all use one header file full of #ifdefs to cope with endianness and compiler variations. If I could mix size-specified enumerations and arrays of non-byte sized structs in I would be ecstatic.

Viz:

enum PacketType:u5 { Zero=0, Initiate=1, Reset=5 ... }
#[repr(packed, reallyPacked)]
struct UglyThing { part_one:u1, part_two:u5 } // 6 bits long
#[repr(packed, reallyPacked)]
struct PacketHeader {
    packet_type: PacketType, // five bits, since the type is a u5
    device_type: u3,
    ugliness[UglyThing; 4], // reallyPacked -> array elements are packed into 24 bits
    ...
}

But really, just being able to declare a struct with non-byte fields would be enough. I'd settle for packet_type being a u8 and having to have my own code to check the value. The UglyThing... they only did that once and if I have to dance in circles to make it work that's just too bad.

[engstad]

As @DigiTester mentions, the situation is pretty bad. @nikomatsakis is currently working on higher level issues, one of which is dealing with constant integers in the type system. I have a feeling that it shouldn't take too much work changing the compilers representation from

pub enum IntegralBits {
    B8, B16, B32, B64, BSize
}

to

pub enum IntegralBits {
    B8, B16, B32, B64, B(U8), BSize
}

Of course, there would be quite some work in making sure bit-sized types don't "escape" into actual storage types.

[RSully]

Has there been any work in this area since 2015? I'm having a hard time tracking all the various issues and history for this proposal.

[weberdaniel]

Erlang Bit Syntax is extremly powerful, it would be awesome to see it in rust.