Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Deprecate type aliases in std::os::*::raw #1415

Merged
merged 1 commit into from
Feb 11, 2016

Conversation

alexcrichton
Copy link
Member

Deprecate type aliases and structs in std::os::$platform::raw in favor of
trait-based accessors which return Rust types rather than the equivalent C type
aliases.

Deprecate type aliases and structs in `std::os::$platform::raw` in favor of
trait-based accessors which return Rust types rather than the equivalent C type
aliases.
@alexcrichton alexcrichton self-assigned this Dec 18, 2015
@alexcrichton alexcrichton added the T-libs-api Relevant to the library API team, which will review and decide on the RFC. label Dec 18, 2015
platform specific modules. All type aliases can be switched over to `u64` and
the `stat` structure could simply be redefined to `stat64` on Linux (minus
keeping the same name). This would, however, explicitly mean that
**std::os::raw is no longer FFI compatible with C**.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but as your motivation says, the current FFI compatibility is a bit of an illusion anyway, thanks to LFS -DFILE_OFFSET_BITS.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's a good point! There's sort of a case to be made with "well they're FFI compatible if you pass no flags to the compiler", but that's shaky at best at this point.

@remram44
Copy link

Rendered

@alexcrichton
Copy link
Member Author

🔔 This RFC is now entering its week-long final comment period 🔔

@alexcrichton alexcrichton added the final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. label Jan 29, 2016
@alexcrichton
Copy link
Member Author

The libs team discussed this today and the conclusion was that there is not a clear enough path moving forward that doesn't involve theoretical breakage. We decided to leave this in FCP while I gather some statistics and flesh out a more concrete plan.

@alexcrichton
Copy link
Member Author

Ok, I've done some analysis of what this change might have. The numbers here were gathered against these changes to the standard library, which I believe fully implement this RFC (enabling us to move to LFS on Linux at the same time).

A crater report reports four regressions, three of which were spurious. I have submitted a PR for the other regression. Specifically, this regression involved code that looked like:

libc::foo(..., value as std::os::raw::off_t)

In other words, the regression came from the reliance that the types in std agreed with those in libc, which this RFC would be breaking.

I then compiled Servo against a standard library with the patches above applied. With the fs2-rs changes Servo compiled for both x86_64-unknown-linux-gnu and arm-linux-androideabi with no other modifications necessary.

Finally, I did a manual survey of all code on crates.io for usage of MetadataExt or the std::os::*::raw types. The investigation revealed two cases which would break

The following usage was all observed but would not break from the change in this RFC (confirmed from the crater run)

  • cargo-extras-0.3.0
    • casts .mtime() to u64 immediately
  • cargo-script-0.1.4
    • casts .mtime() to u64 immediately
  • devicemapper-0.3.2
    • mode() & constant == constant
    • assumes rdev is u64 (ok)
  • filetime-0.1.9
    • immediately casts
    • only user of as_raw_stat
  • fs2-0.2.2
    • immediately casts
  • lalrpop-0.9.0
    • compares mtime() ot mtime()
  • lalrpop-snap-0.9.0
    • compares mtime() ot mtime()
  • logwatcher-0.1.0
    • assumes ino() returns u64 (ok)
  • pnacl-build-helper-1.4.10
    • immediately casts mtime()
  • psutil-1.0.0
    • reads uid/gid
  • systemd-crontab-generator-1.0.0-rc6
    • reads uid
    • casts mtime
  • tango-0.3.3
    • casts mtime immediately
    • prints mtime
  • utime-0.1.3
    • casts atime/mtime immediately
    • equates atime/mtime with constant
  • walkdir-0.1.5
    • compares dev/ino with another dev/ino
  • keyutils-0.2.0
    • uses uid/gid
  • procinfo-0.1.1
    • uses uid/gid/pid
  • tar-0.4.0
    • creates permissions from a mode_t

My conclusion from this is that we can likely take the path set forth in this RFC (aka merge the two commits above at the same time). In summary, they:

  • Deprecate all raw types that aren't in std::os::raw
  • Expand the MetadataExt trait on all platforms for method accessors of all fields. The fields now returned widened types which are the same across platforms (consistency across platforms is not required, however, it's just convenient)
  • [breaking] Change the definition of all std::os::*::raw type aliases to correspond to the newly widened types that are being returned on each platform.
  • [breaking] Change the definition of std::os::*::raw::stat on Linux to match the LFS definitions rather than the standard ones.

The breaking changes are specifically breaking if you assume std/libc or std/gcc agree on the types, which from the above analysis looks like is limited to just a few crates (both of which have PRs to be updated).

@alexcrichton
Copy link
Member Author

We discussed this during libs team triage today, and the conclusion was that given the limited fall out we should push forward with this. Thanks again for the discussion everyone!

@cuviper
Copy link
Member

cuviper commented Feb 11, 2016

🎉

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Feb 13, 2016
This commit is an implementation of [RFC 1415][rfc] which deprecates all types
in the `std::os::*::raw` modules.

[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1415-trim-std-os.md

Many of the types in these modules don't actually have a canonical platform
representation, for example the definition of `stat` on 32-bit Linux will change
depending on whether C code is compiled with LFS support or not. Unfortunately
the current types in `std::os::*::raw` are billed as "compatible with C", which
in light of this means it isn't really possible.

To make matters worse, platforms like Android sometimes define these types as
*smaller* than the way they're actually represented in the `stat` structure
itself. This means that when methods like `DirEntry::ino` are called on Android
the result may be truncated as we're tied to returning a `ino_t` type, not the
underlying type.

The commit here incorporates two backwards-compatible components:

* Deprecate all `raw` types that aren't in `std::os::raw`
* Expand the `std::os::*::fs::MetadataExt` trait on all platforms for method
  accessors of all fields. The fields now returned widened types which are the
  same across platforms (consistency across platforms is not required, however,
  it's just convenient).

and two also backwards-incompatible components:

* Change the definition of all `std::os::*::raw` type aliases to
  correspond to the newly widened types that are being returned on each
  platform.
* Change the definition of `std::os::*::raw::stat` on Linux to match the LFS
  definitions rather than the standard ones.

The breaking changes here will specifically break code that assumes that `libc`
and `std` agree on the definition of `std::os::*::raw` types, or that the `std`
types are faithful representations of the types in C. An [audit] has been
performed of crates.io to determine the fallout which was determined two be
minimal, with the two found cases of breakage having been fixed now.

[audit]: rust-lang/rfcs#1415 (comment)

---

Ok, so after all that, we're finally able to support LFS on Linux! This commit
then simultaneously starts using `stat64` and friends on Linux to ensure that we
can open >4GB files on 32-bit Linux. Yay!

Closes rust-lang#28978
Closes rust-lang#30050
Closes rust-lang#31549
bors added a commit to rust-lang/rust that referenced this pull request Feb 14, 2016
This commit is an implementation of [RFC 1415][rfc] which deprecates all types
in the `std::os::*::raw` modules.

[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1415-trim-std-os.md

Many of the types in these modules don't actually have a canonical platform
representation, for example the definition of `stat` on 32-bit Linux will change
depending on whether C code is compiled with LFS support or not. Unfortunately
the current types in `std::os::*::raw` are billed as "compatible with C", which
in light of this means it isn't really possible.

To make matters worse, platforms like Android sometimes define these types as
*smaller* than the way they're actually represented in the `stat` structure
itself. This means that when methods like `DirEntry::ino` are called on Android
the result may be truncated as we're tied to returning a `ino_t` type, not the
underlying type.

The commit here incorporates two backwards-compatible components:

* Deprecate all `raw` types that aren't in `std::os::raw`
* Expand the `std::os::*::fs::MetadataExt` trait on all platforms for method
  accessors of all fields. The fields now returned widened types which are the
  same across platforms (consistency across platforms is not required, however,
  it's just convenient).

and two also backwards-incompatible components:

* Change the definition of all `std::os::*::raw` type aliases to
  correspond to the newly widened types that are being returned on each
  platform.
* Change the definition of `std::os::*::raw::stat` on Linux to match the LFS
  definitions rather than the standard ones.

The breaking changes here will specifically break code that assumes that `libc`
and `std` agree on the definition of `std::os::*::raw` types, or that the `std`
types are faithful representations of the types in C. An [audit] has been
performed of crates.io to determine the fallout which was determined two be
minimal, with the two found cases of breakage having been fixed now.

[audit]: rust-lang/rfcs#1415 (comment)

---

Ok, so after all that, we're finally able to support LFS on Linux! This commit
then simultaneously starts using `stat64` and friends on Linux to ensure that we
can open >4GB files on 32-bit Linux. Yay!

Closes #28978
Closes #30050
Closes #31549
@Centril Centril added A-repr #[repr(...)] related proposals & ideas A-machine Proposals relating to Rust's abstract machine. labels Nov 23, 2018
@ariasuni
Copy link

ariasuni commented Oct 3, 2019

I’m not sure what’s the procedure for this but, is it possible to plan to remove/forbid calling e.g. MetadataExt::as_raw_stat() in a future edition?

I messed around with the code to be able to use statx (to get creation time on Linux) instead of stat64 but I’m stuck because of this API…

rust-lang/rust#61386

@steveklabnik
Copy link
Member

steveklabnik commented Oct 4, 2019 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-machine Proposals relating to Rust's abstract machine. A-repr #[repr(...)] related proposals & ideas final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. T-libs-api Relevant to the library API team, which will review and decide on the RFC.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants