-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a new wasm32-unknown-wasi target #59464
Conversation
(rust_highfive has picked a reviewer for you, use r? to override) |
This is coupled with rust-lang/libc#1307 for the r? @fitzgen |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎉
//! The wasi target is **very** new in its specification. It's likely going to | ||
//! be a long effort to get it standardized and stable. We'll be following it as | ||
//! best we can with this target. Don't start relying on too much here unless | ||
//! you know what you're getting in to! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this very nice doc comment 👍
} | ||
|
||
pub fn temp_dir() -> PathBuf { | ||
panic!("no filesystem on wasm") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might make sense to have an inline(never)
function that all the unsupported panics defer to. I think this should make code sizes smaller.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps yeah! I figure we can fix that up as it goes along because unconditionally panicking binaries aren't too useful anyway :)
I'm not sure I understand. Is there something that prevents |
Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem. Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
This sort of depends on how the file descriptor situation is settled in wasi's sysroot. It seems like there's the concept of a "wasi file descriptor" as well as a "libc file descriptor", indexing different spaces. For those purposes we'd need to convert between them. This is sort of similar as well to "what if we used |
Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem. Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
Add intial support for wasm32-unknown-wasi This target is [being proposed][LINK] int he rust-lang/rust repository and this is intended to get coupled with that proposal. The definitions here all match the upstream reference-sysroot definitions and the functions all match the reference sysroot as well. The linkage here is described more in detail on the Rust PR itself, but in general it's similar to musl. Automatic verification has been implemented in the same manner as other targets, and it's been used locally to develop this PR and catch errors in the bindings already written (also to help match the evolving sysroot of wasi). The verification isn't hooked up to CI yet though because there is no wasi target distributed via rustup just yet, but once that's done I'll file a follow-up PR to execute verification on CI. [LINK]: rust-lang/rust#59464
|
||
set -ex | ||
|
||
curl http://releases.llvm.org/8.0.0/clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-14.04.tar.xz | \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might make sense to use latest LTS (18.04) instead
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We will eventually! For now though the rest of the image here is 17.10, but when that upgrades we can upgrade this
Add intial support for wasm32-unknown-wasi This target is [being proposed][LINK] int he rust-lang/rust repository and this is intended to get coupled with that proposal. The definitions here all match the upstream reference-sysroot definitions and the functions all match the reference sysroot as well. The linkage here is described more in detail on the Rust PR itself, but in general it's similar to musl. Automatic verification has been implemented in the same manner as other targets, and it's been used locally to develop this PR and catch errors in the bindings already written (also to help match the evolving sysroot of wasi). The verification isn't hooked up to CI yet though because there is no wasi target distributed via rustup just yet, but once that's done I'll file a follow-up PR to execute verification on CI. [LINK]: rust-lang/rust#59464
Add intial support for wasm32-unknown-wasi This target is [being proposed][LINK] int he rust-lang/rust repository and this is intended to get coupled with that proposal. The definitions here all match the upstream reference-sysroot definitions and the functions all match the reference sysroot as well. The linkage here is described more in detail on the Rust PR itself, but in general it's similar to musl. Automatic verification has been implemented in the same manner as other targets, and it's been used locally to develop this PR and catch errors in the bindings already written (also to help match the evolving sysroot of wasi). The verification isn't hooked up to CI yet though because there is no wasi target distributed via rustup just yet, but once that's done I'll file a follow-up PR to execute verification on CI. [LINK]: rust-lang/rust#59464
WASI uses |
@sunfishcode oh ok thanks! So the idea is that a libc file descriptor is interchangeable with a wasi file descriptor? If that's the case sounds like we'll be using a design similar to unix for libstd where every type just internally contains a 32-bit file descriptor (probably the |
Yes, they're interchangeable. (If WASI ever produces an fd >= 231, libc might get confused, but realistically you'll hit other limits well before that happens.) |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
📌 Commit e59ddbf2e8f43385ede4833abc2d2edf3c5e9881 has been approved by |
⌛ Testing commit e59ddbf2e8f43385ede4833abc2d2edf3c5e9881 with merge a31f4e2bc6c22c03455e0a36ceb9ce3a1049fce8... |
@bors retry |
🔒 Merge conflict This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again. How do I rebase?Assuming
You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial. Please avoid the "Resolve conflicts" button on GitHub. It uses Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Error message
|
This commit adds a new wasm32-based target distributed through rustup, supported in the standard library, and implemented in the compiler. The `wasm32-unknown-wasi` target is intended to be a WebAssembly target which matches the [WASI proposal recently announced.][LINK]. In summary the WASI target is an effort to define a standard set of syscalls for WebAssembly modules, allowing WebAssembly modules to not only be portable across architectures but also be portable across environments implementing this standard set of system calls. The wasi target in libstd is still somewhat bare bones. This PR does not fill out the filesystem, networking, threads, etc. Instead it only provides the most basic of integration with the wasi syscalls, enabling features like: * `Instant::now` and `SystemTime::now` work * `env::args` is hooked up * `env::vars` will look up environment variables * `println!` will print to standard out * `process::{exit, abort}` should be hooked up appropriately None of these APIs can work natively on the `wasm32-unknown-unknown` target, but with the assumption of the WASI set of syscalls we're able to provide implementations of these syscalls that engines can implement. Currently the primary engine implementing wasi is [wasmtime], but more will surely emerge! In terms of future development of libstd, I think this is something we'll probably want to discuss. The purpose of the WASI target is to provide a standardized set of syscalls, but it's *also* to provide a standard C sysroot for compiling C/C++ programs. This means it's intended that functions like `read` and `write` are implemented for this target with a relatively standard definition and implementation. It's unclear, therefore, how we want to expose file descriptors and how we'll want to implement system primitives. For example should `std::fs::File` have a libc-based file descriptor underneath it? The raw wasi file descriptor? We'll see! Currently these details are all intentionally hidden and things we can change over time. A `WasiFd` sample struct was added to the standard library as part of this commit, but it's not currently used. It shows how all the wasi syscalls could be ergonomically bound in Rust, and they offer a possible implementation of primitives like `std::fs::File` if we bind wasi file descriptors exactly. Apart from the standard library, there's also the matter of how this target is integrated with respect to its C standard library. The reference sysroot, for example, provides managment of standard unix file descriptors and also standard APIs like `open` (as opposed to the relative `openat` inspiration for the wasi ssycalls). Currently the standard library relies on the C sysroot symbols for operations such as environment management, process exit, and `read`/`write` of stdio fds. We want these operations in Rust to be interoperable with C if they're used in the same process. Put another way, if Rust and C are linked into the same WebAssembly binary they should work together, but that requires that the same C standard library is used. We also, however, want the `wasm32-unknown-wasi` target to be usable-by-default with the Rust compiler without requiring a separate toolchain to get downloaded and configured. With that in mind, there's two modes of operation for the `wasm32-unknown-wasi` target: 1. By default the C standard library is statically provided inside of `liblibc.rlib` distributed as part of the sysroot. This means that you can `rustc foo.wasm --target wasm32-unknown-unknown` and you're good to go, a fully workable wasi binary pops out. This is incompatible with linking in C code, however, which may be compiled against a different sysroot than the Rust code was previously compiled against. In this mode the default of `rust-lld` is used to link binaries. 2. For linking with C code, the `-C target-feature=-crt-static` flag needs to be passed. This takes inspiration from the musl target for this flag, but the idea is that you're no longer using the provided static C runtime, but rather one will be provided externally. This flag is intended to also get coupled with an external `clang` compiler configured with its own sysroot. Therefore you'll typically use this flag with `-C linker=/path/to/clang-script-wrapper`. Using this mode the Rust code will continue to reference standard C symbols, but the definition will be pulled in by the linker configured. Alright so that's all the current state of this PR. I suspect we'll definitely want to discuss this before landing of course! This PR is coupled with libc changes as well which I'll be posting shortly. [LINK]: [wasmtime]:
@bors: r=fitzgen |
📌 Commit ace7124 has been approved by |
Add a new wasm32-unknown-wasi target This commit adds a new wasm32-based target distributed through rustup, supported in the standard library, and implemented in the compiler. The `wasm32-unknown-wasi` target is intended to be a WebAssembly target which matches the [WASI proposal recently announced][LINK]. In summary the WASI target is an effort to define a standard set of syscalls for WebAssembly modules, allowing WebAssembly modules to not only be portable across architectures but also be portable across environments implementing this standard set of system calls. The wasi target in libstd is still somewhat bare bones. This PR does not fill out the filesystem, networking, threads, etc. Instead it only provides the most basic of integration with the wasi syscalls, enabling features like: * `Instant::now` and `SystemTime::now` work * `env::args` is hooked up * `env::vars` will look up environment variables * `println!` will print to standard out * `process::{exit, abort}` should be hooked up appropriately None of these APIs can work natively on the `wasm32-unknown-unknown` target, but with the assumption of the WASI set of syscalls we're able to provide implementations of these syscalls that engines can implement. Currently the primary engine implementing wasi is [wasmtime], but more will surely emerge! In terms of future development of libstd, I think this is something we'll probably want to discuss. The purpose of the WASI target is to provide a standardized set of syscalls, but it's *also* to provide a standard C sysroot for compiling C/C++ programs. This means it's intended that functions like `read` and `write` are implemented for this target with a relatively standard definition and implementation. It's unclear, therefore, how we want to expose file descriptors and how we'll want to implement system primitives. For example should `std::fs::File` have a libc-based file descriptor underneath it? The raw wasi file descriptor? We'll see! Currently these details are all intentionally hidden and things we can change over time. A `WasiFd` sample struct was added to the standard library as part of this commit, but it's not currently used. It shows how all the wasi syscalls could be ergonomically bound in Rust, and they offer a possible implementation of primitives like `std::fs::File` if we bind wasi file descriptors exactly. Apart from the standard library, there's also the matter of how this target is integrated with respect to its C standard library. The reference sysroot, for example, provides managment of standard unix file descriptors and also standard APIs like `open` (as opposed to the relative `openat` inspiration for the wasi ssycalls). Currently the standard library relies on the C sysroot symbols for operations such as environment management, process exit, and `read`/`write` of stdio fds. We want these operations in Rust to be interoperable with C if they're used in the same process. Put another way, if Rust and C are linked into the same WebAssembly binary they should work together, but that requires that the same C standard library is used. We also, however, want the `wasm32-unknown-wasi` target to be usable-by-default with the Rust compiler without requiring a separate toolchain to get downloaded and configured. With that in mind, there's two modes of operation for the `wasm32-unknown-wasi` target: 1. By default the C standard library is statically provided inside of `liblibc.rlib` distributed as part of the sysroot. This means that you can `rustc foo.wasm --target wasm32-unknown-unknown` and you're good to go, a fully workable wasi binary pops out. This is incompatible with linking in C code, however, which may be compiled against a different sysroot than the Rust code was previously compiled against. In this mode the default of `rust-lld` is used to link binaries. 2. For linking with C code, the `-C target-feature=-crt-static` flag needs to be passed. This takes inspiration from the musl target for this flag, but the idea is that you're no longer using the provided static C runtime, but rather one will be provided externally. This flag is intended to also get coupled with an external `clang` compiler configured with its own sysroot. Therefore you'll typically use this flag with `-C linker=/path/to/clang-script-wrapper`. Using this mode the Rust code will continue to reference standard C symbols, but the definition will be pulled in by the linker configured. Alright so that's all the current state of this PR. I suspect we'll definitely want to discuss this before landing of course! This PR is coupled with libc changes as well which I'll be posting shortly. [LINK]: https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/ [wasmtime]: https://github.com/cranestation/wasmtime-wasi
☀️ Test successful - checks-travis, status-appveyor |
Version 1.35.0 (2019-05-23) ========================== Language -------- - [`FnOnce`, `FnMut`, and the `Fn` traits are now implemented for `Box<FnOnce>`, `Box<FnMut>`, and `Box<Fn>` respectively.][59500] - [You can now coerce closures into unsafe function pointers.][59580] e.g. ```rust unsafe fn call_unsafe(func: unsafe fn()) { func() } pub fn main() { unsafe { call_unsafe(|| {}); } } ``` Compiler -------- - [Added the `armv6-unknown-freebsd-gnueabihf` and `armv7-unknown-freebsd-gnueabihf` targets.][58080] - [Added the `wasm32-unknown-wasi` target.][59464] Libraries --------- - [`Thread` will now show its ID in `Debug` output.][59460] - [`StdinLock`, `StdoutLock`, and `StderrLock` now implement `AsRawFd`.][59512] - [`alloc::System` now implements `Default`.][59451] - [Expanded `Debug` output (`{:#?}`) for structs now has a trailing comma on the last field.][59076] - [`char::{ToLowercase, ToUppercase}` now implement `ExactSizeIterator`.][58778] - [All `NonZero` numeric types now implement `FromStr`.][58717] - [Removed the `Read` trait bounds on the `BufReader::{get_ref, get_mut, into_inner}` methods.][58423] - [You can now call the `dbg!` macro without any parameters to print the file and line where it is called.][57847] - [In place ASCII case conversions are now up to 4× faster.][59283] e.g. `str::make_ascii_lowercase` - [`hash_map::{OccupiedEntry, VacantEntry}` now implement `Sync` and `Send`.][58369] Stabilized APIs --------------- - [`f32::copysign`] - [`f64::copysign`] - [`RefCell::replace_with`] - [`RefCell::map_split`] - [`ptr::hash`] - [`Range::contains`] - [`RangeFrom::contains`] - [`RangeTo::contains`] - [`RangeInclusive::contains`] - [`RangeToInclusive::contains`] - [`Option::copied`] Cargo ----- - [You can now set `cargo:rustc-cdylib-link-arg` at build time to pass custom linker arguments when building a `cdylib`.][cargo/6298] Its usage is highly platform specific. Misc ---- - [The Rust toolchain is now available natively for musl based distros.][58575] [59460]: rust-lang/rust#59460 [59464]: rust-lang/rust#59464 [59500]: rust-lang/rust#59500 [59512]: rust-lang/rust#59512 [59580]: rust-lang/rust#59580 [59283]: rust-lang/rust#59283 [59451]: rust-lang/rust#59451 [59076]: rust-lang/rust#59076 [58778]: rust-lang/rust#58778 [58717]: rust-lang/rust#58717 [58369]: rust-lang/rust#58369 [58423]: rust-lang/rust#58423 [58080]: rust-lang/rust#58080 [57847]: rust-lang/rust#57847 [58575]: rust-lang/rust#58575 [cargo/6298]: rust-lang/cargo#6298 [`f32::copysign`]: https://doc.rust-lang.org/stable/std/primitive.f32.html#method.copysign [`f64::copysign`]: https://doc.rust-lang.org/stable/std/primitive.f64.html#method.copysign [`RefCell::replace_with`]: https://doc.rust-lang.org/stable/std/cell/struct.RefCell.html#method.replace_with [`RefCell::map_split`]: https://doc.rust-lang.org/stable/std/cell/struct.RefCell.html#method.map_split [`ptr::hash`]: https://doc.rust-lang.org/stable/std/ptr/fn.hash.html [`Range::contains`]: https://doc.rust-lang.org/std/ops/struct.Range.html#method.contains [`RangeFrom::contains`]: https://doc.rust-lang.org/std/ops/struct.RangeFrom.html#method.contains [`RangeTo::contains`]: https://doc.rust-lang.org/std/ops/struct.RangeTo.html#method.contains [`RangeInclusive::contains`]: https://doc.rust-lang.org/std/ops/struct.RangeInclusive.html#method.contains [`RangeToInclusive::contains`]: https://doc.rust-lang.org/std/ops/struct.RangeToInclusive.html#method.contains [`Option::copied`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.copied
This commit adds a new wasm32-based target distributed through rustup,
supported in the standard library, and implemented in the compiler. The
wasm32-unknown-wasi
target is intended to be a WebAssembly targetwhich matches the WASI proposal recently announced. In summary
the WASI target is an effort to define a standard set of syscalls for
WebAssembly modules, allowing WebAssembly modules to not only be
portable across architectures but also be portable across environments
implementing this standard set of system calls.
The wasi target in libstd is still somewhat bare bones. This PR does not
fill out the filesystem, networking, threads, etc. Instead it only
provides the most basic of integration with the wasi syscalls, enabling
features like:
Instant::now
andSystemTime::now
workenv::args
is hooked upenv::vars
will look up environment variablesprintln!
will print to standard outprocess::{exit, abort}
should be hooked up appropriatelyNone of these APIs can work natively on the
wasm32-unknown-unknown
target, but with the assumption of the WASI set of syscalls we're able
to provide implementations of these syscalls that engines can implement.
Currently the primary engine implementing wasi is wasmtime, but more
will surely emerge!
In terms of future development of libstd, I think this is something
we'll probably want to discuss. The purpose of the WASI target is to
provide a standardized set of syscalls, but it's also to provide a
standard C sysroot for compiling C/C++ programs. This means it's
intended that functions like
read
andwrite
are implemented for thistarget with a relatively standard definition and implementation. It's
unclear, therefore, how we want to expose file descriptors and how we'll
want to implement system primitives. For example should
std::fs::File
have a libc-based file descriptor underneath it? The raw wasi file
descriptor? We'll see! Currently these details are all intentionally
hidden and things we can change over time.
A
WasiFd
sample struct was added to the standard library as part ofthis commit, but it's not currently used. It shows how all the wasi
syscalls could be ergonomically bound in Rust, and they offer a possible
implementation of primitives like
std::fs::File
if we bind wasi filedescriptors exactly.
Apart from the standard library, there's also the matter of how this
target is integrated with respect to its C standard library. The
reference sysroot, for example, provides managment of standard unix file
descriptors and also standard APIs like
open
(as opposed to therelative
openat
inspiration for the wasi ssycalls). Currently thestandard library relies on the C sysroot symbols for operations such as
environment management, process exit, and
read
/write
of stdio fds.We want these operations in Rust to be interoperable with C if they're
used in the same process. Put another way, if Rust and C are linked into
the same WebAssembly binary they should work together, but that requires
that the same C standard library is used.
We also, however, want the
wasm32-unknown-wasi
target to beusable-by-default with the Rust compiler without requiring a separate
toolchain to get downloaded and configured. With that in mind, there's
two modes of operation for the
wasm32-unknown-wasi
target:By default the C standard library is statically provided inside of
liblibc.rlib
distributed as part of the sysroot. This means thatyou can
rustc foo.wasm --target wasm32-unknown-unknown
and you'regood to go, a fully workable wasi binary pops out. This is
incompatible with linking in C code, however, which may be compiled
against a different sysroot than the Rust code was previously
compiled against. In this mode the default of
rust-lld
is used tolink binaries.
For linking with C code, the
-C target-feature=-crt-static
flagneeds to be passed. This takes inspiration from the musl target for
this flag, but the idea is that you're no longer using the provided
static C runtime, but rather one will be provided externally. This
flag is intended to also get coupled with an external
clang
compiler configured with its own sysroot. Therefore you'll typically
use this flag with
-C linker=/path/to/clang-script-wrapper
. Usingthis mode the Rust code will continue to reference standard C
symbols, but the definition will be pulled in by the linker configured.
Alright so that's all the current state of this PR. I suspect we'll
definitely want to discuss this before landing of course! This PR is
coupled with libc changes as well which I'll be posting shortly.