Broader no_std eco-system via clearer paths.

[TheVova]

Im not sure this is the correct place for this, but im sure this issue has crept up for more here than just me.
Rust has a wonderful eco-system and good tooling to allow using 3rd party crates very easily... However, since most of those creates are not no_std, it limits the number of available tools for embedded programmers.

As it stands, the responsibiliy of making a library creates no_std compliant rests with the crate owner/maintainer. Looking around at crates in the wild, there tend to be a number of 'please make this no_std' issues open. The common problem for many crates is usually not any pattern or feature that isnt available on no_std, it's a simple naming issue: use std::x::y....
Having std being a combination of core, alloc, and, shall we call it, 'sys' (all the stuff that actually needs OS support, etc.) sure helps programmers not need to know where everything 'originally' lives, but also kills support for no_std with something as trivial as use std::ops::Add. Macro authors are also advised to use full paths to function/method calls, and more often than not, its std::, even for features that are in core/alloc.

Primarily, it is an awareness issue. People who dont work with embedded dont think about it. And they shouldnt have to go out of their way to ensure no_std is something they support, It should 'just work' for anything that is compatible (i.e doesnt reference/use anything not from core).
l'd love to just make people aware that core and alloc are a thing, and when we say 'use the full path' we mean the backing crate, not the facade.
This applies to the documentation of various functions on docs.rust-lang.org as well. The first search result for a trait or function is std::... .I've had to help several embedded rust newbies to realise that yes, the documentation says std::ops::Add, but its a reexport from core, so we can use it.
another thing is the IDE story. I use intellij-rust often and noticed that the 'auto import' feature uses std as well, for everything. Idont remember if RLS has support for this feature at all, or if it happens with rust-analyzer as well, anybody knows?
Essentially, making embedded better is raising awareness and making people realise embedded is a thing, and is not (well, not always) just a scary minefield of low-level register manipulation and systems with 1k of ram they should ignore. Some of us have operating systems! threads! filesystems! amazing what modern technology can do :)

A more technical approach that might ease our pains in the mean time:
Can we make the compiler see through these exports down to the base path? i.e realise that if I have use std::ops::Add I might as well have written core::ops::Add? or have 'Vec' 'just work' if the code using this library has 'extern crate alloc'. I feel this would help the embedded story quite a lot.

This is maybe more a question for the lang-team,but i'd love to get some input and see if i'm totally off-base here.

[almindor]

This is an interesting take. I agree that std vs core is something that should've been handled better. Sadly the cat is out of the bag now and getting a properly linter "this std can be replaced for core" is not simple. If someone could build a cargo unstd plugin that would probably be the best 1st step.

[therealprof]

@almindor You mean like https://github.com/hobofan/cargo-nono

@TheVova You're not completely off but there's indeed very little we can do here in the EWG. Luckily for us there's another big area where no_std is a hard requirement: WebAssembly, so there's awareness at least.

[TheVova]

@almindor I dont think a linter would do it, was thinking this should maybe done at the language level itself. for example, looking at the source at /libstd/prelude/v1.rs, we find the following comment:

// The file so far is equivalent to src/libcore/prelude/v1.rs,
// and below to src/liballoc/prelude.rs.
// Those files are duplicated rather than using glob imports
// because we want docs to show these re-exports as pointing to within std.

the last line there is important because it seems to say that if we did glob import from core/alloc, the paths wont point to std anymore... which is a step forward, because then i think std::ops::* is the same as core::ops::* (just an example). I realise its not much we can do here at the embedded workgroup, but is this worth checking out or 'nah dont bother'?

[almindor]

cargo-nono seems like a good first step. One issue I think still remains is the "core <-> std" false equivalency. I'm sure there are cases where core::something does not equal to std::something and detecting those automatically is probably a PITA.

[TheVova]

yeah, cargo no-no seemed cool, i did not know about it. As far as the core<->std equivalence, from what i can gather from looking over the rust code, it is in fact a strict superset. (which is has to be, because otherwise the language itself breaks its coherence rules). so yeah, not everything in std:: is in core, but all from core works in std in the same way. so a fully qualified item (crate::mod::maybe_inner_mods::Item(fn/trait/ty) is the same between std and core, while a module doesnt have to be. same applied to alloc. std can add things, but not change how they work. feels to me that a strict superset relation is something that can be easily found (heres a brute force approach: change std to core in every import. if it builds, the item is in core, if it breaks, its not :P)

Thanks anyway for the comments. i just was over looking at the official google flatbuffers implementation for Rust, and the only reason that one isnt no_std is because all the imports use std as the root, and none of those use std only items. slightly annoying. and its not the first crate to do this... death-by-a-thousand-papercuts type of situation.

[jonas-schievink]

IIRC there was some discussion around removing the libstd split and integrating this concept of "support levels" in a different way, but I can't find it now. This is definitely a known issue though, but from what I know there isn't really much bandwidth available to tackle it (it seems to require fairly deep language design work).

As far as the core<->std equivalence, from what i can gather from looking over the rust code, it is in fact a strict superset. (which is has to be, because otherwise the language itself breaks its coherence rules).

In terms of API, yes (at least currently), but for example core::panic! works very differently from std::panic!. It's also unrelated to coherence, which libstd violates freely since it uses a magic attribute to disable it.

There's no inherent reason for why we can't expose a different API in core than in std, but it would make things harder to understand for users. There are use cases though, for example for getting an Error trait in libcore, or io::{Read, Write}. How to deal with these probably requires a pretty in-depth RFC at this point though.

[TheVova]

@jonas-schievink I dont know thats its unrelated to coherence... not internally, but from a user viewpoint, if the same operation behaved differently on the same type between std and core, you essentially lost coherence (im being a bit loose with the term here, i know). i think exposing a different API in core will make it even harder to write 'portable' code, but the comment re io::{Read, Write} hits home.
So i guess tools like cargo-nono to help porting things to no_std is the most viable option. Time to make some PRs... Thanks for the input :)

[jonas-schievink]

Yes, coherence has a specific meaning in Rust.

So i guess tools like cargo-nono to help porting things to no_std is the most viable option. Time to make some PRs...

For now, yeah, this definitely seems like the easiest and most impactful work to be done.

[ryankurte]

another thing that can bite with FFI is dependence on libc which is not implemented for a lot of platforms, so you almost always have to patch the bindgen build to use cty instead of libc. the move of cty types to libc has been proposed japaric/cty#14 and is underway here rust-lang/libc#1820