Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically Usable External Crates #2088

Closed
wants to merge 7 commits into from
Closed
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 136 additions & 0 deletions text/0000-no-more-extern-crate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
- Feature Name: infer-extern-crate
- Start Date: (fill me in with today's date, YYYY-MM-DD)
- RFC PR: (leave this empty)
- Rust Issue: (leave this empty)

# Summary
[summary]: #summary

This RFC reduces redundant boilerplate when including external crates.
`extern crate` declarations will be inferred from the arguments passed to `rustc`.
With this change, projects using Cargo
(or other build systems using the same mechanism)
will no longer have to specify `extern crate`:
dependencies added to `Cargo.toml` will be automatically imported.
Projects which require more flexibility can still use manual `extern crate`
and will be unaffected by this RFC.

# Motivation
[motivation]: #motivation

One of the principles of Rust is that using external crates should be as
easy and natural as using the standard library.
This allows the standard library to be kept small, and allows mature, standard
solutions to be developed by the community.

Currently, however, external crates must be specified twice: once in a build
system such as Cargo and again in the source code using `extern crate`.
When external dependencies are conditional (`cfg`) upon feature flags or the
target platform, the conditional logic must appear in both `Cargo.toml` and
the `extern crate` declarations.

This duplication causes unnecessary effort and results in one more opportunity
for mistakes when working with conditionally-enabled dependencies.
Allowing the omission of the redundant `extern crate` syntax contributes to the
roadmap goals of
[improving Rust's ergonomics](https://github.com/rust-lang/rust-roadmap/issues/17)
and
[providing easy access to high-quality crates.](https://github.com/rust-lang/rust-roadmap/issues/9)

# Guide-Level Explanation
[guide]: #guide

When you add a dependency to your `Cargo.toml`, it is immediately usable within
the source of your crate:

```toml
# Cargo.toml:
name = "my_crate"
version = "0.1.0"
authors = ["Me" <[email protected]>]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this line confuses Github's rendering.


[dependencies]
rand = "0.3"
```

```rust
// src/main.rs:

fn main() {
println!"A random character: {}", rand::random::<char>());
}
```

# Reference-Level Explanation
[reference]: #reference

External crates can be passed to the rust compiler using the
`--extern CRATE_NAME=PATH` flag.
For example, `cargo build`-ing a crate `my_crate` with a dependency on `rand`
results in a call to rustc that looks something like
`rustc --crate-name mycrate src/main.rs --extern rand=/path/to/librand.rlib ...`.

When an external crate is specified this way,
the crate will automatically brought into scope as if an
`extern crate name_of_crate;`
declaration had been added to the current crate root.
This behavior won't occur when including a library using the `-l`
or `-L` flags.

We will continue to support the current `extern crate` syntax,
both for backwards compatibility and to enable users who want to use manual
`extern crate` in order to have more fine grained control-- say, if they wanted
to import an external crate only inside an inner module.
No automatic import will occur if an `extern crate` declaration for the same
external dependency appears anywhere within the crate.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Within the crate before macro expansion or after macro expansion? :)

With explicit extern crates crate names are injected into modules immediately and unconditionally, and can be used in macros before waiting for them to be expanded, so it's reasonable to make this check immediately before expanding anything as well.
On the other hand, if some macro expands to an extern crate item, then you'll have a conflict instead of an override.

(The same issue exists for "items such as modules, types, or functions that conflict with the names of implicitly imported crates will cause the implicit extern crate declaration to be removed")

Copy link
Contributor

@petrochenkov petrochenkov Jul 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be possible to reuse the rules from glob imports - they also "step aside" when conflict with explicitly written names and interactions with macros were already figured out in recent name resolution RFCs.

The "extern crate declaration for the same external dependency appears anywhere within the crate" rule doesn't fit into the glob import analogy though, but I'm not sure how useful it is.

For example, if `rand = "0.3"` is listed as a dependency in Cargo.toml
and `extern crate rand;` appears somewhere in the crate being compiled,
then no implicit `extern crate rand;` will be added.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this special case? It seems like an "if a tree falls in the forest" kind of thing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My personal preference would be that users avoid extern crate as much as possible. In a world with no extern crate, this rule doesn't really matter at all.

I know that there are other people who will disagree, though, and I think that this check will allow them to continue keeping a tight lock on where their external dependencies are used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this portion of the RFC is under-specified, and could also use a bit more analysis around rationale.

re: under-specification: "appears anywhere within the crate" is a bit ambiguous. Are we taking cfg into account? Macros?

re: rationale: you mention back-compat and wanting to control scoping. It might be useful to separate those concerns a bit. Is there something more minimal we could do strictly for back-compat?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is actually impossible to check for „anywhere within the crate“. Or, if I have an example (file in examples) using the library and that one uses an external crate, does it count? How does the rustc compiling the library know that, without being informed about the examples? Or does a crate mean one unit of compilation, therefore the example would be considered a separate crate?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By "anywhere within the crate" I meant "anywhere within the crate source currently being compiled after macro expansion" (including cfgs). This isn't perfect, but I think it's the only truly viable option. I suppose in order for macros to be imported correctly in a macros 2.0 world, you'd have to already know at least some set of external crates that you were going to import. I'll have to think about this-- my gut reaction is to say that external macros expanding to extern crate; could be made an error, but maybe that's too drastic / surprising.

Overall, I feel like I'm pretty open to suggestions on this front. My goal was to support backcompat and leave an "out" for people who had special use cases for extern crate-- perhaps I should have instead focused merely on backcompat.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. That dependency on cfgs, that can make a crate to auto-include or not, might be a source of unexpected failures on different configurations.

#[cfg(a_feature)]
mod submodule {
  extern crate a_crate;
}

fn main() {
  a_crate::do_something();
}

This will stop compiling if the a_feature is activated, but otherwise will compile just fine. And the a_feature can be hidden somewhere deep.

I don't have any new proposal (I think there are some in the discussion), but I find this behaviour a bit complex and magical.

Copy link

@le-jzr le-jzr Jul 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cramertj For backcompat, just leave extern crate working as it does now. Even currently, it's not an error to have extern crate in the root and then also extern crate in several submodules. With the RFC, extern crate lib; and use lib; should be mostly interchangeable in their basic forms.

Copy link

@le-jzr le-jzr Jul 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The motivation for extern crate anywhere disabling the feature is just about not polluting namespace. We've already established that the RFC shouldn't result in automatic inclusion in local namespace. It can possibly be further specified that the automatic path links can be shadowed by root modules (which would make the shadowed crates only accessible via extern crate, but that's not a problem, and it makes everything work backwards compatibly).

If Cargo.toml were to also list another dependency, `log = "0.3"`, and no
`extern crate log;` appears in the crate being compiled,
then an `extern crate log;` would be implicitly added.

Additionally, items such as modules, types, or functions that conflict with
the names of implicitly imported crates will cause the implicit `extern crate`
declaration to be removed.
Note that this is different from the current behavior of the
implicitly-imported `std` module.
Currently, creating a root-level item named `std` results in a name conflict
error. For consistency with other crates, this error will be removed.
Creating a root-level item named `std` will prevent `std` from being included,
and will trigger a warning.

It will still be necessary to use the `extern crate` syntax when using
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is actually kind of a drawback - if there's a long enough period where this RFC is implemented but macros 2.0 isn't, then newcomers to the language won't know about extern crate at all until they want to use a macro. Hopefully that gap won't be all that long though?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's definitely annoying. Since we do have a plan to migrate away from #[macro_use] in the relatively near future, I think it's livable, but the overlap period of "extern crate just for macros" is frustrating.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pub extern crate would also still have to be explicit, I guess.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pub extern crate could potentially be replaced with pub use crate. This could cause problems with having the same item imported twice though.

`#[macro_use]` to import macros from a crate. This is necessary in order to
prevent macros from being automatically brought into scope and changing the
behavior of existing code.
However, as specified in
[RFC 1561](https://github.com/rust-lang/rfcs/blob/master/text/1561-macro-naming.md#importing-macros),
macros 2.0 will no longer require `#[macro_use]`, replacing it with
normal `use` declarations, for which no `extern crate` is required.

One final remaining use case of `extern crate` syntax is for aliasing, i.e.
`extern crate foo as bar;`. There is no way to infer aliasing information from
Cargo.toml, so aliased crates will need to be specied using `extern crate`
syntax.

# Alternatives
[alternatives]: #alternatives

- Don't do this.
- Specify external dependencies using only `extern crate`, rather than only
`Cargo.toml`, by using `extern crate foo = "0.2";` or similar. This would
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the olden days, the pre-Cargo build tool behaved kind of like this, interestingly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do something even simpler. If cargo finds a missing dependency, it can offer to add the latest version from crates.io to Cargo.toml (pinning it to ^latestVersion).

require either `Cargo` or `rustc` to first scan the source before determining
the build dependencies of the existing code, a system which requires fairly
tight coupling between a build system and `rustc`, and which would almost
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think any "tight" coupling is needed for this: just add a mode to rustc which when called only scans for extern crate statements and dumps them via json. Surely the ride is a bit bumpy because deps don't exist yet so no proc macros or similar are supported, but this is not a "tight coupling". External build systems that want to support the new mode can just write a parser for that json code.

certainly interact poorly with third-party build systems.

# Unresolved questions
[unresolved]: #unresolved-questions

- What interactions does this have with future procedural macros?
- Should we lint/warn on local items shadowing implicitly imported crates?
It seems like a useful warning, but it's also a potential
backwards-compatibility hazard for crates which previously depended on a
crate, didn't import it with `extern crate`, and had a root-level item with
an overlapping name (although this seems like an extreme edge case).