Define built-in macros through libcore #62086

petrochenkov · 2019-06-23T20:08:47Z

This PR defines built-in macros through libcore using a scheme similar to lang items (attribute #[rustc_builtin_macro]).
All the macro properties (stability, visibility, etc.) are taken from the source code in libcore, with exception of the expander function transforming input tokens/AST into output tokens/AST, which is still provided by the compiler.

The macros are made available to user code through the standard library prelude ({core,std}::prelude::v1), so they are still always in scope.
As a result built-in macros now have stable absolute addresses in the library, like core::prelude::v1::line!(), this is an insta-stable change.

Right now prelude::v1 is the only publicly available absolute address for these macros, but eventually they can be moved into more appropriate locations with library team approval (e.g. Clone derive -> core::clone::Clone).

Now when built-in macros have canonical definitions they can be imported or reexported without issues (#61687).

Other changes:

You can now define a derive macro with a name matching one of the built-in derives (Built-in macros, derives and attributes have inconsistent interactions with macros of the same name. #52269). This was an artificial restriction that could be worked around with import renaming anyway.

Known regressions:

Empty library crate with a crate-level #![test] attribute no longer compiles without --test. Previously it didn't compile with --test or with the bin crate type.

Fixes #61687
Fixes #61804
r? @eddyb

src/libcore/macros.rs

eddyb

r=me on the last commit, assuming we do want to expose those macros from the top of core and std (has @rust-lang/libs discussed that yet?)

QuietMisdreavus · 2019-07-02T20:49:55Z

Does this PR fix #61804? It looks like it will expose a proper macro location in libcore to the resolver, meaning that rustdoc would be able to link to builtin macros from intra-doc links properly.

petrochenkov · 2019-07-02T20:54:46Z

Maybe?
I'll add the test from #61916 and check whether it works.
UPDATE: It does.

Improve documentation for built-in macros This is the `libcore` part of rust-lang#62086. Right now the only effect is improved documentation. The changes in the last few commits are required to make the `libcore` change compile successfully.

…nkov cleanup: Remove some language features related to built-in macros They are now library features. Cleanup after rust-lang#62086. The unstable book files are moved because tidy complained.

@eddyb

Move special treatment of `derive(Copy, PartialEq, Eq)` from expansion infrastructure to elsewhere As described in #62086 (comment). Reminder: - `derive(PartialEq, Eq)` makes the type it applied to a "structural match" type, so constants of this type can be used in patterns (and const generics in the future). - `derive(Copy)` notifies other derives that the type it applied to implements `Copy`, so `derive(Clone)` can generate optimized code and other derives can generate code working with `packed` types and types with `rustc_layout_scalar_valid_range` attributes. First, the special behavior is now enabled after properly resolving the derives, rather than after textually comparing them with `"Copy"`, `"PartialEq"` and `"Eq"` in `fn add_derived_markers`. The markers are no longer kept as attributes in AST since derives cannot modify items and previously did it through hacks in the expansion infra. Instead, the markers are now kept in a "global context" available from all the necessary places, namely - resolver. For `derive(PartialEq, Eq)` the markers are created by the derive macros themselves and then consumed during HIR lowering to add the `#[structural_match]` attribute in HIR. This is still a hack, but now it's a hack local to two specific macros rather than affecting the whole expansion infra. Ideally we should find the way to put `#[structural_match]` on the impls rather than on the original item, and then consume it in `rustc_mir`, then no hacks in expansion and lowering will be required. (I'll make an issue about this for someone else to solve, after this PR lands.) The marker for `derive(Copy)` cannot be emitted by the `Copy` macro itself because we need to know it *before* the `Copy` macro is expanded for expanding other macros. So we have to do it in resolve and block expansion of any derives in a `derive(...)` container until we know for sure whether this container has `Copy` in it or not. Nasty stuff. r? @eddyb or @matthewjasper

vlad20012 · 2019-08-05T10:18:59Z

Any ETA when these #[cfg(bootstrap)] will be removed from the std?

petrochenkov · 2019-08-05T10:22:42Z

@vlad20012
After the upcoming release, somewhere around August 15th.

vlad20012 · 2019-08-05T10:45:55Z

So these cfgs will remain in the upcoming release?

est31 · 2019-08-05T11:09:28Z

@vlad20012 yes. This is a normal process btw, see the cfg(bootstrap) invocations that each compiler version update PR removes.

@alexcrichton

Give built-in macros stable addresses in the standard library Continuation of rust-lang#62086. Derive macros corresponding to traits from libcore are now available through the same paths as those traits: - `Clone` - `{core,std}::clone::Clone` - `PartialEq` - `{core,std}::cmp::PartialEq` - `Eq` - `{core,std}::cmp::Eq` - `PartialOrd` - `{core,std}::cmp::PartialOrd` - `Ord` - `{core,std}::cmp::Ord` - `Default` - `{core,std}::default::Default` - `Debug` - `{core,std}::fmt::Debug` - `Hash` - `{core,std}::hash::Hash` - `Copy` - `{core,std}::marker::Copy` Fn-like built-in macros are now available through libcore and libstd's root module, by analogy with non-builtin macros defined by libcore and libstd: ```rust {core,std}::{ __rust_unstable_column, asm, assert, cfg, column, compile_error, concat, concat_idents, env, file, format_args, format_args_nl, global_asm, include, include_bytes, include_str, line, log_syntax, module_path, option_env, stringify, trace_macros, } ``` Derive macros without a corresponding trait in libcore or libstd are still available only through prelude (also see rust-lang#62507). Attribute macros also keep being available only through prelude, mostly because they don't have an existing practice to follow. An advice from the library team on their eventual placement would be appreciated. ```rust RustcDecodable, RustcEncodable, bench, global_allocator, test, test_case, ``` r? @alexcrichton

@alexcrichton

Give built-in macros stable addresses in the standard library Continuation of rust-lang#62086. Derive macros corresponding to traits from libcore are now available through the same paths as those traits: - `Clone` - `{core,std}::clone::Clone` - `PartialEq` - `{core,std}::cmp::PartialEq` - `Eq` - `{core,std}::cmp::Eq` - `PartialOrd` - `{core,std}::cmp::PartialOrd` - `Ord` - `{core,std}::cmp::Ord` - `Default` - `{core,std}::default::Default` - `Debug` - `{core,std}::fmt::Debug` - `Hash` - `{core,std}::hash::Hash` - `Copy` - `{core,std}::marker::Copy` Fn-like built-in macros are now available through libcore and libstd's root module, by analogy with non-builtin macros defined by libcore and libstd: ```rust {core,std}::{ __rust_unstable_column, asm, assert, cfg, column, compile_error, concat, concat_idents, env, file, format_args, format_args_nl, global_asm, include, include_bytes, include_str, line, log_syntax, module_path, option_env, stringify, trace_macros, } ``` Derive macros without a corresponding trait in libcore or libstd are still available only through prelude (also see rust-lang#62507). Attribute macros also keep being available only through prelude, mostly because they don't have an existing practice to follow. An advice from the library team on their eventual placement would be appreciated. ```rust RustcDecodable, RustcEncodable, bench, global_allocator, test, test_case, ``` r? @alexcrichton

`tokio::test` and related macros now use the absolute path `::core::prelude::v1::test` to refer to the built-in `test` macro. This absolute path was introduced in rust-lang/rust#62086.

#2503) `tokio::test` and related macros now use the absolute path `::core::prelude::v1::test` to refer to the built-in `test` macro. This absolute path was introduced in rust-lang/rust#62086.

tokio-rs#2503) `tokio::test` and related macros now use the absolute path `::core::prelude::v1::test` to refer to the built-in `test` macro. This absolute path was introduced in rust-lang/rust#62086.

expand/resolve: Turn `#[derive]` into a regular macro attribute This PR turns `#[derive]` into a regular attribute macro declared in libcore and defined in `rustc_builtin_macros`, like it was previously done with other "active" attributes in rust-lang#62086, rust-lang#62735 and other PRs. This PR is also a continuation of rust-lang#65252, rust-lang#69870 and other PRs linked from them, which layed the ground for converting `#[derive]` specifically. `#[derive]` still asks `rustc_resolve` to resolve paths inside `derive(...)`, and `rustc_expand` gets those resolution results through some backdoor (which I'll try to address later), but otherwise `#[derive]` is treated as any other macro attributes, which simplifies the resolution-expansion infra pretty significantly. The change has several observable effects on language and library. Some of the language changes are **feature-gated** by [`feature(macro_attributes_in_derive_output)`](rust-lang#81119). #### Library - `derive` is now available through standard library as `{core,std}::prelude::v1::derive`. #### Language - `derive` now goes through name resolution, so it can now be renamed - `use derive as my_derive; #[my_derive(Debug)] struct S;`. - `derive` now goes through name resolution, so this resolution can fail in corner cases. Crater found one such regression, where import `use foo as derive` goes into a cycle with `#[derive(Something)]`. - **[feature-gated]** `#[derive]` is now expanded as any other attributes in left-to-right order. This allows to remove the restriction on other macro attributes following `#[derive]` (rust-lang/reference#566). The following macro attributes become a part of the derive's input (this is not a change, non-macro attributes following `#[derive]` were treated in the same way previously). - `#[derive]` is now expanded as any other attributes in left-to-right order. This means two derive attributes `#[derive(Foo)] #[derive(Bar)]` are now expanded separately rather than together. It doesn't generally make difference, except for esoteric cases. For example `#[derive(Foo)]` can now produce an import bringing `Bar` into scope, but previously both `Foo` and `Bar` were required to be resolved before expanding any of them. - **[feature-gated]** `#[derive()]` (with empty list in parentheses) actually becomes useful. For historical reasons `#[derive]` *fully configures* its input, eagerly evaluating `cfg` everywhere in its target, for example on fields. Expansion infra doesn't do that for other attributes, but now when macro attributes attributes are allowed to be written after `#[derive]`, it means that derive can *fully configure* items for them. ```rust #[derive()] #[my_attr] struct S { #[cfg(FALSE)] // this field in removed by `#[derive()]` and not observed by `#[my_attr]` field: u8 } ``` - `#[derive]` on some non-item targets is now prohibited. This was accidentally allowed as noop in the past, but was warned about since early 2018 (rust-lang#50092), despite that crater found a few such cases in unmaintained crates. - Derive helper attributes used before their introduction are now reported with a deprecation lint. This change is long overdue (since macro modularization, rust-lang#52226 (comment)), but it was hard to do without fixing expansion order for derives. The deprecation is tracked by rust-lang#79202. ```rust #[trait_helper] // warning: derive helper attribute is used before it is introduced #[derive(Trait)] struct S {} ``` Crater analysis: rust-lang#79078 (comment)

rust-highfive assigned eddyb Jun 23, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 23, 2019

petrochenkov mentioned this pull request Jun 23, 2019

Fix imports of built-in macros (mostly) #61877

Closed