Crates can't migrate to 2018 edition when they invoke coupled procedural macros

[alexcrichton]

Procedural macros are often coupled with "runtime crates" which support the various macros that the procedural macro exports. These runtime crates, however, sometimes also want to invoke the procedural macro itself (think panic! and libstd, which exports it for users and also defines it itself).

Let's say our runtime crate is called foo. The macro today generally expands to paths that look like ::foo::MyType. This doesn't work by default in the crate foo itself, so foo typically includes a module that looks like:

mod foo { pub use super::*; }

And the enables the crate foo to use its own macros internally.

In the 2018 edition, however, the ::foo::MyType path unconditionally requires foo to be a crate, which isn't the case when we're compiling foo! As a result, these sorts of runtime crates don't have a great path forward when migrating to the 2018 edition.

[alexcrichton]

This was first reported at dtolnay/syn#507

[eddyb]

cc @rust-lang/lang @petrochenkov

[joshtriplett]

Shouldn't such macros use $self::MyType or similar instead? (Does that not work for procedural macros, for some reason?)

[dtolnay]

@joshtriplett the procedural macro and the runtime support live in separate crates, think serde_derive vs serde. You would need some way to specify what crate $self would refer to, because in code generated by serde_derive all the types are defined by serde not serde_derive.

[petrochenkov]

Hack: provide a version of macro with an extra argument for internal use and replace ::foo with crate if it's used.

[petrochenkov]

Hack 2: --extern foo=self that adds foo into extern prelude as an alias to crate.

[petrochenkov]

Or, in general, some way to add a name that's associated with the current crate into extern prelude.

[joshtriplett]

@dtolnay Ah, I misunderstood the phrase "the crate foo to use its own macros internally".

[petrochenkov]

Perhaps even #![crate_name = "foo"] and --crate-name NAME can be re-purposed for that.
In general, if the crate knows its name (which is not always true), we can add that name into extern prelude.

EDIT: Nah, perhaps this need is too special, my_crate_name being generally available as an alias to crate in all crates built by Cargo would probably mostly cause confusion.

[petrochenkov]

extern crate foo = PATH; 😄

[eddyb]

@petrochenkov wouldn't that create a bit of a "side-effect" in the system?
Or would you be required to place it in the root of another crate?

[petrochenkov]

@eddyb
Do you mean extern crate foo?
If by side effect you mean that the extern prelude set becomes dynamic and can change in the process of expansion, then yes.
It's not so bad, it's the same thing as #[macro_use] extern crate ... adding names into the macro prelude, so it's trivially implementable in the existing infrastructure.
It breaks the uniform import path scheme with canaries though, so it's one more reason to move the implementation to the proper search in scope sooner (EDIT: or perhaps not and we'll just need more canaries?).

[petrochenkov]

#54658 implements a possible prerequisite (extern crate items adding things into extern prelude).

[stephaneyfx]

This workaround using CARGO_PKG_NAME seems to work, but that does not feel like an elegant solution:

In crate foo:

pub trait Foo {
    fn foo(&self) -> String;
}

In crate foo-derive, the path used in the generated impl block is either ::foo::Foo or crate::Foo based on the value of the CARGO_PKG_NAME environment variable. Tested by using the derive(Foo) attribute within the foo crate and in a separate crate.

#[proc_macro_derive(Foo)]
pub fn derive_foo(input: TokenStream) -> TokenStream {
    let ast = syn::parse::<DeriveInput>(input).unwrap();
    let ty = &ast.ident;
    let ty_name = ty.to_string();
    let in_self = env::var("CARGO_PKG_NAME").unwrap() == "foo";
    let trait_path = if in_self {
        Path {
            leading_colon: None,
            segments: ["crate", "Foo"].iter()
                .map(|&s| Ident::new(s, Span::call_site()))
                .map(PathSegment::from)
                .collect(),
        }
    } else {
        Path {
            leading_colon: Some(Default::default()),
            segments: ["foo", "Foo"].iter()
                .map(|&s| Ident::new(s, Span::call_site()))
                .map(PathSegment::from)
                .collect(),
        }
    };
    let tokens = quote! {
        impl #trait_path for #ty {
            fn foo(&self) -> String {
                format!("I'm a {}", #ty_name)
            }
        }
    };
    tokens.into()
}

[petrochenkov]

#55275 provides a possible solution.

[dhardy]

@stephaneyfx's solution seems to work nicely, but can be simplified:

    let c = if env::var("CARGO_PKG_NAME").unwrap() == "mycrate" {
        quote!( crate )
    } else {
        quote!( mycrate )
    };

Then just use #c inside quote! instead of $crate.

Edit: I realise this will only work where the crate using the macro is using Edition 2018.

[petrochenkov]

@dhardy
::crate::foo with leading :: doesn't currently work (#53347), so there may be issues with disambiguated paths?

[dhardy]

I think I found a problem with this workaround (even using Edition 2018): rustdoc examples. Since the error message is pretty poor I'm not certain, but I think it's using crate within the doc example.

[glandium]

Something I noticed in a 2018 edition crate using a 2015 crate/crate-derive pair, is that the trait needs to be used in the 2018 edition crate, while it wasn't necessary in 2015 code.

[withoutboats]

Something I noticed in a 2018 edition crate using a 2015 crate/crate-derive pair, is that the trait needs to be used in the 2018 edition crate, while it wasn't necessary in 2015 code.

That's expected. Derives were imported with #[macro_use] extern cratein 2015, whereas in 2018 they are imported the same way as any other name would be.

Is this going to be fixed before release or will these kinds of crates be incompatible across editions?

I don't know what the status of this is, but this issue doesn't refer to a crate being "incompatible across editions." That's not possible: there's no way to define a crate so that its "incompatible" with crates using a different edition. Crates using this pattern (which relies on an ambiguity in 2015 edition namespacing) just have to themselves continue to use the 2015 edition or one of the other workarounds discussed in this thread.