Nightly rustc panics when passing r#Self enum variant through proc macro

[JoelCourtney]

This happens when trying to use r#Self as an enum variant on nightly, when the enum is passed through quote! in a proc macro.

Code

src/lib.rs:

extern crate my_macros;

#[my_macros::this_panics]
enum Hello {
    r#Self
}

my_macros/src/lib.rs:

extern crate proc_macro;

use proc_macro::TokenStream;

#[proc_macro_attribute]
pub fn this_panics(_: TokenStream, input: TokenStream) -> TokenStream {
    let ast: syn::DeriveInput = syn::parse(input).unwrap();
    (quote::quote! {#ast}).into()
}

Meta

This only happens on nightly. On stable, it correctly outputs the errors and exits gracefully.

rustc --version --verbose:

rustc 1.51.0-nightly (c8915eebe 2021-01-07)
binary: rustc
commit-hash: c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9
commit-date: 2021-01-07
host: x86_64-apple-darwin
release: 1.51.0-nightly

Error output

error: `Self` cannot be a raw identifier
 --> src/lib.rs:5:5
  |
5 |     r#Self
  |     ^^^^^^

thread 'rustc' panicked at '`Self` cannot be a raw identifier', compiler/rustc_expand/src/proc_macro_server.rs:336:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.51.0-nightly (c8915eebe 2021-01-07) running on x86_64-apple-darwin

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type lib

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack
error: custom attribute panicked
 --> src/lib.rs:3:1
  |
3 | #[my_macros::this_panics]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = help: message: `Self` cannot be a raw identifier

Backtrace

thread 'rustc' panicked at '`Self` cannot be a raw identifier', compiler/rustc_expand/src/proc_macro_server.rs:336:13
stack backtrace:
   0: _rust_begin_unwind
   1: std::panicking::begin_panic_fmt
   2: rustc_expand::proc_macro_server::Ident::new
   3: <proc_macro::bridge::TokenTree<rustc_expand::proc_macro_server::Group,rustc_expand::proc_macro_server::Punct,rustc_expand::proc_macro_server::Ident,rustc_expand::proc_macro_server::Literal> as rustc_expand::proc_macro_server::FromInternal<((rustc_a
st::tokenstream::TokenTree,rustc_ast::tokenstream::Spacing),&rustc_session::parse::ParseSess,&mut alloc::vec::Vec<proc_macro::bridge::TokenTree<rustc_expand::proc_macro_server::Group,rustc_expand::proc_macro_server::Punct,rustc_expand::proc_macro_server:
:Ident,rustc_expand::proc_macro_server::Literal>>)>>::from_internal
   4: <rustc_expand::proc_macro_server::Rustc as proc_macro::bridge::server::TokenStreamIter>::next
   5: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
   6: std::panicking::try
   7: <proc_macro::bridge::server::Dispatcher<proc_macro::bridge::server::MarkedTypes<S>> as proc_macro::bridge::server::DispatcherTrait>::dispatch
   8: <proc_macro::bridge::closure::Closure<A,R> as core::convert::From<&mut F>>::from::call
   9: proc_macro::bridge::closure::Closure<A,R>::call
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/proc_macro/src/bridge/closure.rs:27:18
  10: proc_macro::bridge::client::TokenStreamIter::next::{{closure}}
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/proc_macro/src/bridge/client.rs:244:25
  11: proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::with::{{closure}}
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/proc_macro/src/bridge/client.rs:336:47
  12: proc_macro::bridge::client::BridgeState::with::{{closure}}::{{closure}}
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/proc_macro/src/bridge/client.rs:293:17
  13: proc_macro::bridge::scoped_cell::ScopedCell<T>::replace
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/proc_macro/src/bridge/scoped_cell.rs:75:9
  14: proc_macro::bridge::client::BridgeState::with::{{closure}}
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/proc_macro/src/bridge/client.rs:291:13
  15: std::thread::local::LocalKey<T>::try_with
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/std/src/thread/local.rs:272:16
  16: std::thread::local::LocalKey<T>::with
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/std/src/thread/local.rs:248:9
  17: proc_macro::bridge::client::BridgeState::with
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/proc_macro/src/bridge/client.rs:290:9
  18: proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::with
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/proc_macro/src/bridge/client.rs:329:9
  19: proc_macro::bridge::client::TokenStreamIter::next
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/proc_macro/src/bridge/client.rs:237:17
  20: <proc_macro::token_stream::IntoIter as core::iter::traits::iterator::Iterator>::next
             at /rustc/c8915eebeaaef9f7cc1cff6ffd97f578b03c2ac9/library/proc_macro/src/lib.rs:240:13
  21: <proc_macro2::imp::TokenTreeIter as core::iter::traits::iterator::Iterator>::next
             at /Users/joel/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-1.0.24/src/wrapper.rs:304:46
  22: <proc_macro2::token_stream::IntoIter as core::iter::traits::iterator::Iterator>::next
             at /Users/joel/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-1.0.24/src/lib.rs:1213:13
  23: syn::buffer::TokenBuffer::inner_new
             at /Users/joel/.cargo/registry/src/github.com-1ecc6299db9ec823/syn-1.0.59/src/buffer.rs:53:19
  24: syn::buffer::TokenBuffer::inner_new
             at /Users/joel/.cargo/registry/src/github.com-1ecc6299db9ec823/syn-1.0.59/src/buffer.rs:89:25
  25: syn::buffer::TokenBuffer::new2
             at /Users/joel/.cargo/registry/src/github.com-1ecc6299db9ec823/syn-1.0.59/src/buffer.rs:112:9
  26: <F as syn::parse::Parser>::parse2
             at /Users/joel/.cargo/registry/src/github.com-1ecc6299db9ec823/syn-1.0.59/src/parse.rs:1207:19
  27: syn::parse::Parser::parse
             at /Users/joel/.cargo/registry/src/github.com-1ecc6299db9ec823/syn-1.0.59/src/parse.rs:1161:9
  28: syn::parse
             at /Users/joel/.cargo/registry/src/github.com-1ecc6299db9ec823/syn-1.0.59/src/lib.rs:880:5
  29: my_macros::this_panics
             at ./src/my_macros/src/lib.rs:7:33
30: core::ops::function::FnOnce::call_once
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
  31: proc_macro::bridge::client::Client<fn(proc_macro::TokenStream,proc_macro::TokenStream) .> proc_macro::TokenStream>::expand2::run::{{closure}}
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/proc_macro/src/bridge/client.rs:426:17
  32: proc_macro::bridge::client::run_client::{{closure}}::{{closure}}
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/proc_macro/src/bridge/client.rs:377:26
  33: proc_macro::bridge::scoped_cell::ScopedCell<T>::set::{{closure}}
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/proc_macro/src/bridge/scoped_cell.rs:80:33
  34: proc_macro::bridge::scoped_cell::ScopedCell<T>::replace
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/proc_macro/src/bridge/scoped_cell.rs:75:9
  35: proc_macro::bridge::scoped_cell::ScopedCell<T>::set
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/proc_macro/src/bridge/scoped_cell.rs:80:9
  36: proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::enter::{{closure}}
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/proc_macro/src/bridge/client.rs:325:35
  37: std::thread::local::LocalKey<T>::try_with
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/thread/local.rs:272:16
  38: std::thread::local::LocalKey<T>::with
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/thread/local.rs:248:9
  39: proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::enter
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/proc_macro/src/bridge/client.rs:325:9
  40: proc_macro::bridge::client::run_client::{{closure}}
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/proc_macro/src/bridge/client.rs:370:9
  41: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:322:9
  42: std::panicking::try::do_call
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:379:40
  43: ___rust_try
  44: std::panicking::try
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:343:19
  45: std::panic::catch_unwind
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:396:14
  46: proc_macro::bridge::client::run_client
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/proc_macro/src/bridge/client.rs:369:5
  47: proc_macro::bridge::client::Client<fn(proc_macro::TokenStream,proc_macro::TokenStream) .> proc_macro::TokenStream>::expand2::run
             at /Users/joel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/proc_macro/src/bridge/client.rs:425:13
  48: proc_macro::bridge::server::run_server
  49: <rustc_expand::proc_macro::AttrProcMacro as rustc_expand::base::AttrProcMacro>::expand
  50: rustc_expand::expand::MacroExpander::fully_expand_fragment
  51: rustc_expand::expand::MacroExpander::expand_crate
  52: rustc_session::utils::<impl rustc_session::session::Session>::time
  53: rustc_interface::passes::configure_and_expand_inner
  54: rustc_interface::passes::configure_and_expand::{{closure}}
  55: rustc_data_structures::box_region::PinnedGenerator<I,A,R>::new
  56: rustc_interface::passes::configure_and_expand
  57: rustc_interface::queries::Queries::expansion
  58: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter
  59: rustc_span::with_source_map
  60: rustc_interface::interface::create_compiler_and_run
  61: rustc_span::with_session_globals
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.51.0-nightly (c8915eebe 2021-01-07) running on x86_64-apple-darwin

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type lib

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack

[JoelCourtney]

In the macro, passing input directly through without parsing and re-tokenizing it does not produce this error:

pub fn this_is_fine(_: TokenStream, input: TokenStream) -> TokenStream {
    input
}

[osa1]

I believe this is one of the "delayed error" bugs, where we detect an error and report it, but continue compilation, which causes another code to panic as the error detected earlier breaks an assumption in another code later on.

If I'm right then I suspect fixing this is just a matter of turning the panic! into some kind of "panic if an error is not already reported" call.

[osa1]

I submitted a PR that does what I describe above. I don't know if it's the right thing to do though. Any feedback would be welcome!

[petrochenkov]

Closing as a duplicate of #59998.

[osa1]

Closing as a duplicate of #59998.

That issue says "on non-Linux", but I can reproduce the current issue on Linux, so I'm not sure this a duplicate of #59998.

If it is then perhaps the description of #59998 should be updated.

[petrochenkov]

Yeah, if panics produced by proc macro server turn into ICEs, then it's #59998, regardless of targets.
Could you leave a comment there documenting your case?

[Aaron1011]

This is not really the same as #59998. That issue occurs when a proc-macro explicitly tries to create an invalid Ident. However, this issue occurs when a proc-macro just tries to recollect the input tokens. The TokenStream that we pass to the proc-macro is not really valid, but the actual panic gets delayed until we actually try to convert the invalid rustc Ident into a proc-macro Ident.

[Aaron1011]

This has some conceptual similarities to #76360 (comment). In both cases, we need to decide how to pass a TokenStream to a proc-macro after some form of recovery has taken place (parsing a syntactically invalid attribute target, or creating an invalid raw identifier).