Document compatibility between declarative and procedural macro tokens

[petrochenkov]

This PR assumes rust-lang/rust#92472 is merged.

[dtolnay]

Suggested change

	their equivalent `#[doc = "str"]` attributes when passed to macros.
	their equivalent `#[doc = r"str"]` attributes when passed to macros.

macro_rules! m {
    ($($tt:tt)*) => {
        println!("{}", stringify!($($tt)*));
    };
}

fn main() {
    m!(
        ///str
    );
}

#[doc = r"str"]

[petrochenkov]

I've addressed review comments.
rust-lang/rust#92472 has also landed.

[petrochenkov]

@rustbot ready
(Is the bot working on this repo?)

[rustbot]

Error: The feature shortcut is not enabled in this repository.
To enable it add its section in the triagebot.toml in the root of the repository.

Please let @rust-lang/release know if you're having trouble with this bot.

[petrochenkov]

@rustbot label +S-waiting-on-review

[ehuss]

Thanks!

[ehuss]

Perhaps to be more explicit, as the way the reference dances around the distinction in various places? That is, the TokenTree defined here which references the tokens chapter which makes a distinction between keywords and identifiers.

Suggested change

	- Identifiers (`ident`, `r#ident`)
	- Identifiers and keywords (`ident`, `r#ident`, `fn`)

[petrochenkov]

I'd rather say that the https://doc.rust-lang.org/reference/tokens.html#tokens is not right.

The distinction between Rust keywords and other identifiers starts from the Rust language parser, but each macro has its own language and parser that isn't necessarily aware of the Rust set of keywords, and may reserve other identifiers instead.

E.g. the language introduced by macro_rules! foo { (a) => {}; ($ident:ident) => {} } has only one keyword a that doesn't match any Rust keyword.

[ehuss]

This particular point seemed a little confusing to me. It wasn't immediately clear what it was trying to say. The section below clears it up a little, but I'm wondering if the confusion could be avoided with a few more words. Maybe something like this:

Suggested change

	- All punctuation characters used in operators supported by the language (`+`,
	but not `+=`), and also the single quote `'` character
	- All punctuation characters used in operators supported by the language (`+`,
	but not `+=`), and also the single quote `'` character as part of a lifetime (see below for splitting and joining behavior)

[petrochenkov]

In languages defined by proc macros ' is an independent token, it doesn't necessarily have to be joined with identifiers to form lifetimes.

[petrochenkov]

I've added the changes from #1169 (comment) and #1169 (comment) in a slightly reworded form.

[ehuss]

Thanks!