Allow macro_rules! to also define item and stmt macros

[paulstansifer]

Now it's possible to invoke macros in statement and item position. The extra argument that syntax extensions in item position take is now optional (things defined with macro_rules! cannot see it).

At expansion time, the expansion of a macro defined with macro_rules! is parsed as an expr, an item, or a stmt, depending on where it is.

The delimiters around the RHS of a macro are no longer transcribed (this interfered with the parsing of items). This means that macro_rules! mytuple( () => (1,2,3) ) doesn't work; an extra set of delimiters must be added around the (1,2,3).

Closes #3086.

Outstanding issues:

• Expanding to multiple items is still not done.
• { my_expr_mac!() + 16 } is a parse error because the macro is interpreted as a statement. Workaround: wrap in parentheses.

[paulstansifer]

One problem with this design is that item/expr/stmt macros are not distinguished except by where they are used, so there cannot be a section in the macro tutorial titled "Invocation location notation" ( :

[graydon]

r+, landing along with #4045. Though I'd appreciate a bit of explanation about why the delimiters can't be part of the RHS expansion anymore. I can sorta guess, but if you can say it in clear words I'd appreciate it.

(also: thanks! this is excellent)

[paulstansifer]

With expressions, you can always take a valid expression and wrap it in () or {} and get a valid expression that means the same thing, so the old way of doing things was fine for them. However, items and statements can't be wrapped in delimiters at all. For token-tree reasons, we need to write the RHS with delimiters, but, to make it possible to express items and statements, the delimiters are now ignored.

[graydon]

That's what I figured, ok. Thanks. This landed as a88f9ac.