Stabilize `unrestricted_attribute_tokens` #57367

petrochenkov · 2019-01-06T01:52:50Z

In accordance with a plan described in https://internals.rust-lang.org/t/unrestricted-attribute-tokens-feature-status/8561/3.

Delimited non-macro non-builtin attributes now support the same syntax as macro attributes:

PATH
PATH `(` TOKEN_STREAM `)`
PATH `[` TOKEN_STREAM `]`
PATH `{` TOKEN_STREAM `}`

Such attributes mostly serve as inert proc macro helpers or tool attributes.
To some extent these attributes are de-facto stable due to a hole in feature gate checking (feature gating is done too late - after macro expansion.)
So if macro removes such helper attributes during expansion (and it must remove them, unless it's a derive macro), then the code will work on stable.

Key-value non-macro non-builtin attributes are now restricted to bare minimum required to support what we support on stable - unsuffixed literals (#34981).

PATH `=` LITERAL

(Key-value macro attributes are not supported at all right now.)
Crater run in #57321 found no regressions for this change.
There are multiple possible ways to extend key-value attributes (#57321 (comment)), but I'd expect an RFC for that and it's not a pressing enough issue to block stabilization of delimited attributes.

Built-in attributes are still restricted to the "classic" meta-item syntax, nothing changes here.
#57321 goes further and adds some additional restrictions (more consistent input checking) to built-in attributes.

Closes #55208

petrochenkov · 2019-01-06T01:53:32Z

r? @Centril

src/libsyntax/feature_gate.rs

src/test/ui/attr-eq-token-tree.rs

CAD97 · 2019-01-07T02:18:11Z

Minor note: it might make sense to frame this not as "stabilizing unrestricted_attribute_tokens" but as stabilizing a non-problematic subset. There's still intent (as far as I've seen) to expand the ability of key-value style attributes, but that's much more complicated than the delimited style.

So my "suggestion" is to make the stabilized subset stable under unrestricted_delimited_attributes or similar, and leave the current capability of unrestricted_attribute_tokens to be discussed in the RFC for expanding the "-value" grammar of key-value attributes.

Also, cc rust-lang/wg-grammar#13, @eddyb. This grammar is the current grammar in that (permissive) grammar draft, but this probably should be linked.

bors · 2019-01-07T03:55:12Z

☔ The latest upstream changes (presumably #57379) made this pull request unmergeable. Please resolve the merge conflicts.

Implement basic input validation for built-in attributes Correct top-level shape (`#[attr]` vs `#[attr(...)]` vs `#[attr = ...]`) is enforced for built-in attributes, built-in attributes must also fit into the "meta-item" syntax (aka the "classic attribute syntax"). For some subset of attributes (found by crater run), errors are lowered to deprecation warnings. NOTE: This PR previously included #57367 as well.

Centril · 2019-01-27T01:57:48Z

Stabilization proposal

I propose that we stabilize #![feature(unrestricted_attribute_tokens)].

@rfcbot merge

Version target

The next version is 1.34 which goes into beta around 2019-03-01.

What is stabilized

The formal grammar (in the lyg notation) of attributes becomes:

OuterAttr = "#" attr:Attr;
InnerAttr = "#" "!" attr:Attr;
Attr = "[" path:Path input:AttrInput? "]";
AttrInput =
  | "(" TOKEN_STREAM ")"
  | "[" TOKEN_STREAM "]"
  | "{" TOKEN_STREAM "}"
  | "=" LITERAL // Unsuffixed literals only.
  ;

As a result, users may for example write #[proptest(0..1)] or #[serde(T: Display)].

Motivation

The general motivation for allowing this is to facilitate more natural and expressive EDSLs in attributes as opposed to wrapping things in strings. Moreover, this brings the grammar of attributes closer to that of regular macros.

Tests

Feature gate
Tests
- #[my_attr = !] is rejected.
- Rejecting = $x after interpolation where $x isn't a LITERAL.
- Rejecting a malformed attribute of form #[ident =.
- And of form #[ident*].
- A test that #[my_attr(a b c d) works.
  - (@petrochenkov: can you extend with tests for ..[a b c d] + ..{a b c d} please just to ensure no future regressions; (it works on nightly))

What is not

Users are not yet permitted to write:

#[foo = 1 + 2] since that does not match the grammar above.
#[foo = 42usize] since only unsuffixed literals are permitted per grammar above.

History

Originally proposed: In an internals thread by @petrochenkov and @CAD97.
No RFC was accepted; but there is not much language design and open questions to be done here and this adds immediate value. If an RFC were to be written it would essentially elaborate on what I've written here.
Built-in attributes hardened and validated in Implement basic input validation for built-in attributes #57321 which was reviewed by @Centril and @nikomatsakis.
Tracking issue: Tracking issue for arbitrary token streams in non-macro attributes (unrestricted_attribute_tokens) #55208

rfcbot · 2019-01-27T01:57:49Z

Team member @Centril has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

Centril · 2019-02-11T16:30:00Z

Review pings @cramertj, @joshtriplett, @nikomatsakis, @pnkfelix, @scottmcm, and @withoutboats -- have a look? :)

rfcbot · 2019-02-22T10:55:01Z

🔔 This is now entering its final comment period, as per the review above. 🔔

Centril · 2019-02-25T06:46:18Z

@petrochenkov If you want to rebase quickly, we could try to get this into 1.34 even tho FCP has not completed... at the risk of beta-backporting out if it should not complete, but that risks seems quite small.

As for the implementation, r=me with the additional tests I asked for in #57367 (comment).

@bors p=10

petrochenkov · 2019-02-25T20:47:36Z

@bors r=Centril

bors · 2019-02-25T20:47:37Z

📌 Commit eccc199 has been approved by Centril

bors · 2019-02-25T23:23:15Z

⌛ Testing commit eccc199 with merge 55c173c...

Stabilize `unrestricted_attribute_tokens` In accordance with a plan described in https://internals.rust-lang.org/t/unrestricted-attribute-tokens-feature-status/8561/3. Delimited non-macro non-builtin attributes now support the same syntax as macro attributes: ``` PATH PATH `(` TOKEN_STREAM `)` PATH `[` TOKEN_STREAM `]` PATH `{` TOKEN_STREAM `}` ``` Such attributes mostly serve as inert proc macro helpers or tool attributes. To some extent these attributes are de-facto stable due to a hole in feature gate checking (feature gating is done too late - after macro expansion.) So if macro *removes* such helper attributes during expansion (and it must remove them, unless it's a derive macro), then the code will work on stable. Key-value non-macro non-builtin attributes are now restricted to bare minimum required to support what we support on stable - unsuffixed literals (#34981). ``` PATH `=` LITERAL ``` (Key-value macro attributes are not supported at all right now.) Crater run in #57321 found no regressions for this change. There are multiple possible ways to extend key-value attributes (#57321 (comment)), but I'd expect an RFC for that and it's not a pressing enough issue to block stabilization of delimited attributes. Built-in attributes are still restricted to the "classic" meta-item syntax, nothing changes here. #57321 goes further and adds some additional restrictions (more consistent input checking) to built-in attributes. Closes #55208

bors · 2019-02-26T02:21:45Z

☀️ Test successful - checks-travis, status-appveyor
Approved by: Centril
Pushing 55c173c to master...

bors · 2019-02-26T02:21:55Z

☀️ Test successful - checks-travis, status-appveyor
Approved by: Centril
Pushing 55c173c to master...

withoutboats · 2019-02-28T20:18:34Z

why are only literals allowed with the = syntax?

petrochenkov · 2019-02-28T20:32:12Z

@withoutboats
Because that's the status quo / baseline (set by #34981).
There are multiple possible alternatives (including declaring key-value attributes legacy and not extending them) and I'd prefer to go through the RFC process to choose one of them, rather than do it in a stabilization PR or something like that.

(For key-value attributes we can't "just accept what attribute macros accept", because attribute macros do not accept key-value forms at all, that is blocked on implementing querying currently used delimiter from a proc macro.)

This change is necessary due to the recent change in the rustc parser that the following attribute has become a hard parser error: ```rust struct Foo; ``` cc rust-lang/rust#57367.

Pkgsrc changes: * Bump required rust version to build to 1.33.0. * Adapt patches to changed file locations. * (I worry about 32-bit ports, now that Atomic64 apparently is First-Class; this has been built on NetBSD/amd64 so far) Upstream changes: Version 1.34.0 (2019-04-11) ========================== Language -------- - [You can now use `#[deprecated = "reason"]`][58166] as a shorthand for `#[deprecated(note = "reason")]`. This was previously allowed by mistake but had no effect. - [You can now accept token streams in `#[attr()]`,`#[attr[]]`, and `#[attr{}]` procedural macros.][57367] - [You can now write `extern crate self as foo;`][57407] to import your crate's root into the extern prelude. Compiler -------- - [You can now target `riscv64imac-unknown-none-elf` and `riscv64gc-unknown-none-elf`.][58406] - [You can now enable linker plugin LTO optimisations with `-C linker-plugin-lto`.][58057] This allows rustc to compile your Rust code into LLVM bitcode allowing LLVM to perform LTO optimisations across C/C++ FFI boundaries. - [You can now target `powerpc64-unknown-freebsd`.][57809] Libraries --------- - [The trait bounds have been removed on some of `HashMap<K, V, S>`'s and `HashSet<T, S>`'s basic methods.][58370] Most notably you no longer require the `Hash` trait to create an iterator. - [The `Ord` trait bounds have been removed on some of `BinaryHeap<T>`'s basic methods.][58421] Most notably you no longer require the `Ord` trait to create an iterator. - [The methods `overflowing_neg` and `wrapping_neg` are now `const` functions for all numeric types.][58044] - [Indexing a `str` is now generic over all types that implement `SliceIndex<str>`.][57604] - [`str::trim`, `str::trim_matches`, `str::trim_{start, end}`, and `str::trim_{start, end}_matches` are now `#[must_use]`][57106] and will produce a warning if their returning type is unused. - [The methods `checked_pow`, `saturating_pow`, `wrapping_pow`, and `overflowing_pow` are now available for all numeric types.][57873] These are equivalvent to methods such as `wrapping_add` for the `pow` operation. Stabilized APIs --------------- #### std & core * [`Any::type_id`] * [`Error::type_id`] * [`atomic::AtomicI16`] * [`atomic::AtomicI32`] * [`atomic::AtomicI64`] * [`atomic::AtomicI8`] * [`atomic::AtomicU16`] * [`atomic::AtomicU32`] * [`atomic::AtomicU64`] * [`atomic::AtomicU8`] * [`convert::Infallible`] * [`convert::TryFrom`] * [`convert::TryInto`] * [`iter::from_fn`] * [`iter::successors`] * [`num::NonZeroI128`] * [`num::NonZeroI16`] * [`num::NonZeroI32`] * [`num::NonZeroI64`] * [`num::NonZeroI8`] * [`num::NonZeroIsize`] * [`slice::sort_by_cached_key`] * [`str::escape_debug`] * [`str::escape_default`] * [`str::escape_unicode`] * [`str::split_ascii_whitespace`] #### std * [`Instant::checked_add`] * [`Instant::checked_sub`] * [`SystemTime::checked_add`] * [`SystemTime::checked_sub`] Cargo ----- - [You can now use alternative registries to crates.io.][cargo/6654] Misc ---- - [You can now use the `?` operator in your documentation tests without manually adding `fn main() -> Result<(), _> {}`.][56470] Compatibility Notes ------------------- - [`Command::before_exec` is now deprecated in favor of the unsafe method `Command::pre_exec`.][58059] - [Use of `ATOMIC_{BOOL, ISIZE, USIZE}_INIT` is now deprecated.][57425] As you can now use `const` functions in `static` variables. [58370]: rust-lang/rust#58370 [58406]: rust-lang/rust#58406 [58421]: rust-lang/rust#58421 [58166]: rust-lang/rust#58166 [58044]: rust-lang/rust#58044 [58057]: rust-lang/rust#58057 [58059]: rust-lang/rust#58059 [57809]: rust-lang/rust#57809 [57873]: rust-lang/rust#57873 [57604]: rust-lang/rust#57604 [57367]: rust-lang/rust#57367 [57407]: rust-lang/rust#57407 [57425]: rust-lang/rust#57425 [57106]: rust-lang/rust#57106 [56470]: rust-lang/rust#56470 [cargo/6654]: rust-lang/cargo#6654 [`Any::type_id`]: https://doc.rust-lang.org/std/any/trait.Any.html#tymethod.type_id [`Error::type_id`]: https://doc.rust-lang.org/std/error/trait.Error.html#tymethod.type_id [`atomic::AtomicI16`]: https://doc.rust-lang.org/std/atomic/struct.AtomicI16.html [`atomic::AtomicI32`]: https://doc.rust-lang.org/std/atomic/struct.AtomicI32.html [`atomic::AtomicI64`]: https://doc.rust-lang.org/std/atomic/struct.AtomicI64.html [`atomic::AtomicI8`]: https://doc.rust-lang.org/std/atomic/struct.AtomicI8.html [`atomic::AtomicU16`]: https://doc.rust-lang.org/std/atomic/struct.AtomicU16.html [`atomic::AtomicU32`]: https://doc.rust-lang.org/std/atomic/struct.AtomicU32.html [`atomic::AtomicU64`]: https://doc.rust-lang.org/std/atomic/struct.AtomicU64.html [`atomic::AtomicU8`]: https://doc.rust-lang.org/std/atomic/struct.AtomicU8.html [`convert::Infallible`]: https://doc.rust-lang.org/std/convert/enum.Infallible.html [`convert::TryFrom`]: https://doc.rust-lang.org/std/convert/trait.TryFrom.html [`convert::TryInto`]: https://doc.rust-lang.org/std/convert/trait.TryInto.html [`iter::from_fn`]: https://doc.rust-lang.org/std/iter/fn.from_fn.html [`iter::successors`]: https://doc.rust-lang.org/std/iter/fn.successors.html [`num::NonZeroI128`]: https://doc.rust-lang.org/std/num/struct.NonZeroI128.html [`num::NonZeroI16`]: https://doc.rust-lang.org/std/num/struct.NonZeroI16.html [`num::NonZeroI32`]: https://doc.rust-lang.org/std/num/struct.NonZeroI32.html [`num::NonZeroI64`]: https://doc.rust-lang.org/std/num/struct.NonZeroI64.html [`num::NonZeroI8`]: https://doc.rust-lang.org/std/num/struct.NonZeroI8.html [`num::NonZeroIsize`]: https://doc.rust-lang.org/std/num/struct.NonZeroIsize.html [`slice::sort_by_cached_key`]: https://doc.rust-lang.org/std/slice/fn.sort_by_cached_key [`str::escape_debug`]: https://doc.rust-lang.org/std/primitive.str.html#method.escape_debug [`str::escape_default`]: https://doc.rust-lang.org/std/primitive.str.html#method.escape_default [`str::escape_unicode`]: https://doc.rust-lang.org/std/primitive.str.html#method.escape_unicode [`str::split_ascii_whitespace`]: https://doc.rust-lang.org/std/primitive.str.html#method.split_ascii_whitespace [`Instant::checked_add`]: https://doc.rust-lang.org/std/time/struct.Instant.html#method.checked_add [`Instant::checked_sub`]: https://doc.rust-lang.org/std/time/struct.Instant.html#method.checked_sub [`SystemTime::checked_add`]: https://doc.rust-lang.org/std/time/struct.SystemTime.html#method.checked_add [`SystemTime::checked_sub`]: https://doc.rust-lang.org/std/time/struct.SystemTime.html#method.checked_sub

syntax: Support modern attribute syntax in the `meta` matcher Where "modern" means #57367: ``` PATH PATH `(` TOKEN_STREAM `)` PATH `[` TOKEN_STREAM `]` PATH `{` TOKEN_STREAM `}` ``` Unfortunately, `meta` wasn't future-proofed using the `FOLLOW` token set like other matchers (#34011), so code like `$meta:meta {` or `$meta:meta [` may break, and we need a crater run to find out how often this happens in practice. Closes #49629 (by fully supporting `meta` rather than removing it.)

syntax: Support modern attribute syntax in the `meta` matcher Where "modern" means rust-lang#57367: ``` PATH PATH `(` TOKEN_STREAM `)` PATH `[` TOKEN_STREAM `]` PATH `{` TOKEN_STREAM `}` ``` Unfortunately, `meta` wasn't future-proofed using the `FOLLOW` token set like other matchers (rust-lang#34011), so code like `$meta:meta {` or `$meta:meta [` may break, and we need a crater run to find out how often this happens in practice. Closes rust-lang#49629 (by fully supporting `meta` rather than removing it.)

rust-highfive assigned pnkfelix Jan 6, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 6, 2019

rust-highfive assigned Centril and unassigned pnkfelix Jan 6, 2019

rust-lang deleted a comment from rust-highfive Jan 6, 2019

petrochenkov mentioned this pull request Jan 6, 2019

Implement basic input validation for built-in attributes #57321

Merged

Centril reviewed Jan 6, 2019

View reviewed changes

src/libsyntax/feature_gate.rs Outdated Show resolved Hide resolved

src/test/ui/attr-eq-token-tree.rs Show resolved Hide resolved

Centril added the relnotes Marks issues that should be documented in the release notes of the next release. label Jan 6, 2019

rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Jan 27, 2019

shepmaster mentioned this pull request Jan 27, 2019

Tracking issue for arbitrary token streams in non-macro attributes (unrestricted_attribute_tokens) #55208

Closed

idanarye mentioned this pull request Feb 18, 2019

0.3.0 doesn't support macro invocation in default values idanarye/rust-smart-default#4

Closed

rfcbot added the final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. label Feb 22, 2019

rfcbot removed the proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. label Feb 22, 2019

petrochenkov added 2 commits February 25, 2019 22:40

Restrict value in key-value attributes to literals

8e1b5d8

Stabilize unrestricted_attribute_tokens

eccc199

petrochenkov force-pushed the unrestab branch from fcccd6b to eccc199 Compare February 25, 2019 20:39

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-team Status: Awaiting decision from the relevant subteam (see the T-<team> label). labels Feb 25, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Feb 26, 2019

bors merged commit eccc199 into rust-lang:master Feb 26, 2019

ehuss mentioned this pull request Feb 26, 2019

Document unrestricted_attribute_tokens rust-lang/reference#525

Closed

petrochenkov mentioned this pull request Feb 28, 2019

Meta issue: FFI attributes syntax rust-lang/rfcs#2637

Open

rfcbot added finished-final-comment-period The final comment period is finished for this PR / Issue. and removed final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. labels Mar 4, 2019

topecongiro mentioned this pull request Mar 14, 2019

Update rustc-ap-* crates to 407.0.0 rust-lang/rustfmt#3447

Merged

matklad mentioned this pull request Apr 25, 2019

Bugs fixes And Improvements of MBE rust-lang/rust-analyzer#1209

Merged

Centril added this to the 1.34 milestone Apr 26, 2019

petrochenkov deleted the unrestab branch June 5, 2019 16:33

Centril mentioned this pull request Aug 3, 2019

Tracking issue for syn 1.0 dtolnay/syn#687

Closed

petrochenkov mentioned this pull request Aug 17, 2019

syntax: Support modern attribute syntax in the meta matcher #63674

Merged

Centril mentioned this pull request Dec 7, 2019

[experiment] Expand macros in inert key-value attributes #67121

Closed

ninevra mentioned this pull request Nov 15, 2020

Generalize attribute syntax tree-sitter/tree-sitter-rust#93

Merged

mina86 mentioned this pull request Jan 25, 2022

Add ignore_if RFC rust-lang/rfcs#3221

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stabilize `unrestricted_attribute_tokens` #57367

Stabilize `unrestricted_attribute_tokens` #57367

petrochenkov commented Jan 6, 2019 •

edited

Loading

petrochenkov commented Jan 6, 2019

CAD97 commented Jan 7, 2019

bors commented Jan 7, 2019

Centril commented Jan 27, 2019

rfcbot commented Jan 27, 2019 •

edited by pnkfelix

Loading

Centril commented Feb 11, 2019

rfcbot commented Feb 22, 2019

Centril commented Feb 25, 2019 •

edited

Loading

petrochenkov commented Feb 25, 2019

bors commented Feb 25, 2019

bors commented Feb 25, 2019

bors commented Feb 26, 2019

bors commented Feb 26, 2019

withoutboats commented Feb 28, 2019

petrochenkov commented Feb 28, 2019

Stabilize unrestricted_attribute_tokens #57367

Stabilize unrestricted_attribute_tokens #57367

Conversation

petrochenkov commented Jan 6, 2019 • edited Loading

petrochenkov commented Jan 6, 2019

CAD97 commented Jan 7, 2019

bors commented Jan 7, 2019

Centril commented Jan 27, 2019

Stabilization proposal

Version target

What is stabilized

Motivation

Tests

What is not

History

rfcbot commented Jan 27, 2019 • edited by pnkfelix Loading

Centril commented Feb 11, 2019

rfcbot commented Feb 22, 2019

Centril commented Feb 25, 2019 • edited Loading

petrochenkov commented Feb 25, 2019

bors commented Feb 25, 2019

bors commented Feb 25, 2019

bors commented Feb 26, 2019

bors commented Feb 26, 2019

withoutboats commented Feb 28, 2019

petrochenkov commented Feb 28, 2019

Stabilize `unrestricted_attribute_tokens` #57367

Stabilize `unrestricted_attribute_tokens` #57367

petrochenkov commented Jan 6, 2019 •

edited

Loading

rfcbot commented Jan 27, 2019 •

edited by pnkfelix

Loading

Centril commented Feb 25, 2019 •

edited

Loading