ICE in macro: doc meta with expr on an item, string concat, stringify!(...) #55414
Assignees
Labels
A-macros
Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..)
A-resolve
Area: Name resolution
C-bug
Category: This is a bug.
E-help-wanted
Call for participation: Help is requested to fix this issue.
E-mentor
Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.
glacier
ICE tracked in rust-lang/glacier.
I-ICE
Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️
P-medium
Medium priority
T-compiler
Relevant to the compiler team, which will review and decide on the PR/issue.
Comments
memoryruins
added
the
I-ICE
awaitlink
changed the title
awaitlink
changed the title
jonas-schievink
added
A-macros
|
Still reproduces on current stable and nightly |
|
@petrochenkov this sounds like something you might want to look at. |
|
Minimized further: macro_rules! bug {
($expr:expr) => {
#[rustc_dummy = $expr] // Any key-value attribute, not necessarily `doc`
struct S;
};
}
// Any expressions containing macro call `X` that's more complex than `X` itself.
// Parentheses will work.
bug!((line!())); |
|
The ICE happens due to a mismatch in nonterminal token visiting between
|
petrochenkov
added
the
E-mentor
|
@petrochenkov is the presumed fix here to add 😆 we knew we needed to do that back in 2017: 9f1a8bf |
|
That would fix the ICE, but
|
|
I have a plan on running some crater experiments to find out which kinds of expansions inside interpolated tokens are used in practice, but I need someone else's help to implement it due to lack of time. |
petrochenkov
added
the
E-help-wanted
|
The experiment is to attempt restricting macro expansion inside attributes to Some relevant links are If the experiment goes well, then we'll proceed further in this direction and the end result should be an RFC about supporting macro expansions in key-value attributes like Another link: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Nt.20tokens.20in.20the.20parser So, the problem is that How does that happen? We need to restrict |
This was referenced Nov 27, 2019
|
It's simpler to implement this on top of #66935. |
|
I just ran into this with a doc_comment! {
concat("Create a new ", stringify!($RcBox), "."),
pub fn new(data: T) -> Self
where
T: Sized,
{
unsafe { $RcBox::from_unchecked($Rc::new(data)) }
}
}The typo of This does show the common "abuse" of |
This was referenced Mar 10, 2020
Closed
|
As per Zulip discussion changing priority to |
|
I started working on this in #77271. |
jyn514
added a commit
to jyn514/rust
that referenced
this issue
May 18, 2021
# Stabilization report
## Summary
This stabilizes using macro expansion in key-value attributes, like so:
```rust
#[doc = include_str!("my_doc.md")]
struct S;
#[path = concat!(env!("OUT_DIR"), "/generated.rs")]
mod m;
```
See the changes to the reference for details on what macros are allowed;
see Petrochenkov's excellent blog post [on internals](https://internals.rust-lang.org/t/macro-expansion-points-in-attributes/11455)
for alternatives that were considered and rejected ("why accept no more
and no less?")
This has been available on nightly since 1.50 with no major issues.
## Notes
### Accepted syntax
The parser accepts arbitrary Rust expressions in this position, but any expression other than a macro invocation will ultimately lead to an error because it is not expected by the built-in expression forms (e.g., `#[doc]`). Note that decorators and the like may be able to observe other expression forms.
### Expansion ordering
Expansion of macro expressions in "inert" attributes occurs after decorators have executed, analogously to macro expressions appearing in the function body or other parts of decorator input.
There is currently no way for decorators to accept macros in key-value position if macro expansion must be performed before the decorator executes (if the macro can simply be copied into the output for later expansion, that can work).
## Test cases
- https://github.com/rust-lang/rust/blob/master/src/test/ui/attributes/key-value-expansion-on-mac.rs
- https://github.com/rust-lang/rust/blob/master/src/test/rustdoc/external-doc.rs
The feature has also been dogfooded extensively in the compiler and
standard library:
- rust-lang#83329
- rust-lang#83230
- rust-lang#82641
- rust-lang#80534
## Implementation history
- Initial proposal: rust-lang#55414 (comment)
- Experiment to see how much code it would break: rust-lang#67121
- Preliminary work to restrict expansion that would conflict with this
feature: rust-lang#77271
- Initial implementation: rust-lang#78837
- Fix for an ICE: rust-lang#80563
## Unresolved Questions
~~rust-lang#83366 (comment) listed some concerns, but they have been resolved as of this final report.~~
## Additional Information
There are two workarounds that have a similar effect for `#[doc]`
attributes on nightly. One is to emulate this behavior by using a limited version of this feature that was stabilized for historical reasons:
```rust
macro_rules! forward_inner_docs {
($e:expr => $i:item) => {
#[doc = $e]
$i
};
}
forward_inner_docs!(include_str!("lib.rs") => struct S {});
```
This also works for other attributes (like `#[path = concat!(...)]`).
The other is to use `doc(include)`:
```rust
#![feature(external_doc)]
#[doc(include = "lib.rs")]
struct S {}
```
The first works, but is non-trivial for people to discover, and
difficult to read and maintain. The second is a strange special-case for
a particular use of the macro. This generalizes it to work for any use
case, not just including files.
I plan to remove `doc(include)` when this is stabilized. The
`forward_inner_docs` workaround will still compile without warnings, but
I expect it to be used less once it's no longer necessary.
Dylan-DPC-zz
pushed a commit
to Dylan-DPC-zz/rust
that referenced
this issue
May 18, 2021
…=petrochenkov Stabilize extended_key_value_attributes Closes rust-lang#44732. Closes rust-lang#78835. Closes rust-lang#82768 (by making it irrelevant). # Stabilization report ## Summary This stabilizes using macro expansion in key-value attributes, like so: ```rust #[doc = include_str!("my_doc.md")] struct S; #[path = concat!(env!("OUT_DIR"), "/generated.rs")] mod m; ``` See Petrochenkov's excellent blog post [on internals](https://internals.rust-lang.org/t/macro-expansion-points-in-attributes/11455) for alternatives that were considered and rejected ("why accept no more and no less?") This has been available on nightly since 1.50 with no major issues. ## Notes ### Accepted syntax The parser accepts arbitrary Rust expressions in this position, but any expression other than a macro invocation will ultimately lead to an error because it is not expected by the built-in expression forms (e.g., `#[doc]`). Note that decorators and the like may be able to observe other expression forms. ### Expansion ordering Expansion of macro expressions in "inert" attributes occurs after decorators have executed, analogously to macro expressions appearing in the function body or other parts of decorator input. There is currently no way for decorators to accept macros in key-value position if macro expansion must be performed before the decorator executes (if the macro can simply be copied into the output for later expansion, that can work). ## Test cases - https://github.com/rust-lang/rust/blob/master/src/test/ui/attributes/key-value-expansion-on-mac.rs - https://github.com/rust-lang/rust/blob/master/src/test/rustdoc/external-doc.rs The feature has also been dogfooded extensively in the compiler and standard library: - rust-lang#83329 - rust-lang#83230 - rust-lang#82641 - rust-lang#80534 ## Implementation history - Initial proposal: rust-lang#55414 (comment) - Experiment to see how much code it would break: rust-lang#67121 - Preliminary work to restrict expansion that would conflict with this feature: rust-lang#77271 - Initial implementation: rust-lang#78837 - Fix for an ICE: rust-lang#80563 ## Unresolved Questions ~~rust-lang#83366 (comment) listed some concerns, but they have been resolved as of this final report.~~ ## Additional Information There are two workarounds that have a similar effect for `#[doc]` attributes on nightly. One is to emulate this behavior by using a limited version of this feature that was stabilized for historical reasons: ```rust macro_rules! forward_inner_docs { ($e:expr => $i:item) => { #[doc = $e] $i }; } forward_inner_docs!(include_str!("lib.rs") => struct S {}); ``` This also works for other attributes (like `#[path = concat!(...)]`). The other is to use `doc(include)`: ```rust #![feature(external_doc)] #[doc(include = "lib.rs")] struct S {} ``` The first works, but is non-trivial for people to discover, and difficult to read and maintain. The second is a strange special-case for a particular use of the macro. This generalizes it to work for any use case, not just including files. I plan to remove `doc(include)` when this is stabilized (rust-lang#82539). The `forward_inner_docs` workaround will still compile without warnings, but I expect it to be used less once it's no longer necessary.
jackh726
added a commit
to jackh726/rust
that referenced
this issue
May 19, 2021
…=petrochenkov Stabilize extended_key_value_attributes Closes rust-lang#44732. Closes rust-lang#78835. Closes rust-lang#82768 (by making it irrelevant). # Stabilization report ## Summary This stabilizes using macro expansion in key-value attributes, like so: ```rust #[doc = include_str!("my_doc.md")] struct S; #[path = concat!(env!("OUT_DIR"), "/generated.rs")] mod m; ``` See Petrochenkov's excellent blog post [on internals](https://internals.rust-lang.org/t/macro-expansion-points-in-attributes/11455) for alternatives that were considered and rejected ("why accept no more and no less?") This has been available on nightly since 1.50 with no major issues. ## Notes ### Accepted syntax The parser accepts arbitrary Rust expressions in this position, but any expression other than a macro invocation will ultimately lead to an error because it is not expected by the built-in expression forms (e.g., `#[doc]`). Note that decorators and the like may be able to observe other expression forms. ### Expansion ordering Expansion of macro expressions in "inert" attributes occurs after decorators have executed, analogously to macro expressions appearing in the function body or other parts of decorator input. There is currently no way for decorators to accept macros in key-value position if macro expansion must be performed before the decorator executes (if the macro can simply be copied into the output for later expansion, that can work). ## Test cases - https://github.com/rust-lang/rust/blob/master/src/test/ui/attributes/key-value-expansion-on-mac.rs - https://github.com/rust-lang/rust/blob/master/src/test/rustdoc/external-doc.rs The feature has also been dogfooded extensively in the compiler and standard library: - rust-lang#83329 - rust-lang#83230 - rust-lang#82641 - rust-lang#80534 ## Implementation history - Initial proposal: rust-lang#55414 (comment) - Experiment to see how much code it would break: rust-lang#67121 - Preliminary work to restrict expansion that would conflict with this feature: rust-lang#77271 - Initial implementation: rust-lang#78837 - Fix for an ICE: rust-lang#80563 ## Unresolved Questions ~~rust-lang#83366 (comment) listed some concerns, but they have been resolved as of this final report.~~ ## Additional Information There are two workarounds that have a similar effect for `#[doc]` attributes on nightly. One is to emulate this behavior by using a limited version of this feature that was stabilized for historical reasons: ```rust macro_rules! forward_inner_docs { ($e:expr => $i:item) => { #[doc = $e] $i }; } forward_inner_docs!(include_str!("lib.rs") => struct S {}); ``` This also works for other attributes (like `#[path = concat!(...)]`). The other is to use `doc(include)`: ```rust #![feature(external_doc)] #[doc(include = "lib.rs")] struct S {} ``` The first works, but is non-trivial for people to discover, and difficult to read and maintain. The second is a strange special-case for a particular use of the macro. This generalizes it to work for any use case, not just including files. I plan to remove `doc(include)` when this is stabilized (rust-lang#82539). The `forward_inner_docs` workaround will still compile without warnings, but I expect it to be used less once it's no longer necessary.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
code
description
Any of the following actions stop the panic:
docattribute:// #[doc = $test]stringify!(...):stderr
Show output
rustc
The text was updated successfully, but these errors were encountered: