Repository navigation
[Clang][AST] Introduce ExplicitInstantiationDecl to preserve source info and fix diagnostic locations - #191658
Conversation
…licit template instantiations
|
@llvm/pr-subscribers-clang-tidy @llvm/pr-subscribers-clangir Author: ykiko (16bit-ykiko) ChangesThis is the initial fix of #191442. Following the discussion here #115418 (comment). DescriptionThis PR introduces a new AST node, Background & The ProblemHistorically, Clang's AST lacked a dedicated node to represent the lexical occurrence of an explicit instantiation statement. Instead, This resulted in fragmented behavior across the seven instantiable entity types:
Design Trade-offs EvaluatedBefore settling on the current design, I evaluated a mixed redeclaration-chain approach (similar to how explicit specializations are handled, creating new
The Chosen Architecture:
|
|
@llvm/pr-subscribers-clang-codegen Author: ykiko (16bit-ykiko) ChangesThis is the initial fix of #191442. Following the discussion here #115418 (comment). DescriptionThis PR introduces a new AST node, Background & The ProblemHistorically, Clang's AST lacked a dedicated node to represent the lexical occurrence of an explicit instantiation statement. Instead, This resulted in fragmented behavior across the seven instantiable entity types:
Design Trade-offs EvaluatedBefore settling on the current design, I evaluated a mixed redeclaration-chain approach (similar to how explicit specializations are handled, creating new
The Chosen Architecture:
|
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
🪟 Windows x64 Test Results
✅ The build succeeded and all tests passed. |
🐧 Linux x64 Test Results
✅ The build succeeded and all tests passed. |
mizvekov
left a comment
There was a problem hiding this comment.
Well done, thanks!
Just a few nits here, but great work, I appreciate.
mizvekov
left a comment
There was a problem hiding this comment.
It would also be worth adding support for ast-print with a few tests, that looks fairly simple for this declaration.
After optimization, this class is now quite small. In my experience, explicit instantiations are rarely used in codebases anyway, meaning very few of these nodes will actually be created. Therefore, I believe the impact on compile time will likely be negligible. |
Still worth running the check benchmark though. I hope few of these nodes will be created, but we don't know what we don't know! There could likely be a pathological case here we're not thinking about. |
Hi, here are the test results. It seems nikic has been busy recently, so I asked my friend to help run the tests. It looks like the impact on compile time is negligible—only a few hundredths of a percent (around 0.0x%). |
Thanks for that, I appreciate you doing that! I think I'm happy here, but would like to see @zyn0217 and @mizvekov do the approval, since they did a more thorough review than I did. |
b48a910 to
28ad5ef
Compare
mizvekov
left a comment
There was a problem hiding this comment.
A few nits, but LGTM, Thanks!
|
All tests have passed, but I don't have merge permissions. @mizvekov, could you help merge this? Thanks! |
…e info and fix diagnostic locations (llvm#191658) This is the initial fix of llvm#191442. Following the discussion here llvm#115418 (comment). - Fix llvm#21040 - Fix llvm#52659 - Fix llvm#115418 - Fix llvm#14230 - Fix llvm#21133 ### Description This PR introduces a new AST node, `ExplicitInstantiationDecl`, to systematically fix the long-standing issue of missing or incorrect source location information for explicit template instantiations. #### Background & The Problem Historically, Clang's AST lacked a dedicated node to represent the lexical occurrence of an explicit instantiation statement. Instead, `Sema` tried to shoehorn this information into existing specialization nodes (e.g., `FunctionDecl`, `VarTemplateSpecializationDecl`) or simply returned `nullptr`. This resulted in fragmented behavior across the seven instantiable entity types: * Function & Member Function Templates: Returned `nullptr`, completely losing `SourceRange` and `NestedNameSpecifier` information. * Member Functions & Static Data Members: Mutated existing nodes in-place. Consequently, multiple `template` or `extern template` declarations in the same file would overwrite each other's source locations. * Variable Templates: Suffered from `dyn_cast` bugs and dropped NNS information. #### Design Trade-offs Evaluated Before settling on the current design, I evaluated a mixed redeclaration-chain approach (similar to how explicit *specializations* are handled, creating new `FunctionDecl` nodes and stitching them into the redecl chain). However, this approach had significant flaws: 1. Inconsistency: It couldn't be cleanly applied to member functions or static data members due to `DeclContext` constraints (e.g., a member function shouldn't lexically reside in a namespace `DeclContext`, but placing it in the class context would pollute member lookup). 2. Fragility: It required bypassing standard `FoldingSet` mechanisms (`setFunctionTemplateSpecialization`). 3. Lookup Pollution: Injecting new `NamedDecl` nodes purely for instantiations risked breaking downstream `ASTMatcher`s and altering name lookup behavior. To avoid these pitfalls, this PR introduces `ExplicitInstantiationDecl` as a **purely lexical annotation node**. **Key Design Characteristics:** 1. Inherits from `Decl`, not `NamedDecl`: This is the most crucial design choice. Much like `StaticAssertDecl` or `FriendDecl`, this node lives in a `DeclContext` (making it traversable by `RecursiveASTVisitor` and visible in AST dumps) but remains completely invisible to C++ name lookup. It does not interfere with overload resolution or lookup tables. 2. Unified Representation: A single node type now covers all seven entity types. It holds a pointer (`Specialization`) to the underlying instantiated declaration, unifying how functions, variables, classes, and members are handled. 3. Lexical Fidelity: The node resides in the enclosing namespace or Translation Unit where the explicit instantiation was actually written, perfectly preserving the `SourceRange`, `NestedNameSpecifierLoc`, and the exact locations of the `template` and `extern` keywords. Assisted-by: Claude Code (Anthropic) — used for test writing and checking test results
…e info and fix diagnostic locations (llvm#191658) This is the initial fix of llvm#191442. Following the discussion here llvm#115418 (comment). - Fix llvm#21040 - Fix llvm#52659 - Fix llvm#115418 - Fix llvm#14230 - Fix llvm#21133 ### Description This PR introduces a new AST node, `ExplicitInstantiationDecl`, to systematically fix the long-standing issue of missing or incorrect source location information for explicit template instantiations. #### Background & The Problem Historically, Clang's AST lacked a dedicated node to represent the lexical occurrence of an explicit instantiation statement. Instead, `Sema` tried to shoehorn this information into existing specialization nodes (e.g., `FunctionDecl`, `VarTemplateSpecializationDecl`) or simply returned `nullptr`. This resulted in fragmented behavior across the seven instantiable entity types: * Function & Member Function Templates: Returned `nullptr`, completely losing `SourceRange` and `NestedNameSpecifier` information. * Member Functions & Static Data Members: Mutated existing nodes in-place. Consequently, multiple `template` or `extern template` declarations in the same file would overwrite each other's source locations. * Variable Templates: Suffered from `dyn_cast` bugs and dropped NNS information. #### Design Trade-offs Evaluated Before settling on the current design, I evaluated a mixed redeclaration-chain approach (similar to how explicit *specializations* are handled, creating new `FunctionDecl` nodes and stitching them into the redecl chain). However, this approach had significant flaws: 1. Inconsistency: It couldn't be cleanly applied to member functions or static data members due to `DeclContext` constraints (e.g., a member function shouldn't lexically reside in a namespace `DeclContext`, but placing it in the class context would pollute member lookup). 2. Fragility: It required bypassing standard `FoldingSet` mechanisms (`setFunctionTemplateSpecialization`). 3. Lookup Pollution: Injecting new `NamedDecl` nodes purely for instantiations risked breaking downstream `ASTMatcher`s and altering name lookup behavior. To avoid these pitfalls, this PR introduces `ExplicitInstantiationDecl` as a **purely lexical annotation node**. **Key Design Characteristics:** 1. Inherits from `Decl`, not `NamedDecl`: This is the most crucial design choice. Much like `StaticAssertDecl` or `FriendDecl`, this node lives in a `DeclContext` (making it traversable by `RecursiveASTVisitor` and visible in AST dumps) but remains completely invisible to C++ name lookup. It does not interfere with overload resolution or lookup tables. 2. Unified Representation: A single node type now covers all seven entity types. It holds a pointer (`Specialization`) to the underlying instantiated declaration, unifying how functions, variables, classes, and members are handled. 3. Lexical Fidelity: The node resides in the enclosing namespace or Translation Unit where the explicit instantiation was actually written, perfectly preserving the `SourceRange`, `NestedNameSpecifierLoc`, and the exact locations of the `template` and `extern` keywords. Assisted-by: Claude Code (Anthropic) — used for test writing and checking test results
…ly_mode.cpp` This test seems to have primarily broken due to ``` commit fb02433 Author: ykiko <ykikoykikoykiko@gmail.com> Date: Thu Apr 23 03:21:27 2026 +0800 [Clang][AST] Introduce `ExplicitInstantiationDecl` to preserve source info and fix diagnostic locations (llvm#191658) ``` Explicit template instantiations now appear in the AST so the test case also needed to be updated to account for that. The word `referenced` also started appearing on other AST nodes too so those lines needed updating too. rdar://175193249
The FunctionTemplateDecl node isn't the last node anymore so the FileCheck line stopped matching. The last top level node is now `ExplicitInstantiationDecl`. This was likely caused by: ``` commit fb02433 Author: ykiko <ykikoykikoykiko@gmail.com> Date: Thu Apr 23 03:21:27 2026 +0800 [Clang][AST] Introduce `ExplicitInstantiationDecl` to preserve source info and fix diagnostic locations (llvm#191658) ``` rdar://175912866
|
We have a regression linked to this change: #197797 (comment) |
…nt (#571) ## What The per-feature migrations onto the Semantics node table left each feature re-deriving "is this decl a template instantiation" on its own, and one feature (inlay hints) not deriving it at all. This PR makes the builder the single write point for that fact and cleans up the duplication it left behind. ### Instantiation flag (single write point) - The Semantics builder now sets `in_instantiation` on every node recorded inside a template-instantiation subtree. The decl created by an explicit instantiation directive is itself written and stays unflagged; an implicit instantiation head is not written, so the flag covers it too. The head predicate lives in the shared `decls::is_instantiation`. - **Inlay hints** previously walked instantiated subtrees unguarded; the end-of-pipeline dedup masked it for a single instantiation, but two explicit instantiations stacked contradictory type hints (`: char` and `: int`) on the same dependent `auto`. Fixed and pinned by a new fixture. A side effect consciously accepted: a dependent `auto` local no longer picks up its type from a lone instantiated body (that only ever worked by accident); deducing it properly is tracked by the fixture's `partial` status (clangd#2275). - **Document symbols** and **folding ranges** drop their local TSK re-derivations for the shared predicate (byte-identical snapshots). - The **occurrence layer** (semantic tokens, TU index) deliberately does NOT skip instantiated bodies: clice treats a template as a duck-typed interface and each instantiation as an implementation of it, so a dependent name in the pattern classifies as its actual resolutions — and as a conflict token when instantiations disagree. Pinned by a dedicated fixture and written down in the template resolver design doc ("Instantiations as Implementations"; go-to-implementation over these relations is planned, index-side modeling stays open). The `in_instantiation` flag stays truthful for the members an explicit instantiation delivers as top-level decls. ### Hover comment lookup reuse `decl_for_comment` hand-rolled a weaker version of `decls::instantiated_from` (its `TSK_Undeclared` fallback always chose the primary template). It now chases `instantiated_from` to a fixed point, so an uninstantiated specialization like `Foo<int*>` documents itself with the matching partial specialization's comment. Pinned for both class and variable templates. ### Comment scanning Semantic tokens' `has_logical_newline` re-parsed `/*...*/` syntax by hand to decide where directive context ends; it now consults the comment ranges the Lexer scan already collects, so comment syntax is parsed in one place. ### Explicit instantiation directives (known limitation, now pinned) Function and variable explicit instantiation directives (`template void f<int>(int);`) are invisible today — no semantic token on the name, no outline entry, no occurrence — because clang mislocates them at the pattern. This is fixed upstream by llvm/llvm-project#191658 (`ExplicitInstantiationDecl`, clang 23); until the toolchain pin catches up, every workaround site is tagged `FIXME(explicit-instantiation)` and the current behavior is pinned by `partial` fixtures in the semantic tokens and document symbol corpora. The class form (childless outline node, painted reference) keeps working and is pinned alongside. ## Tests - New fixtures: `inlay_hint/type_conflicting_instantiations`, `semantic_tokens/explicit_instantiation_directives`, `document_symbol/kinds_explicit_instantiations`, plus hover pins for class and variable template comment fallback. - Full local gate: unit (RelWithDebInfo + Debug/ASan), snap (both, standalone + wire), integration, smoke, `npm run check`, docs check — all green.
This is the initial fix of #191442. Following the discussion here #115418 (comment).
Description
This PR introduces a new AST node,
ExplicitInstantiationDecl, to systematically fix the long-standing issue of missing or incorrect source location information for explicit template instantiations.Background & The Problem
Historically, Clang's AST lacked a dedicated node to represent the lexical occurrence of an explicit instantiation statement. Instead,
Sematried to shoehorn this information into existing specialization nodes (e.g.,FunctionDecl,VarTemplateSpecializationDecl) or simply returnednullptr.This resulted in fragmented behavior across the seven instantiable entity types:
nullptr, completely losingSourceRangeandNestedNameSpecifierinformation.templateorextern templatedeclarations in the same file would overwrite each other's source locations.dyn_castbugs and dropped NNS information.Design Trade-offs Evaluated
Before settling on the current design, I evaluated a mixed redeclaration-chain approach (similar to how explicit specializations are handled, creating new
FunctionDeclnodes and stitching them into the redecl chain). However, this approach had significant flaws:DeclContextconstraints (e.g., a member function shouldn't lexically reside in a namespaceDeclContext, but placing it in the class context would pollute member lookup).FoldingSetmechanisms (setFunctionTemplateSpecialization).NamedDeclnodes purely for instantiations risked breaking downstreamASTMatchers and altering name lookup behavior.To avoid these pitfalls, this PR introduces
ExplicitInstantiationDeclas a purely lexical annotation node.Key Design Characteristics:
Decl, notNamedDecl: This is the most crucial design choice. Much likeStaticAssertDeclorFriendDecl, this node lives in aDeclContext(making it traversable byRecursiveASTVisitorand visible in AST dumps) but remains completely invisible to C++ name lookup. It does not interfere with overload resolution or lookup tables.Specialization) to the underlying instantiated declaration, unifying how functions, variables, classes, and members are handled.SourceRange,NestedNameSpecifierLoc, and the exact locations of thetemplateandexternkeywords.Assisted-by: Claude Code (Anthropic) — used for test writing and checking test results