Clone the IR tree so we don't have to re-lower from syntax.

[chsienki]

Due to the reordering of phases in sonic, tag helper lowering happens later in the process. This means if we need to replay back to a point earlier in the process because a TagHelper changed, the IR tree has been mutated and instead we have to perform the initial relowering from syntax again.

This PR introduces a .Clone() mechanism to IR nodes, so that we can clone the DocumentNode before it gets mutated. This means we can just start from that point again instead of having to start over from syntax.

The first commit is big-ish, but also almost entirely mechanical, so you can probably mostly glance over it. Commit 2 uses the clone to keep a copy of the IR tree and restore it when needed. Commit 3 just adds an extra benchmark because during testing I realized we had two different scenarios that we were equating as one and want to be able to measure them independently.

At some point I plan to make the IR nodes fully immutable, so we don't have to worry about cloning to avoid mutation, but this was the expedient way to get it working without an even bigger change.

Microsoft Reviewers: Open in CodeFlow

[azure-pipelines]

Azure Pipelines:
Successfully started running 2 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

This PR adds deep-cloning support to Razor’s intermediate representation (IR) nodes and uses it in the source generator pipeline to preserve an “unresolved” DocumentIntermediateNode snapshot, enabling tag-helper resolution/rewrite to be replayed without re-lowering from syntax. It also adds coverage and benchmarking for the new replay scenario.

Changes:

• Introduce IntermediateNode.Clone() plus per-node CloneNode() implementations for IR node types.
• Update the source generator project engine to cache an unresolved IR tree and resolve/rewrite against a cloned copy on replays.
• Add a new unit test to validate cloning fidelity and add a new microbenchmark/test scenario distinguishing “ignorable” independent edits.

Reviewed changes

Copilot reviewed 34 out of 34 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/Razor/src/Compiler/perf/Microsoft.AspNetCore.Razor.Microbenchmarks.Generator/RazorTests.cs	Adds a (skipped) functional test for the new “independent ignorable edit” scenario.
src/Razor/src/Compiler/perf/Microsoft.AspNetCore.Razor.Microbenchmarks.Generator/RazorBenchmarks.cs	Adds a benchmark for the “independent ignorable edit” scenario and associated input content.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/SourceGenerators/SourceGeneratorProjectEngine.cs	Switches replay behavior to clone the stored unresolved DocumentIntermediateNode before resolution/rewrite.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Mvc/InjectIntermediateNode.cs	Implements CloneNode() for InjectIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/RazorCodeDocument.cs	Adds storage/accessors for an _unresolvedDocumentNode cached on the RazorCodeDocument.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/UsingDirectiveIntermediateNode.cs	Implements CloneNode() for UsingDirectiveIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/UnresolvedExpressionAttributeValueIntermediateNode.cs	Implements CloneNode() for UnresolvedExpressionAttributeValueIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/UnresolvedElementIntermediateNode.cs	Implements CloneNode() for UnresolvedElementIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/UnresolvedAttributeValueIntermediateNode.cs	Implements CloneNode() for UnresolvedAttributeValueIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/UnresolvedAttributeIntermediateNode.cs	Implements CloneNode() for UnresolvedAttributeIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/PropertyDeclarationIntermediateNode.cs	Implements CloneNode() for PropertyDeclarationIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/NamespaceDeclarationIntermediateNode.cs	Implements CloneNode() for NamespaceDeclarationIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/MethodDeclarationIntermediateNode.cs	Implements CloneNode() for MethodDeclarationIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/IntermediateNode.cs	Adds IntermediateNode.Clone() and CloneNode() extensibility point (throws by default).
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/HtmlIntermediateToken.cs	Implements CloneNode() for HtmlIntermediateToken.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/HtmlContentIntermediateNode.cs	Implements CloneNode() for HtmlContentIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/HtmlAttributeValueIntermediateNode.cs	Implements CloneNode() for HtmlAttributeValueIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/HtmlAttributeIntermediateNode.cs	Implements CloneNode() for HtmlAttributeIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/FieldDeclarationIntermediateNode.cs	Implements CloneNode() for FieldDeclarationIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/DocumentIntermediateNode.cs	Implements CloneNode() for DocumentIntermediateNode (shares decl subtree/options/target by reference).
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/DirectiveTokenIntermediateNode.cs	Implements CloneNode() for DirectiveTokenIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/DirectiveIntermediateNode.cs	Implements CloneNode() for DirectiveIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/CSharpIntermediateToken.cs	Implements CloneNode() for CSharpIntermediateToken.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/CSharpExpressionIntermediateNode.cs	Implements CloneNode() for CSharpExpressionIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/CSharpExpressionAttributeValueIntermediateNode.cs	Implements CloneNode() for CSharpExpressionAttributeValueIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/CSharpCodeIntermediateNode.cs	Implements CloneNode() for CSharpCodeIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/CSharpCodeAttributeValueIntermediateNode.cs	Implements CloneNode() for CSharpCodeAttributeValueIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/ClassDeclarationIntermediateNode.cs	Implements CloneNode() for ClassDeclarationIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Extensions/TemplateIntermediateNode.cs	Implements CloneNode() for TemplateIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Extensions/SectionIntermediateNode.cs	Implements CloneNode() for SectionIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Extensions/RazorCompiledItemMetadataAttributeIntermediateNode.cs	Implements CloneNode() for RazorCompiledItemMetadataAttributeIntermediateNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Components/RouteAttributeExtensionNode.cs	Implements CloneNode() for RouteAttributeExtensionNode.
src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Components/ComponentInjectIntermediateNode.cs	Implements CloneNode() for ComponentInjectIntermediateNode.
src/Razor/src/Compiler/Microsoft.AspNetCore.Razor.Language/test/Intermediate/IntermediateNodeCloneTest.cs	Adds a new unit test asserting IR cloning produces a structurally equivalent deep copy (with intentional shared references).

[davidwengier]

Seems like a good comment, and could apply to any property that is itself a node. Can we add a debug assert to each CloneNode method that deals with them, to verify that Children contains the value, to prevent future regressions? Or a test that uses reflection somehow, though presumably creation would be difficult.

[chsienki]

I did an audit and this node was the only case I could find that does this. I've made Clone() virtual so it can just own the entire process rather than trying to delegate back and forth between the base and the derived class.

Added a test that checks if the original node has a property with a value that also occurs in the children collection then the cloned node has the same relationship and doesn't end up with two copies instead.

[davidwengier]

Seems like a good comment, and could apply to any property that is itself a node. Can we add a debug assert to each CloneNode method that deals with them, to verify that Children contains the value, to prevent future regressions? Or a test that uses reflection somehow, though presumably creation would be difficult.

[Copilot]

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated no new comments.

Suppressed comments (3)

src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/UnresolvedAttributeIntermediateNode.cs:75

• HtmlAttributeNode is assigned to reference the last entry in Children during lowering (DefaultRazorIntermediateNodeLoweringPhase.cs:1234). During cloning, the Children list is deep-cloned, but HtmlAttributeNode is handled separately, which can cause HtmlAttributeNode and the corresponding child in Children to diverge. Tag helper resolution mutates HtmlAttributeNode in place, so this can make later phases that walk Children observe stale/unmodified state.

A robust fix is to make HtmlAttributeNode a cached lookup over Children: lowering can set the cache for O(1), but cloned nodes can safely leave it unset and still retrieve the correct child clone.

        visitor.VisitDefault(this);
    }

    protected override IntermediateNode CloneNode()
    {

src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/UnresolvedAttributeIntermediateNode.cs:87

• CloneNode() deep-clones HtmlAttributeNode, but HtmlAttributeNode is a cached reference to a node that also lives in Children (it’s set to Children[^1] during lowering). IntermediateNode.Clone() will also deep-clone Children, so the clone can contain two distinct HtmlAttributeIntermediateNode instances: one reachable via HtmlAttributeNode and another via Children.

If HtmlAttributeNode is changed to be a cached lookup over Children, cloning should avoid cloning it and instead leave it unset (or explicitly null) so it resolves to the child clone.

            AttributeNameSpan = AttributeNameSpan,
            AsTagHelperAttribute = AsTagHelperAttribute?.Clone(),
            AsMarkupAttribute = AsMarkupAttribute?.Clone(),
            HtmlAttributeNode = (HtmlAttributeIntermediateNode?)HtmlAttributeNode?.Clone(),
            IsSynthesizedHelper = IsSynthesizedHelper,

src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/HtmlIntermediateToken.cs:22

• CloneNode() uses the string-valued Content property, which forces evaluation of LazyContent and turns lazy tokens into eager ones (IsLazy becomes false). When cloning large IR trees to avoid re-lowering, this can introduce a lot of extra string materialization that the pipeline might otherwise never pay for.

If possible, consider preserving laziness by cloning the underlying LazyContent when IsLazy is true (same applies to CSharpIntermediateToken.CloneNode).

    protected override IntermediateNode CloneNode()
        => new HtmlIntermediateToken(Content, Source)
        {
            IsSynthesizedHelper = IsSynthesizedHelper,
        };

[Copilot]

Pull request overview

Copilot reviewed 36 out of 36 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/IntermediateNode.cs:110

• CloneNode()'s doc comment currently states that any child nodes held outside Children are deep-cloned. Some node types (e.g. DocumentIntermediateNode sharing DeclDocumentNode) intentionally share known-inert subtrees, so this comment should allow for documented sharing to avoid a misleading contract.

    ///  Creates a copy of this node carrying only its own state -- the node-specific fields (including the
    ///  init-only <see cref="IsSynthesizedHelper"/>) and any child nodes held outside <see cref="Children"/>
    ///  (deep-cloned). The common state and the <see cref="Children"/> are copied by <see cref="Clone"/>.

src/Razor/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Intermediate/IntermediateNode.cs:89

• The XML doc for IntermediateNode.Clone() claims this is always a deep copy of the node and its descendants, but at least DocumentIntermediateNode.CloneNode() intentionally shares DeclDocumentNode by reference. The docs should be tightened to avoid implying a strict deep clone contract for all node-typed side properties.

This issue also appears on line 108 of the same file.

    ///  Returns a deep copy of this node and its descendants. The node-specific state is produced by
    ///  <see cref="CloneNode"/>; this method copies the common state (source span, imported flag,
    ///  diagnostics) and deep-clones the children onto it. A node with a property that aliases one of
    ///  its <see cref="Children"/> overrides this to re-point that property at the cloned child.