Add skill for AOT-compatibility by agocke · Pull Request #122 · dotnet/skills

agocke · 2026-02-25T18:54:44Z

Also adds a real-world project conversion between the before and after tests.

Copilot

Pull request overview

This PR adds a new skill for making .NET projects compatible with Native AOT and trimming by systematically resolving IL trim/AOT analyzer warnings. It includes comprehensive documentation, polyfill examples, and a real-world conversion demonstrating the before and after states of an Azure ResourceManager project.

Changes:

Added dotnet-aot-compat skill with detailed documentation on resolving trim/AOT warnings
Provided comprehensive test fixtures showing AOT-compatible code patterns
Included polyfills for annotation attributes on older target frameworks

Reviewed changes

Copilot reviewed 47 out of 930 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
skills/dotnet-aot-compat/SKILL.md	Complete skill documentation explaining AOT compatibility concepts and step-by-step resolution procedures
skills/dotnet-aot-compat/tests/after/Azure.ResourceManager.csproj	Project file demonstrating AOT compatibility enablement with `<IsAotCompatible>true</IsAotCompatible>`
skills/dotnet-aot-compat/tests/after/ExperimentalAttribute.cs	Polyfill implementation for the ExperimentalAttribute on pre-.NET 8 frameworks
skills/dotnet-aot-compat/tests/after/GenericOperationSource.cs	Example showing proper use of `[DynamicallyAccessedMembers]` annotations for AOT compatibility
skills/dotnet-aot-compat/tests/after/ArmOperation.cs	Demonstrates AOT-compatible operation rehydration with proper member annotations
skills/dotnet-aot-compat/tests/after/ResourceManagerJsonContext.cs	Source-generated JSON serialization context for AOT compatibility
skills/dotnet-aot-compat/tests/after/Extensions/ArmClientBuilderExtensions.cs	Shows use of `[RequiresDynamicCode]` and `[RequiresUnreferencedCode]` attributes
skills/dotnet-aot-compat/tests/after/*/Custom/.cs	Multiple custom resource implementations demonstrating AOT-compatible patterns
skills/dotnet-aot-compat/tests/after/Directory.Packages.props	Central package version management configuration
skills/dotnet-aot-compat/tests/after/Assets/Profile/2020-09-01-hybrid.json	Azure Stack profile configuration data

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-25T18:55:22Z

+        /// <summary> Determines if two <see cref="ArmEnvironment"/> values are the same. </summary>
+        public static bool operator ==(ArmEnvironment left, ArmEnvironment right) => left.Equals(right);
+
+        /// <summary> Determines if two <see cref="ArmEnvironment"/> values are not the same. </summary>internal


The comment internal appears on line 62 before the operator declaration but should either be removed or placed on its own line as a proper comment. As written, it would cause a compilation error.

Suggested change

/// <summary> Determines if two <see cref="ArmEnvironment"/> values are not the same. </summary>internal

/// <summary> Determines if two <see cref="ArmEnvironment"/> values are not the same. </summary>

Copilot · 2026-02-25T18:55:22Z

+            switch (format)
+            {
+                case "J":
+                    return ModelReaderWriter.Write(this, options, AzureResourceManagerContext.Default);


Reference to AzureResourceManagerContext.Default is used, but this type is not defined in any of the provided files. This would cause a compilation error unless this context is defined elsewhere in the project.

Suggested change

return ModelReaderWriter.Write(this, options, AzureResourceManagerContext.Default);

return ModelReaderWriter.Write(this, options, ResourceManagerJsonContext.Default);

Copilot · 2026-02-25T18:55:23Z

+        private void AppendChildObject(StringBuilder stringBuilder, object childObject, ModelReaderWriterOptions options, int spaces, bool indentFirstLine)
+        {
+            string indent = new string(' ', spaces);
+            BinaryData data = ModelReaderWriter.Write(childObject, options, AzureResourceManagerContext.Default);


Same issue as above - AzureResourceManagerContext.Default is referenced but not defined in the provided files, which would cause compilation failure.

Suggested change

BinaryData data = ModelReaderWriter.Write(childObject, options, AzureResourceManagerContext.Default);

BinaryData data = ModelReaderWriter.Write(childObject, options);

jeffschwMSFT · 2026-02-25T21:29:26Z

this is a very large addition, largely test. is there any concern with this source becoming public?
please rebase to the current directory structure src/dotnet/skills/ and src/dotnet/tests/
Curious on the evaluation score.
Finally, please also consider the other proposed skill in this space - I presume we will only keep one. #12

agocke · 2026-02-25T21:34:01Z

Results are still being run -- this takes a long time because of the length of the task.

This project is pulled from the Azure SDK and lightly modified. That's an open-source project, so I don't think there would be a problem with making it public.

I'll try to de-dup with the other skill as well.

jeffschwMSFT · 2026-02-25T21:36:17Z

This project is pulled from the Azure SDK and lightly modified.

got it. thoughts on using a smaller portion? Or is the value at this size? (eg. seeing if we can get the same value with a less expensive CI cost)

agocke · 2026-02-26T18:54:21Z

Both the no-skill and skilled version are terrible -- no skill is 1.0/5.0 for trying to suppress everything with pragmas. Skilled is 1.2/5.0 for making some progress and fixing a few warnings correctly, but then spending the rest of the time scanning the code base before timing out.

I think the real-world test is very good here. It showing that handing a real, large project to the agent is a significant problem. They can get distracted, take shortcuts, and generally do things they wouldn't do in a narrow hand-picked example.

I'm making some changes to the skill and re-running and I'll see if that improves things.

agocke · 2026-02-26T23:51:44Z

Alright, I've hit another problem. The LLM is cheating. It's pulling from the after directory. I will see if I can come up with a fix.

Also adds a real-world project conversion between the before and after tests.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

- Expand SKILL.md with routing signals, severity markers, stop signals, triage step, batch-fix strategy, IJsonModel guidance, and sub-agent dispatch pattern - Extract polyfill code block to references/polyfills.md - Add eval scenario and test fixtures at tests/dotnet/dotnet-aot-compat/ following repo convention (moved from skills/ to tests/<plugin>/) - Eval validates at +25-47% improvement across multiple runs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Warn against adding external package types to JsonSerializerContext (prevents the ResponseError compile-error trap) - Strengthen anti-exploration directive with concrete negative examples - Push sub-agent parallelization after 2 build-fix cycles Validated with 3-run eval (Opus 4.6): pass, rubric 4.4/5. Cross-judge rejudge (Sonnet 4.5, GPT-5.1-Codex, Gemini 3 Pro): all pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

steveisok · 2026-03-01T15:25:27Z

@agocke I couldn't push to your remote directly, so I've pushed improvements to https://github.com/steveisok/skills/tree/aot-skill (https://github.com/steveisok/skills/tree/aot-skill) (2 commits on top of your branch). You'll need to grab the branch yourself.

I was able to push to your branch now.

Here's what changed:

SKILL.md improvements (commit e44121c):

Added INVOKES line to frontmatter description
Strengthened anti-exploration directive — explicit ❌ against find/ls/grep/README before building. The agent was burning time on codebase exploration instead of letting the compiler guide it.
Strategy C step 2 now warns against adding external package types (e.g., ResponseError from Azure.Core) to JsonSerializerContext. The agent was consistently hitting compile errors trying to source-generate for types it doesn't own, then running out the
clock debugging. Gotcha Initial documentation and validation workflow #1 already covered this, but the agent wasn't reading that far — now the warning is inline where the agent encounters the problem.
Sub-agent dispatch now says to parallelize all remaining edits after 2 build-fix cycles instead of continuing sequentially.

Eval tests (commit 98b609f):

Added back tests/dotnet/dotnet-aot-compat/ with eval.yaml, before/after fixtures (Azure.ResourceManager project, 113 files, 180 IL warnings)
Rubric explicitly penalizes #pragma and [UnconditionalSuppressMessage] suppression (baseline was "cheating" this way)
1050s timeout

Validation results:

3-run eval with Opus
4.6 judge: ✅ pass, rubric quality 4.4/5, baseline 2.0 → skilled 3.3
Skilled agent scores range from 2.7 to 4.0 across runs depending on how efficiently the agent triages before the timeout — the variance is in agent behavior, not judge scoring
Used PR Add --keep-sessions flag and rejudge command to skill-validator #149's --keep-sessions and rejudge to cross-validate with 3 additional judge models (Sonnet
4.5, GPT-5.1-Codex, Gemini 3 Pro) — all 4 judges pass and pick the skilled agent as winner in pairwise comparison. Notably, Opus judged more on process (penalizing the baseline for not using build analyzer output), while Sonnet and Gemini judged more on
outcomes (scoring the baseline higher when the final build passed). All agreed directionally that the skill improves agent behavior.
Previous runs without the Strategy C fix showed the agent consistently hitting the ResponseError wall and timing out — that's now resolved

agocke

Overall I think the new changes are good, but I have worries about overfitting. I think we'll likely need to introduce more projects and see how it fairs with something completely different.

agocke · 2026-03-02T18:51:37Z

 ```

-When you annotate a parameter, **all callers** must now pass properly annotated types. This cascades outward — follow each caller and annotate or refactor as needed.
+When you annotate a parameter, **all callers** must now pass properly annotated types. This cascades outward — follow each caller and annotate or refactor as needed. **The caller's annotation must include at least the same member types as the callee's.** If the callee requires `PublicConstructors | NonPublicConstructors`, the caller must specify the same or a superset — using only `NonPublicConstructors` will produce IL2091.


This feels like strangely-specific direction...

It seems without some kind of bound, the models may get lost. Is it too weird?

agocke · 2026-03-02T18:52:29Z

+
+## Common Gotchas
+
+1. **External types without AOT-safe serialization**: When a type comes from a dependency you can't modify (e.g., `ResponseError` from `Azure.Core`) and it lacks a source-generated serializer, `Options.GetConverter<T>()` is reflection-based and will produce IL warnings. First check if the type implements `IJsonModel<T>` (common in Azure SDK) — if so, bypass `JsonSerializer` entirely:


The note about ResponseError looks wrong -- I think it's overfitting to the specific test case.

ResponseError was the last area the models got hung up on and couldn't finish the job. I don't know if taking this away will throw the models back into not being able to finish.

steveisok · 2026-03-03T01:42:59Z

Overall I think the new changes are good, but I have worries about overfitting. I think we'll likely need to introduce more projects and see how it fairs with something completely different.

Do you have a 'medium sized' project? That might be a good way to tell.

agocke · 2026-03-03T22:46:00Z

I ran this locally on Roslyn -- some amount of overfitting. But I don't want to get too into the weeds on this PR. Let's merge it and follow-up.

Copilot AI review requested due to automatic review settings February 25, 2026 18:54

Copilot AI reviewed Feb 25, 2026

View reviewed changes

agocke and others added 5 commits February 27, 2026 14:07

Add skill for AOT-compatibility

e5148f6

Also adds a real-world project conversion between the before and after tests.

Update skills/dotnet-aot-compat/SKILL.md

fc0d1a2

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Improve skill focus

cf2c3ba

Move skill into plugin directory and delete tests

dde6b72

Add references section

c041ef4

steveisok self-requested a review February 28, 2026 04:29

steveisok mentioned this pull request Feb 28, 2026

Add --keep-sessions flag and rejudge command to skill-validator #149

Merged

steveisok and others added 2 commits March 1, 2026 10:06

steveisok requested review from dbreshears and timheuer as code owners March 2, 2026 18:32

agocke commented Mar 2, 2026

View reviewed changes

steveisok approved these changes Mar 3, 2026

View reviewed changes

jeffschwMSFT merged commit 38da20a into dotnet:main Mar 3, 2026
1 check passed

github-actions Bot mentioned this pull request Mar 4, 2026

🏥 Repository Health Dashboard #168

Closed

	/// <summary> Determines if two <see cref="ArmEnvironment"/> values are not the same. </summary>internal
	/// <summary> Determines if two <see cref="ArmEnvironment"/> values are not the same. </summary>

	return ModelReaderWriter.Write(this, options, AzureResourceManagerContext.Default);
	return ModelReaderWriter.Write(this, options, ResourceManagerJsonContext.Default);

	BinaryData data = ModelReaderWriter.Write(childObject, options, AzureResourceManagerContext.Default);
	BinaryData data = ModelReaderWriter.Write(childObject, options);


		## Common Gotchas

		1. External types without AOT-safe serialization: When a type comes from a dependency you can't modify (e.g., `ResponseError` from `Azure.Core`) and it lacks a source-generated serializer, `Options.GetConverter<T>()` is reflection-based and will produce IL warnings. First check if the type implements `IJsonModel<T>` (common in Azure SDK) — if so, bypass `JsonSerializer` entirely:

Uh oh!

Conversation

agocke commented Feb 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jeffschwMSFT commented Feb 25, 2026

Uh oh!

agocke commented Feb 25, 2026

Uh oh!

jeffschwMSFT commented Feb 25, 2026

Uh oh!

agocke commented Feb 26, 2026

Uh oh!

agocke commented Feb 26, 2026

Uh oh!

steveisok commented Mar 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agocke left a comment

Choose a reason for hiding this comment

Uh oh!

agocke Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

steveisok Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

agocke Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

steveisok Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

steveisok commented Mar 3, 2026

Uh oh!

agocke commented Mar 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

steveisok commented Mar 1, 2026 •

edited

Loading