Add migrating-newtonsoft-to-system-text-json skill by mrsharm · Pull Request #200 · dotnet/skills

mrsharm · 2026-03-04T16:42:33Z

Summary

Adds the migrating-newtonsoft-to-system-text-json skill for migrating from Newtonsoft.Json (Json.NET) to System.Text.Json.

Note: Reopened from #89 (accidentally closed, could not reopen due to EMU restrictions).

Skill Validation Results — migrating-newtonsoft-to-system-text-json

Skill	Test	Baseline	With Skill	Δ	Verdict
migrating-newtonsoft-to-system-text-json	Migrate model with Newtonsoft.Json attributes to System.Text.Json	4.0/5	4.3/5	+0.3	✅

Overall improvement: +10.8% (3 runs, not statistically significant)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

3 Iterations.

What the Skill Teaches

Complete attribute mapping table (JsonProperty -> JsonPropertyName, NullValueHandling -> JsonIgnoreCondition, etc.)
Behavioral differences that break silently (default casing, strict parsing, case sensitivity, numbers-in-strings)
Custom converter migration (Newtonsoft JsonConverter -> System.Text.Json JsonConverter with Utf8JsonReader/Writer)
JToken/JObject -> JsonDocument/JsonElement (read-only) or JsonNode (mutable DOM)
Polymorphic serialization with [JsonDerivedType] (.NET 7+)
JsonExtensionData Dictionary value type change (JToken -> JsonElement)

Files

src/dotnet/skills/migrating-newtonsoft-to-system-text-json/SKILL.md
src/dotnet/tests/migrating-newtonsoft-to-system-text-json/eval.yaml

Copilot

Pull request overview

Adds a new .NET skill intended to guide migrations from Newtonsoft.Json (Json.NET) to System.Text.Json, plus an evaluation scenario to validate the skill’s impact.

Changes:

Added a new skill doc covering attribute mappings, behavioral differences, converter migration, and DOM migration guidance.
Added a new eval scenario for migrating a Newtonsoft-attributed model and configuring global JSON options.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 11 comments.

File	Description
src/dotnet/skills/migrating-newtonsoft-to-system-text-json/SKILL.md	New skill documentation for Newtonsoft.Json → System.Text.Json migration guidance
src/dotnet/tests/migrating-newtonsoft-to-system-text-json/eval.yaml	New evaluation scenario exercising attribute/config migration guidance

Comments suppressed due to low confidence (2)

src/dotnet/skills/migrating-newtonsoft-to-system-text-json/SKILL.md:208

The .csproj snippet uses Version="*", which isn't a valid/typical PackageReference version and may confuse readers. Since the intent is "remove these references", consider omitting the Version attribute entirely or showing a realistic pinned/floating version format (e.g., 13.0.3 or 13.*) in examples elsewhere.

```xml
<!-- Remove from .csproj -->
<PackageReference Include="Newtonsoft.Json" Version="*" />
<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="*" />

src/dotnet/tests/migrating-newtonsoft-to-system-text-json/eval.yaml:5

The prompt says the model "uses ... a custom converter", but the snippet only uses StringEnumConverter (a built-in Newtonsoft converter). Either add an actual custom JsonConverter example or reword the prompt to avoid misleading setup.

    prompt: |
      I'm migrating our ASP.NET Core 8 project from Newtonsoft.Json to System.Text.Json. Here's a model class that uses Newtonsoft attributes and a custom converter. Convert this to System.Text.Json:

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

ViktorHofer

@mrsharm please fix the path. These files should be under plugins/... not src

eiriktsarpalis

It is an understatement to say that migrating serializers is a nontrivial task -- it is almost guaranteed that the new serializer is going to behave differently to an extent, so the goal of such a skill is to make sure all steps are taken so that risk is minimized.

An important aspect of minimizing risk in migrations of such scope is having a testing strategy. At a bare minimum, the agent should develop a test suite as it performs the migration comparing STJ serialization/deserialization behavior against the NJ baseline. This would help provide an empirically grounded feedback loop for the agent to apply necessary tweaks to DTOs, their attribute annotations, and their custom convert.

…vement) Teaches migration from Newtonsoft.Json to System.Text.Json: attribute mapping differences, behavioral changes (casing, strictness, null handling), custom converter conversion, JToken->JsonElement/JsonNode migration, and polymorphic serialization with JsonDerivedType. Eval results: +13.8% improvement over baseline (threshold: 10%) Includes eval.yaml with migration scenario + negative test.

…NumberHandling

…LL.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…y structure - Move skill from src/dotnet/skills/ to plugins/dotnet/skills/ - Move eval from src/dotnet/tests/ to tests/dotnet/ - Add CODEOWNERS entries

folder structure addressed

ViktorHofer · 2026-03-06T18:00:12Z

/evaluate

github-actions · 2026-03-06T18:04:33Z

Skill Validation Results

Skill	Scenario	Baseline	With Skill	Δ	Skills Loaded	Overfit	Verdict
migrating-newtonsoft-to-system-text-json	Migrate model with Newtonsoft.Json attributes to System.Text.Json	4.0/5	5.0/5	+1.0	⚠️ NOT ACTIVATED	✅ 0.10	✅

Model: claude-opus-4.6 | Judge: claude-opus-4.6

Full results

github-actions · 2026-03-28T05:32:49Z

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
migrating-newtonsoft-to-system-text-json	Migrate model with Newtonsoft.Json attributes to System.Text.Json	4.0/5 → 5.0/5 🟢	✅ migrating-newtonsoft-to-system-text-json; tools: report_intent, skill / ✅ migrating-newtonsoft-to-system-text-json; tools: report_intent, skill	✅ 0.08	❌ [1]

[1] (Plugin) Quality improved but weighted score is -25.6% due to: judgment, tokens (12523 → 30575), quality, tool calls (0 → 2), time (11.4s → 18.3s)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

📖 See InvestigatingResults.md for how to diagnose failures. Additional debugging guidance may be provided by your workflow.

🔍 Full results — includes quality and agent details

To investigate failures, paste this to your AI coding agent:

For PR 200 in dotnet/skills, download eval artifacts with gh run download 23678436702 --repo dotnet/skills --pattern "skill-validator-results-*" --dir ./eval-results, then fetch https://raw.githubusercontent.com/dotnet/skills/def169f3e4bbac4e4e52552de3398be891332187/eng/skill-validator/InvestigatingResults.md and follow it to analyze the results.json files. Diagnose each failure, suggest fixes to the eval.yaml and skill content, and tell me what to fix first.

danmoseley · 2026-03-28T05:41:56Z

evaluation looks good. @mrsharm you just need review signoffs -- particularly @eiriktsarpalis who made the requested change review. I believe I addressed that
1f79841#diff-c50f8f29f07af40dbed674ea985e909f0c6647686b09dcd2382d657e110e072dR11-R15

and everything else.

eiriktsarpalis · 2026-03-30T12:40:37Z

+> **Important:** Migrating serializers is a nontrivial task. System.Text.Json will almost
+> certainly behave differently from Newtonsoft.Json in subtle ways. Always validate
+> serialization output and deserialization behavior thoroughly with real-world data after
+> migrating. Automated and manual testing of all serialization paths is essential.


Consider adding an explicit instruction to the agent that it should be adding baseline serialization tests for the application models while performing the migration.

eiriktsarpalis · 2026-03-30T12:41:43Z

+| Using `JsonElement` after `JsonDocument` is disposed | JsonElement is invalid after dispose; clone with `element.Clone()` if needed |
+| `[JsonIgnore]` from wrong namespace | Both Newtonsoft and System.Text.Json have `[JsonIgnore]` — wrong `using` = attribute ignored |
+| Custom converter reading past the current token | System.Text.Json reader is strict — must read exactly the right tokens |
+| `JsonExtensionData` type mismatch | Use `Dictionary<string, JsonElement>`, `IDictionary<string, object>`, or `JsonObject` — not `JToken` |


Consider adding links to the official STJ docs at learn.

eiriktsarpalis · 2026-03-30T12:43:47Z

@@ -0,0 +1,48 @@
+scenarios:
+  - name: "Migrate model with Newtonsoft.Json attributes to System.Text.Json"


Shouldn't we add evals covering more scenaria? Custom converters? Naming policies? etc? Consider mining real-world snippets either from dotnet org repos or public github repos in general (e.g. via grep.app)

…rubric items - Rewrite Step 2 to security-first approach (settings commented out with attack-specific warnings for case sensitivity, comments, Preserve) - Add Step 8: baseline serialization testing strategy - Trim Steps 4-8 and behavior table to reduce token overhead - Add reject_tools and 2 new rubric items (security trade-offs, package removal) - Add NullValueHandling default caveat rubric item - Add References section with official Microsoft docs links

mrsharm · 2026-04-06T15:13:05Z

/evaluate

github-actions · 2026-04-06T15:16:41Z

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
migrating-newtonsoft-to-system-text-json	Migrate model with Newtonsoft.Json attributes to System.Text.Json	3.7/5 → 4.0/5 🟢	✅ migrating-newtonsoft-to-system-text-json; tools: report_intent, skill / ✅ migrating-newtonsoft-to-system-text-json; tools: skill, report_intent	✅ 0.12	❌ [1]

[1] ⚠️ High run-to-run variance (CV=18.75) — consider re-running with --runs 5. (Isolated) Quality improved but weighted score is -7.9% due to: tokens (12841 → 28655), tool calls (0 → 1)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions

Replace the diagnosis scenario (failed at -8.0% due to zero quality improvement in isolated mode + token overhead) with a migration review scenario that tests nuanced skill guidance. The new review scenario presents deliberately flawed migration code (NullValueHandling incorrectly added, ReadCommentHandling on external JSON, lingering Newtonsoft import, JToken not replaced). The skill reliably catches the security risk of ReadCommentHandling.Skip that the baseline misses (1.4/5 -> 5.0/5). Both scenarios pass 5-run evals: - Model attributes: 3.6->4.0 (isolated), overfit 0.10 - Migration review: 3.4->4.2 (isolated), overfit 0.10

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-06T15:47:09Z

+  Migrate from Newtonsoft.Json to System.Text.Json, handling behavioral differences,
+  custom converters, and common breaking changes. Use when converting a project from
+  Newtonsoft.Json (Json.NET) to the built-in System.Text.Json serializer.


The frontmatter description doesn't follow the established dotnet-upgrade skill convention of including USE FOR: / DO NOT USE FOR: (and typically INVOKES: / LOADS REFERENCES: when applicable). For consistency and better skill discovery/activation, expand the description to match the pattern used in other skills (e.g., plugins/dotnet-upgrade/skills/migrate-dotnet9-to-dotnet10/SKILL.md:4-18).

Suggested change

Migrate from Newtonsoft.Json to System.Text.Json, handling behavioral differences,

custom converters, and common breaking changes. Use when converting a project from

Newtonsoft.Json (Json.NET) to the built-in System.Text.Json serializer.

USE FOR: Migrating a project from Newtonsoft.Json (Json.NET) to

System.Text.Json; updating serialization and deserialization code to use the

built-in serializer; handling behavioral differences, custom converters, and

common breaking changes encountered during the migration. DO NOT USE FOR:

General JSON serialization guidance unrelated to migration; projects that are

staying on Newtonsoft.Json; unrelated .NET upgrade tasks that do not involve

replacing Newtonsoft.Json with System.Text.Json.

Copilot · 2026-04-06T15:47:10Z

+| `[JsonProperty("name")]` | `[JsonPropertyName("name")]` |
+| `[JsonIgnore]` | `[JsonIgnore]` (same name, different namespace!) |
+| `[JsonProperty(Required = Required.Always)]` | `[JsonRequired]` (.NET 7+) |
+| `[JsonProperty(NullValueHandling = NullValueHandling.Ignore)]` | `[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]` |
+| `[JsonProperty(DefaultValueHandling = DefaultValueHandling.Ignore)]` | `[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]` |


The mapping [JsonProperty(Required = Required.Always)] → [JsonRequired] is not a drop-in equivalent: Newtonsoft's Required.Always enforces both presence and non-null (and has nuanced behavior with defaults), while System.Text.Json's [JsonRequired] primarily enforces presence and interacts differently with nullable annotations/required members. Please add a brief note clarifying the semantic differences and suggesting the usual alternatives (e.g., required members + validation) so readers don't assume identical runtime behavior.

Copilot · 2026-04-06T15:47:10Z

+- Read: `(string)node["key"]!` or `.GetValue<T>()`; Modify: `node["key"] = value`
+- Serialize: `node.ToJsonString()`
+
+For **read-only** scenarios, use `JsonDocument`/`JsonElement` (IDisposable, must clone if keeping past dispose).


This line implies JsonElement is IDisposable ("JsonDocument/JsonElement (IDisposable ...)"). In System.Text.Json only JsonDocument is disposable; JsonElement becomes invalid once its owning JsonDocument is disposed. Reword to avoid suggesting that JsonElement itself should/can be disposed, and keep the guidance about cloning when the document lifetime is shorter than the element lifetime.

Suggested change

For **read-only** scenarios, use `JsonDocument`/`JsonElement` (IDisposable, must clone if keeping past dispose).

For **read-only** scenarios, use `JsonDocument`/`JsonElement`; `JsonDocument` is `IDisposable`, and if a `JsonElement` must outlive its owning document, clone it before disposing the document.

Copilot · 2026-04-06T15:47:10Z

+      ```csharp
+      using System.Text.Json;
+      using System.Text.Json.Serialization;
+      using Newtonsoft.Json;  // keeping this — we still use JObject in one helper


In the scenario prompt, the comment says you're keeping using Newtonsoft.Json; because you still use JObject, but JObject lives in Newtonsoft.Json.Linq. This mismatch can confuse the model (and reviewers) about what the leftover dependency actually is. Consider changing the snippet to using Newtonsoft.Json.Linq; (or update the comment to match the namespace you intend to illustrate).

Suggested change

using Newtonsoft.Json; // keeping this — we still use JObject in one helper

using Newtonsoft.Json.Linq; // keeping this — we still use JObject in one helper

Copilot · 2026-04-06T15:47:10Z

+/plugins/dotnet-upgrade/skills/migrating-newtonsoft-to-system-text-json/ @dotnet/area-system-text-json @mrsharm
+/tests/dotnet-upgrade/migrating-newtonsoft-to-system-text-json/ @dotnet/area-system-text-json @mrsharm


PR description lists files under src/dotnet/..., but the changes in this PR add the skill under plugins/dotnet-upgrade/... and tests under tests/dotnet-upgrade/.... Please align the PR description's file list with the actual paths to avoid confusion for reviewers and future archaeology.

mrsharm · 2026-04-06T16:23:47Z

/evaluate

github-actions · 2026-04-06T16:27:58Z

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
migrating-newtonsoft-to-system-text-json	Migrate model with Newtonsoft.Json attributes to System.Text.Json	3.7/5 → 4.3/5 🟢	✅ migrating-newtonsoft-to-system-text-json; tools: skill, report_intent / ✅ migrating-newtonsoft-to-system-text-json; tools: report_intent, skill	✅ 0.12	❌ [1]
migrating-newtonsoft-to-system-text-json	Review a partially completed Newtonsoft.Json to System.Text.Json migration for correctness	3.0/5 → 4.3/5 🟢	✅ migrating-newtonsoft-to-system-text-json; tools: report_intent, skill	✅ 0.12	✅

[1] ⚠️ High run-to-run variance (CV=1.99) — consider re-running with --runs 5. (Isolated) Quality improved but weighted score is -6.0% due to: tokens (12792 → 28672), tool calls (0 → 2)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions

AbhitejJohn · 2026-05-22T00:24:21Z

@mrsharm : Presuming we are still interested in taking this, it might need a rebase on the latest main. Also tagging @dotnet/skills-upgrade-reviewers to help with reviews.

github-actions · 2026-06-03T23:57:25Z