Skip to content

fix: preserve thematic breaks in overwrite Markdown - #11142

Merged
vicancy merged 3 commits into
dotnet:mainfrom
vicancy:lianwei-microsoft-overwrite-thematic-break-parsing
Sep 17, 2026
Merged

vicancy merged 3 commits into
dotnet:mainfrom
vicancy:lianwei-microsoft-overwrite-thematic-break-parsing

Conversation

@vicancy

@vicancy vicancy commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #11088.

Overwrite files intentionally support multiple UID sections. Their Markdown pipeline enables Markdig's mid-document YAML parser before the thematic-break parser, so a pair of body --- lines currently consumes the enclosed Markdown as YAML. Dictionary deserialization then warns and drops that content.

This change is a bounded reduction in false positives, not an unambiguous way to infer whether an author intended metadata or prose.

Approach

Keep Markdig's delimiter handling and inspect only the first YAML root event for subsequent header candidates:

  • Mapping roots remain overwrite headers. Existing key order, quoted/flow/explicit keys, leading comments and blank lines, block scalars, repeated UIDs, and *content syntax remain supported.
  • Non-mapping roots return to normal Markdown parsing, preserving the reporter's rules, History heading, and numbered list.
  • Header validation on the first line of the file is unchanged. YAML syntax errors encountered during recognition stay on the existing diagnostic path, as do malformed/unsupported mappings and mappings missing uid.
  • Lookahead reads from the source offset without copying the entire remaining document for each candidate.

This is not a first-header-only restriction, a literal uid:/first-line heuristic, or a "deserialization failed, so render Markdown" fallback. Recognizing a mapping commits the block to metadata processing; it does not require the rest of the block to deserialize successfully.

Limitations and compatibility

The same --- delimiters can represent both overwrite headers and thematic breaks. YAML syntax can classify the structure, but cannot establish the author's intent.

Case in a later triple-dashed block Behavior and limitation
Mapping-shaped prose such as Title: Hello Still treated as metadata and reports missing uid. Adding uid makes it an overwrite section, even if the author intended to display it as prose.
Mapping begins correctly but later YAML is malformed Still reports an error; it does not fall back to Markdown.
A root-identification syntax error Keeps the original header/diagnostic path. Not every possible Markdown body is therefore guaranteed to render without a YAML diagnostic.
A malformed intended header such as uid Some.Type, or a sequence such as - uid: Some.Type Intentional diagnostic change: classified as non-mapping body content, with no invalid-header warning for that root. The intended overwrite is not applied. The same invalid header on the first line retains the original validation.
Plain non-mapping content, leading blank lines, or comments Classification follows YAML grammar, not a hand-written prefix rule. Blank lines and comments alone do not rule out a header.

Use *** or ------ for thematic breaks when unambiguous body rendering is required. The expanded Thematic breaks and limitations section in docs/tutorial/intro_overwrite_files.md explains these tradeoffs with an example. This PR does not claim to eliminate every ambiguity or preserve every diagnostic for invalid inputs.

Test structure

  • In-memory syntax coverage: 28 parameterized/fact cases in YamlHeaderTest call the real MarkdigMarkdownService through the existing TestUtility.CreateMarkdownService helper. They check exact rendered HTML, header source ranges, and diagnostics without creating files. The overwrite.md argument is only the logical source name used in diagnostics.
  • Focused reader integration: 5 file-backed cases retain the exact LF/CRLF reporter examples, distinct/repeated UID section splitting with fenced code, metadata and source positions, and missing-UID validation. The new regression cases therefore create 5 files per target framework instead of 31.
  • Existing reader tests remain intact. No fake parser, shared file fixture, production API changes for testing, or new dependencies were introduced.

Validation

Against upstream ad05df46ab615d34af3f81849ae5e45ecb4fd51e:

  • Before: the exact reporter example failed for both LF and CRLF, producing zero horizontal rules instead of two.
  • After: both cases pass through real files, HostService, and MarkdownReader.ReadMarkdownAsOverwrite, preserving the summary, two rules, heading/list content, and source line ranges.
  • Docfx.Build.Common.Tests: 25 passed on each of .NET 8 and .NET 10.
  • Docfx.MarkdigEngine.Tests: 52 passed, 5 existing skips on each of .NET 8 and .NET 10; includes conceptual-mode and initial-header diagnostic coverage.
  • Both test-project dependency graphs compile for net8.0, net9.0, and net10.0, with zero warnings/errors.
  • Changed-file whitespace formatting verification and git diff --check pass.
  • The implementation commit passed upstream Windows/macOS/Linux CI, Lint, CLA, and Codecov checks. Follow-ups clarify limitations and improve test layering without changing the production parser.

.NET 9 runtime execution was unavailable locally; its target compiled successfully. No dependencies, workflows, release/versioning, certificates, or publishing configuration changed.

vicancy and others added 3 commits September 17, 2026 18:01
Distinguish subsequent YAML mappings from Markdown body content without restricting overwrite files to a single UID section. Keep initial-header validation and mapping diagnostics, and cover the reported content loss with reader and rendering regressions.

Fixes dotnet#11088

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: bf666f99-8d30-4763-878d-b566c0ac2f2d
Explain mapping-shaped prose ambiguity, preserved syntax-error handling, and the diagnostic change for malformed non-mapping later headers. Recommend unambiguous thematic-break delimiters.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bf666f99-8d30-4763-878d-b566c0ac2f2d
Move syntax and diagnostic cases to the real Markdown service without file I/O. Keep focused reader integration coverage for the report, repeated and distinct UIDs, metadata, source positions, and missing-UID validation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bf666f99-8d30-4763-878d-b566c0ac2f2d
@vicancy
vicancy merged commit 6b9aeee into dotnet:main Sep 17, 2026
9 checks passed
This was referenced Sep 21, 2026
Benziza pushed a commit to Benziza/queryguard-dotnet that referenced this pull request Sep 21, 2026
Updated [docfx](https://github.com/dotnet/docfx) from 2.78.5 to 2.80.1.

<details>
<summary>Release notes</summary>

_Sourced from [docfx's
releases](https://github.com/dotnet/docfx/releases)._

## 2.80.1

<!-- Release notes generated using configuration in .github/release.yml
at main -->

## What's Changed
### 💪 Other Changes
* fix: preserve thematic breaks in overwrite Markdown by @​vicancy in
dotnet/docfx#11142
* fix: XML comment parse logics for block element that is adjacent to
markdown content by @​filzrev in
dotnet/docfx#10966
* chore: Update Roslyn and Microsoft.Build package dependencies by
@​filzrev in dotnet/docfx#11124
* fix: preserve indentation around XML comment code blocks by @​vicancy
in dotnet/docfx#11143
* fix: stop metadata generation when restore fails by @​vicancy in
dotnet/docfx#11146
* fix: clarify empty .NET API diagnostics by @​vicancy in
dotnet/docfx#11145
* fix: share tool templates across target frameworks by @​vicancy in
dotnet/docfx#11144
* fix: honor TOC metadata while preserving file metadata overrides by
@​vicancy in dotnet/docfx#11140
* fix(deps): update esbuild to 0.28.1 and tsx to 4.22.4 by
@​dependabot[bot] in dotnet/docfx#11077
* feat(pdf): allow footer and header on cover and toc by @​sergimos in
dotnet/docfx#10958
* chore(deps): bump actions/setup-dotnet from 5 to 6 by
@​dependabot[bot] in dotnet/docfx#11116
* chore(deps): bump actions/checkout from 6 to 7 by @​dependabot[bot] in
dotnet/docfx#11115
* Bump Markdig from 1.1.0 to 1.1.1 by @​dependabot[bot] in
dotnet/docfx#11024
* deps: Update JsonSchema.Net dependency to 8.0.5 by @​filzrev in
dotnet/docfx#10964

## New Contributors
* @​sergimos made their first contribution in
dotnet/docfx#10958

**Full Changelog**:
dotnet/docfx@v2.78.6...v2.80.1

## 2.78.6

<!-- Release notes generated using configuration in .github/release.yml
at main -->

## What's Changed
### 💪 Other Changes
* chore(deps): bump dependabot/fetch-metadata from 2.3.0 to 2.5.0 by
@​dependabot[bot] in dotnet/docfx#10956
* chore(deps): bump paulhatch/semantic-version from 5.4.0 to 6.0.1 by
@​dependabot[bot] in dotnet/docfx#10978
* chore(deps-dev): bump minimatch from 3.1.2 to 3.1.5 in /templates by
@​dependabot[bot] in dotnet/docfx#11016
* Bump Jint from 4.6.0 to 4.6.1 by @​dependabot[bot] in
dotnet/docfx#11018
* chore(deps): bump actions/upload-artifact from 6 to 7 by
@​dependabot[bot] in dotnet/docfx#11015
* chore(deps): bump actions/download-artifact from 7 to 8 by
@​dependabot[bot] in dotnet/docfx#11014
* Bump Markdig from 1.0.0 to 1.1.0 by @​dependabot[bot] in
dotnet/docfx#11019
* chore: remove unnecessary using to fix the Lint check by @​lahma in
dotnet/docfx#11085
* Make xref archive download test hermetic by @​vicancy in
dotnet/docfx#11092
* fix: treat different ports as external links by @​vicancy in
dotnet/docfx#11091
* chore: cleanup YamlSerializationTest.cs by @​filzrev in
dotnet/docfx#10974
* chore(deps-dev): bump follow-redirects from 1.15.11 to 1.16.0 in
/templates by @​dependabot[bot] in
dotnet/docfx#11062
* chore(deps): bump postcss from 8.5.6 to 8.5.14 in /templates by
@​dependabot[bot] in dotnet/docfx#11070
* chore(deps): bump uuid and mermaid in /templates by @​dependabot[bot]
in dotnet/docfx#11073
* chore(deps-dev): bump lodash from 4.17.23 to 4.18.1 in /templates by
@​dependabot[bot] in dotnet/docfx#11061
* chore(deps): bump fast-uri from 3.1.0 to 3.1.2 in /templates by
@​dependabot[bot] in dotnet/docfx#11071
* chore(deps): bump mermaid from 11.15.0 to 11.16.1 in /templates by
@​dependabot[bot] in dotnet/docfx#11093
* chore(deps): bump dompurify from 3.3.1 to 3.4.13 in /templates by
@​dependabot[bot] in dotnet/docfx#11096
* chore(deps-dev): bump js-yaml from 4.1.1 to 4.3.1 in /templates by
@​dependabot[bot] in dotnet/docfx#11095
* chore(deps): bump postcss from 8.5.14 to 8.5.26 in /templates by
@​dependabot[bot] in dotnet/docfx#11094
* chore(deps): bump fast-uri from 3.1.2 to 3.1.5 in /templates by
@​dependabot[bot] in dotnet/docfx#11097
* chore(deps-dev): bump socket.io-parser from 4.2.5 to 4.2.7 in
/templates by @​dependabot[bot] in
dotnet/docfx#11098
* chore(deps): bump ws, engine.io-client, engine.io and
socket.io-adapter in /templates by @​dependabot[bot] in
dotnet/docfx#11101
* chore(deps-dev): bump brace-expansion from 1.1.12 to 1.1.18 in
/templates by @​dependabot[bot] in
dotnet/docfx#11099
* deps: Update playwright version to 1.60.0 by @​filzrev in
dotnet/docfx#11074
* deps: Update roslyn package versions to 5.6.0 by @​filzrev in
dotnet/docfx#11047
* fix(deps): apply compatible npm security updates by @​vicancy in
dotnet/docfx#11105
* Bump the spectre group with 2 updates by @​dependabot[bot] in
dotnet/docfx#11102
* Bump the xunit group with 1 update by @​dependabot[bot] in
dotnet/docfx#11107
* Fix typo in Schema Document Processor overview by @​smartcaveman in
dotnet/docfx#11005
* Bump coverlet.collector from 8.0.0 to 8.0.1 by @​dependabot[bot] in
dotnet/docfx#11045
* chore(deps): bump paulhatch/semantic-version from 6.0.1 to 6.0.2 by
@​dependabot[bot] in dotnet/docfx#11036
* chore(deps): bump dependabot/fetch-metadata from 2.5.0 to 3.0.0 by
@​dependabot[bot] in dotnet/docfx#11051
* chore(deps): bump actions/deploy-pages from 4 to 5 by
@​dependabot[bot] in dotnet/docfx#11049
* chore(deps): bump azure/login from 2 to 3 by @​dependabot[bot] in
dotnet/docfx#11035
* Bump Jint from 4.6.1 to 4.6.3 by @​dependabot[bot] in
dotnet/docfx#11032
* chore(deps): bump dorny/test-reporter from 2.5.0 to 3.0.0 by
@​dependabot[bot] in dotnet/docfx#11043
* Bump AwesomeAssertions from 9.4.0 to 9.5.0 by @​dependabot[bot] in
dotnet/docfx#11114
* chore(deps): bump dependabot/fetch-metadata from 3.0.0 to 3.1.0 by
@​dependabot[bot] in dotnet/docfx#11110
* chore(deps): bump paulhatch/semantic-version from 6.0.2 to 6.0.3 by
@​dependabot[bot] in dotnet/docfx#11112
* chore(deps): bump actions/upload-pages-artifact from 4 to 5 by
@​dependabot[bot] in dotnet/docfx#11109
* chore(deps): bump codecov/codecov-action from 5 to 7 by
@​dependabot[bot] in dotnet/docfx#11113
* chore(deps): bump actions/github-script from 8 to 9 by
@​dependabot[bot] in dotnet/docfx#11111
* Bump the xunit group with 1 update by @​dependabot[bot] in
dotnet/docfx#11117
* Bump coverlet.collector from 8.0.1 to 10.0.1 by @​dependabot[bot] in
dotnet/docfx#11119
* Bump Microsoft.NET.Test.Sdk from 18.0.1 to 18.9.0 by @​dependabot[bot]
in dotnet/docfx#11122
 ... (truncated)

Commits viewable in [compare
view](dotnet/docfx@v2.78.5...v2.80.1).
</details>

[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=docfx&package-manager=nuget&previous-version=2.78.5&new-version=2.80.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Markdown to Override API fails if it has multiple "---"

2 participants