chore(coverage): move CommonMark conformance tests to xtask/coverage

[jfmcdowell]

Note

AI Assistance Disclosure: This PR was developed with assistance from Claude Code.

Summary

Moves CommonMark spec compliance tests from crates/biome_markdown_parser/tests/ to xtask/coverage/src/markdown/ to align with how other language conformance tests are organized.

• Add CommonMarkTestSuite with load_all() for embedded test suites
• Add just update-commonmark-spec <version> for one-command spec updates
• Move test infrastructure from biome_markdown_parser to xtask/coverage

CommonMark publishes spec.json at versioned URLs so we embed the spec (140KB, 652 tests). The update script downloads the spec and auto-updates the provenance comment in commonmark.rs, providing a one-command update workflow.

Follow-up to PR #8525 and is a part of #3718.

Test Plan

# Verify update script works
  rm xtask/coverage/src/markdown/spec.json
  just update-commonmark-spec 0.31.2

  # Verify provenance auto-updates
  # Set bogus values in commonmark.rs:
  #   Version: OLD_VERSION
  #   Downloaded: 1999-01-01
  #   Examples: 999

  just update-commonmark-spec 0.31.2
  # Confirm provenance updated to 0.31.2, today's date, 652 examples

  # Verify tests pass
  just test-markdown-conformance
  # Expected: 652 tests, 455 passed (69.79%)

[changeset-bot]

⚠️ No Changeset found

Latest commit: 92d6326

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[codspeed-hq]

Merging this PR will not alter performance

✅ 58 untouched benchmarks
⏩ 95 skipped benchmarks¹

---

_{Comparing jfmcdowell:chore/md-coverage (92d6326) with next (5701ead)}

Footnotes

1. 95 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[coderabbitai]

Walkthrough

This PR removes the CommonMark test harness file from crates/biome_markdown_parser/tests and relocates CommonMark compliance testing into the xtask/coverage framework. It adds a new markdown coverage module and a CommonMark test suite that loads an embedded spec.json, introduces a script to update spec.json, updates the justfile to run coverage for markdown, adjusts xtask/coverage Cargo manifest and wiring (modules, runner extension), and updates CLI help text.

Possibly related PRs

• None with a strong code-level connection.

Suggested reviewers

• ematipico
• dyc3

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarises the main change: moving CommonMark conformance tests from biome_markdown_parser to xtask/coverage.
Description check	✅ Passed	The description clearly relates to the changeset, detailing the rationale, implementation approach, and test plan for moving the test suite.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@scripts/update-commonmark-spec.sh`:
- Around line 41-44: The script currently treats missing arguments the same as
help flags and exits 0; change the conditional so that if "$1" is "-h" or
"--help" it calls print_help and exits 0, but if there are no arguments (i.e.,
$# -lt 1) call print_help and exit with a non-zero status (e.g., exit 1 or exit
2) so missing required arguments return a failure code; update the if/elif/else
around the existing print_help invocation to reflect this in the block
containing print_help and the exit call.

🧹 Nitpick comments (3)

xtask/coverage/src/runner.rs (1)

354-356: Consider simplifying the reporting loop.

Iterating solely to call test_loaded() for each item is a bit verbose. If the reporter only needs a count, a single call with the length might suffice—though this depends on whether test_loaded() has side effects beyond counting.

♻️ Optional simplification

-        for _ in &tests {
-            context.reporter.test_loaded();
-        }
+        (0..tests.len()).for_each(|_| context.reporter.test_loaded());

Or if the reporter can accept a batch count, that would be cleaner still.

xtask/coverage/src/markdown/commonmark.rs (1)

86-107: Edge case: same-line <pre>...</pre> handling.

If a line contains both <pre and </pre> (e.g., <pre>code</pre>), the current logic sets in_pre = true then immediately in_pre = false, so it processes the line as inside a <pre> block. This happens to be correct for preserving whitespace on that line, but the logic is fragile. Consider adding a brief comment or a test case to document this behaviour.

justfile (1)

237-239: Consider Windows portability for the update script.

The ./scripts/update-commonmark-spec.sh invocation won't work on Windows without WSL. The justfile already handles platform differences for _touch (lines 213-219). You could apply a similar pattern here, or add a comment noting WSL/Git Bash is required.

That said, this is an infrequent maintenance task, so it's not a blocker.

[dyc3]

LGTM