fix(parser): ignore fenced code blocks when parsing delta specs - #1151
Conversation
Requirement headers, delta section headers, scenarios and REMOVED/RENAMED entries written inside fenced code blocks were parsed as real content by the delta-spec parser. A fenced `### Requirement:` example became a phantom requirement, producing spurious `validate` errors and risking incorrect `archive` output. Fence detection was duplicated across MarkdownParser and spec-structure but missing entirely from requirement-blocks (which powers both validate and archive). Extract a single shared `buildCodeFenceMask` helper and make the delta-spec parser and validator block helpers honor it, so all parsers treat fenced code consistently. Co-authored-by: Cursor <cursoragent@cursor.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughFence-aware parsing is centralized in a shared mask utility. Requirement, delta-spec, and fenced-content parsing now ignore Markdown structures inside code fences, with regression tests covering requirements, directives, renamed or removed entries, and validation scenarios. ChangesFence-aware Delta-spec Parsing
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
test/core/validation.test.ts (1)
651-686: ⚡ Quick winAdd a fenced-only scenario regression here.
This test still passes if
countScenarios()starts counting#### Scenario:lines inside fences again, because Line 663 already provides a real scenario. Add a sibling case with no unfenced scenario and assert the validator still reports the missing-scenario error.Example follow-up case
+ it('does not count scenario headers inside fenced code blocks toward the required scenario count', async () => { + const changeDir = path.join(testDir, 'test-change-fenced-scenario-only'); + const specsDir = path.join(changeDir, 'specs', 'test-spec'); + await fs.mkdir(specsDir, { recursive: true }); + + const deltaSpec = `# Test Spec + +## ADDED Requirements + +### Requirement: Documentation Generator +The system SHALL render a delta example in its output. + +\`\`\`markdown +#### Scenario: Example scenario +\`\`\` +`; + + const specPath = path.join(specsDir, 'spec.md'); + await fs.writeFile(specPath, deltaSpec); + + const report = await new Validator(true).validateChangeDeltaSpecs(changeDir); + + expect(report.valid).toBe(false); + expect( + report.issues.some((i) => i.message.includes('must include at least one scenario')) + ).toBe(true); + });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/core/validation.test.ts` around lines 651 - 686, The test currently only covers a case where a fenced code block contains a faux requirement+scenario but an unfenced scenario exists so a regression in countScenarios() could be missed; add a sibling test case in test/core/validation.test.ts that writes a spec where the only "#### Scenario:" is inside a fenced block (no unfenced scenario), call Validator.validateChangeDeltaSpecs on that changeDir, and assert the validator reports the missing-scenario error (e.g., report.valid is false, report.summary.errors > 0 and report.issues includes a message about missing scenario). This ensures countScenarios() and the Validator class logic correctly ignore fenced code blocks and surface the missing-scenario error when appropriate.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@test/core/validation.test.ts`:
- Around line 651-686: The test currently only covers a case where a fenced code
block contains a faux requirement+scenario but an unfenced scenario exists so a
regression in countScenarios() could be missed; add a sibling test case in
test/core/validation.test.ts that writes a spec where the only "#### Scenario:"
is inside a fenced block (no unfenced scenario), call
Validator.validateChangeDeltaSpecs on that changeDir, and assert the validator
reports the missing-scenario error (e.g., report.valid is false,
report.summary.errors > 0 and report.issues includes a message about missing
scenario). This ensures countScenarios() and the Validator class logic correctly
ignore fenced code blocks and surface the missing-scenario error when
appropriate.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 725c321b-78bf-4210-a01d-d12e1da4475b
📒 Files selected for processing (8)
.changeset/fence-aware-delta-parsing.mdsrc/core/parsers/code-fence.tssrc/core/parsers/markdown-parser.tssrc/core/parsers/requirement-blocks.tssrc/core/parsers/spec-structure.tssrc/core/validation/validator.tstest/core/parsers/requirement-blocks.test.tstest/core/validation.test.ts
alfred-openspec
left a comment
There was a problem hiding this comment.
Reviewed the parser/validator paths and this looks good to merge.
The shared buildCodeFenceMask keeps the existing MarkdownParser/spec-structure behavior while wiring the missing fence awareness into requirement-blocks.ts, which is the important archive/validate path. I also checked the CodeRabbit scenario-count nit: the current implementation does ignore fenced-only #### Scenario: lines and still reports the missing-scenario error, so I do not think that extra test should block merge. It would still be a nice regression to add if you want the belt-and-suspenders coverage.
Local verification:
pnpm exec vitest run test/core/parsers/requirement-blocks.test.ts test/core/validation.test.tspnpm exec vitest run test/specs/source-specs-normalization.test.ts test/core/parsers/markdown-parser.test.ts test/core/parsers/change-parser.test.ts- fenced-only scenario throwaway fixture against built validator
pnpm run buildpnpm run lintpnpm test→ 1659 passed
Add a regression test asserting that a `#### Scenario:` appearing only inside a fenced code block does not count toward the required scenario count, so the validator still reports the missing-scenario error. This guards the fence awareness of `countScenarios()`: the existing fenced-example test always includes a real (unfenced) scenario, so it would not catch a regression that began counting fenced scenario headers. Addresses the review suggestion on PR Fission-AI#1151. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Addressed the scenario-count nit from the review. Pushed Test-only change: no source/behavior changes, and no new changeset (the existing Verification:
|
alfred-openspec
left a comment
There was a problem hiding this comment.
Re-reviewed after the latest commits and this is still ready to merge.
The new fenced-only scenario regression covers the previous CodeRabbit nit directly: fenced #### Scenario: examples no longer count toward the required scenario count, so real missing-scenario errors still fire. I re-ran the focused parser/validator tests and build locally on head 0816db3, all green.
|
Hi, is there anything I can do on my side to move forward with this PR? |
|
Thanks for checking in. Nothing needed from your side right now, this is already approved and mergeable. I’m leaving the actual merge to Tabish, but the PR itself looks ready. |
Deterministic, CLI-enforced prevention of silent spec drop in the spec-driven workflow, addressing one root fault (correctness decisions in agent prompts) across four layers: schema-aware delta gate shared by validate+archive (Fission-AI#997), the non-transitive apply-gate loop fix (incorporates Fission-AI#1250), the keystone (all archive templates call `openspec archive` — Fission-AI#656/Fission-AI#863), and a specced archived-drift audit. Hardened across three review rounds (alfred-openspec, CodeRabbit), an internal adversarial pass, and an open-PR coordination scan: - reconciles Fission-AI#977 (allow-specless) via the schema-aware gate + --skip-specs, deliberately not allowing silent specless archives under spec-driven; - coordinates with Fission-AI#902 (propose/ff spec discovery), the unify-template-generation-pipeline manifest, add-change-stacking-awareness (provides/touches markers + overlap warnings), and add-artifact- regeneration-support (complementary staleness); - rebases onto approved Fission-AI#1186/Fission-AI#1151/Fission-AI#1153/Fission-AI#1252; Fission-AI#1252 is a prerequisite. Closes Fission-AI#1212 Fission-AI#1260 Fission-AI#1222 Fission-AI#1264 Fission-AI#799 Fission-AI#656 Fission-AI#863 Fission-AI#913. Supersedes PRs Fission-AI#1250 Fission-AI#1271 Fission-AI#1241 Fission-AI#1233. Addresses Fission-AI#997 Fission-AI#977 Fission-AI#902 Fission-AI#164 Fission-AI#426 Fission-AI#911. Out of scope Fission-AI#1246 Fission-AI#1112 Fission-AI#1252(prereq) Fission-AI#1120 Fission-AI#827 Fission-AI#1265. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alfred-openspec
left a comment
There was a problem hiding this comment.
Revalidated the shared fence mask through parser, validator, scenario-count, and archive-facing paths; the only changes since the prior approval are main merges. Full current CI is green.
…-audit follow-ups (Fission-AI#1475) * fix(archive): make the scenario-drift check fence-aware parseScenarioBlocks matched #### Scenario: headers on raw lines while the validator's countScenarios masks fenced code blocks (Fission-AI#1151). The drift check (Fission-AI#1391) inherited the raw scan, so a fenced scenario example in the current spec aborted an archive that validate had passed, and a fenced name in the MODIFIED block counted as keeping a scenario the block had actually dropped. Build the shared code-fence mask and skip masked lines in both the header scan and the block-end scan. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(update): tear down the redirected request when the budget expires The overall request budget was armed inside the first send() and its callback closed over that hop's request. After a redirect the timer destroyed the already-dead first request, so a redirect target that trickled bytes kept resetting its idle timeout and held the socket open until the body-size cap. Track the in-flight request and have the budget timer destroy whichever one is open. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(release): add changesets for user-facing changes missing from the 1.7.0 notes 18 feat/fix commits merged since v1.6.0 without a changeset, so the pending Version Packages PR would have released them silently: five tool integrations (ZCode, Hermes, CodeArts, Kimi Code rename, Codex skills-only), skills.sh distribution, symlinked schema dirs, nested spec discovery, drift multiplicity, checkbox markers, Windows welcome input, npx avoidance, doctor store drift, local dates, missing-core-workflows warning, store-aware main specs, open-questions guidance, and spec content guidance. Plus changesets for this branch's two fixes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(adapters): escape TOML-active characters in Gemini command files The gemini adapter interpolated the description into a TOML basic string and the body into a multiline basic string with no escaping. Every current template value happens to be safe; the first description with a double quote or backslash would silently produce invalid TOML for all Gemini command files. Escape both contexts (Fission-AI#1447 fixed the same class for the YAML adapters but scoped itself to YAML). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(update): harden install detection and redirect handling Three follow-ups from the release audit: - A path segment literally named volta (a user or project directory) classified the install as volta-managed and swallowed the upgrade offer. The undotted spelling now requires volta's own tools/image layout, matching how pnpm and yarn already demand corroboration. - The Windows npm-ownership fallback checked that the npm prefix exists, which is true of any X\node_modules\pkg tree, hand-copied ones included. Corroborate with the openspec.cmd shim npm actually writes. - A https registry redirecting to plain http was followed; a MITM on that reply controls the newer-version answer. Refuse the downgrade. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(cli): export zcodeAdapter from the barrel and sync a completion description zcode was registered but missing from the adapters barrel (its test imported the module directly), and the completion registry still carried the pre-Fission-AI#1062 description for the instructions command. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(parser): strip a UTF-8 BOM before parsing specs and deltas A BOM-prefixed delta spec (Windows editors, PowerShell Out-File) failed validate and archive with 'No delta sections found' because the first line never matched '## ADDED Requirements'. Strip the BOM in both normalizers, the same way tool detection already does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(cli): reject over-long change names with a validation message A 300-character change name surfaced two raw ENAMETOOLONG errno dumps from stat and mkdir. Bound the name at 200 characters in validateChangeName so the failure is a normal validation error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(archive): finish the early-sync no-op rules for MODIFIED and RENAMED Two asymmetries left over from the Fission-AI#1376/Fission-AI#1386/Fission-AI#1437 no-op work: - MODIFIED counted every delta as applied even when the block was byte-equal to the main spec, so a fully early-synced change rewrote the file (normalization churn), printed '~ N modified', and reported specsUpdated: true where its ADDED/REMOVED/RENAMED twins print 'Specs already in sync; no files changed.' Count only real replacements. - RENAMED's already-synced skip (source gone, target present) had no near-miss guard: a case/whitespace variant of the source still in the spec means a typo'd header, and REMOVED already hard-aborts on that signal. Apply the same guard, excluding the target itself so a case-only rename still no-ops. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(validate): stop reporting an unreadable specs dir as 'no deltas' The delta-validation loop swallowed every error as 'if no specs dir, treat as no deltas', so an EACCES capability folder produced the misleading 'Change must have at least one delta' while archive let the same error propagate. Tolerate only ENOENT and ENOTDIR (a stray specs file); anything else stays loud, matching discoverSpecFiles' documented fail-loud contract. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(update): say when commands-only delivery leaves a tool with nothing Under delivery: commands, update removed the skills of adapterless skills-only tools (Hermes, Kimi Code, Vibe, CodeArts, ForgeCode) without a word — leaving zero OpenSpec artifacts while the tool's detection dir kept re-suggesting an init that would also generate nothing. Print the same per-tool configuration correction init already prints, pointing at 'openspec config set delivery both'. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(completion): honor $ZSH and $ZSH_CUSTOM for Oh My Zsh installs The installer used a set $ZSH only as an is-installed signal and then wrote to ~/.oh-my-zsh regardless, so a custom OMZ location got a freshly created ~/.oh-my-zsh tree that no shell ever loads — and isInstalled/uninstall looked in the same wrong place. Route every path through the $ZSH/$ZSH_CUSTOM-aware helpers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(init): make the static welcome screen wait for the Enter it asks for The static branch printed 'Press Enter to select tools...' and returned immediately, so the Enter landed in the tool picker and submitted the pre-selected set sight-unseen. Fission-AI#1462 routed reduced-motion, OPENSPEC_NO_ANIMATION, --no-animation, NO_COLOR, and narrow-terminal users onto this path. Wait in a TTY; drop the prompt line when there is no TTY to wait on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(feedback): keep the manual fallback on every gh failure Only missing-gh and unauthenticated flows showed the formatted feedback and pre-filled submission URL; issues-disabled, network, or rate-limit failures printed gh's stderr and discarded the path to submit what the user had already typed. Route those through the same manual fallback, preserving gh's exit code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(update): model the npm shim in the Windows prefix fixture The ownership corroboration now checks for the openspec.cmd shim npm writes beside node_modules; the Homebrew-prefix fixture built the layout without it, so the test failed on windows-pwsh. Write the shim in the fixture and pin the inverse: the same shape with nothing npm wrote (a hand-copied portable tree) is not an npm install. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(update): require volta's full tools/image layout for the undotted spelling The corroboration used has('tools', 'image'), which is some() — volta AND (tools OR image) — so /srv/volta/tools/apps/... still classified as a Volta install and swallowed the upgrade offer. Require both segments, matching the real %LOCALAPPDATA%\Volta\tools\image layout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(adapters): escape control characters in Gemini multiline prompts escapeTomlMultilineBasicString handled backslashes and quote-triples but not the C0 controls that are as invalid in a multiline basic string as in a single-line one. Reuse TOML_CONTROL_CHARS, applied last so the escapes it introduces are not re-doubled. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(completion): finish the $ZSH_CUSTOM support and isolate it in tests The fpath verification advice still grepped the literal custom/completions, which a relocated $ZSH_CUSTOM need never contain — grep the actual directory instead. The installer tests cleared only $ZSH, so on a machine exporting $ZSH_CUSTOM they would have written into (and deleted from) the developer's real OMZ custom dir — the same leakage class Fission-AI#1400 fixed for $ZSH. Clear/restore both, and pin the custom-location paths with two new tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(release): correct the hermes and zcode changeset wording Hermes is skills-only (no command adapter), and zcode's namespaced commands register /opsx:<id>, not /opsx-* — the release notes must not reintroduce the invocation-spelling confusion Fission-AI#1471 removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(feedback): pin the manual fallback on a non-label gh failure The new reportGhFailure output (formatted feedback + pre-filled URL) had no coverage; the network-failure test now asserts it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(completion): match fpath entries as literal strings in the OMZ guidance The verification advice interpolated the completions dir into grep "<dir>" where regex metacharacters make the check unreliable and quotes could break the displayed command. Print one fpath entry per line and match with grep -F on a shell-quoted literal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(adapters): never emit a bare carriage return in Gemini TOML prompts A lone CR is illegal in a multiline basic string — Python 3.13 tomllib rejects the file — and the control-char pass deliberately skipped it on the assumption it only appears as CRLF. Normalize CRLF to LF and escape any remaining CR as \r. The escaping guarantee is now parser-backed: smol-toml (new devDependency) round-trips every hostile body in the regression matrix (lone CR, CRLF, CR before a quote run, NUL/VT/FF, trailing backslash, four- and five-quote runs), and the same outputs were verified against Python tomllib. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * build(nix): update the pnpm deps hash for the smol-toml devDependency The lockfile changed, so the fixed-output derivation hash moved; value taken from the CI mismatch report. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Summary
Markdown structure written inside fenced code blocks was being parsed as
real content by the delta-spec parser (
src/core/parsers/requirement-blocks.ts).For example, documenting the delta format inside a scenario:
…caused
### Requirement: Example onlyto be parsed as a second, phantomrequirement. Because that phantom block has no body text,
openspec validatereported a spurious
ADDED "Example only" is missing requirement texterror,and the same mis-parse feeds
archive(specs-apply.ts), risking incorrectspec output.
Root cause
Fenced-code detection already existed in
MarkdownParserandspec-structure.ts(and
ChangeParseruses it), but it was duplicated in each place andmissing entirely from
requirement-blocks.ts, which powers bothvalidateand
archive.Changes
buildCodeFenceMaskhelper (src/core/parsers/code-fence.ts).parseDeltaSpecandextractRequirementsSectionhonor fenced code whendetecting section/requirement/removed/renamed headers.
countScenarios/extractRequirementTextfence-aware too.MarkdownParserandspec-structure.tsto reuse the shared helperinstead of their own copies (no behavior change there; removes the drift that
caused this bug). Public/protected APIs are unchanged.
No new features, no config or output format changes.
Test plan
test/core/parsers/requirement-blocks.test.ts(fenced requirement headers, REMOVED bullets and RENAMED pairs are ignored;
main-spec requirements section too).
test/core/validation.test.ts(fenced example no longer triggers a spurious error).
node build.js(tsc) passes.eslint src/passes.patchchangeset.Made with Cursor
Summary by CodeRabbit