Skip to content

feat(core): align SEO severities with their evidence - #428

Merged
oekazuma merged 3 commits into
mainfrom
fix/p2-severity-alignment
Aug 9, 2026
Merged

oekazuma merged 3 commits into
mainfrom
fix/p2-severity-alignment

Conversation

@oekazuma

@oekazuma oekazuma commented Aug 8, 2026 •

Copy link
Copy Markdown
Owner

The final batch from the 2026-08-09 v1.0 rule-validity review (Priority-2 rows 9-12), maintainer-approved with an explicit re-verify-safety instruction — hence the measured gate/score consequences below.

The four changes

  1. seo/description-presence: critical → warning. critical now uniformly means deploy-blocking (four crash/security rules — orphan-effect, orphan-lifecycle, server-browser-global, handler-state-write — plus title-presence). Gate consequences, independently measured via a full truth-table (all four movements declared in the changeset): desc-missing 1→0 under the default --fail-on critical; under --fail-on warning, og-url-missing 0→1 (the one tightening — can turn previously-green CI red), og-desc-missing 1→0, multi-h1 1→0.
  2. seo/og-url info → warning, seo/og-description warning → info — matches OGP's required/optional split (re-fetched ogp.me during implementation). The old "covered by canonical" rationale is preserved as historical context in the og-url docs (canonical targets search engines; og:url targets social platforms).
  3. seo/single-h1 severity split: zero-h1 stays warning; multiple-h1 demoted to info (no official source documents a penalty). Implemented per-result — the rule's registered severity stays warning, so the scoring denominator is untouched; the split's coherence was traced through buildInventory/computeScore (the Math.max clamp already handles result-vs-rule severity divergence — the treatDynamicAs promotion path uses the same mechanism) and verified end-to-end: an isolated multi-h1 route no longer trips --fail-on warning, an isolated zero-h1 route still does. A global severity override flattens both arms (documented).
  4. seo/hreflang: wording only — the x-default arm now presents Google's "consider, especially for selector/redirect pages" framing instead of implying a defect; severity unchanged, malformed-code arm untouched.

Measured safety verification

  • Pre/post fixture table in the executor report: basic-project exit unchanged (1→1); the description-only scratch fixture flips 1→0 as designed; no other fixture's exit code moved (full suite green both sides).
  • Scoring cascade traced and disclosed: the seo::route pair's inventory weight moves 110 → 100 (description −10, og-url +4, og-description −4), so scores can shift a point or two with zero finding changes — 9 score/report tests recomputed by hand against the new denominator, not pasted from runner output; the critical-carrier fixture in those tests swapped to title-presence (still genuinely critical).
  • findingKey carries no severity (no suppressions/baseline keying risk, verified); io-budget unchanged.
  • Changeset: @svelte-vitals/core minor, exit-code consequences first.

An independent fresh-context review ran a full pre/post gate truth-table over isolated one-rule fixtures, independently re-derived the scoring arithmetic and 7 of the 9 hand-recomputed test expectations, and swept every severity consumer (reporters, suppressions, baseline, the action repo). Verdict: code merge-safe; its text findings (the undeclared og-gate movements, a now-impossible README sample, a rule-count error that originated in the review record) are all fixed in the latest commits.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • SEO findings now use clearer, more consistent severity levels.
    • Missing descriptions are warnings instead of deployment-blocking critical issues.
    • Open Graph URL issues are warnings, while missing Open Graph descriptions are informational.
    • Missing and multiple <h1> findings now receive appropriate severities.
    • Hreflang guidance better distinguishes invalid values from conditionally missing x-default declarations.
  • Documentation

    • Updated English and Japanese SEO rule documentation to reflect revised severities, behavior, and scoring.

…sence, og:url/og:description, single-h1 split)

2026-08-09 v1 rule-validity review, Priority-2 rows 9-12:
- seo/description-presence: critical -> warning (Google only "sometimes" uses it;
  critical is now deploy-blocking rules + title-presence only)
- seo/og-url: info -> warning, seo/og-description: warning -> info (matches OGP's
  own required/optional split)
- seo/single-h1: zero-h1 stays warning, multiple-h1 demoted to info, plus softened
  wording (no ranking-penalty claim for multiple h1s)
- seo/hreflang: wording only on the missing-x-default message/recommendation
  (Google frames it as a recommendation for selector/redirect pages, not a defect)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 410b4595-2fa0-4f14-a1e0-707571f49f0c

📥 Commits

Reviewing files that changed from the base of the PR and between bd06523 and b4f3046.

📒 Files selected for processing (12)
  • .changeset/seo-severity-alignment.md
  • docs/src/content/docs/ja/rules/index.mdx
  • docs/src/content/docs/ja/rules/seo/hreflang.md
  • docs/src/content/docs/ja/rules/seo/index.mdx
  • docs/src/content/docs/ja/rules/seo/og-description.md
  • docs/src/content/docs/rules/index.mdx
  • docs/src/content/docs/rules/seo/hreflang.md
  • docs/src/content/docs/rules/seo/index.mdx
  • docs/src/content/docs/rules/seo/og-description.md
  • docs/superpowers/specs/2026-08-09-v1-rule-validity-review.md
  • packages/cli/README.md
  • packages/vite/test/integration.test.ts
🚧 Files skipped from review as they are similar to previous changes (10)
  • docs/src/content/docs/rules/seo/og-description.md
  • docs/src/content/docs/rules/seo/index.mdx
  • docs/src/content/docs/ja/rules/seo/hreflang.md
  • docs/src/content/docs/rules/seo/hreflang.md
  • docs/src/content/docs/ja/rules/seo/og-description.md
  • packages/vite/test/integration.test.ts
  • .changeset/seo-severity-alignment.md
  • docs/src/content/docs/ja/rules/seo/index.mdx
  • docs/src/content/docs/ja/rules/index.mdx
  • docs/src/content/docs/rules/index.mdx

📝 Walkthrough

Walkthrough

The PR aligns SEO severities for descriptions, Open Graph metadata, and single-h1 findings. It separates hreflang recommendations, updates scoring and report tests, and synchronizes English and Japanese documentation.

Changes

SEO severity alignment

Layer / File(s) Summary
Rule severity and finding behavior
packages/core/src/rules/seo/*.ts
Description presence changes to warning. og-url changes to warning. og-description changes to info. single-h1 assigns warning to missing headings and info to multiple headings. hreflang uses condition-specific recommendations.
Report, gate, and score validation
packages/cli/test/run.test.ts, packages/core/test/*, packages/vite/test/integration.test.ts
Tests update report fixtures, gate behavior, score inventories, Open Graph severities, and single-h1 result severities.
Rule documentation and release metadata
docs/src/content/docs/rules/**, docs/src/content/docs/ja/rules/**, .changeset/seo-severity-alignment.md, packages/cli/README.md, docs/superpowers/specs/2026-08-09-v1-rule-validity-review.md
English and Japanese documentation, release notes, CLI examples, and rule-count notes reflect the revised severities and recommendations.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: aligning SEO rule severities with supporting evidence.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.changeset/seo-severity-alignment.md:
- Around line 5-8: Update the severity-alignment changelog to document the
`seo/og-url` change from info to warning and `seo/og-description` change from
warning to info, including their effects on `--fail-on warning`; alternatively,
limit the existing “no other gate behavior changes” statement to `--fail-on
critical`.

In `@docs/src/content/docs/ja/rules/seo/hreflang.md`:
- Around line 12-13: Update the hreflang validity descriptions in both English
and Japanese documentation to match the subset enforced by isValidHreflang:
x-default or language-script-region tags without variants or extensions. Do not
describe all well-formed BCP-47 codes as valid, including examples such as
de-DE-1996 or en-US-u-hc-h12.
- Around line 15-19: Update the hreflang rule wording in hreflang.md so
x-default is presented as recommended only when a fallback is needed, not
universally required for fixed alternate lists without unsupported-language
handling. Revise the frontmatter summary and rule-check text accordingly, then
regenerate the English and Japanese rule-index tables so their summaries match
the updated guidance.

In `@docs/src/content/docs/rules/seo/hreflang.md`:
- Around line 13-19: Update the source metadata for the hreflang rule so its
index description reflects that fixed language alternates may omit x-default,
then run the documentation generation workflow to regenerate the rule indexes.
Do not edit the generated blocks in rules/index.mdx or rules/seo/index.mdx
directly; locate the hreflang rule’s source definition and preserve the rest of
the index content.

In `@docs/src/content/docs/rules/seo/og-description.md`:
- Around line 16-17: Update the check description in
docs/src/content/docs/rules/seo/og-description.md (lines 16-17) to use
recommended or optional wording for og:description. Regenerate the generated
indexes so docs/src/content/docs/rules/index.mdx (line 62) and
docs/src/content/docs/rules/seo/index.mdx (line 39) use the same optional
recommendation wording; do not edit those generated blocks directly.

In `@packages/vite/test/integration.test.ts`:
- Around line 14-20: Update the fixture comments above the writeFile call to
state that the page includes a meta description and that only the missing title
supplies the critical seo/title-presence finding; remove the inaccurate claim
that the description is missing while preserving the severity context.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fbf07b4a-d732-4fc8-b8c6-83fa729471c0

📥 Commits

Reviewing files that changed from the base of the PR and between 72d908d and bd06523.

📒 Files selected for processing (30)
  • .changeset/seo-severity-alignment.md
  • docs/src/content/docs/ja/rules/index.mdx
  • docs/src/content/docs/ja/rules/seo/description-presence.md
  • docs/src/content/docs/ja/rules/seo/hreflang.md
  • docs/src/content/docs/ja/rules/seo/index.mdx
  • docs/src/content/docs/ja/rules/seo/og-description.md
  • docs/src/content/docs/ja/rules/seo/og-url.md
  • docs/src/content/docs/ja/rules/seo/single-h1.md
  • docs/src/content/docs/rules/index.mdx
  • docs/src/content/docs/rules/seo/description-presence.md
  • docs/src/content/docs/rules/seo/hreflang.md
  • docs/src/content/docs/rules/seo/index.mdx
  • docs/src/content/docs/rules/seo/og-description.md
  • docs/src/content/docs/rules/seo/og-url.md
  • docs/src/content/docs/rules/seo/single-h1.md
  • packages/cli/test/run.test.ts
  • packages/core/src/rules/seo/description-presence.ts
  • packages/core/src/rules/seo/hreflang.ts
  • packages/core/src/rules/seo/og-description.ts
  • packages/core/src/rules/seo/og-url.ts
  • packages/core/src/rules/seo/single-h1.ts
  • packages/core/test/agent-report.test.ts
  • packages/core/test/console-report.test.ts
  • packages/core/test/head-rules.test.ts
  • packages/core/test/json-report.test.ts
  • packages/core/test/sarif-report.test.ts
  • packages/core/test/score.test.ts
  • packages/core/test/seo-head-completeness.test.ts
  • packages/core/test/seo-static-gaps-rules.test.ts
  • packages/vite/test/integration.test.ts

Comment thread .changeset/seo-severity-alignment.md Outdated
Comment thread docs/src/content/docs/ja/rules/seo/hreflang.md Outdated
Comment thread docs/src/content/docs/ja/rules/seo/hreflang.md
Comment thread docs/src/content/docs/rules/seo/hreflang.md
Comment thread docs/src/content/docs/rules/seo/og-description.md
Comment thread packages/vite/test/integration.test.ts
oekazuma and others added 2 commits August 9, 2026 04:21
…ty rule count

Independent review of PR #428 caught four issues, text-only (code and
scoring arithmetic already confirmed correct):

- Changeset falsely claimed "no other gate behavior changes" — under
  --fail-on warning, og-url-missing tightens 0->1 (can newly fail CI on
  upgrade) and og-description-missing/multi-h1 loosen 1->0. All four gate
  movements are now enumerated, with the og-url tightening called out.
- "Three crash/security rules" was a miscount (source has four:
  orphan-effect, orphan-lifecycle, server-browser-global,
  handler-state-write); fixed in the changeset and appended as a dated
  correction to the review record (historical record left otherwise intact).
- packages/cli/README.md sample output showed description-presence under
  Critical (2), a combination this PR makes impossible; trimmed to the
  still-possible Critical (1) shape.
- seo/hreflang's doc frontmatter still commanded declaring x-default;
  softened to match the body's "recommended for selector/redirect pages"
  framing, and regenerated the rules-index tables that embed it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-description wording, stale comment)

Three valid CodeRabbit findings on PR #428, text-only:

- seo/hreflang docs claimed "well-formed BCP-47" but HREFLANG_RE only
  accepts language(-script)(-region) — BCP-47 variants/extensions (e.g.
  de-DE-1996, en-US-u-hc-h12) get flagged. Narrowed the "What it checks"
  wording and added a Limitations section (en+ja). Detection unchanged;
  the rule's own source strings already said "pragmatic BCP-47 subset",
  so no rule-code or changeset update needed.
- seo/og-description's "must include" / "flagged" phrasing read as a
  hard requirement despite the rule now being info/optional-framed;
  reworded to match the sibling info-severity rules' "should include" /
  informational-finding phrasing (en+ja). Frontmatter already used
  "should" wording, so no rules-index regen needed.
- packages/vite/test/integration.test.ts: fixed a stale comment claiming
  the fixture's description is missing — it isn't (title is the missing,
  critical-triggering field); the fixture and test behavior were already
  correct.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant