Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .changeset/health-report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
'@svelte-vitals/core': minor
'svelte-vitals': minor
---

Add the combined **Health Report** (#10): a single weighted Health score across the
SEO, Performance, and Accessibility categories (equal weights by default, overridable
via `Config.weights`), surfaced as the headline in the console/agent reporters and the
MCP `analyze` output, with an optional `--min-health <0-100>` CI gate.

**Breaking (JSON report):** the top-level `score` is now the combined Health score (it
was the SEO score); the top-level `scoreModel` is removed; a `weights` field is added.
Per-category scores remain under `categories` (e.g. `categories.seo.score` /
`categories.seo.scoreModel`).
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,10 +149,11 @@ The project advances along two axes: **mode maturity** and **category coverage**
- **MCP server** (`@svelte-vitals/mcp`) — exposes `analyze` and `explain_rule` tools over stdio so an agent can run analysis in its tool loop and receive structured, fixable findings.
- **Performance checks** (`0.4`) — static `<img>` analysis: `width`/`height` (CLS) and a `loading` advisory, scored as a separate Performance category alongside SEO.
- **Accessibility checks** (`0.5`) — aggregates the Svelte compiler's `a11y_*` warnings (alt text, label association, ARIA, …) into a scored Accessibility category.
- **Health Report** — a single weighted **Health** score combining SEO, Performance, and Accessibility (equal weights by default), shown as the headline in every reporter and the MCP `analyze` output; gate CI on it with `--min-health`.

**Upcoming**

- **More categories** ([#10](https://github.com/oekazuma/svelte-vitals/issues/10)) — Upgrade checks, then a combined weighted Health Report — landing across `0.x` ahead of the `1.0` polish.
- **Toward `1.0`** — rule-reference docs, a config file, and polish. The Upgrade/deprecation category was dropped (covered by official Svelte tooling — the compiler, the Svelte MCP, and `sv migrate`).

See the design document for the full vision.

Expand Down
562 changes: 562 additions & 0 deletions docs/superpowers/plans/2026-06-23-health-report.md

Large diffs are not rendered by default.

102 changes: 102 additions & 0 deletions docs/superpowers/specs/2026-06-23-health-report-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# Design: Combined weighted Health Report (1.0 capstone)

Issue: [#10](https://github.com/oekazuma/svelte-vitals/issues/10) (roadmap epic). The differentiated capstone: synthesize the per-category scores (SEO, Performance, A11y) into one weighted **Health** score, surfaced across reporters and the MCP `analyze` tool. This is the headline number for `1.0` and is svelte-vitals' own synthesis — not something official Svelte tooling provides.

## Goal

Add a single **Health** score = the weighted average of the present category scores, with **equal default weights** (1/3 each, re-normalized over whichever categories are present). Surface it as the top-level `score` in the JSON report, a console/agent headline, and (for free) the MCP `analyze` output. Add an optional `--min-health <0-100>` CI gate. Keep severity-based exit codes unchanged.

Reuses the multi-category foundation from Performance v0.4 (`scoresByCategory`, category-aware reporters). Static + plugin modes (the report is computed from `Result[]`, mode-independent).

## Decisions (settled in brainstorming)

- **Default weights: equal** (each present category weight 1). Overridable via `Config.weights`; a CLI flag is deferred to the config-file feature (roadmap C).
- **JSON top-level `score` becomes the Health score** (was the SEO score) — a deliberate breaking change, announced for `1.0` via the changeset. Per-category scores remain in `categories`.
- **Exit codes stay severity-based** (`hasFailureAtOrAbove`); Health is informational unless the new optional `--min-health` flag is set.

## core — `computeHealth` + `Config.weights`

- **`Config`** gains `weights?: Partial<Record<Category, number>>`. `defineConfig` passes it through; `defaultConfig` leaves it `undefined` (= equal). Effective weight: `weightOf(cat) = config.weights?.[cat] ?? 1`.
- **`computeHealth(results, config): HealthResult`** (new, in `packages/core/src/scoring/score.ts`):
- `const byCat = scoresByCategory(results, config)` — only categories with findings/seeds (so a suppressed a11y category, or a project with no routes, is excluded).
- `health = round( Σ_present(score_c × weightOf(c)) / Σ_present(weightOf(c)) )`. When no categories are present, `health = 100` (consistent with `computeScore`'s empty → 100).
- Returns `{ health: number; categories: Partial<Record<Category, ScoreResult>>; weights: Partial<Record<Category, number>> }` where `weights` is the **effective** weight actually used per present category (so the report can show how Health was derived).
- Exported from `@svelte-vitals/core`'s index alongside `computeScore`/`scoresByCategory`.

```ts
export interface HealthResult {
health: number;
categories: Partial<Record<Category, ScoreResult>>;
weights: Partial<Record<Category, number>>;
}
export function computeHealth(results: Result[], config: Config): HealthResult;
```

## core — JSON report reshape (`buildJsonReport`)

The report object changes shape (breaking, for 1.0):

- top-level **`score`** = `computeHealth(...).health` (was the SEO subset score).
- top-level **`scoreModel`** is **removed** (it was the SEO route-average model and is now redundant with `categories.seo.scoreModel`).
- add top-level **`weights`** = the effective per-category weights used for Health.
- **`categories`** unchanged: `{ seo: { score, scoreModel }, performance: {…}, a11y: {…} }` (only present categories).
- `summary`, `routes`, `siteIssues` unchanged.

Result shape:

```jsonc
{
"version": "1.0.0",
"score": 91, // Health (weighted avg of present categories)
"weights": { "seo": 1, "performance": 1, "a11y": 1 },
"categories": {
"seo": { "score": 86, "scoreModel": { … } },
"performance": { "score": 95, "scoreModel": { … } },
"a11y": { "score": 92, "scoreModel": { … } }
},
"summary": { … },
"routes": [ … ],
"siteIssues": [ … ]
}
```

`formatJsonReport` keeps stringifying `buildJsonReport`. The **MCP `analyze`** tool returns `buildJsonReport`'s object as `structuredContent`, so it surfaces `score`(=Health) + `weights` + `categories` with **no MCP code change**.

## core — reporters

- **console**: prepend a headline `Svelte Vitals · Health: N/100 (static mode)` (or a dedicated `Health: N/100` line at the very top), then the existing per-category score lines (`SEO Score:`, `Performance Score:`, `Accessibility Score:`), then findings. The per-category lines and findings are unchanged.
- **agent**: add a single leading line `Health: N/100` after the `# svelte-vitals — fixes` heading (cheap, informative for the agent).
- **sarif / github**: **unchanged** — a single overall score has no per-finding SARIF/annotation representation; these reporters keep emitting findings only.

## cli — `--min-health` gate + wiring

- **`bin.ts`**: parse `--min-health <n>` (string → number). Invalid or out-of-range value is a usage error — prints an error to stderr and exits 2 (consistent with how unknown `--reporter`/`--rules` ids are handled).
- **`RunOptions`/`AnalyzeOptions`** gain `minHealth?: number`; `analyzeProject` does not need it (it returns raw results), but `run()` computes Health and applies the gate.
- **`run()`** exit logic: compute `const { health } = computeHealth(results, config)`. Return `1` when `hasFailureAtOrAbove(summary, config.failOn)` **OR** (`opts.minHealth != null && health < opts.minHealth`). Exit `2` (execution error) unchanged. So `--min-health` adds a score gate on top of the existing severity gate; without it, behavior is unchanged.
- Help text documents `--min-health`.

> Weights are **not** a CLI flag in this increment (no `--weights`); they default to equal and are overridable programmatically via `defineConfig({ weights })`, with a config-file surface arriving in roadmap item C.

## Backward-compat & migration

- The JSON `score` semantic change (SEO → Health) and the removal of top-level `scoreModel` are breaking for JSON consumers. Per-category SEO data moves to `categories.seo`. Announce clearly in the changeset (this lands as the lead-in to `1.0`).
- console/agent gain a headline line (additive to human output; no existing console assertion pins the absence of a Health line — verify and update the console/agent tests for the new headline).
- sarif/github/exit-codes (default) unchanged.

## Testing (TDD)

- **core**: `computeHealth` — equal-weight mean of present categories; `Config.weights` override changes the result; a suppressed/absent category is excluded and weights re-normalize; no categories → 100; `weights` field reflects effective weights. `buildJsonReport` — `score` === `computeHealth().health`, top-level `scoreModel` gone, `weights` present, `categories` intact; `formatJsonReport` === `JSON.stringify(buildJsonReport)`.
- **cli**: `run()` returns 1 when `--min-health` threshold is unmet even with no failing severity; returns 0 when Health ≥ threshold and no failing severity; severity gate still fires independently; invalid `--min-health` exits 2 (usage error). console/agent show the Health headline.
- **mcp**: `analyze` `structuredContent.score` is the Health value and `weights` is present (the report flows through unchanged).
- Existing per-category/score tests updated for the reshaped top-level (the `categories` map already carries per-category data, so most assertions move from top-level to `categories.seo`).

## Roadmap / release

- README roadmap: move the **combined Health Report** to Shipped; explicitly drop the Upgrade category (redundant with official Svelte tooling — compiler/MCP/Skills/`sv migrate`); state that `1.0` is the polish/stabilization of SEO + Performance + A11y + Health.
- Changeset: `@svelte-vitals/core` **minor** (`computeHealth`, `Config.weights`, JSON reshape) and `svelte-vitals` **minor** (`--min-health`, console/agent headline). Call out the JSON `score`/`scoreModel` breaking change prominently.

## Non-goals / follow-ups (post-Health, toward/after 1.0)

- Rule-reference docs / fixing the `svelte-vitals.dev/rules/<id>` dead `docsUrl` links (roadmap item B — 1.0-required, next).
- Config-file support + `--weights` CLI flag (roadmap item C — 1.0-required).
- Upgrade/deprecation category (declined), plugin-mode Performance parity, layout breakouts (#12) — post-1.0.
16 changes: 14 additions & 2 deletions packages/cli/src/bin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Options:
--json Alias for --reporter=json
--fail-on <severity> Fail (exit 1) when any finding reaches this severity: critical | warning | info
--fail-on-warning Alias for --fail-on=warning
--min-health <0-100> Fail (exit 1) when the combined Health score is below this value
--rules <ids> Comma-separated rule ids to enable (all others disabled)
--ignore <ids> Comma-separated rule ids to disable
-h, --help Show this help
Expand All @@ -34,7 +35,7 @@ async function main(): Promise<void> {
const argv = mri(process.argv.slice(2), {
alias: { h: 'help', v: 'version' },
boolean: ['by-route', 'json', 'fail-on-warning'],
string: ['meta-components', 'treat-dynamic-as', 'route', 'fail-on', 'reporter', 'rules', 'ignore']
string: ['meta-components', 'treat-dynamic-as', 'route', 'fail-on', 'reporter', 'rules', 'ignore', 'min-health']
});

if (argv.help) {
Expand All @@ -51,7 +52,18 @@ async function main(): Promise<void> {
for (const e of errors) console.error(e);
if (!options) process.exit(2);

const code = await run(options);
const minHealthRaw = argv['min-health'];
let minHealth: number | undefined;
if (minHealthRaw !== undefined) {
const n = Number(minHealthRaw);
if (!Number.isFinite(n) || n < 0 || n > 100) {
console.error(`svelte-vitals: invalid --min-health '${minHealthRaw}'; expected a number 0-100.`);
process.exit(2);
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
minHealth = n;
}

const code = await run({ ...options, minHealth });
process.exit(code);
}

Expand Down
12 changes: 11 additions & 1 deletion packages/cli/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import {
formatGithubReport,
summarize,
hasFailureAtOrAbove,
computeHealth,
defineConfig,
selectRules,
applyRuleSeverities,
Expand Down Expand Up @@ -37,6 +38,8 @@ export interface RunOptions {
rules?: Record<string, RuleSetting>;
/** Override process.env for reporter auto-detection (mainly useful in tests). */
env?: NodeJS.ProcessEnv;
/** Fail (exit 1) when the combined Health score is below this value (0–100). */
minHealth?: number;
}

export function routeMatcher(glob: string | undefined): (route: string) => boolean {
Expand Down Expand Up @@ -107,6 +110,11 @@ export async function run(opts: RunOptions = {}): Promise<number> {
const log = opts.log ?? ((line: string) => console.log(line));
const errorLog = opts.errorLog ?? ((line: string) => console.error(line));

if (opts.minHealth != null && (!Number.isFinite(opts.minHealth) || opts.minHealth < 0 || opts.minHealth > 100)) {
errorLog(`svelte-vitals: invalid minHealth '${opts.minHealth}'; expected a number 0-100.`);
return 2;
}

let analysis: AnalyzeResult;
try {
analysis = await analyzeProject({
Expand Down Expand Up @@ -155,7 +163,9 @@ export async function run(opts: RunOptions = {}): Promise<number> {
log(formatConsoleReport(results, config, { byRoute: opts.byRoute ?? false }));
}
const summary = summarize(results, config);
return hasFailureAtOrAbove(summary, config.failOn) ? 1 : 0;
const failBySeverity = hasFailureAtOrAbove(summary, config.failOn);
const failByHealth = opts.minHealth != null && computeHealth(results, config).health < opts.minHealth;
return failBySeverity || failByHealth ? 1 : 0;
Comment thread
coderabbitai[bot] marked this conversation as resolved.
} catch (err) {
errorLog(`svelte-vitals: ${err instanceof Error ? err.message : String(err)}`);
return 2;
Expand Down
36 changes: 36 additions & 0 deletions packages/cli/test/run.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,42 @@ describe('run() performance rules', () => {
});
});

describe('run() --min-health validation', () => {
it('returns exit 2 for an out-of-range minHealth (150)', async () => {
const cap = capture();
const code = await run({ cwd: fixtureDir, minHealth: 150, log: cap.log, errorLog: cap.errorLog, env: CLEAN_ENV });
expect(code).toBe(2);
expect(cap.err.join('\n')).toContain('invalid minHealth');
});

it('returns exit 2 for a NaN minHealth', async () => {
const cap = capture();
const code = await run({ cwd: fixtureDir, minHealth: NaN, log: cap.log, errorLog: cap.errorLog, env: CLEAN_ENV });
expect(code).toBe(2);
expect(cap.err.join('\n')).toContain('invalid minHealth');
});
});

describe('run() --min-health gate', () => {
it('--min-health fails (exit 1) when Health is below the threshold', async () => {
const cap = capture();
// 100 is unreachable for the fixture (it has SEO failures), so the gate trips.
const code = await run({ cwd: fixtureDir, minHealth: 100, log: cap.log, errorLog: cap.errorLog, env: CLEAN_ENV });
expect(code).toBe(1);
});

it('--min-health passes (exit 0) when Health meets the threshold and no failing severity', async () => {
const cap = capture();
// 0 is always met; with default failOn=critical the fixture's criticals still gate,
// so use a project-less assertion: a threshold of 0 must not be the cause of a failure.
const code = await run({ cwd: fixtureDir, minHealth: 0, log: cap.log, errorLog: cap.errorLog, env: CLEAN_ENV });
// The fixture has a critical SEO finding, so severity still gates to 1; min-health=0 does not add a failure.
// Assert min-health=0 alone never forces 1 by comparing to the baseline (no minHealth).
const baseline = await run({ cwd: fixtureDir, log: capture().log, errorLog: capture().errorLog, env: CLEAN_ENV });
expect(code).toBe(baseline);
});
});

describe('run() accessibility rules', () => {
it('reports an Accessibility finding for an <img> without alt', async () => {
const cap = capture();
Expand Down
4 changes: 2 additions & 2 deletions packages/core/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -57,5 +57,5 @@ export { formatGithubReport } from './reporter/github.js';

export { selectRules, applyRuleSeverities } from './config-apply.js';

export type { ScoreModel, ScoreResult, ScoreOptions } from './scoring/score.js';
export { computeScore, scoresByCategory } from './scoring/score.js';
export type { ScoreModel, ScoreResult, ScoreOptions, HealthResult } from './scoring/score.js';
export { computeScore, scoresByCategory, computeHealth } from './scoring/score.js';
4 changes: 3 additions & 1 deletion packages/core/src/reporter/agent.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import type { Config, Result, Severity } from '../types.js';
import { classify, effectiveSeverity } from '../summary.js';
import { computeHealth } from '../scoring/score.js';

/** Sort order so the most severe findings (and the groups holding them) surface first. */
const SEVERITY_RANK: Record<Severity, number> = { critical: 0, warning: 1, info: 2 };
Expand All @@ -17,7 +18,8 @@ function mdTags(text: string): string {
/** Render failing findings as an agent-actionable Markdown remediation document (issue #18). */
export function formatAgentReport(results: Result[], config: Config): string {
const failing = results.filter((r) => classify(r, config) === 'fail');
const lines: string[] = ['# svelte-vitals — fixes', ''];
const { health } = computeHealth(results, config);
const lines: string[] = ['# svelte-vitals — fixes', '', `Health: ${health}/100`, ''];

if (failing.length === 0) {
lines.push('No issues to fix.', '');
Expand Down
6 changes: 3 additions & 3 deletions packages/core/src/reporter/console.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type { Category, Config, Result, Severity } from '../types.js';
import { classify, summarize, effectiveSeverity } from '../summary.js';
import { computeScore, scoresByCategory, type ScoreResult } from '../scoring/score.js';
import { computeScore, computeHealth, type ScoreResult } from '../scoring/score.js';

const RULE = '────────────────────────';
const SEVERITY_TITLE: Record<Severity, string> = {
Expand Down Expand Up @@ -56,9 +56,9 @@ function byRouteTree(results: Result[], config: Config): string[] {
*/
export function formatConsoleReport(results: Result[], config: Config, options: ConsoleReportOptions = {}): string {
const summary = summarize(results, config);
const byCat = scoresByCategory(results, config);
const { health, categories: byCat } = computeHealth(results, config);
const present = CATEGORY_ORDER.filter((c) => byCat[c] !== undefined);
Comment thread
oekazuma marked this conversation as resolved.
const header: string[] = [`Svelte Vitals · ${options.mode ?? 'static mode'}`, ''];
const header: string[] = [`Svelte Vitals · ${options.mode ?? 'static mode'}`, '', `Health: ${health}/100`];
for (const c of present) {
header.push(scoreLine(CATEGORY_LABEL[c] ?? c, byCat[c]!));
}
Expand Down
Loading