docs: update project direction based on ablation experiments#122
Merged
docs: update project direction based on ablation experiments#122
Conversation
- CLAUDE.md: rewrite Core Goal — from "can AI implement?" to "help teams implement with zero unnecessary AI cost". Add design-tree format spec. Update calibration guidance (large fixtures only, structure is dominant). - README.md: update tagline and Why section with experiment evidence. design-tree achieves 94% accuracy with 5× fewer tokens than raw JSON. - SCORING.md: rewrite with empirical data from ablation experiments. Add category impact table, update density/diversity formulas to reflect current calculatedScore-based weighting. Evidence: wiki experiments 01-04, issues #108 #117 #121 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughRefocuses the project on a "design-tree" first approach and zero/unnecessary-AI-token implementation, updates docs and scoring/calibration to reflect ablation results and token-aware metrics, disables two rules in rule-config, tweaks one test fixture, and adds Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
…store calibration docs - Disable unnecessary-node: false positives on icon wrappers and intentional layout wrappers - Disable prototype-link-in-design: REST API can't access prototype data (private beta) - SCORING.md: restore calibration loop docs alongside ablation experiments - Fix test that depended on unnecessary-node being enabled (use GROUP type instead) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 21
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/core/engine/rule-engine.test.ts (1)
165-190: 🧹 Nitpick | 🔵 TrivialKeep a positive control for the non-excluded sibling.
This only proves that
i:1disappears. IfexcludeNodeNamesstarts suppressing sibling nodes too, the test still passes. Assert thatn:1still produces issues in the excluded run.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/core/engine/rule-engine.test.ts` around lines 165 - 190, The test currently only asserts that issues for node "i:1" vanish when analyzeFile is called with excludeNodeNames, so add a positive control asserting that the sibling node "n:1" still produces issues in the excluded run: after calling result = analyzeFile(file, { excludeNodeNames: ["IgnoreMe"] }), filter result.issues for i => i.violation.nodeId === "n:1" and expect that length to be > 0; reference the makeNode-created nodes (ignoredNode / normalNode) and the analyzeFile function to locate where to insert this assertion.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CLAUDE.md`:
- Around line 236-241: Reword the release steps paragraph so it no longer
instructs pushing directly to main but instead clarifies that tagging should be
done only after the commit has been merged via the PR workflow: update the
bullet list (the four-step release steps block) to replace "Commit and push to
main" with language like "Merge the approved PR to main (or ensure the release
commit is already merged to main)," and change the heading sentence "Release
steps bypass the PR workflow below." to state that releases should be performed
by tagging the already-approved commit on main (e.g., "Create and push a git tag
for the already-merged commit on main; do not bypass the PR approval process"),
ensuring the PR Workflow section and the tagging/publishing steps are
consistent.
In `@docs/SCORING.md`:
- Around line 40-43: The fenced code block containing the formulas for
diversity_ratio and diversity_score needs an explicit language tag to satisfy
markdownlint; update the fence that contains "diversity_ratio = sum(|score| of
triggered rules) / sum(|score| of all category rules)" and "diversity_score = (1
- diversity_ratio) * 100" to use a language like text (e.g., ```text) so the
linter stops flagging it.
- Around line 38-45: Update the SCORING.md diversity section to match the
implementation in src/core/engine/scoring.ts: replace "weighted by severity"
with explicit wording that the diversity ratio uses the per-rule absolute scores
(sum of |score| of triggered rules divided by sum of |score| of all category
rules) as shown by the diversity_ratio/diversity_score formulas, and clarify
that rule score magnitudes (e.g., -10 vs -1) drive the weighting; reference the
diversity_ratio and diversity_score formulas so the prose aligns with the actual
scoring logic in scoring.ts.
In `@package.json`:
- Line 59: The dependency "@anthropic-ai/sdk" is only used in scripts/
experiment files and should be moved out of runtime dependencies; remove the
"@anthropic-ai/sdk" entry from the top-level "dependencies" object in
package.json and add the same version string under "devDependencies" instead,
then update the lockfile by running the project's package manager
(npm/yarn/pnpm) to reflect the change; confirm no production code imports it
(only scripts/) and that the "files" publish list still excludes scripts as
intended.
In `@scripts/run-ablation-experiment.ts`:
- Around line 101-102: The current write in scripts/run-ablation-experiment.ts
overwrites results.json (const output = { baseline: good, deltas, raw: results
}; writeFileSync(...)) and thus drops any existing visualCompare data from
scripts/run-visual-compare-experiment.ts; change to a merge-before-write: if a
RUN_DIR/results.json exists (use fs.existsSync/readFileSync), parse it, merge
its top-level fields into a new object but overwrite baseline/deltas/raw with
the new values, and then write the merged object back with
writeFileSync(JSON.stringify(...)); ensure you reference the existing variables
output, RUN_DIR and the results.json filename so only non-conflicting keys
(e.g., visualCompare) are preserved.
- Around line 60-64: The returned result objects from the
run-ablation-experiment flows are missing input token counts; update each return
to include inputTokens: response.usage.input_tokens (alongside thinkingTokens
and outputTokens) so the bad-token ablation can measure input-token
deltas—locate the return expressions that build the result object (the block
returning thinkingTokens, outputTokens, html and the similar returns around the
noted spots) and add inputTokens: response.usage.input_tokens to each.
In `@scripts/run-all-500.ts`:
- Around line 29-35: The current global regex in the test.strip375 branch
replaces every "width: 375px" in the whole document; change it to only target
the root element: parse the HTML (e.g., with cheerio/jsdom/parse5) using
htmlPath, locate the root element (<html> or the element with id/class used as
the root), update only its inline style or the specific root CSS rule to "width:
100%;" and serialize/write that output to fixedPath (keep variables
test.strip375, htmlPath, fixedPath, RUN_DIR unchanged); if you cannot guarantee
the root is the first match, do not use a global regex—explicitly select and
mutate the root node, and if this behavioral change alters documented
expectations, update the README/CLAUDE.md/JSDoc accordingly.
- Around line 10-18: The TESTS array currently hardcodes dated run directories
and embedded MCP percentages, causing stale comparisons; change TESTS to use
parameterized directory variables (e.g., baselineRunDir and currentRunDir
supplied via CLI args or environment vars) and replace any hardcoded MCP values
by reading the MCP numbers from the run artifacts (parse the result HTML/JSON
produced by the run) at runtime; update any code that references TESTS and
strip375 to construct file paths from these variables and remove literal
percentages so the comparison always uses the actual MCP values extracted from
artifacts (also apply same change to the other test list block around the second
array).
In `@scripts/run-input-comparison.ts`:
- Around line 55-84: runOne duplicates the response parsing (aggregating
thinkingText, extracting text blocks, stripping fenced code and trimming) which
causes drift; refactor by extracting that logic into a shared function (eg.
parseAnthropicResponse or normalizeResponse) that accepts the
finalMessage/response.content and response.usage and returns { thinkingTokens,
inputTokens, outputTokens, html }; replace the in-function parsing in runOne
with a call to that shared function and update the other script that currently
repeats the same logic to import and use the same helper so both paths share one
canonical parser.
In `@scripts/run-mcp-500.ts`:
- Around line 16-20: Check for the renderer build up front and fail fast instead
of serving a 404 page: before starting the HTTP server (where createServer is
used and distDir is referenced), verify dist/index.html exists by attempting to
readFile(join(distDir, "index.html")) and throw or exit if it fails;
additionally, update the request handler (the async function passed to
createServer) to return a 500 or close the server when the build is missing
rather than silently serving 404, and add a pre-screenshot validation step that
loads the index.html (or does an HTTP GET against the local server) and errors
out if the response is not 200 so screenshot/similarity logic never runs on the
error page.
In `@scripts/run-mcp-all-vite-v2.ts`:
- Line 10: Replace the hard-coded MCP_DIR constant with a parameterized value:
read from an environment variable (e.g. process.env.MCP_DIR) or accept a CLI
argument (e.g. parse process.argv) and fall back to a sensible repo-relative
default; update the MCP_DIR symbol usage accordingly so CI and other developers
can override the path without committing personal filesystem locations.
- Around line 44-46: The current logic using matches =
[...code.matchAll(/function\s+(\w+)/g)] and defaulting rootName to "App" causes
infinite recursion when the code only contains arrow function components; change
rootName resolution to first try the last named function from
matchAll(/function\s+(\w+)/g), then fall back to the last named arrow/const
component via a regex like /(?:const|let|var)\s+(\w+)\s*=\s*.*=>/ (search the
code for such matches), and if neither yields a name pick a safe wrapper name
that does not collide with existing identifiers (e.g. "AppWrapper" or generate a
unique name by checking code.includes(name)). Update the code that builds
appCode (the export default function App {...}) to use this resolved rootName
symbol so the wrapper does not render itself.
In `@scripts/run-mcp-all-vite.ts`:
- Around line 10-12: The hardcoded machine-local path in the constant
MCP_RESULTS_DIR will break CI and other devs; change MCP_RESULTS_DIR to read
from an environment variable or CLI flag (e.g., process.env.MCP_RESULTS_DIR or a
parsed --mcp-results option) with a sensible fallback that resolves a
repo-relative path using process.cwd() or __dirname; update any code that uses
MCP_RESULTS_DIR (and keep RUN_DIR and RENDERER_DIR behavior intact) so the
script fails with a clear error if neither env/flag nor repo-relative path
exists.
- Around line 29-52: The extractCode function currently parses toolResultPath
into an any[] and assumes each block has .type and .text; replace that with a
Zod-validated schema (e.g., BlockSchema = z.object({ type: z.string(), text:
z.string().optional() }) and an array schema) and use z.infer<typeof
BlockArraySchema> to type the parsed value, then validate JSON.parse(raw)
against this schema before iterating; if validation fails, fall back to
returning raw (or handle the error) and ensure when concatenating blocks you
only append block.text when it exists to avoid appending undefined.
- Around line 55-63: findRootComponent currently returns the hardcoded "App"
when its regex fails, causing wrong rendered component; change findRootComponent
(the function in scripts/run-mcp-all-vite.ts) to return null instead of "App" on
no match and update callers (e.g., buildAndRender which receives rootComponent)
to use the returned value with the nullish coalescing operator (foundRoot ??
rootComponent) so the explicit rootComponent passed per fixture (like
ExamplesProductDetailPage or BadNaming) is used as the fallback; keep the
existing regex logic but ensure arrow/other export patterns can still fall
through to null.
In `@scripts/run-mcp-no375-500.ts`:
- Around line 15-20: The current createServer handler concatenates req.url into
join("/tmp/mcp-renderer/dist", ...) allowing path traversal; change the request
handling in the createServer callback so you first compute a fixed baseDir
("/tmp/mcp-renderer/dist"), parse and sanitize req.url, resolve the requested
path with path.resolve(baseDir, decodeURIComponent(req.url || "/")), then verify
the resolved path startsWith the baseDir (reject with 403 or 404 if not); only
then readFile the resolved path and serve with MIME[extname(...)]; also bind the
server to 127.0.0.1 in the srv.listen call instead of all interfaces; apply the
same fix to the other duplicated run-mcp-* scripts mentioned.
In `@scripts/run-mcp-vite-compare.ts`:
- Around line 19-31: The temporary HTTP handler using createServer should
validate and constrain requested paths to DIST_DIR and bind the server to
loopback: parse and sanitize the request URL (e.g., via new URL(req.url,
'http://localhost')), decode and normalize the pathname with path.resolve
against DIST_DIR, then verify the resolved path starts with the resolved
DIST_DIR value before calling readFile; if it does not, return a 403/Not allowed
response instead of reading a file. Also change server.listen(4173, ...) to bind
explicitly to 127.0.0.1 (e.g., server.listen(4173, '127.0.0.1', ...)). Refer to
the filePath variable, DIST_DIR, readFile, MIME, createServer and server.listen
when making these changes.
In `@scripts/run-raw-all.ts`:
- Around line 46-49: The file currently hardcodes a stale baseline row into the
results object (const results ... "good": {...}) which must be recomputed each
run; remove the baked-in "good" entry and instead compute it via the same
measurement flow used for other rows (i.e., invoke the existing measurement
logic/functions that produce thinkingTokens/inputTokens/outputTokens/similarity
and insert that result into the results Record<string, any>). Ensure the
computed entry uses the same keys ("thinkingTokens", "inputTokens",
"outputTokens", "similarity") and is added to the results object rather than a
hardcoded literal.
In `@scripts/run-raw-v2.ts`:
- Around line 65-76: After stripping fences from the model response (the html
variable in scripts/run-raw-v2.ts), validate the final string using a Zod schema
so non-HTML or empty responses fail fast: add import { z } from "zod", define
generatedHtmlSchema (a trimmed, min(1) string that refines with a regex like
/<\s*(html|body|div|main|section|style)\b/i to assert it contains HTML) near the
existing constants, and replace the final html = html.trim() with html =
generatedHtmlSchema.parse(html.trim()) so refusals/blank outputs throw and are
handled as generation failures.
In `@scripts/run-viewport-test.ts`:
- Around line 106-113: The table output in the console uses hard-coded 375px
percentages while the script only computes 500px scores—update the printing
logic around the console.error calls (the lines that reference results and print
the markdown table) to avoid synthetic `@375px` values: either load the real 375px
scores from the existing results artifact (e.g., results["design-tree-good-375"]
etc.) or measure both widths in this script and populate results with both keys
before printing; replace the fixed numbers (94%, 84%, 79%, 55%, 94%) with the
corresponding results lookups and ensure the printed keys match the stored
result property names.
In `@src/core/rules/rule-config.ts`:
- Around line 94-97: The strict preset application
(getConfigsWithPreset("strict")) currently forces enabled: true on every rule
and thus re-enables rules explicitly disabled (e.g., "unnecessary-node"); update
the merging logic inside getConfigsWithPreset (and the related preset-merge code
referenced around the other block) so that when applying the "strict" preset you
do not overwrite an existing enabled: false on a rule — merge by preserving an
explicit enabled flag from the target config and only set enabled: true for
rules that do not already have an explicit enabled value (or explicitly check
for enabled === undefined before setting); ensure this change touches both
places noted (the main strict merge and the secondary merge around the other
block).
---
Outside diff comments:
In `@src/core/engine/rule-engine.test.ts`:
- Around line 165-190: The test currently only asserts that issues for node
"i:1" vanish when analyzeFile is called with excludeNodeNames, so add a positive
control asserting that the sibling node "n:1" still produces issues in the
excluded run: after calling result = analyzeFile(file, { excludeNodeNames:
["IgnoreMe"] }), filter result.issues for i => i.violation.nodeId === "n:1" and
expect that length to be > 0; reference the makeNode-created nodes (ignoredNode
/ normalNode) and the analyzeFile function to locate where to insert this
assertion.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 11405da5-4708-4f62-b92f-cb4de56d5aa8
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml,!pnpm-lock.yaml
📒 Files selected for processing (24)
CLAUDE.mdREADME.mddocs/SCORING.mdpackage.jsonscripts/run-ablation-experiment.tsscripts/run-all-500.tsscripts/run-dt-no375.tsscripts/run-input-comparison.tsscripts/run-mcp-500-fix.tsscripts/run-mcp-500.tsscripts/run-mcp-all-vite-v2.tsscripts/run-mcp-all-vite.tsscripts/run-mcp-all375-500.tsscripts/run-mcp-bad-structure-500.tsscripts/run-mcp-comparison.tsscripts/run-mcp-direct.tsscripts/run-mcp-no375-500.tsscripts/run-mcp-vite-compare.tsscripts/run-raw-all.tsscripts/run-raw-v2.tsscripts/run-viewport-test.tsscripts/run-visual-compare-experiment.tssrc/core/engine/rule-engine.test.tssrc/core/rules/rule-config.ts
scripts/run-ablation-experiment.ts
Outdated
Comment on lines
+60
to
+64
| return { | ||
| thinkingTokens: Math.ceil(thinkingText.length / 4), | ||
| outputTokens: response.usage.output_tokens, | ||
| html, | ||
| }; |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's find and examine the target file
fd "run-ablation-experiment.ts" --type fRepository: let-sunny/canicode
Length of output: 96
🏁 Script executed:
# Read the relevant file
cat -n scripts/run-ablation-experiment.tsRepository: let-sunny/canicode
Length of output: 4665
🏁 Script executed:
# Check the other comparison scripts mentioned in the review
fd "run-mcp-comparison.ts" --type f
fd "run-raw-all.ts" --type fRepository: let-sunny/canicode
Length of output: 114
🏁 Script executed:
# Check run-mcp-comparison.ts for input token handling
cat -n scripts/run-mcp-comparison.ts | head -150Repository: let-sunny/canicode
Length of output: 4628
🏁 Script executed:
# Check run-raw-all.ts for input token handling
cat -n scripts/run-raw-all.ts | head -150Repository: let-sunny/canicode
Length of output: 5606
Capture inputTokens for the bad-token ablation.
This script returns only thinkingTokens and outputTokens, missing response.usage.input_tokens. The ablation cannot measure input-token deltas, which is critical for the token-heavy fixtures. Other scripts in this directory—scripts/run-mcp-comparison.ts and scripts/run-raw-all.ts—already capture input tokens correctly.
Suggested shape update
-async function runOne(name: string, client: Anthropic): Promise<{thinkingTokens: number, outputTokens: number, html: string}> {
+async function runOne(name: string, client: Anthropic): Promise<{ thinkingTokens: number; inputTokens: number; outputTokens: number; html: string }> {
@@
return {
thinkingTokens: Math.ceil(thinkingText.length / 4),
+ inputTokens: response.usage.input_tokens,
outputTokens: response.usage.output_tokens,
html,
};
}
@@
results[name] = {
thinkingTokens: result.thinkingTokens,
+ inputTokens: result.inputTokens,
outputTokens: result.outputTokens,
};
@@
deltas[name] = {
...r,
+ deltaI: r.inputTokens - good.inputTokens,
deltaO: r.outputTokens - good.outputTokens,
deltaO_pct: Math.round((r.outputTokens - good.outputTokens) / good.outputTokens * 100),
};Also applies to: 82-85, 94-98, 104-106
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/run-ablation-experiment.ts` around lines 60 - 64, The returned result
objects from the run-ablation-experiment flows are missing input token counts;
update each return to include inputTokens: response.usage.input_tokens
(alongside thinkingTokens and outputTokens) so the bad-token ablation can
measure input-token deltas—locate the return expressions that build the result
object (the block returning thinkingTokens, outputTokens, html and the similar
returns around the noted spots) and add inputTokens: response.usage.input_tokens
to each.
scripts/run-mcp-vite-compare.ts
Outdated
Comment on lines
+19
to
+31
| const server = createServer(async (req, res) => { | ||
| const filePath = join(DIST_DIR, req.url === "/" ? "index.html" : req.url!); | ||
| try { | ||
| const data = await readFile(filePath); | ||
| res.writeHead(200, { "Content-Type": MIME[extname(filePath)] || "application/octet-stream" }); | ||
| res.end(data); | ||
| } catch { | ||
| res.writeHead(404); | ||
| res.end("Not found"); | ||
| } | ||
| }); | ||
|
|
||
| await new Promise<void>(r => server.listen(4173, r)); |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n scripts/run-mcp-vite-compare.tsRepository: let-sunny/canicode
Length of output: 3324
Constrain the temporary server to DIST_DIR.
Path traversal is possible: join(DIST_DIR, req.url) normalizes but does not restrict path segments, allowing .. to escape into parent directories. The server binding also exposes the port to all interfaces instead of loopback; listen(4173) without specifying a host defaults to all IPv4/IPv6 addresses.
Normalize the pathname using resolve(), validate the resolved file stays under DIST_DIR, and bind explicitly to 127.0.0.1.
Suggested hardening
+ const distRoot = resolve(DIST_DIR);
const server = createServer(async (req, res) => {
- const filePath = join(DIST_DIR, req.url === "/" ? "index.html" : req.url!);
+ const pathname = new URL(req.url ?? "/", "http://127.0.0.1").pathname;
+ const relativePath = pathname === "/" ? "index.html" : pathname.slice(1);
+ const filePath = resolve(distRoot, relativePath);
+ if (!filePath.startsWith(`${distRoot}/`)) {
+ res.writeHead(403);
+ res.end("Forbidden");
+ return;
+ }
try {
const data = await readFile(filePath);
res.writeHead(200, { "Content-Type": MIME[extname(filePath)] || "application/octet-stream" });
res.end(data);
} catch {
res.writeHead(404);
res.end("Not found");
}
});
- await new Promise<void>(r => server.listen(4173, r));
+ await new Promise<void>(r => server.listen(4173, "127.0.0.1", r));🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/run-mcp-vite-compare.ts` around lines 19 - 31, The temporary HTTP
handler using createServer should validate and constrain requested paths to
DIST_DIR and bind the server to loopback: parse and sanitize the request URL
(e.g., via new URL(req.url, 'http://localhost')), decode and normalize the
pathname with path.resolve against DIST_DIR, then verify the resolved path
starts with the resolved DIST_DIR value before calling readFile; if it does not,
return a 403/Not allowed response instead of reading a file. Also change
server.listen(4173, ...) to bind explicitly to 127.0.0.1 (e.g.,
server.listen(4173, '127.0.0.1', ...)). Refer to the filePath variable,
DIST_DIR, readFile, MIME, createServer and server.listen when making these
changes.
scripts/run-raw-all.ts
Outdated
Comment on lines
+46
to
+49
| // Skip good — already done (thinking=615, input=58898, output=10944, similarity=78%) | ||
| const results: Record<string, any> = { | ||
| "good": { thinkingTokens: 615, inputTokens: 58898, outputTokens: 10944, similarity: 78 }, | ||
| }; |
Contributor
There was a problem hiding this comment.
Recompute good in this run instead of baking in an old row.
This results.json mixes one hand-entered baseline with newly generated measurements. If the prompt, model, or fixture changes, good is no longer comparable to the other rows and this run will not contain matching HTML/diff artifacts for it.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/run-raw-all.ts` around lines 46 - 49, The file currently hardcodes a
stale baseline row into the results object (const results ... "good": {...})
which must be recomputed each run; remove the baked-in "good" entry and instead
compute it via the same measurement flow used for other rows (i.e., invoke the
existing measurement logic/functions that produce
thinkingTokens/inputTokens/outputTokens/similarity and insert that result into
the results Record<string, any>). Ensure the computed entry uses the same keys
("thinkingTokens", "inputTokens", "outputTokens", "similarity") and is added to
the results object rather than a hardcoded literal.
scripts/run-raw-v2.ts
Outdated
Comment on lines
+65
to
+76
| let thinkingText = ""; | ||
| let html = ""; | ||
| for (const block of response.content) { | ||
| if (block.type === "thinking") thinkingText += block.thinking; | ||
| else if (block.type === "text") html += block.text; | ||
| } | ||
| html = html.trim(); | ||
| if (html.startsWith("```html")) html = html.slice(7); | ||
| else if (html.startsWith("```")) html = html.slice(3); | ||
| if (html.endsWith("```")) html = html.slice(0, -3); | ||
| html = html.trim(); | ||
|
|
Contributor
There was a problem hiding this comment.
Fail fast when the model does not return HTML.
After fence stripping, any remaining text is treated as a valid sample. A refusal or blank response will be written to disk and scored as a low-similarity run instead of being marked as a generation failure.
🛡️ Minimal fix
if (html.endsWith("```")) html = html.slice(0, -3);
- html = html.trim();
+ html = generatedHtmlSchema.parse(html.trim());import { z } from "zod";
const generatedHtmlSchema = z
.string()
.trim()
.min(1)
.refine((value) => /<\s*(html|body|div|main|section|style)\b/i.test(value), "Model did not return HTML");Add import { z } from "zod"; near the existing imports and define generatedHtmlSchema next to the constants. As per coding guidelines, "Validate all external inputs with Zod schemas" and "Use Zod for validation".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/run-raw-v2.ts` around lines 65 - 76, After stripping fences from the
model response (the html variable in scripts/run-raw-v2.ts), validate the final
string using a Zod schema so non-HTML or empty responses fail fast: add import {
z } from "zod", define generatedHtmlSchema (a trimmed, min(1) string that
refines with a regex like /<\s*(html|body|div|main|section|style)\b/i to assert
it contains HTML) near the existing constants, and replace the final html =
html.trim() with html = generatedHtmlSchema.parse(html.trim()) so refusals/blank
outputs throw and are handled as generation failures.
scripts/run-viewport-test.ts
Outdated
Comment on lines
+106
to
+113
| console.error("\n=== 500px Viewport Test ==="); | ||
| console.error("| Method | @375px | @500px |"); | ||
| console.error("|---|---|---|"); | ||
| console.error(`| design-tree good | 94% | ${results["design-tree-good"]}% |`); | ||
| console.error(`| design-tree bad-structure | 84% | ${results["design-tree-bad-structure"]}% |`); | ||
| console.error(`| figma-raw good | 79% | ${results["figma-raw-good"]}% |`); | ||
| console.error(`| figma-raw bad-structure | 55% | ${results["figma-raw-bad-structure"]}% |`); | ||
| console.error(`| MCP Vite good | 94% | ${results["mcp-vite-good"]}% |`); |
Contributor
There was a problem hiding this comment.
Don't print a synthetic @375px column as current-run output.
This script only computes 500px scores, but the report mixes those fresh values with hard-coded 375px numbers. After any fixture, prompt, or model change, that table becomes stale evidence. Load the 375px values from a results artifact or measure both widths here.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/run-viewport-test.ts` around lines 106 - 113, The table output in the
console uses hard-coded 375px percentages while the script only computes 500px
scores—update the printing logic around the console.error calls (the lines that
reference results and print the markdown table) to avoid synthetic `@375px`
values: either load the real 375px scores from the existing results artifact
(e.g., results["design-tree-good-375"] etc.) or measure both widths in this
script and populate results with both keys before printing; replace the fixed
numbers (94%, 84%, 79%, 55%, 94%) with the corresponding results lookups and
ensure the printed keys match the stored result property names.
- CLAUDE.md: deployment steps — don't bypass PR workflow, tag merged commit - SCORING.md: diversity uses per-rule score magnitude, not severity weight - SCORING.md: add language tag to fenced code block - scripts/run-mcp-all-vite-v2.ts: fix infinite recursion when MCP code has only arrow functions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove 18 one-off experiment scripts (hardcoded paths, not reusable) - Move @anthropic-ai/sdk to devDependencies (experiment use only) - Fix strict preset re-enabling disabled rules (respect enabled: false) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CLAUDE.md`:
- Around line 184-190: The fenced code block containing the hero-section example
lacks a language tag and triggers markdownlint MD040; update that fenced block
by adding the language specifier "text" after the opening backticks (i.e.,
change the opening "```" to "```text" for the block that begins with "Hero
Section (FRAME, 375x960) ...") so the markdown linter recognizes it as plain
text.
In `@docs/SCORING.md`:
- Around line 112-118: Add a short clarifying sentence below the JSON sample in
docs/SCORING.md stating that the values (e.g., "no-auto-layout": { "score": -15
}) are illustrative override examples and not the current defaults, and
reference where the real defaults live (rule-config.ts) so readers know to
consult that file for canonical values; update the text near the JSON block that
shows "no-auto-layout" to include this note.
In `@scripts/run-mcp-all-vite-v2.ts`:
- Around line 85-86: Wrap the cleanup of browser and srv in a try/finally so the
HTTP server is always closed even if browser.close() throws: surround the await
browser.close() call with try { await browser.close(); } finally { await
srv.close(); } (if srv.close is callback-based, convert it to a Promise or use
util.promisify) so both resources are reliably released; update the code around
browser.close() and srv.close() accordingly.
- Around line 103-111: The loop printing the 3-way comparison indexes FIXTURES
against hardcoded baseline maps dt and raw, but accesses dt[name] and raw[name]
without guarding for missing keys (symbols: dt, raw, FIXTURES, results), causing
"undefined%" if a new fixture is added; fix by either deriving dt and raw
entries from the same source as FIXTURES or add a safe lookup when formatting
the table (e.g., compute baselineDt = dt[name] ?? "N/A" or a numeric default and
baselineRaw = raw[name] ?? "N/A" before interpolating, or skip/flag fixtures
missing baseline data) so undefined values are never printed.
- Around line 23-30: The extractCode function currently calls JSON.parse and
iterates over data assuming it's an array of objects with a type/text shape;
wrap the JSON.parse in a try/catch to handle parsing errors and return a clear
fallback or rethrow a descriptive error, then validate that the parsed value
(data) is an Array before iterating, and within the loop check each item (the
variable b) is an object and has the expected properties (b.type === "text" and
typeof b.text === "string")—skip or log invalid entries rather than letting
exceptions bubble up; ensure the trimming/substring logic (idx, lb) still
operates on a safe string and return a predictable empty string on malformed
input.
- Around line 63-68: The current random port selection (const port = 4200 +
Math.floor(Math.random()*100)) risks collisions; change to let the OS pick an
available port by binding to port 0 and then read the actual assigned port from
the server instance (srv.address() or the result of server.listen callback)
before printing/using it; update any logic that references the `port` variable
to use the runtime-assigned value and keep references to `srv` and
`createServer` as the server object to perform this binding.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 0dc1dd06-74b1-47f8-a000-a1d746a61bd0
📒 Files selected for processing (3)
CLAUDE.mddocs/SCORING.mdscripts/run-mcp-all-vite-v2.ts
Comment on lines
112
to
118
| ```json | ||
| { | ||
| "rules": { | ||
| "no-auto-layout": { "score": 20, "severity": "blocking" }, | ||
| "no-auto-layout": { "score": -15, "severity": "blocking" }, | ||
| "raw-color": { "enabled": false } | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Config override example is illustrative, not a default reference.
The example shows "score": -15 for no-auto-layout, while the actual default in rule-config.ts is -10. This is acceptable since the section demonstrates override syntax, but consider adding a note that these are example values, not current defaults.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/SCORING.md` around lines 112 - 118, Add a short clarifying sentence
below the JSON sample in docs/SCORING.md stating that the values (e.g.,
"no-auto-layout": { "score": -15 }) are illustrative override examples and not
the current defaults, and reference where the real defaults live
(rule-config.ts) so readers know to consult that file for canonical values;
update the text near the JSON block that shows "no-auto-layout" to include this
note.
scripts/run-mcp-all-vite-v2.ts
Outdated
Comment on lines
+23
to
+30
| function extractCode(path: string): string { | ||
| const data = JSON.parse(readFileSync(path, "utf-8")); | ||
| let code = ""; | ||
| for (const b of data) { if (b.type === "text") code += b.text; } | ||
| const idx = code.indexOf("SUPER CRITICAL"); | ||
| if (idx > 0) { code = code.substring(0, idx); const lb = code.lastIndexOf("}"); if (lb > 0) code = code.substring(0, lb+1); } | ||
| return code.trim(); | ||
| } |
Contributor
There was a problem hiding this comment.
Validate external JSON input and handle malformed data.
JSON.parse can throw, and the parsed data is used without type validation. The iteration assumes data is an array with objects containing a type property, but malformed tool-results JSON will cause cryptic runtime errors.
Suggested defensive approach
+import { z } from "zod";
+
+const McpBlockSchema = z.array(z.object({ type: z.string(), text: z.string().optional() }));
+
function extractCode(path: string): string {
- const data = JSON.parse(readFileSync(path, "utf-8"));
+ let data: unknown;
+ try {
+ data = JSON.parse(readFileSync(path, "utf-8"));
+ } catch {
+ throw new Error(`Failed to parse JSON from ${path}`);
+ }
+ const blocks = McpBlockSchema.parse(data);
let code = "";
- for (const b of data) { if (b.type === "text") code += b.text; }
+ for (const b of blocks) { if (b.type === "text" && b.text) code += b.text; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/run-mcp-all-vite-v2.ts` around lines 23 - 30, The extractCode
function currently calls JSON.parse and iterates over data assuming it's an
array of objects with a type/text shape; wrap the JSON.parse in a try/catch to
handle parsing errors and return a clear fallback or rethrow a descriptive
error, then validate that the parsed value (data) is an Array before iterating,
and within the loop check each item (the variable b) is an object and has the
expected properties (b.type === "text" and typeof b.text === "string")—skip or
log invalid entries rather than letting exceptions bubble up; ensure the
trimming/substring logic (idx, lb) still operates on a safe string and return a
predictable empty string on malformed input.
scripts/run-mcp-all-vite-v2.ts
Outdated
Comment on lines
+63
to
+68
| const port = 4200 + Math.floor(Math.random()*100); | ||
| const srv = createServer(async (req: IncomingMessage, res: ServerResponse) => { | ||
| const fp = join(RENDERER, "dist", req.url === "/" ? "index.html" : req.url!); | ||
| try { const d = await readFile(fp); res.writeHead(200, {"Content-Type": MIME[extname(fp)]||"application/octet-stream"}); res.end(d); } | ||
| catch { res.writeHead(404); res.end(); } | ||
| }); |
Contributor
There was a problem hiding this comment.
Random port selection risks collision on concurrent runs.
The port range 4200 + Math.floor(Math.random()*100) is narrow. If multiple script instances run concurrently (e.g., parallel CI jobs), they may bind to the same port.
Suggested fix using port 0 for OS assignment
- const port = 4200 + Math.floor(Math.random()*100);
const srv = createServer(async (req: IncomingMessage, res: ServerResponse) => {
const fp = join(RENDERER, "dist", req.url === "/" ? "index.html" : req.url!);
try { const d = await readFile(fp); res.writeHead(200, {"Content-Type": MIME[extname(fp)]||"application/octet-stream"}); res.end(d); }
catch { res.writeHead(404); res.end(); }
});
- await new Promise<void>(r => srv.listen(port, r));
+ await new Promise<void>(r => srv.listen(0, r));
+ const port = (srv.address() as { port: number }).port;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/run-mcp-all-vite-v2.ts` around lines 63 - 68, The current random port
selection (const port = 4200 + Math.floor(Math.random()*100)) risks collisions;
change to let the OS pick an available port by binding to port 0 and then read
the actual assigned port from the server instance (srv.address() or the result
of server.listen callback) before printing/using it; update any logic that
references the `port` variable to use the runtime-assigned value and keep
references to `srv` and `createServer` as the server object to perform this
binding.
scripts/run-mcp-all-vite-v2.ts
Outdated
Comment on lines
+85
to
+86
| await browser.close(); | ||
| srv.close(); |
Contributor
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Resource cleanup not wrapped in try/finally.
If browser.close() throws, the HTTP server remains bound. For a one-off experiment script this is tolerable, but wrapping cleanup in try/finally prevents port leaks during debugging.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/run-mcp-all-vite-v2.ts` around lines 85 - 86, Wrap the cleanup of
browser and srv in a try/finally so the HTTP server is always closed even if
browser.close() throws: surround the await browser.close() call with try { await
browser.close(); } finally { await srv.close(); } (if srv.close is
callback-based, convert it to a Promise or use util.promisify) so both resources
are reliably released; update the code around browser.close() and srv.close()
accordingly.
scripts/run-mcp-all-vite-v2.ts
Outdated
Comment on lines
+103
to
+111
| const dt: Record<string,number> = { good:94, "bad-structure":84, "bad-token":90, "bad-component":91, "bad-naming":94, "bad-behavior":94 }; | ||
| const raw: Record<string,number> = { good:78, "bad-structure":3, "bad-token":69, "bad-component":66, "bad-naming":78, "bad-behavior":80 }; | ||
|
|
||
| console.error("\n=== 3-way comparison ==="); | ||
| console.error("| Fixture | MCP Vite | design-tree | figma-raw |"); | ||
| console.error("|---|---|---|---|"); | ||
| for (const name of Object.keys(FIXTURES)) { | ||
| console.error(`| ${name} | ${results[name]}% | ${dt[name]}% | ${raw[name]}% |`); | ||
| } |
Contributor
There was a problem hiding this comment.
Baseline lookups lack undefined checks per noUncheckedIndexedAccess.
If FIXTURES gains a new key, dt[name] and raw[name] will be undefined, producing undefined% in the output table. Either guard the access or derive the baseline maps from the same source of truth as FIXTURES.
Suggested guard
for (const name of Object.keys(FIXTURES)) {
- console.error(`| ${name} | ${results[name]}% | ${dt[name]}% | ${raw[name]}% |`);
+ const dtVal = dt[name] ?? "N/A";
+ const rawVal = raw[name] ?? "N/A";
+ console.error(`| ${name} | ${results[name]}% | ${dtVal}% | ${rawVal}% |`);
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| const dt: Record<string,number> = { good:94, "bad-structure":84, "bad-token":90, "bad-component":91, "bad-naming":94, "bad-behavior":94 }; | |
| const raw: Record<string,number> = { good:78, "bad-structure":3, "bad-token":69, "bad-component":66, "bad-naming":78, "bad-behavior":80 }; | |
| console.error("\n=== 3-way comparison ==="); | |
| console.error("| Fixture | MCP Vite | design-tree | figma-raw |"); | |
| console.error("|---|---|---|---|"); | |
| for (const name of Object.keys(FIXTURES)) { | |
| console.error(`| ${name} | ${results[name]}% | ${dt[name]}% | ${raw[name]}% |`); | |
| } | |
| const dt: Record<string,number> = { good:94, "bad-structure":84, "bad-token":90, "bad-component":91, "bad-naming":94, "bad-behavior":94 }; | |
| const raw: Record<string,number> = { good:78, "bad-structure":3, "bad-token":69, "bad-component":66, "bad-naming":78, "bad-behavior":80 }; | |
| console.error("\n=== 3-way comparison ==="); | |
| console.error("| Fixture | MCP Vite | design-tree | figma-raw |"); | |
| console.error("|---|---|---|---|"); | |
| for (const name of Object.keys(FIXTURES)) { | |
| const dtVal = dt[name] ?? "N/A"; | |
| const rawVal = raw[name] ?? "N/A"; | |
| console.error(`| ${name} | ${results[name]}% | ${dtVal}% | ${rawVal}% |`); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/run-mcp-all-vite-v2.ts` around lines 103 - 111, The loop printing the
3-way comparison indexes FIXTURES against hardcoded baseline maps dt and raw,
but accesses dt[name] and raw[name] without guarding for missing keys (symbols:
dt, raw, FIXTURES, results), causing "undefined%" if a new fixture is added; fix
by either deriving dt and raw entries from the same source as FIXTURES or add a
safe lookup when formatting the table (e.g., compute baselineDt = dt[name] ??
"N/A" or a numeric default and baselineRaw = raw[name] ?? "N/A" before
interpolating, or skip/flag fixtures missing baseline data) so undefined values
are never printed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CLAUDE.md`:
- Line 25: Update the sentence asserting "Structure (auto-layout) is the most
impactful category" to accurately reflect the data in rule-config: either change
it to state that the "no-auto-layout" rule is the single most impactful rule
(referencing the rule name no-auto-layout) or rephrase to indicate that the
structure category shows the highest average only marginally above component
(referencing the structure and component averages from rule-config), so the
claim becomes: "the no-auto-layout rule is the single highest-impact rule;
structure's category-level advantage is marginal."
- Around line 7-11: Update the unsupported empirical claims in CLAUDE.md: either
add verifiable ablation experiment artifacts (e.g., a new ablation-results
document and raw data + summary tables) that back the specific numbers "94%
pixel accuracy", "5× token reduction", and "95%→63% viewport drop" and link them
from the doc, or change those sentences to hedged language like "early evidence
suggests" and remove the exact percentages; also ensure the repository's
calibration/scoring documentation references or links to the new ablation
evidence so readers can verify the claims.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: e3d03b87-21f8-4272-adcf-723efbe3bcba
📒 Files selected for processing (1)
CLAUDE.md
- CLAUDE.md: remove specific percentages from Core Goal (data in wiki, not repo). Change "early evidence suggests" instead of asserting exact numbers. - CLAUDE.md: "structure is most impactful category" → "no-auto-layout is single highest-impact rule" (category averages are close to component) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CLAUDE.md`:
- Around line 184-206: Update the design-tree spec examples and wording to match
the actual renderer: replace the invalid color example `#2C` with a valid hex
(e.g., `#2C2C2C`), change the component annotation example from `[component:
Key=Value, ...]` to the actual emitted form `[component: <component-name>]`, and
correct the statement about internal node sizing to say nodes use `width: 100%`
only when `layoutSizingHorizontal === "FILL"` (i.e., reflect the conditional
behavior). Also mention the renderer token format exactly as shown in the code
and reference the `layoutSizingHorizontal` variable to help locate the
implementation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: e4173ba3-3bd6-4c17-8f56-485af132cc6b
⛔ Files ignored due to path filters (33)
fixtures/ablation-large-good/data.jsonis excluded by!fixtures/**fixtures/ablation-large-good/images/image@2x.pngis excluded by!**/*.png,!fixtures/**,!**/*.pngfixtures/ablation-large-good/images/mapping.jsonis excluded by!fixtures/**fixtures/ablation-large-good/images/shape-2@2x.pngis excluded by!**/*.png,!fixtures/**,!**/*.pngfixtures/ablation-large-good/images/shape-3@2x.pngis excluded by!**/*.png,!fixtures/**,!**/*.pngfixtures/ablation-large-good/images/shape@2x.pngis excluded by!**/*.png,!fixtures/**,!**/*.pngfixtures/ablation-large-good/screenshot.pngis excluded by!**/*.png,!fixtures/**,!**/*.pngfixtures/ablation-large-good/vectors/I562-10932;189-27028.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10932;2322-7007;34-12257;68-15864.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10933;348-15160;565-15194;68-15644.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10933;348-15161;565-15194;68-15644.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10933;348-15164;34-12248;7758-11716.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10933;375-13171;7753-4753;149-9553;7758-11236.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15217;638-14361;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15217;638-14362;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15217;638-14363;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15217;638-14364;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15217;638-14365;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15218;638-14361;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15218;638-14362;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15218;638-14363;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15218;638-14364;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15218;638-14365;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15219;638-14361;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15219;638-14362;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15219;638-14363;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15219;638-14364;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15219;638-14365;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10936;321-11076.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10936;325-12995.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10936;325-12997.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10936;325-13118.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10936;325-13226.svgis excluded by!**/*.svg,!fixtures/**
📒 Files selected for processing (1)
CLAUDE.md
- Fix invalid hex color #2C → #2C2C2C - Component annotation: show both formats (name only + key=value) - Responsive: "by default width: 100%" → "width: 100% when FILL mode" Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Owner
Author
|
@coderabbitai resume |
Contributor
✅ Actions performedReviews resumed. |
Owner
Author
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CLAUDE.md`:
- Line 196: The docs claim a variant properties format `[component: Key=Value,
...]` which is not implemented; update the documentation to only document the
actual annotation produced (`[component: ComponentName]`) and remove the
`Key=Value` variant example so it matches the runtime output that uses comp.name
(the generator that currently emits `[component: ${comp.name}]`). Ensure the
CLAUDE.md line is changed to the single supported format and wording aligned
with the component annotation behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 9020fbe7-afbe-43f9-b43e-775715bed1c2
⛔ Files ignored due to path filters (33)
fixtures/ablation-large-good/data.jsonis excluded by!fixtures/**fixtures/ablation-large-good/images/image@2x.pngis excluded by!**/*.png,!fixtures/**,!**/*.pngfixtures/ablation-large-good/images/mapping.jsonis excluded by!fixtures/**fixtures/ablation-large-good/images/shape-2@2x.pngis excluded by!**/*.png,!fixtures/**,!**/*.pngfixtures/ablation-large-good/images/shape-3@2x.pngis excluded by!**/*.png,!fixtures/**,!**/*.pngfixtures/ablation-large-good/images/shape@2x.pngis excluded by!**/*.png,!fixtures/**,!**/*.pngfixtures/ablation-large-good/screenshot.pngis excluded by!**/*.png,!fixtures/**,!**/*.pngfixtures/ablation-large-good/vectors/I562-10932;189-27028.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10932;2322-7007;34-12257;68-15864.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10933;348-15160;565-15194;68-15644.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10933;348-15161;565-15194;68-15644.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10933;348-15164;34-12248;7758-11716.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10933;375-13171;7753-4753;149-9553;7758-11236.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15217;638-14361;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15217;638-14362;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15217;638-14363;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15217;638-14364;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15217;638-14365;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15218;638-14361;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15218;638-14362;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15218;638-14363;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15218;638-14364;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15218;638-14365;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15219;638-14361;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15219;638-14362;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15219;638-14363;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15219;638-14364;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10934;348-15219;638-14365;7758-12316.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10936;321-11076.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10936;325-12995.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10936;325-12997.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10936;325-13118.svgis excluded by!**/*.svg,!fixtures/**fixtures/ablation-large-good/vectors/I562-10936;325-13226.svgis excluded by!**/*.svg,!fixtures/**
📒 Files selected for processing (1)
CLAUDE.md
CodeRabbit noted [component: Key=Value] isn't explicitly implemented. It's actually comp.name output — Figma variant components naturally have Key=Value format names. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This was referenced Mar 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Update project documentation to reflect the new direction established through ablation experiments 01-04.
Changes
CLAUDE.md
README.md
docs/SCORING.md
Evidence
Closes #121
Test plan
🤖 Generated with Claude Code
Summary by CodeRabbit
Documentation
Tests
Chores