feat: set CATEGORY_WEIGHT from ablation data (#169)

[let-sunny]

Summary

• CATEGORY_WEIGHT set based on ablation experiment data (PR experiment: ablation Phase 1 results #149, experiment: ablation Phase 2 — generalization across 6 new fixtures #150):
  responsive-critical 3.0, pixel-critical 2.5, code-quality/token-management 1.0, interaction 0.5, minor 0.3
• missing-prototype disabled — interactionDestinations data missing from fixtures (Pre-dry-run: rebuild PROMPT.md + fixtures after design-tree changes #139 rebuild pending)
• non-standard-naming score -1 → -3 — non-standard state names break the interaction detection pipeline downstream

Before/After (5 fixtures)

Fixture	Before	After	Δ
desktop-product-detail	F 47%	F 44%	-3
desktop-landing-page	D 53%	D 53%	0
desktop-shop	D 51%	D 50%	-1
mobile-product-detail	F 47%	F 44%	-3
mobile-landing-page	D 54%	D 54%	0

Scores drop slightly because responsive-critical (low scores, 518%) now weighs 3x more. This is directionally correct — responsive issues have the highest ablation impact (ΔV +15.9%). Interaction scores improved (+1427%) due to missing-prototype being disabled.

Test plan

• All 606 tests pass
• Scoring test updated for weighted average
• REFERENCE.md synced with rule-config.ts
• Before/after fixture score comparison documented

Closes #169

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

• New Features

  • Scoring system now weights critical issues more heavily for more accurate assessment prioritization.

• Bug Fixes

  • Increased penalty for non-standard naming violations to better reflect impact on code quality.
  • Disabled prototype missing checks pending fixture data updates.

• Documentation

  • Updated rule reference documentation to reflect scoring changes.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR implements calibration changes based on ablation experiment data: it updates category weights in the scoring engine (exporting CATEGORY_WEIGHT with differentiated weights for responsive and pixel concerns), disables the missing-prototype rule due to missing fixture data, increases the non-standard-naming penalty from -1 to -3, and refactors the test to validate weighted score calculations.

Changes

Cohort / File(s)	Summary
Scoring Engine Configuration src/core/engine/scoring.ts	Exported CATEGORY_WEIGHT and updated weights based on ablation data: pixel-critical (1.0→2.5), responsive-critical (1.0→3.0), interaction (1.0→0.5), minor (1.0→0.3). Others unchanged. Replaced documentation with experiment evidence references.
Rule Configuration src/core/rules/rule-config.ts	Disabled missing-prototype rule (missing fixture data) and increased non-standard-naming penalty from -1 to -3 with justification comments.
Scoring Tests src/core/engine/scoring.test.ts	Updated to import CATEGORY_WEIGHT and rewrote weighted average test to dynamically compute expected score using category weights instead of hardcoded values.
Documentation docs/REFERENCE.md	Updated non-standard-naming rule table entry: default score changed from -1 to -3.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• feat: non-standard-naming rule — detect dev-unfriendly state names (#159) #162: Introduces the non-standard-naming rule whose default score is now being adjusted from -1 to -3 in this PR.
• feat: interaction category — missing-interaction-state + missing-prototype (#151) #158: Also modifies CATEGORY_WEIGHT and RULE_CONFIGS entries, including the missing-prototype rule being disabled here.
• fix: sync hardcoded rule values and add auto-sync tooling #112: Updates rule metadata and configuration consistency through both RULE_CONFIGS and documentation files, similar scope to this PR.

Poem

🐰 The weights have shifted, the scales now align,
Responsive and pixels get their time to shine!
Experiments whispered of balance and care,
While missing-prototype steps with grace from the air. ✨
Each category weighted, each score now refined,
A more truthful measure of impact we find!

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately reflects the primary change: setting CATEGORY_WEIGHT values based on ablation experiment data, which is the main objective.
Linked Issues check	✅ Passed	All coding requirements from issue #169 are met: CATEGORY_WEIGHT values updated with ablation data, missing-prototype disabled, non-standard-naming score adjusted, and tests updated.
Out of Scope Changes check	✅ Passed	All changes directly support the linked issue objectives: CATEGORY_WEIGHT configuration, missing-prototype disabling, non-standard-naming adjustment, and related test/documentation updates.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/category-weight-169

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/core/rules/rule-config.ts`:
- Around line 134-140: Remove inline calibration rationale comments from the
auto-managed RULE_CONFIGS object: delete the comment after the enabled property
(the "// disabled: interactionDestinations data missing from fixtures (`#139`)")
and the comment after the score for "non-standard-naming" (and any similar
rationale comments in RULE_CONFIGS). Keep the configuration values unchanged,
and ensure no other inline calibration rationale comments remain in the
RULE_CONFIGS declaration (move any rationale to external docs if needed).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 2a253fa2-312d-4622-9830-44385c8f6839

📥 Commits

Reviewing files that changed from the base of the PR and between 162b593 and d751d5c.

📒 Files selected for processing (4)

• docs/REFERENCE.md
• src/core/engine/scoring.test.ts
• src/core/engine/scoring.ts
• src/core/rules/rule-config.ts

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Remove inline calibration rationale comments from auto-managed config.

Line 134 and Line 140 add inline rationale in RULE_CONFIGS. Keep this file comment-free for calibration rationale to avoid churn from nightly auto-adjustments.

Proposed cleanup

   "missing-prototype": {
     severity: "missing-info",
     score: -3,
-    enabled: false, // disabled: interactionDestinations data missing from fixtures (`#139`)
+    enabled: false,
   },
@@
   "non-standard-naming": {
     severity: "suggestion",
-    score: -3, // higher than other naming rules: non-standard state names break interaction detection pipeline
+    score: -3,
     enabled: true,
   },

Based on learnings: In let-sunny/canicode, src/core/rules/rule-config.ts is auto-adjusted by a nightly calibration pipeline, and inline calibration rationale should not be added there.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/core/rules/rule-config.ts` around lines 134 - 140, Remove inline
calibration rationale comments from the auto-managed RULE_CONFIGS object: delete
the comment after the enabled property (the "// disabled:
interactionDestinations data missing from fixtures (`#139`)") and the comment
after the score for "non-standard-naming" (and any similar rationale comments in
RULE_CONFIGS). Keep the configuration values unchanged, and ensure no other
inline calibration rationale comments remain in the RULE_CONFIGS declaration
(move any rationale to external docs if needed).