Skip to content

fix(core): fail loud on un-graduatable code-condition rule updates (#566)#587

Merged
laofahai merged 5 commits into
mainfrom
fix/rule-update-graduation-guard
Jun 13, 2026
Merged

fix(core): fail loud on un-graduatable code-condition rule updates (#566)#587
laofahai merged 5 commits into
mainfrom
fix/rule-update-graduation-guard

Conversation

@laofahai

@laofahai laofahai commented Jun 13, 2026

Copy link
Copy Markdown
Owner

What

Graduating an approved update_rule proposal for a code-condition rule silently wrote a corrupt defineRule({ "name": "…" }) stub instead of valid source. ProposalFileWriter now fails loud before writing.

Why (the silent-corruption hole, #566)

The procurement manager_approval_threshold rule has a CodeCondition function condition (threshold baked into code). When NL says "把经理审批阈值改成2万", the resolver (schema-intent-rule-updater) honestly refuses to fabricate a definition — it emits an update change with no definition and requiresCodeChange: true.

At graduation, defaultCodegen's ?? { name: change.name } fallback then serialized that to:

export default defineRule({ "name": "manager_approval_threshold" });

— a stub missing trigger / condition / effect that fails validation once written. JSON.stringify also silently drops any function-typed condition.

The fix

assertGraduatable(proposal, change) (scoped to rule) runs in the write path before any mkdir/writeFile, only when there is no trusted generatedSource:

  • (a) throws if the effective definition (change.definition ?? {}) lacks trigger/condition/effect — the requiresCodeChange / code-condition case.
  • (b) throws if a required field is a function (JSON.stringify would drop it).

Errors name the proposal id, change name, target, and the fix (a requiresCodeChange update needs a materialized generatedSource). The trusted generatedSource path is untouched (materialized source still writes verbatim); declarative rule updates with a full definition still graduate normally.

Tests

  • code-condition update (no definition, no generatedSource) → throws, no corrupt dir created
  • update whose condition is a function → throws
  • update with generatedSource → writes verbatim
  • full declarative update → writes a valid defineRule(...) (no regression)

Several existing rule fixtures were upgraded from { name }-only stubs (themselves the corrupt shape the guard now rejects) to complete declarative definitions — behavior-preserving (path/filename/w-flag overwrite/updated marker all pinned). Full packages/core suite: 4323 pass / 0 fail.

Scope

Closes the silent-corruption half of #566. The AI materializer that regenerates the condition function (so "把阈值改成2万" graduates to valid code) is a tracked follow-up — not in this PR.

Part of #566.

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • Bug Fixes

    • Added validation to ensure rule definitions include all required fields and prevent graduation of incomplete rule scaffolds. The system now validates that required rule components are properly defined and cannot be functions before processing.
  • Tests

    • Expanded test coverage for rule graduation and validation behavior, including new guard tests for invalid rule scenarios.

)

Graduating an approved `update_rule` proposal for a CODE-condition rule
(e.g. the procurement `manager_approval_threshold`, whose condition is a
`CodeCondition` function) silently wrote a corrupt
`defineRule({ "name": "…" })` stub — missing trigger/condition/effect —
because `defaultCodegen`'s `?? { name: change.name }` fallback ran over a
definition the NL resolver intentionally left undefined (`requiresCodeChange:
true`), and `JSON.stringify` drops function-typed conditions.

`ProposalFileWriter` now refuses such a change BEFORE any mkdir/writeFile:
`assertGraduatable` (scoped to `rule`) throws when the effective definition
lacks a required field (trigger/condition/effect) or carries a function where
source needs data, naming the proposal id, change name, target and the fix
(a `requiresCodeChange` update needs a materialized `generatedSource`). The
trusted `generatedSource` path is untouched — materialized source still writes
verbatim. Declarative rule updates with a full definition still graduate.

Closes the silent-corruption half of #566. The AI materializer that
regenerates the condition function (so "把阈值改成2万" graduates to valid
code) is a tracked follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@laofahai, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 19 minutes and 52 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 31d564d7-a0fb-4a2e-8e11-c2bad37cb1a3

📥 Commits

Reviewing files that changed from the base of the PR and between 1b5030d and 70e55a7.

📒 Files selected for processing (2)
  • packages/core/__tests__/proposal-file-writer.test.ts
  • packages/core/src/engine/proposal-file-writer.ts
📝 Walkthrough

Walkthrough

ProposalFileWriter adds deterministic-serialization validation for rule changes via a new assertGraduatable() guard that rejects incomplete definitions or function-typed fields. Test fixtures are modernized to use complete declarative rules, integration ensures the guard runs only when trusted generatedSource is absent, and comprehensive test coverage validates guard failures, bypasses, and code generation.

Changes

Rule Graduatability Guard

Layer / File(s) Summary
Test fixtures for complete declarative rules
packages/core/__tests__/proposal-file-writer.test.ts (lines 49–110)
makeApprovedProposal() default rule is updated from minimal stub to complete declarative definition with label, trigger, condition, and effect; new declarativeRuleChange() helper builds serializable rule changes for test consistency.
Rule graduation validation guard
packages/core/src/engine/proposal-file-writer.ts (lines 241–306)
assertGraduatable() enforces that rule definitions include required trigger, condition, and effect fields and rejects function-typed values, preventing silent data loss during JSON serialization.
Integration and comprehensive guard coverage
packages/core/src/engine/proposal-file-writer.ts (lines 439–449), packages/core/__tests__/proposal-file-writer.test.ts (lines 372–492)
writeApprovedProposal() uses hasGeneratedSource to decide whether to skip validation (when generatedSource is present/non-blank) or run assertGraduatable() before codegen; test suite covers throw cases (missing definition, function-typed condition), verbatim generatedSource writes, and valid declarative rule code generation.
Fixture adoption in existing tests
packages/core/__tests__/proposal-file-writer.test.ts (lines 264–345)
Multi-change proposal, collision-prevention, and overwrite-on-update tests refactored to use declarativeRuleChange() helper, standardizing fixture construction across the suite.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

  • laofahai/linchkit#495: Both PRs modify ProposalFileWriter.writeApprovedProposal to branch on whether generatedSource is present/non-blank, using it verbatim when available and otherwise generating deterministic TypeScript.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: adding a guard to fail loud when graduating un-graduatable code-condition rule updates, directly matching the core fix in the changeset.
Description check ✅ Passed The description comprehensively covers the problem, solution, tests, and scope, matching most template sections; however, the Quality Gates and Changeset checklist sections are not explicitly confirmed as completed.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/rule-update-graduation-guard

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a graduatability guard (assertGraduatable) in ProposalFileWriter to prevent writing corrupt rule stubs when required fields (trigger, condition, effect) are missing or contain non-serializable functions. It also updates the test suite with helper functions and new test cases to validate this guard. The reviewer points out that running assertGraduatable unconditionally might break custom codegen implementations that are designed to handle these cases, and suggests restricting the guard to only run when the default codegen is used.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread packages/core/src/engine/proposal-file-writer.ts Outdated
@claude

claude Bot commented Jun 13, 2026

Copy link
Copy Markdown

Code Review

Reviewed packages/core/src/engine/proposal-file-writer.ts and the accompanying test suite.

No bugs found. The guard is correctly implemented:

  • Placement is right: assertGraduatable runs in the else branch (no generatedSource) before any mkdir/writeFile, so it is unreachable from the delete-skip path (line 419) and the revert-skip path — the two cases that bail out early in the loop.
  • Scope is right: the early if (target !== 'rule') return keeps all other targets unaffected; ProposalChangeOperation is exactly "create" | "update" | "delete" with no hidden variants.
  • Checks are right: the undefined | null gate catches the code-condition draft (no definition at all), and the typeof value === 'function' gate catches the silent-drop-by-JSON.stringify case. Both map to the exact failure modes described in Procurement scenario Phase 3 — NL rule-update loop end-to-end on the procurement domain #566.
  • Tests are right: four new cases — throw on missing definition, throw on function condition, verbatim write with generatedSource, valid declarative write — cover the full decision matrix. Existing fixtures correctly upgraded from stub-only shapes to complete declarative definitions.

One minor observation (not blocking): the fix string in assertGraduatable says "requiresCodeChange update" even when change.operation is "create". The message is still actionable, but a future reader creating a code-condition rule from scratch might find the wording slightly off. Worth a follow-up doc-level tweak, not a blocker.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/core/__tests__/proposal-file-writer.test.ts (1)

1-885: 🛠️ Refactor suggestion | 🟠 Major | 🏗️ Heavy lift

File exceeds 500-line guideline — split into focused modules.

This file is 885 lines, significantly exceeding the ~500-line guideline. Based on learnings, test files under __tests__/ should be split into multiple focused modules when approaching the limit (e.g., separate files for write behavior, filename format, formatter options, and engine hooks), with shared fixtures extracted to __tests__/helpers/*-fixtures.ts.

The new declarativeRuleChange() helper is a good start, but the file remains 77% over the limit.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/__tests__/proposal-file-writer.test.ts` around lines 1 - 885,
The test file is too large; split it into focused test modules and extract
shared fixtures/helpers: move makeApprovedProposal, declarativeRuleChange,
viewChange, selfContainedEntityChange, FIXED_NOW/DATE_STAMP, TEST_* constants
and todayUtcDateStamp into a new __tests__/helpers/proposal-fixtures.ts (or
similar) and import them; then create separate test files for the main concerns
— e.g. proposal-file-writer.writeApprovedProposal.test.ts (tests in the
"ProposalFileWriter.writeApprovedProposal" describe),
proposal-file-writer.filename.test.ts (the "filename format" describe),
proposal-file-writer.formatter.test.ts (the "formatter option" describe), and
proposal-engine.onApproved.test.ts (the "ProposalEngine.onApproved hook"
describe) — each should import ProposalFileWriter, createProposalEngine, and the
shared fixtures, keeping tests grouped by describe blocks like
writeApprovedProposal, filename format, formatter option, and onApproved; ensure
any per-test tmpDir setup/teardown (beforeEach/afterEach) is preserved or moved
into a shared test helper to avoid duplication and update imports in all new
files accordingly.

Sources: Coding guidelines, Learnings

🧹 Nitpick comments (1)
packages/core/src/engine/proposal-file-writer.ts (1)

1-529: 💤 Low value

File length exceeds 500-line guideline (529 lines).

The addition of assertGraduatable pushed this file slightly over the limit. The current structure is cohesive—the guard is tightly coupled to the write path—so splitting may not be necessary immediately. Consider extracting the validation constants and assertGraduatable helper to a separate proposal-validation.ts module if more guards are added in the future.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/engine/proposal-file-writer.ts` around lines 1 - 529, File
exceeds the 500-line guideline because assertGraduatable and its constants were
added; extract the validation bits into a new module and import them to shrink
the file: move RULE_REQUIRED_FIELDS and the function assertGraduatable (and any
helpers they need like short error message fragments) into a new
proposal-validation.ts module that exports them, update ProposalFileWriter to
import { assertGraduatable, RULE_REQUIRED_FIELDS } from that module, keep the
behaviour/signature identical, and run project tests/typechecks to ensure no
breakage.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@packages/core/__tests__/proposal-file-writer.test.ts`:
- Around line 1-885: The test file is too large; split it into focused test
modules and extract shared fixtures/helpers: move makeApprovedProposal,
declarativeRuleChange, viewChange, selfContainedEntityChange,
FIXED_NOW/DATE_STAMP, TEST_* constants and todayUtcDateStamp into a new
__tests__/helpers/proposal-fixtures.ts (or similar) and import them; then create
separate test files for the main concerns — e.g.
proposal-file-writer.writeApprovedProposal.test.ts (tests in the
"ProposalFileWriter.writeApprovedProposal" describe),
proposal-file-writer.filename.test.ts (the "filename format" describe),
proposal-file-writer.formatter.test.ts (the "formatter option" describe), and
proposal-engine.onApproved.test.ts (the "ProposalEngine.onApproved hook"
describe) — each should import ProposalFileWriter, createProposalEngine, and the
shared fixtures, keeping tests grouped by describe blocks like
writeApprovedProposal, filename format, formatter option, and onApproved; ensure
any per-test tmpDir setup/teardown (beforeEach/afterEach) is preserved or moved
into a shared test helper to avoid duplication and update imports in all new
files accordingly.

---

Nitpick comments:
In `@packages/core/src/engine/proposal-file-writer.ts`:
- Around line 1-529: File exceeds the 500-line guideline because
assertGraduatable and its constants were added; extract the validation bits into
a new module and import them to shrink the file: move RULE_REQUIRED_FIELDS and
the function assertGraduatable (and any helpers they need like short error
message fragments) into a new proposal-validation.ts module that exports them,
update ProposalFileWriter to import { assertGraduatable, RULE_REQUIRED_FIELDS }
from that module, keep the behaviour/signature identical, and run project
tests/typechecks to ensure no breakage.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 1af9cfc9-4d93-446c-9625-4d965460887f

📥 Commits

Reviewing files that changed from the base of the PR and between f9d81ed and 1b5030d.

📒 Files selected for processing (2)
  • packages/core/__tests__/proposal-file-writer.test.ts
  • packages/core/src/engine/proposal-file-writer.ts

laofahai and others added 2 commits June 13, 2026 17:24
)

A caller-supplied `codegen` (ProposalFileWriterOptions.codegen) is an escape
hatch for unusual ChangeDefinition shapes and may legitimately handle a
code-condition / definition-less rule update itself. `assertGraduatable`
encodes the limits of `defaultCodegen` specifically (JSON.stringify drops
functions; a rule needs trigger/condition/effect), so running it
unconditionally would block custom generators.

Gate the guard on `usesDefaultCodegen` (true only when no custom codegen was
supplied). Adds a test: a custom codegen receives a definition-less
code-condition rule update without the guard throwing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread packages/core/src/engine/proposal-file-writer.ts Outdated
Comment thread packages/core/src/engine/proposal-file-writer.ts
@claude

claude Bot commented Jun 13, 2026

Copy link
Copy Markdown

Review summary

The guard logic (assertGraduatable) is sound and the four new tests cover the right scenarios. Two issues worth addressing:

  1. Guard fires before custom codegen (see inline comment on line 447) — assertGraduatable runs unconditionally, so a consumer using options.codegen as the documented escape hatch for code-condition rules gets the guard's throw before their codegen can run. Scoping the guard to this.codegen === defaultCodegen (or an opt-out flag) preserves the contract.

  2. Error message says "update" even for create operations (inline on lines 277-279) — the fix string is shared by both throw branches and hardcodes the word "update", which is misleading when the failing change is a create.

No other correctness or security issues found. The core fix — failing loud instead of writing a defineRule({ "name": "…" }) stub — is correct.

Comment thread packages/core/src/engine/proposal-file-writer.ts
@claude

claude Bot commented Jun 13, 2026

Copy link
Copy Markdown

Code Review Summary

The fix is correct and well-scoped. The guard runs before any mkdir/writeFile, so the corrupt-stub path is cleanly blocked with no partial I/O. The two detected cases (missing required field; function-valued field) are both tested and cover the real failure modes from #566. The escape hatch for custom codegen is a good design choice and is correctly tested.

One issue filed inline: the diagnostic message in assertGraduatable hard-codes the word 'update' and would mislead a developer who hits the guard on a create operation.

Everything else looks good: guard placement, generatedSource short-circuit, sequential writes, wx-flag atomicity for creates, and test coverage of all four paths.

laofahai and others added 2 commits June 13, 2026 17:37
assertGraduatable fires on create AND update, but the hint string hardcoded
"update" — misleading when a brand-new rule (operation: "create") carries a
function condition. Use `change.operation` so the message reads "code-condition
create needs…" / "code-condition update needs…" accurately. Adds a test pinning
the create wording.

(claude's other thread — guard pre-empting a custom codegen — was already fixed
in e5c126f by scoping the guard to usesDefaultCodegen.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@laofahai laofahai merged commit 4e4f984 into main Jun 13, 2026
8 checks passed
@laofahai laofahai deleted the fix/rule-update-graduation-guard branch June 13, 2026 09:43
@claude

claude Bot commented Jun 13, 2026

Copy link
Copy Markdown

Code review — no bugs found. Reviewed proposal-file-writer.ts and the test suite. The guard is correct on all axes I checked: (1) RULE_REQUIRED_FIELDS including condition is consistent with validation-engine.ts which requires condition before a rule can be approved; (2) delete operations are skipped at line 432, before the assertGraduatable call, so the guard never fires on deletes; (3) the usesDefaultCodegen flag is safe — defaultCodegen is unexported so a caller cannot accidentally re-pass it and bypass the guard; (4) the guard fires before mkdir/writeFile so no partial state is left on disk on throw; (5) the upgraded declarativeRuleChange test helper correctly carries trigger/condition/effect and satisfies the guard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant