feat(core): validation Phase 4 — generated-source contract check (G5)

[laofahai]

What

Validation Phase 4 — generated-source CONTRACT check (G5). Replaces the
skipped Phase 4 stub with validatePhase4: a static, execution-free check
that any AI-materialized generatedSource (from the #495 materializer) actually
defines the change's declared target/name — the right define*() call, a
reference to its name, and an import from @linchkit/core. It catches a class of
AI errors that pass the Phase 2 syntax gate yet are wrong (empty scaffold, wrong
target, mismatched name) before a human reviews the candidate.

• Warn-only by default; ValidationContext.strictGeneratedContract escalates to
  block. All-declarative proposals (no generatedSource) stay "skipped".
• Wired into validateProposal (was the M1 skip stub); exported from
  @linchkit/core/server (+ barrel-surface test).
• action is the only materializable target today (matches the materializer's
  MATERIALIZABLE_TARGETS); the map is extensible.

Safety

EXECUTION-FREE by design ("AI never modifies production directly"): it never
evals, imports, transpiles-and-runs, or otherwise executes the generated
source — only static string/structural heuristics. A true execution-based
dry-run (running a generated handler against sample/historical data) requires a
locked-down sandbox to run untrusted AI code and is intentionally out of
scope here — deferred to a separate, sandbox-gated step.

Review

3 rounds of codex review converged to clean. Hardenings applied along the way:

• Tokens appearing only in a comment (// defineAction() or string literal no
  longer satisfy the call/import checks (strip comments, and strings for the
  call check; match real import/call structure via anchored regexes).
• The name check uses a word-boundary match so a different name that merely
  CONTAINS the declared one (do_thing_v2 for do_thing) no longer passes.

Verification

• bun run check / bun run typecheck — clean
• bun run test — PASS (8 Phase 4 unit tests incl. comment/string and
  substring-name hardening; barrel surface test; packages/core 4170 tests green)

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added Phase 4 validation to verify AI-generated source code properly imports from the core library and references declared definitions by name.
  • Validation is warn-only by default but can be enforced strictly via configuration.

• Tests

  • Added comprehensive test suite for generated source code contract validation.

[coderabbitai]

Warning

Review limit reached

@laofahai, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 15 minutes and 22 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9f5abbde-642c-4c2b-89c7-93070acbb86a

📥 Commits

Reviewing files that changed from the base of the PR and between bd03122 and 4b2d364.

📒 Files selected for processing (2)

• packages/core/src/__tests__/engine/validation-phase4.test.ts
• packages/core/src/engine/validation-phase4.ts

📝 Walkthrough

Walkthrough

This PR introduces Phase 4 validation, a static contract check for AI-materialized generatedSource that verifies generated code defines the declared target, references the declared name as a whole word, and imports from @linchkit/core. The check is warn-only by default and can be enforced via ValidationContext.strictGeneratedContract. It is wired into validateProposal for all-declarative proposals.

Changes

Phase 4 Generated-Source Contract Validation

Layer / File(s)	Summary
Phase 4 validation contract and implementation packages/core/src/engine/validation-phase4.ts	Phase 4 performs three static checks on non-empty generatedSource: presence of the expected define* call, whole-word reference to declared name (comments stripped, strings preserved), and actual import/require from @linchkit/core. Returns skipped when no generated source is present; otherwise collects findings and gates them as warnings or errors based on strictGeneratedContract.
Validation engine wiring packages/core/src/engine/validation-engine.ts	Extends ValidationContext with optional strictGeneratedContract flag and replaces the Phase 4 "skipped" placeholder in validateProposal with an actual validatePhase4() call, passing through the context's strictness setting.
Phase 4 contract test suite packages/core/src/__tests__/engine/validation-phase4.test.ts	Comprehensive tests covering skip behavior (no/empty generatedSource), passing validation (well-formed defineAction with core import), warn-only default, missing name reference and missing import findings, comment/string hardening (mentions not satisfied by comments/strings), name word-boundary matching (substring containment fails), and strict mode escalation to errors.
Public exports and documentation packages/core/src/exports/server/engines.ts, packages/core/__tests__/barrel-public-surface.test.ts, .changeset/g5-phase4-contract.md	Server barrel exports updated to include ValidatePhase4Options and validatePhase4; public-surface test snapshot includes new export; changeset documents the feature as warn-only by default with optional strictGeneratedContract enforcement.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• laofahai/linchkit#487: Wires validatePhase3 into the same validation-engine integration pattern, extending ValidationContext with strictCompatibility.
• laofahai/linchkit#495: Adds Phase 2 validatePhase2 to the validation flow with generatedSource checks, using the same engine wiring approach.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately and specifically describes the main change: implementing validation Phase 4 for generated-source contract checking.
Description check	✅ Passed	The PR description is comprehensive and well-structured, covering what the change does, safety considerations, and verification results, though it partially follows the template structure.
Docstring Coverage	✅ Passed	Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/g5-phase4-contract

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request introduces Phase 4 validation, which statically verifies that AI-materialized generated source code conforms to its declared contract (e.g., calling the correct definition helper, referencing its declared name, and importing @linchkit/core). Feedback on the implementation highlights a parsing bug where URLs or slashes inside string literals are incorrectly stripped as comments, which breaks validation. Additionally, the reviewer recommends escaping the dot in the core import regex, pre-compiling regular expressions at the module level to improve performance, and adding a test case to prevent regressions.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[coderabbitai]

🧹 Nitpick comments (1)

packages/core/src/engine/validation-engine.ts (1)

1-717: Split packages/core/src/engine/validation-engine.ts to satisfy the 500-line guideline

• packages/core/src/engine/validation-engine.ts is ~717 lines (> 500).
• Searches for existing tracking issues mentioning “validation-engine” / “validation engine” / “validation-engine.ts” didn’t surface one for splitting this file; consider creating an issue to break it into smaller per-phase/per-validator modules.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/engine/validation-engine.ts` around lines 1 - 717, The file
validation-engine.ts exceeds the 500-line guideline; split it into smaller
modules by extracting Phase 1 runner and each validator into separate files and
re-exporting a compact API: create a file for the phase runner (export
validatePhase1 and validateProposal), a module for helpers/state resolution
(resolveStateMachine, ValidationContext utilities), and separate validator
modules for validateEntity, validateAction, validateRule, validateStateDef,
validateEventDef, and validateViewDef; update imports/exports so the original
symbols (validatePhase1, validateProposal, validateEntity, validateAction,
validateRule, validateStateDef, validateEventDef, validateViewDef,
resolveStateMachine, ValidationContext) remain available from the engine
package, add a short tracking issue referencing “validation-engine.ts” for the
refactor, and ensure tests/builds still pass after replacing local references
with the new module paths.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/core/src/engine/validation-engine.ts`:
- Around line 1-717: The file validation-engine.ts exceeds the 500-line
guideline; split it into smaller modules by extracting Phase 1 runner and each
validator into separate files and re-exporting a compact API: create a file for
the phase runner (export validatePhase1 and validateProposal), a module for
helpers/state resolution (resolveStateMachine, ValidationContext utilities), and
separate validator modules for validateEntity, validateAction, validateRule,
validateStateDef, validateEventDef, and validateViewDef; update imports/exports
so the original symbols (validatePhase1, validateProposal, validateEntity,
validateAction, validateRule, validateStateDef, validateEventDef,
validateViewDef, resolveStateMachine, ValidationContext) remain available from
the engine package, add a short tracking issue referencing
“validation-engine.ts” for the refactor, and ensure tests/builds still pass
after replacing local references with the new module paths.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9a02bee2-1ad2-4024-b413-987b9c237835

📥 Commits

Reviewing files that changed from the base of the PR and between e9795d0 and bd03122.

📒 Files selected for processing (6)

• .changeset/g5-phase4-contract.md
• packages/core/__tests__/barrel-public-surface.test.ts
• packages/core/src/__tests__/engine/validation-phase4.test.ts
• packages/core/src/engine/validation-engine.ts
• packages/core/src/engine/validation-phase4.ts
• packages/core/src/exports/server/engines.ts