Skip to content

feat(core): implement proposal-validation Phase 3 compatibility checks (Spec 09 §4.5)#487

Merged
laofahai merged 2 commits into
mainfrom
feat/validation-phase3-compatibility
Jun 7, 2026
Merged

feat(core): implement proposal-validation Phase 3 compatibility checks (Spec 09 §4.5)#487
laofahai merged 2 commits into
mainfrom
feat/validation-phase3-compatibility

Conversation

@laofahai

@laofahai laofahai commented Jun 6, 2026

Copy link
Copy Markdown
Owner

What & why

Proposal validation (validateProposal) only ran Phase 1 (static checks) — Phase 2/3/4 were hardcoded status: "skipped" stubs. So a proposal that removes a field an action/view/rule depends on passed validation. This makes the "AI never modifies production directly" guardrail shallow (G3).

Change

Implement Phase 3 (compatibility / breaking-reference detection), Spec 09 §4.5:

  • New packages/core/src/engine/validation-phase3.ts (validatePhase3). Detects, against the current meta-model:
    • entity-field delete still referenced by a view / rule → BREAKING_FIELD_DELETE
    • whole-entity / action / state delete with dependents (via OntologyRegistry impact graph) → BREAKING_ELEMENT_DELETE
    • field type changeBREAKING_FIELD_TYPE_CHANGE
    • required field losing its default (while staying required)BREAKING_REQUIRED_DEFAULT_DROP
    • enum value removalBREAKING_ENUM_VALUE_REMOVED
  • Wired into validateProposal (replaces the phase3 stub). ValidationContext gains optional ontology + strictCompatibility.

Gating (fail-closed per Spec 09 §4.1) — ⚠️ changes production behavior

  • Default: warn-only — findings are warnings; passed is NOT affected.
  • strictCompatibility escalates to blocking errors. New features.strictCompatibility env flag — true in production and staging, false in dev/test (sibling of strictValidation; note isProduction covers both production and staging) — threaded through mountProposalAPI. So production and staging now refuse proposals that break existing references; dev/test stays advisory. Flagged here explicitly because it changes pre-prod/prod validation behavior. The detector is deterministic and conservative (reference checks via the impact graph; skips opaque code conditions and refs it cannot resolve) → it under-reports rather than false-positive-blocks.

Backward compatible

When the context carries no ontology, Phase 3 returns "skipped" — identical to the prior stub. Existing validation tests pass untouched.

Known limitation (documented, follow-up)

The impact DAG does not yet model StateDefinition.transitions[].action / entity state-field machine edges, so deleting an action used only as a state transition may not surface a dependent (under-report). Noted in code; tracked for a future ontology-DAG enhancement.

Review

Cross-model reviewed (codex + PR bots gemini-code-assist & CodeRabbit). All findings addressed:

  • codex (3× P2): whole-entity delete now consults the impact graph; state-transition under-reporting documented; fixed a false positive — dropping a default is no longer flagged when the field also becomes optional. Regression tests added.
  • gemini-code-assist: defensive view.fields?. and rule?.trigger guards added; the "enum options can be strings" note was verified non-applicable (EnumField.options is typed Array<{value,label?}>).
  • CodeRabbit: corrected the staging wording (staging is blocking, not advisory).

Tests

validation-phase3.test.ts — 18 tests (unit + validateProposal integration): degradation/skip, field-delete warn vs strict-block, whole-entity delete, action/state delete, type/enum/default narrowing, required→optional loosening (no false positive), and the integration warn-vs-block path.

Gates

bun run check, bun run typecheck, full bun run test (batched) all green. Stays strictly within the validation boundary — no approval/commit/deploy/graduation/file-write path touched.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jun 6, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This pull request introduces Phase 3 proposal validation for detecting breaking references against the current meta-model. It detects deleted fields referenced by views/rules, deleted actions/states with dependents, and narrowing changes (type changes, enum removal, required-default drops). Findings are warn-only by default and become blocking when strictCompatibility environment flag is enabled. The feature integrates into the validation engine, is wired through server API mounting, and is covered by comprehensive tests.

Changes

Phase 3 Validation Feature

Layer / File(s) Summary
Phase 3 Breaking-Reference Detection Implementation
packages/core/src/engine/validation-phase3.ts
validatePhase3 implements deletion and update-narrowing breakage detection. Gracefully returns skipped when ontology is absent. Deletion detection compares proposal-surviving fields vs. current entity descriptor and scans views/rules for references, or consults ontology.impactAnalysis for whole-element deletion dependents. Update narrowing detects breaking field type changes, removed enum values, and dropped required defaults. Findings are emitted as errors (strictCompatibility true) or warnings (false), with recursive condition parsing for rule field references.
Validation Engine Phase 3 Integration
packages/core/src/engine/validation-engine.ts
ValidationContext gains optional ontology and strictCompatibility fields. validateProposal invokes validatePhase3 instead of hard-coding skipped, passing changes and context options; Phase 2 and Phase 4 remain skipped.
Environment Strict Compatibility Feature Flag
packages/core/src/deployment/environment.ts
EnvironmentFeatureFlags adds strictCompatibility boolean (defaults to true in production/staging, false in dev/test). buildConfig initializes the flag based on environment classification.
Public API Exports
packages/core/src/engine/index.ts, packages/core/src/exports/server/engines.ts, packages/core/__tests__/barrel-public-surface.test.ts
validatePhase3 and ValidatePhase3Options are re-exported through core engine and server engines surfaces. Barrel-public-surface test snapshot is updated to reflect new exports.
Server Proposal API Integration
addons/adapter-server/cap-adapter-server/src/proposal-api.ts
Introduces MountProposalAPIOptions interface with optional executionLogger, ontology, and strictCompatibility. mountProposalAPI signature updated to accept options or bare ExecutionLogger with runtime discrimination via log method presence. Shared ValidationContext is constructed and threaded through proposal approval/rejection auto-submit flows.
Server Initialization with Proposal API
addons/adapter-server/cap-adapter-server/src/server.ts
createServer mounts mountProposalAPI with options object containing executionLogger, ontology from opts.ontologyRegistry, and strictCompatibility from environment feature flag (prod/staging blocking, dev/test warn-only).
Comprehensive Test Suite
packages/core/src/__tests__/engine/validation-phase3.test.ts
Mock ontology fixtures and test helpers for ProposalChange/ProposalDefinition construction. Tests verify degradation when ontology missing, field deletion impact detection (warn-only by default, escalated to errors under strictCompatibility), action/state deletion via impact graph, and update-narrowing for type changes, enum removal, required-default drops. Integration tests confirm Phase 3 gating behavior and interaction with overall proposal validation.
Release Notes
.changeset/validation-phase3-compatibility.md
Documents Phase 3 compatibility feature, breaking-reference detection capabilities, warn-only default behavior, strictCompatibility blocking mode, and ValidationContext extensions for version bumps.

🎯 3 (Moderate) | ⏱️ ~28 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and specifically summarizes the main change: implementing proposal-validation Phase 3 compatibility checks per Spec 09 §4.5, which is the primary objective of this PR.
Docstring Coverage ✅ Passed Docstring coverage is 90.91% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The pull request description is comprehensive and well-structured, covering the summary, related specs, type, test plan, and quality gates.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/validation-phase3-compatibility

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements Phase 3 of proposal validation, which checks for breaking compatibility and reference changes (such as deleting referenced fields, actions, or states, changing field types, or narrowing enum values) against the current meta-model. These checks are warn-only by default but can escalate to blocking errors in production environments via the strictCompatibility flag. The review feedback highlights several critical robustness improvements in validation-phase3.ts, including potential runtime crashes when handling optional view.fields or nullish rule triggers, and a functional gap where string-based enum options are not correctly mapped.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread packages/core/src/engine/validation-phase3.ts Outdated
Comment thread packages/core/src/engine/validation-phase3.ts Outdated
Comment thread packages/core/src/engine/validation-phase3.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/core/src/engine/validation-engine.ts (1)

642-646: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Outdated docstring — Phase 3 now runs.

The docstring still says "Currently only runs Phase 1 (static checks)" but Phase 3 is now wired in. Update the comment to reflect this.

📝 Suggested fix
 /**
- * Validate a full proposal. Currently only runs Phase 1 (static checks).
- * Phase 2-4 will be added in later milestones.
+ * Validate a full proposal. Runs Phase 1 (static checks) and Phase 3
+ * (compatibility / breaking-reference checks). Phase 2/4 are skipped for M1.
  */
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/engine/validation-engine.ts` around lines 642 - 646, Update
the docstring that currently reads "Validate a full proposal. Currently only
runs Phase 1 (static checks)." (the comment above the full-proposal validation
function) to accurately state that Phase 3 is now executed as well — e.g.,
"Validate a full proposal. Runs Phase 1 (static checks) and Phase 3 (runtime
checks); Phases 2 and 4 are planned for later." Keep the descriptive text near
the same comment block so it clearly documents the behavior of the full-proposal
validation routine.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.changeset/validation-phase3-compatibility.md:
- Line 8: Update the changeset sentence that currently states "dev/staging stays
advisory" to correctly reflect the actual wiring of features.strictCompatibility
(which sets strictCompatibility = true for production and staging, false for
dev/test); reword it to indicate that staging is blocking like production while
dev remains advisory, and ensure the text mentions mountProposalAPI and
ValidationContext.strictCompatibility behavior so the release note matches
runtime behavior.

---

Outside diff comments:
In `@packages/core/src/engine/validation-engine.ts`:
- Around line 642-646: Update the docstring that currently reads "Validate a
full proposal. Currently only runs Phase 1 (static checks)." (the comment above
the full-proposal validation function) to accurately state that Phase 3 is now
executed as well — e.g., "Validate a full proposal. Runs Phase 1 (static checks)
and Phase 3 (runtime checks); Phases 2 and 4 are planned for later." Keep the
descriptive text near the same comment block so it clearly documents the
behavior of the full-proposal validation routine.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d4fbee8b-5bc4-4f80-9b38-a17f9d7d383e

📥 Commits

Reviewing files that changed from the base of the PR and between 812c5e8 and b69a41d.

📒 Files selected for processing (10)
  • .changeset/validation-phase3-compatibility.md
  • addons/adapter-server/cap-adapter-server/src/proposal-api.ts
  • addons/adapter-server/cap-adapter-server/src/server.ts
  • packages/core/__tests__/barrel-public-surface.test.ts
  • packages/core/src/__tests__/engine/validation-phase3.test.ts
  • packages/core/src/deployment/environment.ts
  • packages/core/src/engine/index.ts
  • packages/core/src/engine/validation-engine.ts
  • packages/core/src/engine/validation-phase3.ts
  • packages/core/src/exports/server/engines.ts

Comment thread .changeset/validation-phase3-compatibility.md Outdated
…s (Spec 09 §4.5)

Proposal validation only ran Phase 1 (static checks); Phase 2/3/4 were
hardcoded "skipped" stubs, so a proposal that removes a field an action/view/
rule depends on passed validation. This makes the "AI never modifies production
directly" guardrail shallow.

Implement Phase 3 (compatibility / breaking-reference detection):
- New `packages/core/src/engine/validation-phase3.ts` exporting `validatePhase3`.
  It inspects delete/narrowing changes against the current meta-model via the
  OntologyRegistry impact graph (action/state deletes) + a field-reference scan
  (entity-field deletes referenced by views/rules) + field-shape comparison
  (type change, required-default drop, enum-value removal).
- Wire it into `validateProposal` (replace the phase3 skipped stub).
- Extend `ValidationContext` with optional `ontology` + `strictCompatibility`.

Gating (low-regret, fail-closed per Spec 09 §4.1):
- DEFAULT warn-only — findings are warnings, `passed` is NOT affected.
- `strictCompatibility` escalates findings to blocking errors. New
  `features.strictCompatibility` env flag (true in prod, false in dev/test,
  sibling of `strictValidation`) threads through `mountProposalAPI` so
  PRODUCTION refuses breaking proposals while dev/staging stays advisory.

Backward compatible: when the context carries no ontology, Phase 3 returns
"skipped" — identical to the prior stub. The detector is conservative
(deterministic reference checks; skips opaque code conditions and cross-entity
refs it cannot resolve), so it under-reports rather than false-positive-blocks.
Stays strictly within the validation boundary — no approval/commit/deploy/
graduation/file-write path is touched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@laofahai laofahai force-pushed the feat/validation-phase3-compatibility branch from b69a41d to 1cedd34 Compare June 7, 2026 00:56
@laofahai laofahai merged commit 9626920 into main Jun 7, 2026
6 checks passed
@laofahai laofahai deleted the feat/validation-phase3-compatibility branch June 7, 2026 01:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant