Feat/requirements doors db - #695
Conversation
Add standalone SQLite schema for tracking requirements with IBM DOORS-style traceability and change control. Three tables: requirement (core object with GitHub issue origin and implementation refinement), requirement_link (typed directional traceability with suspect flagging), and requirement_change (append-only field-level change log). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Populate the requirements database with 134 requirements derived from cath-service GitHub issues that have moved to "Refined Tickets" or beyond on the CaTH Kanban board (project 43). - seed.sql: 134 requirement rows (REQ-0001..REQ-0134, ordered by issue number) plus one 'created' change row each. Generated from the board and issues, not hand-edited; later changes go in migrations/. - schema.sql: add 'in_progress' status (distinguishes ready-to-pick-up from in-flight work); replace MoSCoW priority with the repo's actual label scale (highest/high/medium/low/lowest); add nullable granularity (epic/story/task). - scripts/init_db.sh builds the SQLite DB (schema -> seed -> migrations); new_migration.sh scaffolds numbered migrations. - requirements:build yarn script wraps init_db.sh. - gitignore requirements/*.db (build artefact; rebuilt from SQL). - biome: ignore .claude/ so the post-write hook stops tripping on worktrees. kind classified from ticket content (113 functional, 12 constraint, 9 non_functional). priority/granularity from labels, NULL where absent. requirement_link left empty: cross-references were not trivially typeable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Populate requirement_link and the impl_* columns, both derived from real GitHub data rather than inferred from topic similarity. - requirement_link: 21 links. 16 derives_from from native GitHub sub-issue parents (where both ends are in the loaded set); 5 depends_on from cross-references verified in the issue bodies. Topic-similarity links were deliberately not invented. - impl_commit_sha / impl_paths: from each issue's merged closing PR(s). impl_paths is the union of source files across all merged PRs (lockfiles, CI, helm, docs and generated files filtered out); impl_commit_sha is the latest merge commit. Only 19 of 134 have a merged PR — the migrated JIRA tickets were closed administratively and carry no commit trace. This lets the DB answer both "what is this requirement connected to" and "which code is affected by this requirement". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add 66 content-derived traceability links between requirements, kept strictly separable from the 21 structural (fact) links. Schema: - requirement_link gains origin (github_subissue/issue_reference/inferred), confidence, and rationale columns. Structural links are facts; inferred links are judgement and carry is_suspect=1 until a human confirms them. - seed.sql structural links now record their origin (16 github_subissue, 5 issue_reference). migrations/001_inferred_links.sql: - 66 links (57 depends_on, 9 refines) derived from requirement content by a multi-agent analysis: cluster into feature areas -> propose links per cluster -> adversarial verification (each candidate had to survive a skeptic defaulting to rejection). 35 high-confidence (>=0.8). - All origin='inferred', is_suspect=1, with confidence + rationale so they can be reviewed (WHERE origin='inferred' ORDER BY confidence DESC), confirmed (is_suspect=0), or rejected (DELETE) individually. The 21 structural links are untouched. Querying the graph now answers "what is connected to this requirement" with verified and proposed links distinguishable by origin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a scheduled workflow (04:00 UTC, after the nightly E2E run) that keeps the requirements DB in sync with GitHub Project #43. It builds the current DB, reads the live board via the existing GitHub App token (gh api graphql), and has Claude write a numbered migration capturing drift, then opens a PR for human review. Nothing enters the source of truth unreviewed. Drift detected: new requirements crossing the Refined gate, board status changes, and implementation changes (merged closing PRs -> impl_commit_sha / impl_paths). Out of scope: title/statement/priority edits. Mirrors claude.yml (App token, AWS OIDC, Bedrock Opus) and nightly.yml (schedule, node setup). Installs sqlite3 (not present in CI by default). The prompt mandates: no PR when there is no drift; never hard-delete or down-status a requirement; treat NULL-vs-NULL impl as no change (most requirements have no merged PR). A final build + PRAGMA integrity_check/foreign_key_check step runs independently of Claude so a malformed migration cannot sit in a green PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a SQLite requirements traceability schema with build and migration scripts, two data migrations populating the database with inferred links and live GitHub state, a Claude-driven scheduled sync workflow that scaffolds migrations and enforces integrity checks, Claude command specifications for issue shipping and ticket listing workflows, and supporting configuration updates. ChangesRequirements Traceability Database System
Claude Workflow Command Specifications
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🎭 Playwright E2E Test Results84 tests 52 ✅ 3m 53s ⏱️ Results for commit f63cf3f. ♻️ This comment has been updated with latest results. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
package.json (1)
49-50:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUse exact version pinning for all dependencies.
The following dependencies use caret (^) ranges instead of exact version pinning:
@biomejs/cli-linux-arm64: "^2.3.6"happy-dom: "^20.0.5"accessible-autocomplete: "^3.0.1"notifications-node-client: "^8.2.1"As per coding guidelines, all dependencies except peer dependencies must use specific version pinning (e.g.,
"2.3.6"not"^2.3.6").📌 Proposed fix to pin dependency versions
- "`@biomejs/cli-linux-arm64`": "^2.3.6", + "`@biomejs/cli-linux-arm64`": "2.3.6",- "happy-dom": "^20.0.5", + "happy-dom": "20.0.5",- "accessible-autocomplete": "^3.0.1", + "accessible-autocomplete": "3.0.1",- "notifications-node-client": "^8.2.1", + "notifications-node-client": "8.2.1",Also applies to: 58-58, 86-87
Source: Coding guidelines
🧹 Nitpick comments (2)
requirements/schema.sql (1)
54-55: 💤 Low valueConsider enforcing ISO8601 format for timestamps.
created_atandupdated_atare documented as ISO8601 but accept any TEXT. While SQLite'sstrftimeis forgiving, enforcing a basic pattern would prevent malformed timestamps.♻️ Example timestamp validation CHECK constraint
- created_at TEXT NOT NULL, -- ISO8601 - updated_at TEXT NOT NULL, + created_at TEXT NOT NULL CHECK (created_at GLOB '[0-9][0-9][0-9][0-9]-[0-1][0-9]-[0-3][0-9]T*'), + updated_at TEXT NOT NULL CHECK (updated_at GLOB '[0-9][0-9][0-9][0-9]-[0-1][0-9]-[0-3][0-9]T*'),requirements/scripts/init_db.sh (1)
20-24: 💤 Low valueMigration ordering relies on lexicographic glob expansion.
The script applies migrations in the order returned by the glob
*.sql, which is lexicographic. This works correctly with the three-digit zero-padded naming fromnew_migration.sh(001_, 002_, etc.), but the dependency is implicit.Consider adding a comment documenting the ordering assumption, or explicitly sort:
♻️ Optional explicit sort
shopt -s nullglob -for migration in "$REQ_DIR"/migrations/*.sql; do +for migration in "$REQ_DIR"/migrations/[0-9]*.sql; do echo "Applying $(basename "$migration")"Or with explicit sorting:
-for migration in "$REQ_DIR"/migrations/*.sql; do +for migration in $(ls "$REQ_DIR"/migrations/[0-9]*.sql 2>/dev/null | sort -V); do
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: fd95a492-37b3-4674-8b29-3e8a879e434f
📒 Files selected for processing (10)
.github/workflows/requirements-sync.yml.gitignorebiome.jsonpackage.jsonrequirements/migrations/.gitkeeprequirements/migrations/001_inferred_links.sqlrequirements/schema.sqlrequirements/scripts/init_db.shrequirements/scripts/new_migration.shrequirements/seed.sql
| uses: actions/create-github-app-token@v3 | ||
| with: | ||
| app-id: ${{ secrets.CLAUDE_CODE_APP_ID }} | ||
| private-key: ${{ secrets.CLAUDE_CODE_PRIVATE_KEY }} | ||
|
|
||
| - name: Checkout repository | ||
| uses: actions/checkout@v6 | ||
| with: | ||
| token: ${{ steps.app-token.outputs.token }} | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Configure Git | ||
| run: | | ||
| git config user.name "claude-code-bot[bot]" | ||
| git config user.email "claude-code-bot[bot]@users.noreply.github.com" | ||
|
|
||
| - name: Install sqlite3 | ||
| run: sudo apt-get update && sudo apt-get install -y sqlite3 | ||
|
|
||
| - name: Enable Corepack | ||
| run: corepack enable | ||
|
|
||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v6 | ||
| with: | ||
| node-version-file: '.nvmrc' | ||
| cache: 'yarn' | ||
| cache-dependency-path: '**/yarn.lock' | ||
|
|
||
| - name: Install dependencies | ||
| run: yarn install --immutable | ||
|
|
||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v6 | ||
| with: | ||
| role-to-assume: arn:aws:iam::313941174580:role/HMCTSClaudeGitHubActionsRole | ||
| aws-region: eu-west-1 | ||
|
|
||
| - name: Run Claude Code - Requirements Sync | ||
| id: claude | ||
| uses: anthropics/claude-code-action@v1 |
There was a problem hiding this comment.
Pin actions to commit SHAs for supply chain security.
Static analysis flags multiple actions using mutable version tags (@v1, @v3, @v6). Pinning to full commit SHAs prevents tag hijacking attacks where a malicious actor could push a compromised version to an existing tag.
🔒 Example pinning pattern
- - uses: actions/create-github-app-token@v3
+ - uses: actions/create-github-app-token@<full-sha> # v3
- - uses: actions/checkout@v6
+ - uses: actions/checkout@<full-sha> # v6
- - uses: actions/setup-node@v6
+ - uses: actions/setup-node@<full-sha> # v6
- - uses: aws-actions/configure-aws-credentials@v6
+ - uses: aws-actions/configure-aws-credentials@<full-sha> # v6
- - uses: anthropics/claude-code-action@v1
+ - uses: anthropics/claude-code-action@<full-sha> # v1Add a trailing comment with the version tag for readability.
🧰 Tools
🪛 zizmor (1.25.2)
[warning] 44-48: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 39-39: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 45-45: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 62-62: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 72-72: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 79-79: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 39-39: dangerous use of GitHub App tokens (github-app): app token inherits blanket installation permissions
(github-app)
Source: Linters/SAST tools
There was a problem hiding this comment.
Won't fix here. The repo's other ~22 workflows (claude.yml, nightly.yml, etc.) all use version tags, so SHA-pinning only this file would be inconsistent for little security benefit while the others stay unpinned. SHA-pinning is worth doing repo-wide as a separate change/ticket rather than piecemeal on this PR.
There was a problem hiding this comment.
@SarahLittlejohn, understood — pinning only one file while the rest remain on version tags is inconsistent. Would you like me to open a GitHub issue to track the repo-wide SHA-pinning work as a follow-up?
- schema.sql: CHECK(json_valid(impl_paths)) so the nightly sync's LLM-written JSON can't corrupt downstream json_each queries; CHECK confidence in [0,1] matching the inferred-link semantics. Both verified against existing data (no violations) and confirmed to reject bad input. - schema.sql: comment clarifying change_type='deleted' is reserved for manual operations and intentionally unused by the automated sync. - requirements-sync.yml: persist-credentials: false on checkout; set the App token on the origin remote explicitly in Configure Git so the bot push still works without git storing credentials in .git/config. Declined (out of scope / low value): SHA-pinning actions (kept @v6 tags to match the repo's other workflows), package.json exact-pinning of pre-existing deps, timestamp GLOB CHECK, and init_db.sh glob-ordering tweak. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CodeRabbit review — response summaryAddressed in
Declined, with rationale:
All schema changes verified: |
Reconciles the requirements baseline with the live state of GitHub Project #43 as of 2026-06-18, as migration 002: - 3 new requirements: REQ-0135 (#569), REQ-0136 (#716), REQ-0137 (#729) - 6 status changes: REQ-0105, 0107, 0109, 0112, 0121, 0132 - 4 status + impl changes: REQ-0111, 0119, 0133, 0134 (now verified, with merged-PR commit sha and impl_paths) - 4 regressions recorded as status_changed history (DOORS-faithful): REQ-0126 -> draft, REQ-0129/0130/0131 -> proposed; flagged for review DB builds clean at 137 requirements; integrity_check ok and foreign_key_check empty. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7cbb7142-3491-4115-aaac-783245ce2ae9
📒 Files selected for processing (2)
.claude/commands/qk-ship.md.claude/commands/qk-tickets.md
| ## Usage | ||
|
|
||
| ``` | ||
| /list-tickets |
| **Worktree Setup:** | ||
|
|
||
| Branch name: issue-$ARGUMENT | ||
| Worktree path: ../cath-$ARGUMENT |
There was a problem hiding this comment.
typically I think it will create worktrees in .claude/worktrees/[some-name]
| - Conflicts: docs/tickets/$ARGUMENT/conflicts.md | ||
| - Checklist: docs/tickets/$ARGUMENT/checklist.md | ||
| - Codebase Findings: docs/tickets/$ARGUMENT/findings.md | ||
| - Architecture: @CLAUDE.md |
There was a problem hiding this comment.
You don't need to tell it to read CLAUDE.md its always in context
| ## Testing Approach | ||
| - Unit tests required | ||
| - E2E test scenarios | ||
| - Accessibility testing |
There was a problem hiding this comment.
careful on this one, it may opt to create separate a11y tests - I believe they are currently embedded in the e2e tests.
| - Don't rely on memory - actually check the files | ||
| - If checklist says 'register in X', verify X was modified | ||
|
|
||
| **CRITICAL - Do NOT return until COMPLETE:** |
There was a problem hiding this comment.
I think we've overused CRITICAL
| - You are working in a git worktree at ../worktrees/issue-$ARGUMENT | ||
| - This is a separate working directory with its own branch | ||
| - The main repository is unaffected by your changes | ||
| - Multiple agents can work in parallel in their own worktrees |
There was a problem hiding this comment.
You should add the note about not using stash.
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.claude/commands/qk-tickets.md (1)
16-21:⚠️ Potential issue | 🔴 CriticalCorrect the link type references in the implementation prompt.
The
issue_numberfield exists in the schema, but the prompt references link types that are not defined. The requirement_link.type column allows only'derives_from','refines','satisfies','depends_on', and'conflicts_with'. The prompt uses'blocks'and'related_to', which will cause runtime SQL constraint violations. Replace these with valid types from the schema or update the schema to include them.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d05c3119-d2fb-4756-9e98-55344569e3f4
📒 Files selected for processing (2)
.claude/commands/qk-ship.md.claude/commands/qk-tickets.md
✅ Files skipped from review due to trivial changes (1)
- .claude/commands/qk-ship.md
Add .mcp.json pointing at GitHub's hosted MCP server (read-only, scoped to issues/pull_requests/repos/projects). run.sh exports GITHUB_MCP_TOKEN from the already-authenticated gh CLI — ephemeral, never written to disk. CI is unchanged and continues to use the gh CLI directly. Documented in docs/GITHUB_MCP.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Multi-agent workflow that infers traceability links between requirements: cluster into feature areas -> propose candidate links per cluster -> adversarially verify each candidate. Output is inserted into the requirements DB as origin='inferred', is_suspect=1 links (see migrations/001_inferred_links.sql). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two inferred traceability links for the requirements added by the 002 GitHub sync, derived from live issue text read via the GitHub MCP server: 135 -> 127 depends_on (storage.tf presupposes the bootstrap Key Vault infra) 137 -> 86 refines (footer font size refines the header/footer update) Both origin='inferred', is_suspect=1, following 001's conventions. None of the three new issues (569/716/729) has sub-issues, so there are no structural links to record. Weak topic-only candidates were considered and dropped (documented in the migration header). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The database is derived entirely from schema.sql + seed.sql + migrations/*.sql, which init_db.sh applies in order. The migrations describe what is in it, so the binary adds nothing a clone cannot rebuild — and committing it means every sync PR carries a ~2MB binary diff that no reviewer can read. .gitignore has listed requirements/*.db since #695, but the file was already tracked by then: it was swept into #772 (an unrelated list-types PR) the same day, and gitignore does not apply to tracked files. Remove it from the index; it stays on disk and is now correctly ignored. Rebuild with: yarn requirements:build Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
4 participants
Jira link
See PROJ-XXXXXX
Change description
Testing done
Security Vulnerability Assessment
CVE Suppression: Are there any CVEs present in the codebase (either newly introduced or pre-existing) that are being intentionally suppressed or ignored by this commit?
Checklist
Summary by CodeRabbit
requirements:buildcommand, and updated ignore/format settings for generated DB artefacts and.claudepaths./qk-ticketscommand to list approved tickets from the requirements database.