Skip to content

feat(security): add issue inventory and evidence-classification foundation - #911

Closed
seonghobae wants to merge 12 commits into
developfrom
agent/issue-detection-contract
Closed

feat(security): add issue inventory and evidence-classification foundation#911
seonghobae wants to merge 12 commits into
developfrom
agent/issue-detection-contract

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Scope

This PR is an inventory and evidence-classification foundation, not completed issue-level detector efficacy.

Current machine-checked measurements:

  • registered issue inventory: 414/414;
  • registered claim rows: 417;
  • unique (classifier_family, claim_id) semantics: 20;
  • classifier families: 17;
  • formally cause-bound issues: 0/414;
  • independently validated direct-detector claims: 0/417; and
  • protected-develop operational proof: 0/414.

A failure collector, issue number, title/body digest, generic family fixture, log regex, caller-provided Boolean/list, or signed opaque upstream outcome does not count as AppGuardrail directly detecting the underlying condition.

What this PR adds

  • read-only complete issue inventory and requirement-digest reconciliation;
  • closed family-classifier schemas and fail-closed unknown handling;
  • bounded workflow-observation provenance and multi-result classification;
  • CLI/package/release contracts;
  • a canonical documentation graph: Architecture, PRD, TRD, five indexed ADRs, UML, conceptual and physical ERDs, threat model, test strategy, operability guide, incident runbook, and bidirectional requirement traceability;
  • docs/issue-detection-traceability.json plus a deliberately bounded topology/count/status validator;
  • exact event-head checkout and assertion in both issue-classification documentation workflows;
  • reconciled AGENTS/CLAUDE guidance, methodology capability states, historical collector boundaries, standards references, changelog ownership, and public HTTPS examples.

Independent documentation re-audit

Protected develop@0d07baae44a40edfcaec5e42c7fb9351510ca9f0 was PARTIAL/MISSING for the canonical issue-detection documentation graph. The active PR now provides an honest, status-bearing reconstruction, including:

  • ADR-0001 through ADR-0005 separating inventory/collector boundaries, evidence authority, typed aggregation, independent oracles/mutation proof, and legacy/canonical-v2 persistence;
  • ERD corrections for composite claim/obligation identity and actual purge preview-to-receipt cardinality;
  • explicit ACTIVE_PR, PARTIAL, and MISSING states in Architecture, threat model, operability, and incident response;
  • ownership, escalation, evidence locations, commands, recovery gaps, and the absence of a staffed on-call/SLO and managed backup/restore proof;
  • a narrower validator that checks documentation topology and declared-state consistency without claiming prose truth, detector efficacy, remote ancestry, or standards currency.

The implementation remains incomplete:

  • issue→atomic cause→obligation→source probe/acquirer→typed outcome→independent oracle→test/evidence binding is missing;
  • current adapters largely classify caller-provided Boolean/list/log/signed outcomes instead of acquiring source-authoritative evidence;
  • workflow DetectionResult and family FamilyAssessment taxonomies are not unified;
  • legacy v1 envelopes remain classifier-only, while v2 now HMAC-binds repository and source-artifact SHA-256 identity; source acquisition and independent artifact verification are still missing;
  • retention/audit v2 schema exists, but main control-plane/purge/API integration is partial;
  • exact branch coverage, live alerting, staffed incident response, recovery objectives, and protected-main live efficacy are not evidenced.

Verification at exact head

Local verification on the exact tree published as a71a51daca45dfc47a362a26e09a60a035c92a91:

  • full repository: 987 passed, 1,805 subtests passed;
  • focused issue-detection and release suite: 58 passed, 1,805 subtests passed;
  • appguardrail_core/issue_detection.py: 1,363/1,363 statements (100%);
  • focused documentation-contract suite: 19 passed;
  • appguardrail_core/issue_detection_docs.py: 389/389 statements (100%);
  • JSON parsing, compile, and git diff --check validation passed.

The seven published remote blobs exactly match the locally tested Git object IDs and there are 0 unresolved review threads. Exact-head hosted checks for the new head are pending and must not be inferred as passing. Even successful checks validate the foundation only; they cannot satisfy the direct-efficacy blockers below.

Merge blockers / next queue

  1. Add claim-specific trusted probe_ref/acquirer_ref, independently acquired source identity, atomic cause, obligation binding, and typed assessments.
  2. Add an independent reviewed source-evidence corpus; registry-generated fixtures cannot be their own oracle.
  3. Add source-bound black-box and production-predicate mutation tests, including real [security-failure] ContextualWisdomLab/.github: OpenCode Review Dispatch ContextualWisdomLab/.github#701@3a15867168d39a248b92c14f6db0e63584e8dc22 #815/[security-failure] ContextualWisdomLab/newsdom-api: Strix Security Scan ContextualWisdomLab/newsdom-api#501@440c702b29614649d12924643687e5d028521f6d #813/[security-failure] ContextualWisdomLab/.github: OpenCode Review Dispatch ContextualWisdomLab/scopeweave#386@644e9fc5cb3adfb96e2948152f92c61f8661e6d3 #763 replays and multi-cause order/duplicate behavior.
  4. Execute the same production path in a live exact-head audit.
  5. Obtain required independent review and verify the merged protected-develop commit.

The PR must remain Draft until these boundaries are implemented and exact-head gates permit promotion.

@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9a064271-4db7-4bb8-82cd-2f641cfcf7eb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Comment thread appguardrail_core/issue_detection.py Fixed
Comment thread appguardrail_core/issue_detection.py Fixed
@seonghobae
seonghobae marked this pull request as ready for review August 9, 2026 12:15
@seonghobae seonghobae changed the title feat(security): make every issue an executable detection contract feat(security): add issue inventory and evidence-classification foundation Aug 9, 2026
@seonghobae
seonghobae marked this pull request as draft August 9, 2026 13:11
seonghobae added a commit that referenced this pull request Aug 12, 2026
Merge protected develop into the canonical documentation branch, promote the separately verified webhook prevention and bounded scanner-detection controls, keep PR #911 labelled active, and update executable documentation contracts.

Copy link
Copy Markdown
Contributor Author

Closing this draft and preserving the branch as an evidence source rather than merging or rebasing it wholesale. The PR itself records 0/414 cause-bound issues, 0/417 independently validated direct-detector claims, and 0/414 protected-branch operational proofs; it is also a 20k-line, non-mergeable change based on an obsolete develop. Those are architecture-level merge blockers, not a finite review cleanup. The replacement execution unit is issue #938: one source-authoritative detector vertical slice with independent oracle, production-path mutation proof, 100% coverage/docstrings, and exact-head security/approval gates. Reuse only minimal reviewed pieces from this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant