feat: v0.13.0 hot policy reload, OVERT Phase 3 IAP, named detectors, HTML dashboard#82
Merged
Conversation
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (26)
📝 WalkthroughWalkthroughRelease v0.13.0 adds OVERT Phase 3 reference attestation provider with in-process transparency log, named injection/PII detectors, hot policy reload with atomic semantics, HTTP endpoints for detection and reload, compliance dashboard HTML renderer, and CLI commands for operator tooling. Package version increments to 0.13.0. ChangesVaara v0.13.0 Release
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| print(f"vaara policy reload: HTTP {exc.code}", file=sys.stderr) | ||
| try: | ||
| print(exc.read().decode("utf-8"), file=sys.stderr) | ||
| except Exception: |
vaaraio
force-pushed
the
feat/hot-policy-reload
branch
from
944a3e6 to
101afb4
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🧹 Nitpick comments (4)
src/vaara/server/schemas.py (2)
208-213: ⚡ Quick winAdd bounds to
PolicyReloadResponseprimitives.
versionand counts should be non-negative/monotonic at the schema layer, consistent with the API contract.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/vaara/server/schemas.py` around lines 208 - 213, The PolicyReloadResponse schema allows negative values for version and count fields; update class PolicyReloadResponse so that version, sequence_count, action_class_count, and escalation_route_count are constrained to be non-negative (e.g., use pydantic Field(..., ge=0) or an appropriate non-negative int type) to enforce the API contract at the schema layer; modify the field declarations for version, sequence_count, action_class_count, and escalation_route_count in PolicyReloadResponse accordingly.
173-192: ⚡ Quick winTighten response enums and numeric bounds to match the documented contract.
The current models accept broader values than the OpenAPI contract (
backend,category,offset,length), which can hide regressions.✅ Suggested tightening
class DetectInjectionResponse(BaseModel): detected: bool score: float = Field(ge=0, le=1) threshold: float = Field(ge=0, le=1) bundle_version: str - backend: str + backend: Literal["vaara_adversarial", "heuristic"] class DetectPIIFinding(BaseModel): - category: str + category: Literal["email", "phone", "ssn", "ipv4", "credit_card", "iban"] value: str - offset: int - length: int + offset: int = Field(ge=0) + length: int = Field(ge=1)tests/test_policy_reload_http.py (1)
94-98: ⚡ Quick winAssert the error code for the “neither path nor body” branch.
This test currently checks only HTTP status; asserting
error.codewould lock the API contract like the sibling negative tests do.🧪 Suggested assertion
def test_reload_rejects_neither_path_nor_body(client_with_controller): client, _ = client_with_controller r = client.post("/v1/policy/reload", json={}) assert r.status_code == 400 + assert r.json()["error"]["code"] == "invalid_request"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_policy_reload_http.py` around lines 94 - 98, Update test_reload_rejects_neither_path_nor_body to assert the JSON error code in addition to the HTTP 400: after calling client.post("/v1/policy/reload", json={}) parse r.json(), ensure there is an "error" object and assert its "code" equals the same error code used by the sibling negative tests for the "neither path nor body" branch (match the value used in those tests), referencing the test function name test_reload_rejects_neither_path_nor_body and the POST to "/v1/policy/reload" to locate where to add the assertion.docs/openapi.yaml (1)
561-577: ⚡ Quick winEncode
path/bodyXOR in schema (not only description text).Right now the constraint is prose-only; schema-based clients can’t enforce it preflight.
📌 Proposed spec patch
PolicyReloadRequest: type: object description: | Exactly one of `path` or `body` must be supplied. `path` lets the server read the policy file directly; `body` carries an inline parsed document for cases where the operator runs on a different host than the server. + oneOf: + - required: [path] + - required: [body] properties: path: { type: string, maxLength: 4096 } body: type: object additionalProperties: true🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/openapi.yaml` around lines 561 - 577, PolicyReloadRequest currently enforces the "exactly one of path or body" rule only in prose; update the schema to express this using oneOf so tools can validate it: replace the flat properties block with a oneOf containing two subschemas — one that requires "path" (and defines path: {type: string, maxLength: 4096} and allows "format") and one that requires "body" (and defines body: {type: object, additionalProperties: true} and allows "format"); keep the existing "format" enum/nullable and the original path/body property schemas (including maxLength and additionalProperties) inside those subschemas so validators enforce the XOR constraint for PolicyReloadRequest.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CHANGELOG.md`:
- Around line 9-73: The CHANGELOG entry header "## [0.13.0] - 2026-05-17" has
the wrong date; verify whether the release was actually on 2026-05-17 or still
pending and update the header in CHANGELOG.md accordingly (either set the
correct release date or use an "Unreleased"/date-to-be-filled marker), making
sure the version header "## [0.13.0] - 2026-05-17" is edited to the confirmed
value.
In `@docs/openapi.yaml`:
- Line 171: Top-level OpenAPI tags are missing entries for the tags used
elsewhere; add global tag declarations for "detect" and "policy" in the
document's top-level tags array so generated docs can show their descriptions
and grouping metadata. Specifically, create entries named detect and policy
(e.g., tags: - name: detect description: "..." and - name: policy description:
"...") with appropriate descriptions (and optional externalDocs) so the API
operations that reference the detect and policy tags pick up their metadata.
In `@src/vaara/attestation/iap.py`:
- Around line 147-149: The append followed by inclusion_proof is non-atomic
(transparency_log.append(envelope_cbor) then
transparency_log.inclusion_proof(log_entry.log_index)), so concurrent appends
can produce a proof from a different tree snapshot; implement a single atomic
operation on the log (e.g., add a new method
InProcessTransparencyLog.append_with_inclusion_proof(envelope_cbor) that is
executed under the same lock) that returns both LogEntry and InclusionProof
together, then replace both call sites (the current append + inclusion_proof
pairs around envelope_cbor/log_entry.log_index, including the similar block at
lines 163-166) to call this new append_with_inclusion_proof so log_entry and
proof are derived from the same snapshot.
- Around line 230-245: The structural-independence check that ensures the
envelope's "key_identifier" differs from attestation.notary_key_identifier must
run regardless of whether arbiter_public_key_raw is provided: move the decoded
cbor parsing and the comparisons using decoded.get("key_identifier") and
_sha256(arbiter_public_key_raw) (and the check against
attestation.notary_key_identifier) out of the arbiter_public_key_raw-only branch
so the notary-vs-arbiter collision check is unconditional; keep only the arbiter
signature-specific verification (e.g., importing cbor2 solely for signature
verification and any signature validation logic that depends on
arbiter_public_key_raw) inside the conditional so callers can omit arbiter
signature verification but cannot bypass the structural independence check.
In `@src/vaara/compliance/dashboard.py`:
- Line 91: Remove the unnecessary f-string prefix on the HTML literal causing
Ruff F541: replace the f'<div class="card">' occurrence in
src/vaara/compliance/dashboard.py with a normal string literal '<div
class="card">'. Locate the same change wherever the identical f-prefixed static
HTML fragment appears (e.g., inside the HTML-building block or function that
constructs the card markup) to eliminate the redundant f-string usage and
unblock CI.
In `@src/vaara/detect/injection.py`:
- Line 106: The code currently coerces threshold to float (th =
_DEFAULT_THRESHOLD if threshold is None else float(threshold)) but does not
validate it; update the handling of the input parameter `threshold` (and
resulting local `th`) to explicitly check that the value is finite and within
[0, 1], using math.isfinite after conversion and raising a ValueError with a
clear message if the check fails; keep the None fallback to _DEFAULT_THRESHOLD
and perform these checks on the final `th` before it is used for scoring
(reference the symbols `threshold` and `th` in injection.py).
In `@src/vaara/policy/controller.py`:
- Around line 67-70: The current code appends a listener to self._listeners
before calling listener(self._policy), which can leave a broken listener
registered if its initial apply raises; change the logic in the method in
controller.py so you first invoke listener(self._policy) and only if that call
succeeds append the listener to self._listeners (use self._lock around the
append to keep thread-safety); if the initial call raises, do not append the
listener (either re-raise or log/handle the exception as appropriate).
- Around line 102-105: The code path handling fmt currently only checks for
"yaml" and "json" and silently falls through to auto-detection for any other
value; update the reload handler (the branch that examines the fmt variable in
src/vaara/policy/controller.py) to explicitly reject unsupported formats by
raising a clear exception (e.g., ValueError) when fmt is not "json" or "yaml"
instead of falling back, and include the invalid fmt value in the error message;
keep the existing calls to from_yaml(source) and from_json(source) for the
supported cases.
In `@src/vaara/scorer/adaptive.py`:
- Around line 687-700: The apply_policy method currently annotates its parameter
as the string literal "Policy"; remove the unnecessary quotes and use the real
type name Policy in the signature (def apply_policy(self, policy: Policy) ->
None) since the module already conditionally imports Policy from
vaara.policy.schema; update the annotation only (no other logic changes) and
keep the local import/check and isinstance(policy, Policy) validation intact.
In `@tests/test_detect.py`:
- Around line 123-129: The test currently catches ImportError across both
imports which can hide real failures; update the try/except so only the optional
dependency import for TestClient is caught and handled (e.g., try: from
fastapi.testclient import TestClient except ImportError: TestClient = None)
while importing create_app from vaara.server stays outside that except so any
ImportError from create_app propagates and fails the tests; reference TestClient
and create_app in tests/test_detect.py to locate and adjust the import blocks
accordingly.
---
Nitpick comments:
In `@docs/openapi.yaml`:
- Around line 561-577: PolicyReloadRequest currently enforces the "exactly one
of path or body" rule only in prose; update the schema to express this using
oneOf so tools can validate it: replace the flat properties block with a oneOf
containing two subschemas — one that requires "path" (and defines path: {type:
string, maxLength: 4096} and allows "format") and one that requires "body" (and
defines body: {type: object, additionalProperties: true} and allows "format");
keep the existing "format" enum/nullable and the original path/body property
schemas (including maxLength and additionalProperties) inside those subschemas
so validators enforce the XOR constraint for PolicyReloadRequest.
In `@src/vaara/server/schemas.py`:
- Around line 208-213: The PolicyReloadResponse schema allows negative values
for version and count fields; update class PolicyReloadResponse so that version,
sequence_count, action_class_count, and escalation_route_count are constrained
to be non-negative (e.g., use pydantic Field(..., ge=0) or an appropriate
non-negative int type) to enforce the API contract at the schema layer; modify
the field declarations for version, sequence_count, action_class_count, and
escalation_route_count in PolicyReloadResponse accordingly.
In `@tests/test_policy_reload_http.py`:
- Around line 94-98: Update test_reload_rejects_neither_path_nor_body to assert
the JSON error code in addition to the HTTP 400: after calling
client.post("/v1/policy/reload", json={}) parse r.json(), ensure there is an
"error" object and assert its "code" equals the same error code used by the
sibling negative tests for the "neither path nor body" branch (match the value
used in those tests), referencing the test function name
test_reload_rejects_neither_path_nor_body and the POST to "/v1/policy/reload" to
locate where to add the assertion.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: ff522a8a-d9e8-4f24-b8d7-6b50e412f666
📒 Files selected for processing (26)
CHANGELOG.mdCOMPLIANCE.mdREADME.mddocs/openapi.yamlpyproject.tomlsrc/vaara/__init__.pysrc/vaara/attestation/__init__.pysrc/vaara/attestation/iap.pysrc/vaara/attestation/transparency_log.pysrc/vaara/cli.pysrc/vaara/compliance/dashboard.pysrc/vaara/detect/__init__.pysrc/vaara/detect/injection.pysrc/vaara/detect/pii.pysrc/vaara/policy/__init__.pysrc/vaara/policy/controller.pysrc/vaara/scorer/adaptive.pysrc/vaara/server/app.pysrc/vaara/server/routes.pysrc/vaara/server/schemas.pysrc/vaara/server/state.pytests/test_attestation_iap.pytests/test_compliance_dashboard.pytests/test_detect.pytests/test_policy_controller.pytests/test_policy_reload_http.py
Comment on lines
+9
to
+73
| ## [0.13.0] - 2026-05-17 | ||
|
|
||
| **Theme: operator surface + OVERT Phase 3 path.** Four additions that | ||
| close the most legible competitive gaps without diluting the kernel | ||
| position. Hot policy reload meets the Galileo Agent Control selling | ||
| point on its own ground. The OVERT 1.0 Phase 3 Independent Attestation | ||
| Provider (IAP) reference closes the AAL-3 → AAL-4 promotion path that | ||
| v0.11.0's Provisional Receipt opens, so Vaara owns the full path | ||
| without forcing dependence on an external IAP vendor. Named injection | ||
| and PII detectors expose existing scoring surface under buyer-visible | ||
| labels. A static HTML article-coverage dashboard adds the auditor- | ||
| facing visual artefact that the peer set has converged on. | ||
|
|
||
| ### Added | ||
| - **Hot policy reload.** New `vaara.policy.controller.PolicyController` | ||
| owns the live `Policy` and runs registered listeners under a write | ||
| lock on `reload()`. `AdaptiveScorer.apply_policy(policy)` rebinds | ||
| thresholds and sequence patterns atomically under the scorer's own | ||
| RLock; an `evaluate()` call in flight on another thread either sees | ||
| the old `(allow, deny)` pair or the new one, never a torn half. | ||
| Conformal calibration, MWU expert state, and agent profiles are | ||
| preserved across reloads. Malformed reloads are rejected with the | ||
| previous policy left live. `POST /v1/policy/reload` accepts a | ||
| server-side path or an inline body; `vaara serve --policy PATH` | ||
| enables the endpoint; `vaara policy reload POLICY_PATH` triggers | ||
| reload over HTTP from the operator's shell. | ||
| - **OVERT 1.0 AAL-4 Phase 3 IAP reference.** New | ||
| `vaara.attestation.iap` ships a `Phase3Attestation` dataclass that | ||
| wraps a Vaara `BaseEnvelope` with a notary Ed25519 signature (over a | ||
| domain-separated prefix + canonical-CBOR of the inner envelope | ||
| including its signature) and a transparency-log inclusion proof. | ||
| Structural independence between the Arbiter key and the notary key | ||
| is enforced at both emit and verify. New | ||
| `vaara.attestation.transparency_log.InProcessTransparencyLog` | ||
| implements an RFC 6962-style binary Merkle tree with domain- | ||
| separated leaf and internal hashes; `append()` / | ||
| `inclusion_proof()` / `root_hash` match the shape a sigstore Rekor | ||
| adapter would expose, so a production deployment can swap in Rekor | ||
| at the same call sites without changing the IAP contract. | ||
| - **Named injection + PII detector aliases.** `vaara.detect.detect_injection` | ||
| routes free text through the same AdversarialClassifier behind | ||
| vaara-bench-v1's published numbers (heuristic fallback when the ml | ||
| extra is absent; the `backend` field reports which path served the | ||
| call). `vaara.detect.detect_pii` is a zero-dependency regex extractor | ||
| over six categories — email, phone, US SSN, IPv4, credit_card | ||
| (Luhn-checked), IBAN (mod-97 checksum). `POST /v1/detect/injection` | ||
| and `POST /v1/detect/pii` mirror the CLI. `vaara detect injection` | ||
| and `vaara detect pii` read text from `--text`, `--file`, or | ||
| `--stdin` and exit non-zero when the detector fires. | ||
| - **Static HTML article-coverage dashboard.** | ||
| `vaara.compliance.dashboard.render_html` produces a single | ||
| self-contained HTML page with embedded CSS, no JavaScript, no | ||
| external assets, no network calls. Same content as the Markdown | ||
| renderer (system metadata, audit-trail integrity, summary, critical | ||
| gaps, per-domain article tables, detailed per-article sections) with | ||
| status badges as colored pills and a print-friendly stylesheet. | ||
| `vaara compliance dashboard --db PATH --out PATH` writes the page; | ||
| a trailing slash or existing directory drops `index.html` inside. | ||
| - **OpenAPI spec coverage.** `docs/openapi.yaml` adds | ||
| `/v1/detect/injection`, `/v1/detect/pii`, and `/v1/policy/reload` | ||
| with full request and response schemas. The spec remains the | ||
| authoritative integration surface. | ||
| - 53 new tests (14 PolicyController + reload HTTP; 11 IAP + | ||
| transparency-log; 19 detect (injection + PII + HTTP); 9 HTML | ||
| dashboard). Total 586 passing, 12 skipped. |
There was a problem hiding this comment.
Verify the release date.
The changelog shows release date 2026-05-17, but according to the PR context the current date is May 16, 2026. If this PR hasn't merged yet, the release date should either match the actual release date or be updated when the release is cut.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@CHANGELOG.md` around lines 9 - 73, The CHANGELOG entry header "## [0.13.0] -
2026-05-17" has the wrong date; verify whether the release was actually on
2026-05-17 or still pending and update the header in CHANGELOG.md accordingly
(either set the correct release date or use an "Unreleased"/date-to-be-filled
marker), making sure the version header "## [0.13.0] - 2026-05-17" is edited to
the confirmed value.
|
|
||
| /v1/detect/injection: | ||
| post: | ||
| tags: [detect] |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Check if the file exists and get its size
ls -lh docs/openapi.yaml && wc -l docs/openapi.yamlRepository: vaaraio/vaara
Length of output: 145
🏁 Script executed:
# Read the global tags section (around lines 30-32)
sed -n '25,45p' docs/openapi.yamlRepository: vaaraio/vaara
Length of output: 778
🏁 Script executed:
# Read line 171 to verify the detect tag usage
sed -n '165,180p' docs/openapi.yamlRepository: vaaraio/vaara
Length of output: 655
🏁 Script executed:
# Read line 215 to verify the policy tag usage
sed -n '210,220p' docs/openapi.yamlRepository: vaaraio/vaara
Length of output: 515
🏁 Script executed:
# Find all tags used in operation definitions
rg 'tags:\s*\[' docs/openapi.yaml -o | sort | uniqRepository: vaaraio/vaara
Length of output: 65
🏁 Script executed:
# Get all tags used in operation definitions with context
rg 'tags:\s*\[(\w+)\]' docs/openapi.yaml -o --replace '$1' | sort | uniqRepository: vaaraio/vaara
Length of output: 90
🏁 Script executed:
# Verify the exact global tags declaration
sed -n '30,35p' docs/openapi.yamlRepository: vaaraio/vaara
Length of output: 251
Declare detect and policy in the global OpenAPI tags list.
Lines 171 and 215 reference tags not declared in the top-level catalog (lines 30–32). Without these declarations, generated documentation loses tag descriptions and grouping metadata.
📌 Proposed spec patch
tags:
- { name: score, description: Risk evaluation }
- { name: audit, description: Hash-chained audit trail }
- { name: server, description: Server identity and liveness }
+ - { name: detect, description: Prompt-injection and PII detection }
+ - { name: policy, description: Runtime policy operations }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/openapi.yaml` at line 171, Top-level OpenAPI tags are missing entries
for the tags used elsewhere; add global tag declarations for "detect" and
"policy" in the document's top-level tags array so generated docs can show their
descriptions and grouping metadata. Specifically, create entries named detect
and policy (e.g., tags: - name: detect description: "..." and - name: policy
description: "...") with appropriate descriptions (and optional externalDocs) so
the API operations that reference the detect and policy tags pick up their
metadata.
Comment on lines
+147
to
+149
| log_entry: LogEntry = transparency_log.append(envelope_cbor) | ||
| proof: InclusionProof = transparency_log.inclusion_proof(log_entry.log_index) | ||
|
|
There was a problem hiding this comment.
Make append + inclusion-proof capture atomic.
Line 147 and Line 148 are separate calls, so concurrent appends can produce inclusion_proof_siblings for a newer tree while log_tree_size/log_root_at_append remain at append-time values. That can create internally inconsistent attestations and intermittent verification failures.
Suggested direction
- log_entry: LogEntry = transparency_log.append(envelope_cbor)
- proof: InclusionProof = transparency_log.inclusion_proof(log_entry.log_index)
+ log_entry, proof = transparency_log.append_with_inclusion_proof(envelope_cbor)Implement append_with_inclusion_proof() inside InProcessTransparencyLog under one lock so log_entry and proof are derived from the same tree snapshot.
Also applies to: 163-166
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/vaara/attestation/iap.py` around lines 147 - 149, The append followed by
inclusion_proof is non-atomic (transparency_log.append(envelope_cbor) then
transparency_log.inclusion_proof(log_entry.log_index)), so concurrent appends
can produce a proof from a different tree snapshot; implement a single atomic
operation on the log (e.g., add a new method
InProcessTransparencyLog.append_with_inclusion_proof(envelope_cbor) that is
executed under the same lock) that returns both LogEntry and InclusionProof
together, then replace both call sites (the current append + inclusion_proof
pairs around envelope_cbor/log_entry.log_index, including the similar block at
lines 163-166) to call this new append_with_inclusion_proof so log_entry and
proof are derived from the same snapshot.
Comment on lines
+230
to
+245
| if arbiter_public_key_raw is not None: | ||
| try: | ||
| import cbor2 | ||
| except ImportError as exc: | ||
| raise EnvelopeError( | ||
| "cbor2 not installed. Install with: pip install " | ||
| "'vaara[attestation]'" | ||
| ) from exc | ||
|
|
||
| decoded = cbor2.loads(attestation.envelope_cbor) | ||
| if not isinstance(decoded, dict): | ||
| return False | ||
| if _sha256(arbiter_public_key_raw) != decoded.get("key_identifier"): | ||
| return False | ||
| if decoded.get("key_identifier") == attestation.notary_key_identifier: | ||
| return False |
There was a problem hiding this comment.
Enforce structural independence even when arbiter signature verification is skipped.
Right now the arbiter/notary key-collision check is gated by arbiter_public_key_raw is not None (Line 230). That allows a caller to bypass the Phase 3 structural-independence validation by omitting the arbiter key. The independence check should be unconditional; only inner arbiter signature verification should remain optional.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/vaara/attestation/iap.py` around lines 230 - 245, The
structural-independence check that ensures the envelope's "key_identifier"
differs from attestation.notary_key_identifier must run regardless of whether
arbiter_public_key_raw is provided: move the decoded cbor parsing and the
comparisons using decoded.get("key_identifier") and
_sha256(arbiter_public_key_raw) (and the check against
attestation.notary_key_identifier) out of the arbiter_public_key_raw-only branch
so the notary-vs-arbiter collision check is unconditional; keep only the arbiter
signature-specific verification (e.g., importing cbor2 solely for signature
verification and any signature validation logic that depends on
arbiter_public_key_raw) inside the conditional so callers can omit arbiter
signature verification but cannot bypass the structural independence check.
| """ | ||
| if not isinstance(text, str): | ||
| raise TypeError(f"text must be str, got {type(text).__name__}") | ||
| th = _DEFAULT_THRESHOLD if threshold is None else float(threshold) |
There was a problem hiding this comment.
Validate threshold bounds before scoring.
At Line 106, threshold is coerced to float but not checked for finiteness or [0, 1] bounds, which can silently produce invalid decisions.
Suggested fix
+import math
...
- th = _DEFAULT_THRESHOLD if threshold is None else float(threshold)
+ th = _DEFAULT_THRESHOLD if threshold is None else float(threshold)
+ if not math.isfinite(th) or not (0.0 <= th <= 1.0):
+ raise ValueError(f"threshold must be finite and in [0, 1], got {th!r}")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| th = _DEFAULT_THRESHOLD if threshold is None else float(threshold) | |
| th = _DEFAULT_THRESHOLD if threshold is None else float(threshold) | |
| if not math.isfinite(th) or not (0.0 <= th <= 1.0): | |
| raise ValueError(f"threshold must be finite and in [0, 1], got {th!r}") |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/vaara/detect/injection.py` at line 106, The code currently coerces
threshold to float (th = _DEFAULT_THRESHOLD if threshold is None else
float(threshold)) but does not validate it; update the handling of the input
parameter `threshold` (and resulting local `th`) to explicitly check that the
value is finite and within [0, 1], using math.isfinite after conversion and
raising a ValueError with a clear message if the check fails; keep the None
fallback to _DEFAULT_THRESHOLD and perform these checks on the final `th` before
it is used for scoring (reference the symbols `threshold` and `th` in
injection.py).
Comment on lines
+102
to
+105
| if fmt == "yaml": | ||
| return from_yaml(source) | ||
| if fmt == "json": | ||
| return from_json(source) |
There was a problem hiding this comment.
Reject unsupported format values explicitly.
reload(..., format=...) documents only "json" and "yaml", but unsupported values currently fall through to auto-detection, which is surprising and hard to debug.
Suggested fix
def _load(source: Union[str, Path, dict], fmt: Optional[str]) -> Policy:
+ if fmt is not None and fmt not in {"json", "yaml"}:
+ raise PolicyError(f"unsupported policy format: {fmt}")
if isinstance(source, dict):
return from_dict(source)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/vaara/policy/controller.py` around lines 102 - 105, The code path
handling fmt currently only checks for "yaml" and "json" and silently falls
through to auto-detection for any other value; update the reload handler (the
branch that examines the fmt variable in src/vaara/policy/controller.py) to
explicitly reject unsupported formats by raising a clear exception (e.g.,
ValueError) when fmt is not "json" or "yaml" instead of falling back, and
include the invalid fmt value in the error message; keep the existing calls to
from_yaml(source) and from_json(source) for the supported cases.
Comment on lines
+687
to
+700
| def apply_policy(self, policy: "Policy") -> None: | ||
| """Rebind thresholds and sequence patterns from a loaded policy. | ||
|
|
||
| Runs under the scorer's RLock so an evaluate() in flight on | ||
| another thread either sees the old or the new pair completely. | ||
| The conformal calibrator, MWU expert state, agent profiles, and | ||
| cross-agent history are intentionally preserved — the operator | ||
| intent of a hot reload is "change the policy I'm enforcing," | ||
| not "wipe the model state I've been calibrating against." | ||
| """ | ||
| from vaara.policy.schema import Policy # local import to avoid cycles | ||
| if not isinstance(policy, Policy): | ||
| raise TypeError( | ||
| f"apply_policy requires a Policy, got {type(policy).__name__}" |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify the failing pattern and absence/presence of type-only import.
set -e
rg -n 'def apply_policy\(self, policy:\s*"Policy"\)' src/vaara/scorer/adaptive.py
rg -n 'TYPE_CHECKING|from vaara\.policy\.schema import Policy' src/vaara/scorer/adaptive.pyRepository: vaaraio/vaara
Length of output: 316
Remove unnecessary string literal quotes from Policy annotation.
Line 690 uses policy: "Policy" as a string literal. Since TYPE_CHECKING and the conditional import from vaara.policy.schema are already present in the module (lines 34–37), the annotation can use the direct type name without quotes.
Change line 690 from:
def apply_policy(self, policy: "Policy") -> None:to:
def apply_policy(self, policy: Policy) -> None:🧰 Tools
🪛 GitHub Actions: CI / 0_tests (3.11).txt
[error] 687-687: Ruff (F821): Undefined name Policy.
🪛 GitHub Actions: CI / 1_tests (3.10).txt
[error] 687-687: ruff check . reported F821 Undefined name Policy (type annotation string "Policy" but Policy is not defined/imported).
🪛 GitHub Actions: CI / 2_tests (3.13).txt
[error] 687-687: ruff check: F821 Undefined name Policy.
🪛 GitHub Actions: CI / 4_tests (3.12).txt
[error] 687-687: ruff F821 Undefined name Policy
🪛 GitHub Actions: CI / tests (3.10)
[error] 687-687: ruff check: F821 Undefined name Policy (type name referenced as string annotation but not defined/imported in scope).
🪛 GitHub Actions: CI / tests (3.11)
[error] 687-687: Ruff (F821) Undefined name Policy (type annotation string).
🪛 GitHub Actions: CI / tests (3.12)
[error] 687-687: ruff check failed (F821): Undefined name Policy in type annotation apply_policy(self, policy: "Policy").
🪛 GitHub Actions: CI / tests (3.13)
[error] 687-687: ruff check: F821 Undefined name Policy.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/vaara/scorer/adaptive.py` around lines 687 - 700, The apply_policy method
currently annotates its parameter as the string literal "Policy"; remove the
unnecessary quotes and use the real type name Policy in the signature (def
apply_policy(self, policy: Policy) -> None) since the module already
conditionally imports Policy from vaara.policy.schema; update the annotation
only (no other logic changes) and keep the local import/check and
isinstance(policy, Policy) validation intact.
…HTML dashboard Theme: operator surface + OVERT Phase 3 path. Four additions that close the most legible competitive gaps without diluting the kernel position. Hot policy reload. PolicyController owns the live Policy and runs registered listeners under a write lock on reload. AdaptiveScorer .apply_policy rebinds thresholds and sequence patterns atomically under the scorer's own RLock; an evaluate() in flight either sees the old (allow, deny) pair or the new one, never a torn half. Conformal calibration, MWU expert state, and agent profiles are preserved across reloads. Malformed reloads are rejected with the previous policy left live. POST /v1/policy/reload accepts a server-side path or an inline body; vaara serve --policy PATH enables the endpoint; vaara policy reload POLICY_PATH triggers it over HTTP. OVERT 1.0 AAL-4 Phase 3 IAP reference. vaara.attestation.iap ships a Phase3Attestation that wraps a Vaara BaseEnvelope with a notary Ed25519 signature over canonical CBOR of all nine envelope fields (inner Arbiter signature bound by reference) plus a transparency-log inclusion proof. Structural independence between the Arbiter key and the notary key is enforced at both emit and verify. InProcessTransparencyLog is an RFC 6962-style binary Merkle log with domain-separated leaf and internal hashes; append, inclusion_proof, and root_hash match the shape a sigstore Rekor adapter would expose, so a production deployment can swap in Rekor at the same call sites. Named injection + PII detector aliases. vaara.detect.detect_injection routes free text through the same AdversarialClassifier behind vaara-bench-v1's published numbers (heuristic fallback when the ml extra is absent; the backend field reports which path served). vaara.detect.detect_pii is a zero-dependency regex extractor over email, phone, US SSN, IPv4, credit_card (Luhn-checked), and IBAN (mod-97 checksum). POST /v1/detect/injection and POST /v1/detect/pii mirror the CLI. vaara detect injection and vaara detect pii read text from --text, --file, or --stdin and exit non-zero when the detector fires. Static HTML article-coverage dashboard. vaara.compliance.dashboard.render_html produces a single self-contained HTML page with embedded CSS, no JavaScript, no external assets, no network calls. Same content as render_markdown (system metadata, audit-trail integrity, summary, critical gaps, per-domain article tables, per-article detail) with status badges as colored pills and a print-friendly stylesheet. vaara compliance dashboard --db PATH --out PATH writes the page. docs/openapi.yaml adds /v1/policy/reload, /v1/detect/injection, and /v1/detect/pii with full request and response schemas. COMPLIANCE.md TOOL-1.4 and TOOL-5 rows updated to reflect that Phase 3 is now in tree; the AAL-4 requirement that the notary keys live with an independent operator is preserved. 586 tests pass, 12 skipped.
vaaraio
force-pushed
the
feat/hot-policy-reload
branch
from
101afb4 to
f20cc7a
Compare
| import pytest | ||
|
|
||
| try: | ||
| import cbor2 # noqa: F401 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Theme: operator surface + OVERT Phase 3 path. Four additions that close the most legible competitive gaps without diluting the kernel position.
PolicyController+AdaptiveScorer.apply_policy;POST /v1/policy/reloadandvaara policy reload POLICY_PATH. Atomic swap under the scorer's RLock so in-flightevaluate()calls never see a torn (allow, deny) pair. Conformal calibration, MWU experts, and agent profiles preserved across reloads.vaara.attestation.iap.Phase3Attestationwraps a VaaraBaseEnvelopewith a notary Ed25519 signature + transparency-log inclusion proof. Structural independence (notary key ≠ Arbiter key) enforced at emit and verify.InProcessTransparencyLogis RFC 6962-style; production deployments can swap in sigstore Rekor at the same call sites.vaara.detect.detect_injectionroutes through the AdversarialClassifier behind vaara-bench-v1's numbers.vaara.detect.detect_piicovers email, phone, US SSN, IPv4, credit_card (Luhn-checked), IBAN (mod-97).POST /v1/detect/{injection,pii}andvaara detect {injection,pii}mirror each other and exit non-zero on detection for CI gates.vaara.compliance.dashboard.render_htmlproduces a single self-contained HTML page (embedded CSS, no JS, no external assets, no network calls).vaara compliance dashboard --db PATH --out PATHwrites it.OpenAPI spec, CHANGELOG, COMPLIANCE.md (TOOL-1.4, TOOL-5, OVERT-position section), and README updated.
Test plan
vaara serve --policy PATH+vaara policy reload POLICY_PATHround-trip.vaara detect injection --stdinexits 1 on injection patterns, 0 on clean text.vaara compliance dashboard --out site/renders a print-friendly HTML with no remote URLs.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation