docs: OVERT 1.0 status and public-doc cleanup#80
Conversation
📝 WalkthroughWalkthroughThis pull request updates project documentation across release notes, compliance alignment, and benchmarking guidance. Changes normalize formatting and typography while expanding technical specification details and tool comparison coverage, with no impact to code or APIs. ChangesDocumentation Refinement and Specification Details
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
README.md (1)
78-78: ⚡ Quick winUse the specific publication date (March 25, 2026) instead of "March 2026".
The OVERT 1.0 specification was published on March 25, 2026 by Glacis Technologies, Inc. The current statement is accurate but can be more precise. Update "published in March 2026" to "published March 25, 2026" for stronger provenance documentation. The canonical citation URL (overt.is) is already present.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@README.md` at line 78, Update the README sentence that currently reads "published in March 2026" to the exact publication date "published March 25, 2026" so the OVERT 1.0 description becomes: "OVERT 1.0 is an open standard for runtime trust in AI systems, authored by Glacis Technologies and published March 25, 2026." Locate the sentence in the README (the paragraph starting "Vaara implements the OVERT 1.0 ...") and replace only the date fragment to preserve the rest of the wording and links.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@bench/COMPARISON.md`:
- Around line 24-42: Add an "as-of" date to the matrix header and footnote each
comparative claim with authoritative sources; for each row (e.g., "OVERT 1.0
Base Envelope emission (RFC 8949 CBOR)", "RFC 6962 Merkle inclusion proof
integration", "Glacis Python SDK", "Validates LLM *output text*", etc.) add one
or more URL citations to vendor/docs or standards and append a footnote marker
in the header/rows, and update the prose block referencing the matrix to cite
the same sources; specifically verify the "Glacis Python SDK" entry—if you
cannot find an authoritative public doc as-of May 2026, remove that column or
replace it with the correct project name/URL and mark the matrix with "as of
[YYYY-MM-DD]".
In `@COMPLIANCE.md`:
- Around line 265-267: The sentence in COMPLIANCE.md that cites "OVERT Annex
F.2" is incorrect; locate the line containing "Per OVERT Annex F.2 this
mapping..." and either replace the bogus citation with the correct OVERT 1.0
reference (if you can find the exact annex/section in the OVERT 1.0 spec) or
remove the OVERT attribution entirely and rephrase as general guidance (e.g.,
"This mapping does not establish legal compliance with any regulation; it
records technical correspondence."). Ensure the edited text removes the "OVERT
Annex F.2" token or substitutes the accurate annex identifier.
---
Nitpick comments:
In `@README.md`:
- Line 78: Update the README sentence that currently reads "published in March
2026" to the exact publication date "published March 25, 2026" so the OVERT 1.0
description becomes: "OVERT 1.0 is an open standard for runtime trust in AI
systems, authored by Glacis Technologies and published March 25, 2026." Locate
the sentence in the README (the paragraph starting "Vaara implements the OVERT
1.0 ...") and replace only the date fragment to preserve the rest of the wording
and links.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: a224fd2f-ea47-4cc4-9949-2b4c632a73fe
📒 Files selected for processing (7)
CHANGELOG.mdCOMPLIANCE.mdCONTRIBUTING.mdREADME.mdSECURITY.mdbench/COMPARISON.mdbench/README.md
| | Concern | Vaara | NeMo Guardrails | Guardrails AI | OpenAI Guardrails | LangChain callbacks | OWASP LLM Top 10 | Glacis Python SDK | MS Agent Governance Toolkit | | ||
| | ------------------------------------------------ | :---: | :-------------: | :-----------: | :---------------: | :-----------------: | :--------------: | :---------------: | :-------------------------: | | ||
| | Validates tool-call **arguments** at runtime | ✓ | ✗ | ✗ | ✗ | observes only | not software | ✗ | ✓ | | ||
| | Probabilistic / conformal risk scoring per call | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | ||
| | Detects temporal **sequence** patterns | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | ||
| | Hash-chained, regulator-exportable audit trail | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | partial (Merkle) | partial (logging) | | ||
| | EU AI Act Art. 12 / 14 / 26 evidence mapping | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | ||
| | OVERT 1.0 Base Envelope emission (RFC 8949 CBOR) | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | ||
| | RFC 6962 Merkle inclusion proof integration | ext. IAP | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ (hosted) | ✗ | | ||
| | Validates LLM *output text* (PII, toxicity, etc) | ✗ | ✓ | ✓ | ✓ | ✗ | advisory only | ✗ | ✗ | | ||
| | Validates LLM *input prompt* (jailbreak etc) | ✗ | ✓ | ✓ | ✓ | ✗ | advisory only | ✗ | ✗ | | ||
| | Structured-output validation (schema / regex) | partial| ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | partial | | ||
| | Zero-trust agent identity primitives | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | | ||
| | Capability-based access control | policy schema | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | | ||
| | Execution sandboxing | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | | ||
| | Multi-language SDKs | Python only | N/A | Python | Python (Agents) | Python / JS | N/A | Python only | ✓ | | ||
| | Self-hostable Python library (no SaaS required) | ✓ | ✓ | ✓ | ✓ | ✓ | document | ✓ | ✓ | | ||
| | License | Apache-2.0 | Apache-2.0 | Apache-2.0 | MIT | MIT | CC-BY | Apache-2.0 | MIT | | ||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
For each tool in the matrix (NeMo Guardrails, Guardrails AI, OpenAI Guardrails, LangChain callbacks, OWASP LLM Top 10, Glacis Python SDK, Microsoft Agent Governance Toolkit), provide current official-source URLs confirming: license, tool-call/runtime governance capabilities, identity/capability/sandboxing support, and any OVERT/RFC 6962-related attestation support.
💡 Result:
The following matrix summarizes the requested information for the specified AI governance and guardrail tools as of May 16, 2026. Tool Matrix Summary 1. NVIDIA NeMo Guardrails - License: Open-source (Apache 2.0) [1][2] - Governance: Provides programmable guardrails for topic control, PII detection, RAG grounding, and jailbreak prevention [2]. Supports runtime orchestration of input/output checks [2][3]. - Identity/Sandboxing: Focuses on application-level orchestration; relies on underlying infrastructure (e.g., Kubernetes/NIM) for isolation [1][2]. - Attestation: No native RFC 6962/OVERT attestation support; integrates with external security APIs for enterprise enforcement [4]. - URL: https://docs.nvidia.com/nemo/guardrails/latest/about/overview.html 2. Guardrails AI - License: Apache 2.0 [5] - Governance: Validates and structures LLM outputs using Pydantic-style schemas; supports input/output guards [5][6]. - Identity/Sandboxing: Primarily a data validation framework; does not provide native agent sandboxing or identity management [5][6]. - Attestation: No native RFC 6962/OVERT attestation support [5]. - URL: https://www.guardrailsai.com/docs 3. OpenAI Guardrails (Agents SDK) - License: Proprietary (OpenAI Terms of Service) [7][8] - Governance: Provides input, output, and tool-level guardrails; supports human-in-the-loop approvals for tool calls [7][8]. - Identity/Sandboxing: Uses RBAC and safety identifiers for user/org-level scoping [9][10]. - Attestation: No native RFC 6962/OVERT attestation support [7][8]. - URL: https://openai.github.io/openai-agents-js/guides/guardrails/ 4. LangChain Callbacks - License: MIT [LangChain GitHub] - Governance: Provides an observability and event-hook mechanism to intercept LLM lifecycle events; used to build custom governance logic [LangChain Docs]. - Identity/Sandboxing: Does not provide native identity or sandboxing; relies on the host application [LangChain Docs]. - Attestation: No native RFC 6962/OVERT attestation support. - URL: https://python.langchain.com/ 5. OWASP LLM Top 10 - License: Creative Commons Attribution-ShareAlike 4.0 [11] - Governance: This is a security framework/guidance project, not a software tool [12][11]. It provides the standard for identifying risks (e.g., Excessive Agency, Sensitive Information Disclosure) [13]. - Identity/Sandboxing: N/A (Guidance only) [12]. - Attestation: N/A (Guidance only) [12]. - URL: https://genai.owasp.org/llm-top-10/ 6. Glacis Python SDK - Note: No authoritative documentation found for a tool named "Glacis Python SDK" in the context of AI governance as of May 2026. 7. Microsoft Agent Governance Toolkit - License: MIT [14][15] - Governance: Provides deterministic policy enforcement, capability modeling, and SRE for autonomous agents; covers all 10 OWASP Agentic Top 10 risks [14][16]. - Identity/Sandboxing: Features zero-trust identity (Ed25519/ML-DSA-65) and execution sandboxing with privilege rings [14][16]. - Attestation: Aligns with RFC 9334 (RATS) for remote attestation [16]. - URL: https://microsoft.github.io/agent-governance-toolkit/
Citations:
- 1: https://docs.nvidia.com/nemo/guardrails/latest/about/overview.html
- 2: https://developer.nvidia.com/nemo-guardrails
- 3: https://docs.nvidia.com/nemo/microservices/26.3.0/guardrails/concepts/inference.html
- 4: https://www.paloaltonetworks.com/blog/network-security/securing-genai-with-ai-runtime-security-and-nvidia-nemo-guardrails/
- 5: https://github.com/guardrails-ai/guardrails/
- 6: https://guardrailsai.com/docs/getting_started/quickstart/
- 7: https://developers.openai.com/api/docs/guides/agents/guardrails-approvals
- 8: https://openai.github.io/openai-agents-js/guides/guardrails/
- 9: http://developers.openai.com/api/docs/guides/safety-checks
- 10: https://developers.openai.com/api/docs/guides/rbac
- 11: https://genai.owasp.org/resource/owasp-top-10-for-llm-applications-2025/
- 12: https://owasp.org/www-project-top-10-for-large-language-model-applications
- 13: https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-v2025.pdf
- 14: https://aka.ms/agt-install
- 15: https://github.com/microsoft/Agent-Governance-Toolkit
- 16: https://microsoft.github.io/agent-governance-toolkit/
Add source footnotes and "as-of" date to comparative claims; remove or verify Glacis Python SDK reference.
The matrix and prose (lines 97–119) contain vendor/tool-specific assertions (features, protocols, licensing) that need citation and a timestamp to prevent drift. Official documentation exists for most tools (NeMo Guardrails, Guardrails AI, OpenAI Guardrails, LangChain, OWASP, Microsoft) but Glacis Python SDK has no authoritative public documentation found as of May 2026—either remove this column or confirm the correct name/URL.
For remaining tools, add "as of [date]" marker and footnote each row (or the matrix header) with source URLs. Technical claims about RFC 6962, OVERT 1.0, and Merkle proofs require explicit citations.
🧰 Tools
🪛 LanguageTool
[style] ~33-~33: In American English, abbreviations like “etc.” require a period.
Context: ...dates LLM output text (PII, toxicity, etc) | ✗ | ✓ | ✓ ...
(ETC_PERIOD)
[style] ~34-~34: In American English, abbreviations like “etc.” require a period.
Context: ...Validates LLM input prompt (jailbreak etc) | ✗ | ✓ | ...
(ETC_PERIOD)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@bench/COMPARISON.md` around lines 24 - 42, Add an "as-of" date to the matrix
header and footnote each comparative claim with authoritative sources; for each
row (e.g., "OVERT 1.0 Base Envelope emission (RFC 8949 CBOR)", "RFC 6962 Merkle
inclusion proof integration", "Glacis Python SDK", "Validates LLM *output
text*", etc.) add one or more URL citations to vendor/docs or standards and
append a footnote marker in the header/rows, and update the prose block
referencing the matrix to cite the same sources; specifically verify the "Glacis
Python SDK" entry—if you cannot find an authoritative public doc as-of May 2026,
remove that column or replace it with the correct project name/URL and mark the
matrix with "as of [YYYY-MM-DD]".
| explicit gap-to-deployer or future-work (◯). Per OVERT Annex F.2 this | ||
| mapping does not establish legal compliance with any regulation; it | ||
| mapping does not establish legal compliance with any regulation. It | ||
| records technical correspondence. |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Find the official OVERT 1.0 Annex F.2 location (stable URL and section anchor, if available) suitable for direct citation in documentation.
💡 Result:
The OVERT 1.0 standard (Observable Verification Evidence for Runtime Trust) does not contain an Annex F.2. The official specification, published by Glacis Technologies, Inc., is structured to define runtime control systems, attestation providers, and assessor requirements, but it does not utilize an Annex F.2 structure [1]. It is possible you are conflating this with other technical or policy documents that use similar naming conventions, such as: - The UK government's Use of CCTV (Overt Closed-Circuit Television system) Policy Framework, which contains an Annex F regarding the retention of CCTV footage [2]. - The Joint Staff Manual CJCSM 3320.01D, which contains various enclosures and appendices, including Enclosure F (Joint Electromagnetic Spectrum Operations) [3]. If you are referencing a specific implementation guide or a different standard, please verify the document title, as "OVERT" is also commonly used as an acronym in unrelated research benchmarks (e.g., for text-to-image model evaluation) [4][5] and general terminology [2][6].
Citations:
- 1: https://overt.is/
- 2: https://www.gov.uk/government/publications/use-of-cctv-overt-closed-circuit-television-system-policy-framework
- 3: https://www.jcs.mil/Portals/36/Documents/Library/Manuals/CJCSM%203320.01D.pdf
- 4: http://arxiv.org/abs/2505.21347v2
- 5: https://par.nsf.gov/biblio/10657918-overt-benchmark-over-refusal-evaluation-text-image-models
- 6: https://www.echa.europa.eu/documents/10162/ad7a4e8d-da70-da9d-ff69-ab84b8b2738d
🏁 Script executed:
# First, locate and examine COMPLIANCE.md around lines 265-267
if [ -f "COMPLIANCE.md" ]; then
echo "=== Lines 260-275 of COMPLIANCE.md ==="
sed -n '260,275p' COMPLIANCE.md | cat -n
echo -e "\n=== Broader context: lines 250-280 ==="
sed -n '250,280p' COMPLIANCE.md | cat -n
fiRepository: vaaraio/vaara
Length of output: 2855
🏁 Script executed:
# Also search for any OVERT references throughout the file to understand context
echo "=== All OVERT references in COMPLIANCE.md ==="
rg -i "overt" COMPLIANCE.md -B 2 -A 2 || echo "No OVERT references found"Repository: vaaraio/vaara
Length of output: 2553
Remove or correct the "OVERT Annex F.2" reference.
The official OVERT 1.0 specification (Glacis Technologies) does not contain an Annex F.2. The cited section does not exist in the standard. Either locate the correct annex reference within OVERT 1.0 or remove this attribution entirely if the statement is general guidance rather than a normative citation.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@COMPLIANCE.md` around lines 265 - 267, The sentence in COMPLIANCE.md that
cites "OVERT Annex F.2" is incorrect; locate the line containing "Per OVERT
Annex F.2 this mapping..." and either replace the bogus citation with the
correct OVERT 1.0 reference (if you can find the exact annex/section in the
OVERT 1.0 spec) or remove the OVERT attribution entirely and rephrase as general
guidance (e.g., "This mapping does not establish legal compliance with any
regulation; it records technical correspondence."). Ensure the edited text
removes the "OVERT Annex F.2" token or substitutes the accurate annex
identifier.
1 participant
Summary
OVERT 1.0 implementation status reference plus a public-prose cleanup pass across the public docs and a
bench/COMPARISON.mdextension.Changes
README.md: clarifies OVERT 1.0 implementation status. Glacis Technologies cited as the open-standard author of OVERT 1.0 (runtime trust in AI systems, published March 2026).COMPLIANCE.md: removes "Relation to Glacis's own published SDK" subsection that had drifted from the document's compliance scope.bench/COMPARISON.md: extends the capability matrix with Glacis Python SDK and Microsoft Agent Governance Toolkit, plus two paragraphs in the existing voice. New rows cover OVERT 1.0 Base Envelope emission, RFC 6962 Merkle inclusion proof integration, zero-trust agent identity primitives, capability-based access control, execution sandboxing, and multi-language SDK surface.pyproject.toml: broadenskeywordsfor runtime AI governance discoverability. Addsagent-governance,article-14,overt,attestation,runtime-monitoring,risk-scoring,audit-trail. Drops subsumed entries.README.md,COMPLIANCE.md,CHANGELOG.md,CONTRIBUTING.md,SECURITY.md,bench/COMPARISON.md,bench/README.md,docs/formal_specification.md,docs/signing-keys.md,docs/vaara-audit-cli.md. Em-dashes converted to hyphens. Semicolons converted to sentence breaks with capitalization. Fenced code blocks preserved.Test plan
Summary by CodeRabbit