Skip to content

feat(agent): record coding verification evidence - #52285

Merged
OutThisLife merged 2 commits into
mainfrom
bb/verify-ledger
Jun 25, 2026
Merged

feat(agent): record coding verification evidence#52285
OutThisLife merged 2 commits into
mainfrom
bb/verify-ledger

Conversation

@OutThisLife

@OutThisLife OutThisLife commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a profile-scoped verification evidence ledger for canonical project checks detected by agent.coding_context
  • record foreground terminal test/lint/typecheck/build results as scoped evidence (full vs targeted, pass vs fail)
  • key durable state by conversation session_id while preserving raw task_id for sandbox/file resolution
  • mark prior evidence stale after successful write_file / patch edits, using the edited file's project root when available
  • match common real command spellings (pnpm test, bash scripts/run_tests.sh, uv run pytest) while avoiding echoed command text false positives
  • bound storage growth: output summaries are capped, changed-path lists are capped, old per-session/workspace events are pruned, global pruning only deletes events no state row points at, and evidence naturally expires after 30 days

Notes

This is intentionally passive: it records evidence, not guarantees. A targeted test stays targeted, lint/typecheck/build are classified separately, and no stop-loop or desktop UI policy is added in this PR.

Test plan

  • scripts/run_tests.sh tests/agent/test_verification_evidence.py tests/agent/test_coding_context.py -- -q

@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

🔎 Lint report: bb/verify-ledger vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 11261 on HEAD, 11259 on base (🆕 +2)

🆕 New issues (2):

Rule Count
unresolved-attribute 2
First entries
tests/run_agent/test_credits_notices_toggle.py:76: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_credits_session_start_micros` on type `AIAgent`
run_agent.py:2984: [unresolved-attribute] unresolved-attribute: Object of type `Self@get_credits_spent_micros` has no attribute `_credits_session_start_micros`

✅ Fixed issues (1):

Rule Count
invalid-assignment 1
First entries
tests/run_agent/test_credits_notices_toggle.py:76: [invalid-assignment] invalid-assignment: Object of type `None` is not assignable to attribute `_credits_session_start_micros` of type `int`

Unchanged: 5940 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/file File tools (read, write, patch, search) tool/terminal Terminal execution and process management P3 Low — cosmetic, nice to have labels Jun 25, 2026
Record foreground verification commands in a bounded, profile-scoped ledger and mark evidence stale when code edits change the workspace.
Exercise command classification, session scoping, stale edits, bounded retention, and natural expiry for recorded verification evidence.
@OutThisLife
OutThisLife merged commit da0320b into main Jun 25, 2026
27 checks passed
@OutThisLife
OutThisLife deleted the bb/verify-ledger branch June 25, 2026 04:07

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Well-structured verification evidence ledger. The module is intentionally passive — recording evidence without making guarantees. Good design with bounded storage, scoped evidence (full vs targeted), and natural expiration. Comprehensive test coverage.

Changes:

  • agent/verification_evidence.py: New 547-line module with SQLite-backed evidence ledger
  • tests/agent/test_verification_evidence.py: 313 lines of tests
  • tools/file_tools.py: 44 additions for file mutation tracking
  • tools/terminal_tool.py: 29 additions for terminal result recording

LGTM.

Reviewed by Hermes Agent

waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
feat(agent): record coding verification evidence
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
feat(agent): record coding verification evidence
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
feat(agent): record coding verification evidence
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
feat(agent): record coding verification evidence
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
feat(agent): record coding verification evidence
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have tool/file File tools (read, write, patch, search) tool/terminal Terminal execution and process management type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants