Skip to content

feat(review): multi-agent review consensus engine (verify + dedup + tag) - #143

Merged
getappz merged 3 commits into
masterfrom
feat/review-consensus
Jul 11, 2026
Merged

feat(review): multi-agent review consensus engine (verify + dedup + tag)#143
getappz merged 3 commits into
masterfrom
feat/review-consensus

Conversation

@getappz

@getappz getappz commented Jul 11, 2026

Copy link
Copy Markdown
Owner

Stacked on #141 (needs src/db.rs / agentflare.db). Base is
feat/claim-ledger; retarget to master once #141 merges.

What

Step 2 of the GitHub-across-agents roadmap: the deterministic consensus core
for multi-agent PR review. Finders (agents running /code-review, ponytail,
subagents) submit findings; agentflare verifies, dedups, and tags them into one
review.

Scope decision

agentflare agents launch is an interactive single-agent launcher, not a
headless "run N agents and collect JSON" primitive — so this PR owns only the
deterministic, reusable consensus layer. Finders stay external and just submit.
(Headless fan-out and PR-posting are separate later steps.)

How

  • Storagereview_findings table in ~/.agentflare/agentflare.db
    (reuses src/db.rs). Re-submit by an agent replaces its prior set for the
    round, so a re-run never double-counts.
  • Verifychanged_lines() parses the unified diff (git diff base...head,
    default master...HEAD) into the new-side line set per file; a finding whose
    file:line isn't a changed line is UNVERIFIED.
  • Dedup + tagconsensus() clusters verified findings by file + nearby
    line (±3), then tags each cluster CONFIRMED (≥2 distinct agents) /
    UNIQUE (1) / DISPUTED (≥2 agents, conflicting severity) /
    UNVERIFIED. Ranked most-confident first; markdown + JSON renderings.

Surface

  • MCP (flare server): review_submit, review_consensus, review_list,
    review_clear.
  • CLI: agentflare review {submit|consensus|list|clear} [--pr] [--repo] [--base --head].

Tests

10 unit tests (diff parsing incl. new-file hunks, unverified-off-diff, 2-agent
clustering → CONFIRMED, single → UNIQUE, same-agent-twice ≠ CONFIRMED,
conflicting-severity → DISPUTED, distant lines don't cluster, ranking,
submit-replaces, round-scoped load/clear). Full suite: 278 passed, 0 failed.

Driven end-to-end via the CLI on this branch's real diff: two finders citing
nearby committed lines → one CONFIRMED cluster (db.rs:5-7), singles → UNIQUE,
off-diff citation → UNVERIFIED.

Closes #142 · related #137 (posting later), #141 (db)

Summary by CodeRabbit

  • New Features
    • Added multi-agent code review workflows through the CLI and MCP tools.
    • Submit, list, clear, and combine review findings into consensus results.
    • Added verification against changed lines, agreement status, severity conflicts, and Markdown or JSON output.
    • Review findings are stored per repository and pull request, with repeat submissions from the same agent replacing previous results.

@getappz getappz added enhancement New feature or request rust Pull requests that update rust code labels Jul 11, 2026
@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0faeea78-2b9a-4cea-a26e-2b7cf2f2aa1c

📥 Commits

Reviewing files that changed from the base of the PR and between b73e749 and 86c3f6e.

📒 Files selected for processing (6)
  • src/cli/mod.rs
  • src/cli/review.rs
  • src/db.rs
  • src/main.rs
  • src/mcp_server.rs
  • src/review.rs

📝 Walkthrough

Walkthrough

Adds a multi-agent review consensus engine with SQLite-backed findings, changed-line verification, clustering and verdicts, Markdown/JSON output, CLI commands, and MCP tools for review submission and round management.

Changes

Review consensus workflow

Layer / File(s) Summary
Consensus data, storage, and aggregation
src/review.rs
Defines finding models, persists per-agent submissions, verifies changed lines, clusters findings, assigns verdicts, renders Markdown, computes diffs, and tests these behaviors.
Review module and database migration wiring
src/main.rs, src/db.rs
Registers the review module and runs its migration during shared database initialization.
CLI review commands
src/cli/mod.rs, src/cli/review.rs
Adds review submit, consensus, list, and clear actions with repository/round resolution and JSON or Markdown output.
MCP review tools
src/mcp_server.rs
Adds MCP request models and handlers for submitting, consolidating, listing, and clearing review findings.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Finder
  participant CLI_or_MCP
  participant ReviewStorage
  participant Git
  participant Consensus
  Finder->>CLI_or_MCP: submit findings
  CLI_or_MCP->>ReviewStorage: store findings by repo and round
  CLI_or_MCP->>Git: compute base...head diff
  Git-->>Consensus: changed lines
  ReviewStorage-->>Consensus: stored findings
  Consensus-->>CLI_or_MCP: ranked items and rendered Markdown
Loading

Possibly related PRs

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/review-consensus

Comment @coderabbitai help to get the list of available commands.

getappz pushed a commit that referenced this pull request Jul 11, 2026
Review follow-up on #143. changed_lines() treated any non +/-/@@ line while a
file was open as a context line — so in a multi-file diff the 'diff --git' /
'index' headers between file sections inserted a phantom changed-line into the
PRECEDING file, falsely verifying a finding that cited it (and, in step 3,
inflating that agent's accuracy). Reset the file at each 'diff --git' boundary
and ignore '\ No newline at end of file' markers. Regression tests added.
shiva added 2 commits July 11, 2026 14:24
Finders submit findings for a round; agentflare mechanically verifies each cited
file:line against the git diff (base...head), clusters nearby findings from
different agents, and tags each CONFIRMED (>=2 agents) / UNIQUE (1) / DISPUTED
(conflicting severity) / UNVERIFIED (line not in the diff). Stored in a
review_findings table in agentflare.db; re-submit replaces an agent's prior set.

Surface: MCP tools review_submit/review_consensus/review_list/review_clear on
the flare server, plus `agentflare review {submit|consensus|list|clear}`.
Deterministic core only — finders stay external; posting to the PR (via the
step-0 github-mcp-server) is a later step.

Closes #142
Review follow-up on #143. changed_lines() treated any non +/-/@@ line while a
file was open as a context line — so in a multi-file diff the 'diff --git' /
'index' headers between file sections inserted a phantom changed-line into the
PRECEDING file, falsely verifying a finding that cited it (and, in step 3,
inflating that agent's accuracy). Reset the file at each 'diff --git' boundary
and ignore '\ No newline at end of file' markers. Regression tests added.
@getappz
getappz force-pushed the feat/review-consensus branch from 18a5eab to b690436 Compare July 11, 2026 09:15
Base automatically changed from feat/claim-ledger to master July 11, 2026 13:29
@getappz
getappz merged commit 7d68564 into master Jul 11, 2026
@getappz
getappz deleted the feat/review-consensus branch July 11, 2026 13:31
getappz added a commit that referenced this pull request Aug 18, 2026
#549)

Headless CLI agents run in print mode and emit no incremental stdout —
only one blob once the whole turn finishes — so idle detection can't
tell a hung process from one legitimately still computing (e.g. a
cargo build/test mid-turn). DEFAULT_IDLE_TIMEOUT_SECS defaulted to 300,
so any real SDD turn taking longer than 5 minutes died every time,
confirmed live via items #143/#150/#151 failing identically 8 times in
a row. It now defaults to the same 21600s (6h) ceiling as --timeout,
the only backstop that still means what its doc comment says.

Agentflare-Agent: claude-code
Agentflare-Branch: task/143-idle-timeout-inherit-hard-cap
Agentflare-Item: 143

Co-authored-by: shiva <shiva@gosysinfo.tech>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request rust Pull requests that update rust code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(review): multi-agent review consensus engine (verify + dedup + tag)

1 participant