feat(conductor): multi-agent fan-out + verifier-gated merge worker (Rust)

[rohitg00]

Summary

Adds conductor/, a Rust binary worker that fans out a task across N agent CLIs in parallel, each in its own git worktree (local and remote), runs configurable verifier gates against every result, and picks the agent that finished first among the eligible set.

Four iii functions:

Function	Purpose
conductor::dispatch	Fan out task across agents[], store run state, run gates per agent
conductor::status	Read a RunState by run_id
conductor::list	List all runs
conductor::merge	Smallest-finished_at winner pick over the eligible set, loser worktree cleanup

All four are registered with metadata.public = true so iii-worker-manager can expose them through iii-mcp and iii-a2a. JSON-Schema request/response shapes are declared on every registration so MCP tool discovery returns full type info.

Agent kinds

• Local CLI shellouts: claude, codex, gemini, aider, cursor, amp, opencode, qwen. Default arg vector per kind, overridable via bin / args.
• remote: any iii function_id — typically registered by iii-mcp-client (mcp.<server>::<tool>) or iii-a2a-client (a2a.<session>::<skill>). Remote agents now also get a per-agent worktree, and the worktree path is passed as cwd in the trigger payload, so remote handlers that write to cwd produce a real diff and can win the merge.

Verifier gates

Gates are themselves iii functions of the shape (input: { cwd }) -> { ok: bool, reason?: string }. The conductor invokes each gate with the agent's worktree as cwd and treats ok: false as a stop. Gate results are stored as an ordered Vec<GateRunResult> so duplicate function_ids with different descriptions (e.g. verify::tests for unit and again for integration) are preserved end-to-end. Recommended pairings already in this repo: eval, guardrails, proof.

Run state

Stored under state::set scope conductor, key runs::<run_id>. Persisted after every agent transitions, not only at start and end — fan-out uses FuturesUnordered and writes the run back to state as each agent completes its gates, so a worker restart mid-run preserves the progress of every agent that already finished.

State helpers follow the same envelope-tolerant pattern as llm-router/state.rs so both 0.11.0 ({ items: [...] }) and 0.11.2 (bare arrays) engine responses are accepted.

Files

conductor/
├── Cargo.toml
├── Cargo.lock
├── README.md
├── iii.worker.yaml          # language: rust, deploy: binary, bin: iii-conductor
├── src/
│   ├── agents.rs            # local CLI shellout (tokio::process)
│   ├── dispatch.rs          # fan-out + gates + state writes per transition
│   ├── gates.rs             # gate runner -> Vec<GateRunResult>, all_passed
│   ├── git.rs               # tokio::process git wrapper, worktree create/remove, diff
│   ├── lib.rs               # module re-exports
│   ├── main.rs              # clap entry, register_worker + four register_function_with calls
│   ├── state.rs             # state::set/get/list helpers (envelope-tolerant)
│   └── types.rs             # AgentSpec, GateSpec, GateRunResult, RunState, MergeResult, ...
└── tests/
    └── merge.rs             # 9 tests: gate aggregator, completion-order winner pick,
                             #         duplicate function_id preservation, all-failed branch,
                             #         empty-diff skip, serde Vec round-trip, ...

Verification

• cargo fmt --all -- --check — clean
• cargo clippy --all-targets --all-features -- -D warnings — clean
• cargo test — 9 tests pass

CodeRabbit findings (from the earlier TS scaffold) — all addressed

#	Severity	Status
1 Dockerfile non-root	major	N/A — Rust binary deploy, no Dockerfile
2 Remote agent isolation/diff/gates	critical	fixed: shared worktree path applied for remote too
3 Mid-run state writes	major	fixed: FuturesUnordered loop persists after every agent
4 Winner = completion order	major	fixed: min_by_key(finished_at) over eligible set
5 Gate results as ordered list	minor	fixed: Vec<GateRunResult> with function_id/description preserved
6 Create worktree root before git worktree add	critical	already covered: create_dir_all(parent) runs before add

Test plan

• cargo build --release produces target/release/iii-conductor
• cargo test — 9 tests pass
• Run iii engine + iii-worker-manager (expose_functions: [match("conductor::*")]) + iii-mcp + iii-conductor
• MCP Inspector / Claude Desktop sees conductor_dispatch, conductor_status, conductor_list, conductor_merge tools
• Round-trip: dispatch a 2-agent run with one no-op verifier, confirm conductor::merge picks the agent with the smallest finished_at
• Round-trip with kind: "remote" registered via iii-a2a-client: confirm a non-empty diff is captured and the remote agent can win

Out of scope (follow-ups)

• Streaming agent stdout to an iii-stream channel for live UI consumers — easy follow-up.
• Idempotent run_id from a fingerprint of (task, cwd, agents) — currently UUID v4. Required if dispatch must be safely retriable.
• A separate worktree worker. Claude Code now ships native --worktree and isolation: worktree for subagents; reimplementing it adds nothing. The conductor uses git worktree add directly.

[coderabbitai]

Warning

Rate limit exceeded

@rohitg00 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 3 minutes and 23 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0fcc616a-d103-464f-a906-b4ed9f9dfd74

📥 Commits

Reviewing files that changed from the base of the PR and between c330932 and 9ed5cb4.

📒 Files selected for processing (14)

• conductor/.gitignore
• conductor/CHANGELOG.md
• conductor/Cargo.toml
• conductor/README.md
• conductor/iii.worker.yaml
• conductor/src/agents.rs
• conductor/src/dispatch.rs
• conductor/src/gates.rs
• conductor/src/git.rs
• conductor/src/lib.rs
• conductor/src/main.rs
• conductor/src/state.rs
• conductor/src/types.rs
• conductor/tests/merge.rs

📝 Walkthrough

Walkthrough

Introduces a new conductor worker package for orchestrating multi-agent execution. It dispatches agents in parallel via git worktrees, executes sequential gate validations per agent, persists run state, and exposes public functions for dispatch, status, list, and merge operations via TypeScript/Node.js.

Changes

Cohort / File(s)	Summary
Configuration & Build conductor/Dockerfile, conductor/package.json, conductor/iii.worker.yaml, conductor/tsconfig.json	Adds Node.js/TypeScript build configuration with multi-stage Docker image, ESM package setup with TypeScript/ESM tooling dependencies, worker manifest declaring image deployment mode and resource limits, and strict TypeScript compiler settings.
Type Definitions conductor/src/types.ts	Defines complete domain types including AgentKind union (9 agent types), AgentSpec and GateSpec configurations, DispatchInput/DispatchResult contracts, AgentRunState with status progression and per-gate results, RunState for persisting entire runs, and MergeResult for winner selection outcomes.
Infrastructure Utilities conductor/src/git.ts	Provides Git command execution via spawn with timeout/signal handling, high-level helpers for worktree creation/removal (under ~/.iii/conductor/worktrees), diff computation against base ref, and branch detection.
Execution Modules conductor/src/agents.ts, conductor/src/gates.ts	Implements agent execution by resolving binaries and constructing CLI arguments for local agents (claude, codex, gemini with fallback logic), and gate validation by invoking remote functions with 600s timeout, normalizing outcomes, and aggregating pass/fail status.
Workflow Orchestration conductor/src/dispatch.ts	Core logic: dispatch validates input, runs agents in parallel per worktree, executes gates sequentially on finished agents, computes diffs, and persists run state; mergeRun selects first finished agent with non-empty diff and all gates passing, removes loser worktrees, and records winner index.
State Management conductor/src/state.ts	Abstracts SDK state operations with writeRun, readRun (by ID), and listRuns using scope conductor and key pattern runs::<run.id>.
Worker Registration & Entrypoint conductor/src/iii.ts, conductor/src/index.ts	Registers Conductor III worker with WebSocket engine URL from III_URL environment variable; exposes four public functions (conductor::dispatch, conductor::status, conductor::list, conductor::merge) with OpenTelemetry tracking.
Tests & Documentation conductor/tests/dispatch.test.ts, conductor/tests/gates.test.ts, conductor/README.md	Validates mergeRun selection logic (all-failed, mixed-finished, gate filtering scenarios), allPassed gate aggregation, and documents public function interfaces, schemas, end-to-end flow, example usage, RBAC exposure, dependencies, and configuration.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Conductor as Conductor Worker
    participant Agents
    participant RemoteFn as Remote Functions
    participant Git as Git Worktrees
    participant State as Persistent State

    Client->>Conductor: dispatch(task, agents, cwd)
    Conductor->>State: writeRun (initial RunState)
    
    par Parallel Agent Execution
        Conductor->>Agents: runLocalAgent (Agent 1)
        Agents->>Git: createWorktree (branch-sanitized)
        Git-->>Agents: worktree path
        Agents-->>Conductor: {ok, stdout, stderr}
        
        Conductor->>RemoteFn: sdk.trigger (Remote Agent)
        RemoteFn-->>Conductor: result or error
    end
    
    Conductor->>Git: diffAgainst baseRef (per agent worktree)
    Git-->>Conductor: diff content
    
    loop For Each Finished Agent
        Conductor->>RemoteFn: runGate (gate function)
        RemoteFn-->>Conductor: {ok, reason?}
    end
    
    Conductor->>State: writeRun (completed RunState)
    Conductor-->>Client: {ok, run_id, agents, gates}
    
    Client->>Conductor: merge(run_id)
    Conductor->>State: readRun
    State-->>Conductor: RunState
    
    alt Has Finished Agent with Non-Empty Diff & Passing Gates
        Conductor->>Git: removeWorktree (losers)
        Git-->>Conductor: {ok}
        Conductor->>State: writeRun (winnerIndex set)
        Conductor-->>Client: {ok: true, winner, losers}
    else No Eligible Agent
        Conductor-->>Client: {ok: false, reason, losers: []}
    end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~30 minutes

Possibly related PRs

• Phase 1: delegate exposure to iii-sdk RBAC, remove handler-level gates (BREAKING) #48: Addresses worker metadata and function exposure via RBAC, directly related to this PR's use of metadata.public = true and III worker registration patterns.

Suggested reviewers

• sergiofilhowz
• ytallo

Poem

🐰 A conductor's baton orchestrates the dance,
Agents waltz in parallel—each takes a chance,
Git worktrees bloom like gardens of code,
Gates stand sentinel on the success road,
The first to cross the line wins the embrace,
Merge selects the victor with algorithmic grace! ✨

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check	⚠️ Warning	The title mentions 'Rust' but the changeset introduces a Node.js/TypeScript worker, not Rust code. This is a factual mismatch.	Update the title to remove '(Rust)' and replace it with an accurate description, such as 'feat(conductor): multi-agent fan-out + verifier-gated merge worker (Node.js)' or similar.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/conductor

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Review rate limit: 0/1 reviews remaining, refill in 3 minutes and 23 seconds.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@conductor/Dockerfile`:
- Around line 9-16: The image currently runs as root; create and switch to a
non-root runtime user by adding a dedicated user/group and changing ownership of
/app before the final image CMD. Update the Dockerfile to add a non-root user
(e.g., create a user like "conductor" or "appuser"), chown /app (or the copied
files) to that user after copying files (refer to the COPY and WORKDIR steps),
and add a USER instruction to run the container as that non-root user prior to
the CMD invocation so node /app/dist/index.js runs without root privileges.

In `@conductor/src/dispatch.ts`:
- Around line 27-53: The remote-agent branch (when spec.kind === 'remote')
currently calls sdk.trigger with the shared cwd and never records worktreePath
or diff, so remote agents share the checkout and produce no mergeable result;
modify this branch to create a per-agent worktreePath (e.g., create a temporary
worktree or per-agent checkout), pass that worktreePath as cwd in the payload to
sdk.trigger (instead of the shared cwd), capture the remote result and compute
the git diff for that worktree, and return a result object that includes agent:
spec, status, startedAt, finishedAt, exitCode/output, and the new worktreePath
and diff fields so mergeRun() can accept and merge remote agents; also keep the
existing error handling around sdk.trigger but ensure errors include the
worktree cleanup and accurate timestamps.
- Around line 128-143: The current winner selection iterates run.agents in input
order (variable a) which picks the first eligible agent instead of the one that
actually finished first; change the logic to pick the agent with the earliest
completion timestamp instead: filter agents to those with status === 'finished',
allPassed(a.gateResults) (or true if absent), and a.diff non-empty, then choose
the agent with the smallest finishedAt/ completionTime/finishedTimestamp field
(or if such a field does not exist, add and set finishedAt when status
transitions to 'finished'); set winner to that agent ({ index, agent: a.agent,
diff: a.diff, branch: a.branch }) and put all other eligible indices into losers
to preserve current semantics.
- Around line 106-119: The run currently only persists the run once at the end,
so intermediate agent state transitions (from runAgentInWorktree and
runAllGates) are lost if the worker crashes; after each agent state is updated
in the loop (i.e., after setting run.agents[i] = state and after gateResults are
attached), call await writeRun(sdk, run) to persist the partial progress so
completed agents and gate results survive restarts; keep using
runAgentInWorktree, runAllGates and writeRun as the points to snapshot state.

In `@conductor/src/gates.ts`:
- Around line 25-29: The current runAllGates function stores results in a Record
keyed by GateSpec.function_id which can overwrite entries when function_id is
duplicated; change runAllGates to preserve order and duplicates by returning an
array of outcomes (e.g., GateOutcome[]) or a list of objects that include the
original GateSpec metadata and outcome. Update runAllGates to iterate gates,
call await runGate(sdk, gate, cwd) for each, and push a result entry that
contains gate (or its description and function_id) plus the GateOutcome into an
array, then return that array instead of the Record keyed by function_id; ensure
callers expecting Record<string,GateOutcome> are updated to handle the new array
shape.

In `@conductor/src/git.ts`:
- Around line 38-46: The createWorktree function may call git worktree add
before the worktree root exists; ensure the directory root (and any parent dirs)
exist before invoking git by creating the directory (use fs.mkdir with {
recursive: true }) for the computed path or at least for root, handle any mkdir
errors and return { ok: false, reason: <error message> } if creation fails, then
proceed to call git(repoCwd, ['worktree', 'add', '-b', branch, path]) as before;
reference createWorktree, root, path and git to locate where to add the mkdir
and error handling.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8771e350-d55c-44dc-823c-a35ffaf597ab

📥 Commits

Reviewing files that changed from the base of the PR and between 7bfc033 and c330932.

📒 Files selected for processing (15)

• conductor/Dockerfile
• conductor/README.md
• conductor/iii.worker.yaml
• conductor/package.json
• conductor/src/agents.ts
• conductor/src/dispatch.ts
• conductor/src/gates.ts
• conductor/src/git.ts
• conductor/src/iii.ts
• conductor/src/index.ts
• conductor/src/state.ts
• conductor/src/types.ts
• conductor/tests/dispatch.test.ts
• conductor/tests/gates.test.ts
• conductor/tsconfig.json