Skip to content

Wire detection plane + enrich status for agent visibility (#3596) - #3600

Closed
james-in-a-box[bot] wants to merge 60 commits into
mainfrom
egg/issue-3596-v2/work
Closed

Wire detection plane + enrich status for agent visibility (#3596)#3600
james-in-a-box[bot] wants to merge 60 commits into
mainfrom
egg/issue-3596-v2/work

Conversation

@james-in-a-box

@james-in-a-box james-in-a-box Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Operators cannot distinguish a working agent from a wedged one via get_status — every agent reports WORKING with no forward-progress signal. The codebase has 27 deterministic detectors in the detection plane, but none can fire because (1) the plane is never invoked from the runtime tick, and (2) the snapshot builder populates only 5 of 13 top-level fields and 3 of 7 RunningAgent fields. This plan wires the detection plane into the runtime tick, enriches the snapshot builder with data from 8 existing data sources, adds a forward-progress detector, fixes the peer-progress gate, and enriches get_status with per-agent progress signals and active alerts.

Test Plan

  • Automated: per-slice test suites covering detection plane wiring, snapshot enrichment, forward-progress detector, peer-progress gate fix, and status endpoint enrichment. make test narrows to reachable suites.
  • Manual: verify /status response includes progress sub-object and alerts array; verify detection plane findings appear on the event bus.

Manual Steps

Pre-merge: none. Post-merge: watch the first real pipeline run to confirm detection plane findings appear in /health/alerts and get_status surfaces progress signals.

Pipeline context

Per-phase BRC transcripts: refine, plan.

egg-orchestrator and others added 22 commits July 24, 2026 23:26
Faithful jargon-free companion to the refiner's analysis-draft.
Covers: forward progress gap, alerts not in status, per-invocation
accounting, post-mortem durability, dead-code detector, and proposed
approach with risks.
…nt state)

Plan: 6 tasks across 6 slices to make forward-progress state visible and
make the system act on it. Key insight: the codebase already has 25+
deterministic detectors, a HealthMonitor, ProgressStore, agent_log_store,
and working_heartbeat emitter — but the detection plane is never invoked
from the runtime tick, the snapshot builder only populates 3 of 10
RunningAgent fields, get_status doesn't surface health alerts or per-agent
progress metrics, and the peer-progress gate is dependency-blind.

Task 1 (critical): wire detection plane into runtime tick + enrich snapshot
builder. This unlocks all 25+ dormant detectors.
Task 2: add forward-progress detector (commit count + worktree activity).
Task 3: fix peer-progress gate to be dependency-aware (#3595 root cause 1).
Task 4: enrich get_status with progress signals + active alerts.
Task 5: add consumption breaker (uncached tokens + call count).
Task 6: record sampling params in cost_callback.

Already verified as existing (do NOT rebuild): agent_log_store (#3547),
GET /health/alerts, evidence_rescue, commit_authorship_store, agent_salvage,
AgentExitInfo (#2205), working_heartbeat (#3341), driver_heartbeat (#3540),
DriverLivenessCheck, PhaseOutputPresenceCheck, _live_event_agents (#3230),
OverseerSelfMonitor (#2270 §5).

Confirmed dead code: detect_heartbeat_stall — snapshot_from_health_context
never populates last_tool_call_age_s/last_heartbeat_age_s, and role=str(cid)
puts a container ID in the role field.

Co-Authored-By: Claude <noreply@anthropic.com>
…ase gate)

The orchestrator's phase gate expects the architect-output artifact at
.egg-state/agent-outputs/issue-3596-v2-architect-output.json. The slices
file is also moved to agent-outputs for consistency.

Co-Authored-By: Claude <noreply@anthropic.com>
8 risks identified, 4 HIGH severity:
- R1: Detection plane wiring conflict with operator's cq-1 resolution
- R2: role=str(cid) defect confirmed (container UUID in role field)
- R3: Snapshot builder only populates 2 of 10 required fields
- R4: detect_heartbeat_stall is dead code (last_tool_call_age_s never populated)
- R5: Plan claims '25+ dormant detectors' — count needs auditing
- R6: Consumption breaker (task-5) has no data source for cost_counters
- R7: Peer-progress gate fix targets old HealthMonitor path
- R8: Session transcripts only pushed on exit

Verdict: PROCEED_WITH_MITIGATIONS. Registered cq-2 for the detection plane wiring conflict.

Co-Authored-By: Claude <noreply@anthropic.com>
…ewer_plan)

Addresses all 7 NACK points:
- R1: cq-2 was registered and resolved by operator — detection plane is NOT wired
- R3: task-1 split into 9 sub-tasks (1a-1g) by data source
- R5: full detector audit — ALL 27 detectors are starved
- R6: task-5 (consumption breaker) deferred — no cost counter store exists
- R7: HealthMonitor IS still active in production (12+ call sites)

Key findings:
- snapshot_from_health_context populates only 5 of 13 top-level fields
  and 3 of 7 RunningAgent fields
- ALL 27 registered detection-plane detectors are starved (cannot fire)
- role=str(cid) defect confirmed (container UUID in role field)
- health_checks/README.md:88 falsely documents plane as wired
- No cost counter store exists (cost_callback logs to stdout)

Co-Authored-By: Claude <noreply@anthropic.com>
Records the NACK resolution, detector audit results, and proposal status.
All 27 detection-plane detectors confirmed starved.

Co-Authored-By: Claude <noreply@anthropic.com>
…deferred candidates

Incorporates the operator's pre-answers from the overseer STATUS message:
- A (commit counting scope): count LOCAL worktree commits ahead of phase_start_sha
- B (tool-call counting proxy): reuse existing progress-event data, no new event type
- C (alert surface format): capped LIST, newest-first, cap at 10, configurable
- D (log retention TTL): configurable via PipelineConfig, default 24h unchanged

Also corrects the detector audit: ALL 27 detectors are starved (not 26),
and adds the nack_resolution section documenting how each NACK point was addressed.

Co-Authored-By: Claude <noreply@anthropic.com>
Faithful jargon-free companion to the task_planner's plan-draft.
Covers: 7 visibility gaps, 6-slice implementation plan, deferred items,
execution order, testing approach, and operator constraints (null != zero,
don't rebuild existing infra).
@james-in-a-box james-in-a-box Bot added the egg label Jul 25, 2026
egg-orchestrator added 26 commits July 25, 2026 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant