Add inner latency spans for agent loop executor

[serrrfirat]

Summary

• add live latency TRACE spans around canonical agent-loop executor stages
• break down planned-driver driver_run into cancel, budget, input, prompt, checkpoint, model, capability, post-capability, stop, exit, and ack stages
• keep fields scoped to operational IDs and iteration; no prompt/message contents are logged

Change Type

• Observability-only runtime instrumentation
• No intentional product behavior change

Linked Issue

• None

Security / DB Impact

• Security: trace fields are limited to operational IDs, scope IDs, run/turn/thread IDs, operation name, and iteration. No prompt text, tool payloads, model output, or raw errors are emitted.
• Database: no schema changes and no new writes.

Blast Radius

• Reborn agent-loop executor tracing only.
• Logs appear only when live latency tracing is enabled through the existing observability/RUST_LOG path.

Rollback Plan

• Revert this PR to remove the new executor TRACE spans and the ironclaw_observability dependency from ironclaw_agent_loop.
• Runtime fallback is to disable ironclaw_latency=trace logging.

Review Follow-Through

• Gemini/CodeRabbit iteration-attribution comments addressed by preserving active loop iteration for post-capability exit cases and exit-path ACK spans.
• CodeRabbit macro drift concern addressed by centralizing the stage macro in executor::latency instead of defining it locally in canonical.rs.

Reborn Checklist

• Uses the existing Reborn executor/driver path
• Does not alter model/tool routing or capability execution semantics
• Does not log user content, prompts, tool arguments, or raw model output

Validation

• cargo fmt --check
• cargo check -p ironclaw_agent_loop -p ironclaw_reborn
• cargo test -p ironclaw_agent_loop --lib
• git diff --check

[coderabbitai]

Caution

Review failed

An error occurred during the review process. Please try again later.

📝 Walkthrough

Walkthrough

Adds a latency-tracing helper module (started_at, operation_ok, operation_error, result) and introduces a trace_stage! macro in the executor's canonical pipeline, wrapping stage calls (cancel, budget, prompt, model, reply/capability, stop, exit, ack) with latency instrumentation. Adds TurnCompletedStep::iteration() and the ironclaw_observability dependency.

Changes

Latency instrumentation of executor pipeline

Layer / File(s)	Summary
Latency helper module and dependency crates/ironclaw_agent_loop/src/executor/latency.rs, crates/ironclaw_agent_loop/src/executor.rs, crates/ironclaw_agent_loop/Cargo.toml	Adds started_at, operation_ok, operation_error, result for emitting success/failure latency traces via ironclaw_observability; declares mod latency; and adds the new crate dependency.
TurnCompletedStep iteration accessor crates/ironclaw_agent_loop/src/executor.rs	Adds iteration() returning state.iteration for Continue or 0 for Exit, used as trace context.
trace_stage! macro and import wiring crates/ironclaw_agent_loop/src/executor/canonical.rs	Introduces the local trace_stage! macro and reorders the super::{...} import list.
Pre-model stages: progress, input drain, prompt, checkpoint crates/ironclaw_agent_loop/src/executor/canonical.rs	Wraps cancel/budget checks, progress emission, input drain, prompt, checkpoint, and pre-model ack/model-processing calls with trace_stage!/latency helpers.
Reply and capability output handling crates/ironclaw_agent_loop/src/executor/canonical.rs	Traces reply admission, assistant reply processing, capability/post-capability processing, stop.observe, and follow-up input drain.
Stop decision and exit/ack tracing crates/ironclaw_agent_loop/src/executor/canonical.rs	Wraps stop.decide, exit.process, and pre-exit ack delivery with traced latency calls on the Stop branch.
Prompt resume and skip-model branch tracing crates/ironclaw_agent_loop/src/executor/canonical.rs	Applies equivalent traced instrumentation to resume-path capability/stop/exit calls and the SkipModel branch's ack/exit calls.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
    participant Pipeline as DefaultExecutorPipeline
    participant TraceStage as trace_stage! macro
    participant Latency as latency module
    participant Observability as ironclaw_observability

    Pipeline->>TraceStage: invoke stage (e.g. model.process, exit.process)
    TraceStage->>Latency: started_at()
    TraceStage->>Pipeline: await stage future
    Pipeline-->>TraceStage: Result<T, E>
    TraceStage->>Latency: result(operation, context, iteration, started_at, result)
    alt Ok
        Latency->>Observability: operation_ok trace event
    else Err
        Latency->>Observability: operation_error trace event
    end
    TraceStage-->>Pipeline: return original Result

Loading

No repo invariant docs (CLAUDE.md/AGENTS.md/.claude/rules) provided in this diff for citation. Flagging purely from the diff:

• operation_ok/operation_error early-return when started_at is None — verify trace_stage! guarantees started_at is always captured before the awaited future runs, otherwise traces silently drop for the first stage in a chain.
• operation_error<E: ?Sized> takes _error: &E (unused) — confirm no error detail (secrets/payload) is intended to leak into trace fields later; currently it's inert but the parameter shape invites someone to wire raw error content into observability without redaction review.
• trace_stage! is defined locally in canonical.rs rather than in latency.rs — check this isn't duplicated per-file elsewhere in the crate (macro drift risk).

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the observability-focused executor tracing changes, even though it is not Conventional Commits style.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The PR description covers all required template sections with concrete summary, impact, rollback, validation, and review notes.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[gemini-code-assist]

Code Review

This pull request integrates the "ironclaw_observability" crate to add latency tracing across various stages of the agent loop executor. It introduces a "trace_stage!" macro to measure and log the duration of operations. The review feedback correctly identifies three locations where the iteration number is hardcoded to "0" instead of using "state.iteration", which is necessary for accurate metrics tracking.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The iteration number is hardcoded to 0 here, but the actual iteration is available in state.iteration. Using state.iteration ensures that the latency metrics for this ACK operation are correctly associated with the specific iteration of the loop.

Suggested change

	trace_stage!("ack_pending_input_before_exit", 0, ack.ack(host))?;
	trace_stage!("ack_pending_input_before_exit", state.iteration, ack.ack(host))?;

[gemini-code-assist]

The iteration number is hardcoded to 0 here, but the actual iteration is available in state.iteration. Using state.iteration ensures that the latency metrics for this ACK operation are correctly associated with the specific iteration of the loop.

Suggested change

	trace_stage!("ack_pending_input_before_exit_resume", 0, ack.ack(host))?;
	trace_stage!("ack_pending_input_before_exit_resume", state.iteration, ack.ack(host))?;

[gemini-code-assist]

The iteration number is hardcoded to 0 here, but the actual iteration is available in state.iteration. Using state.iteration ensures that the latency metrics for this ACK operation are correctly associated with the specific iteration of the loop.

Suggested change

	trace_stage!(
	"ack_pending_input_before_exit_skip_model",
	0,
	ack.ack(host)
	)?;
	trace_stage!(
	"ack_pending_input_before_exit_skip_model",
	state.iteration,
	ack.ack(host)
	)?;

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/ironclaw_agent_loop/src/executor.rs`:
- Around line 194-201: The TurnCompletedStep::iteration helper is synthesizing a
sentinel 0 for Exit, which causes canonical.rs tracing to lose the live loop
iteration when labeling post_capability and post_capability_resume. Update
TurnCompletedStep::iteration and the callers around assistant_reply/capabilities
so Exit carries or preserves the current iteration from the caller/state instead
of returning 0, and use that value when emitting the trace boundary.

In `@crates/ironclaw_agent_loop/src/executor/canonical.rs`:
- Around line 329-334: The exit-path ack spans are being recorded with a
hardcoded iteration of 0, which makes the latency traces look like iteration-0
events instead of the active turn. Update the `trace_stage!` calls in the
`StopStep::Stop` exit path within `canonical.rs`—including the
`ack_pending_input_before_exit` variants—to use `state.iteration` (the same
iteration already available in each arm) rather than 0, so the spans are
attributed to the correct step.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: cc4bd4d9-f857-4446-895d-c8659b411ce9

📥 Commits

Reviewing files that changed from the base of the PR and between 940ca7a and 7362e1d.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock, !**/Cargo.lock

📒 Files selected for processing (4)

• crates/ironclaw_agent_loop/Cargo.toml
• crates/ironclaw_agent_loop/src/executor.rs
• crates/ironclaw_agent_loop/src/executor/canonical.rs
• crates/ironclaw_agent_loop/src/executor/latency.rs

[coderabbitai]

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Don't synthesize iteration 0 for TurnCompletedStep::Exit.

Line 198 forces Exit(_) to 0, but canonical.rs uses completed.iteration() to label post_capability and post_capability_resume. Any exit coming out of assistant_reply or capabilities will therefore be traced under iteration 0 instead of the live loop iteration. Carry the caller's current iteration through that trace boundary instead of encoding a sentinel here.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/ironclaw_agent_loop/src/executor.rs` around lines 194 - 201, The
TurnCompletedStep::iteration helper is synthesizing a sentinel 0 for Exit, which
causes canonical.rs tracing to lose the live loop iteration when labeling
post_capability and post_capability_resume. Update TurnCompletedStep::iteration
and the callers around assistant_reply/capabilities so Exit carries or preserves
the current iteration from the caller/state instead of returning 0, and use that
value when emitting the trace boundary.

[railway-app]

🚅 Deployed to the ironclaw-pr-5487 environment in ironclaw-ci-preview

Service	Status	Web	Updated (UTC)
ironclaw	✅ Success (View Logs)	Web	Jul 1, 2026 at 7:57 am