Skip to content

feat(agent-loop): content-digest plumbing for output-aware progress (PR2) - #5000

Merged
serrrfirat merged 6 commits into
mainfrom
firat/no-progress-pr2-content-digest
Jun 17, 2026
Merged

serrrfirat merged 6 commits into
mainfrom
firat/no-progress-pr2-content-digest

Conversation

@serrrfirat

Copy link
Copy Markdown
Collaborator

Stacked on #4993 (PR1). Review against the base branch firat/elastic-driscoll-86596f so the diff shows only this change; retarget to main once #4993 merges.

What

PR2 of the no-progress redesign. This is inert plumbing: it adds a ContentDigest of each completed capability output so a later change (PR3) can treat "the output moved" as real progress. This PR does not change any detection/stop behavior.

  • ContentDigest newtype (keyed-BLAKE3, low-8-bytes-LE — mirrors ArgsHash) in ironclaw_turns::run_profile::content_digest. normalize_for_hash is moved there and re-exported from agent_loop::strategies::progress, so CapabilityCallSignature hashing is byte-for-byte unchanged (no checkpoint-signature break).
  • output_digest: Option<ContentDigest> on CapabilityResultMessage, #[serde(default)] (None = back-compat). Computed host-side in the single write_capability_result chokepoint alongside byte_len. Best-effort: a digest-compute failure degrades to None and never fails an otherwise-successful capability write.
  • seen_capability_output_digests: BoundedRing<_, 64> on LoopExecutionState, populated at append_completed_capability_result but not consumed. Additive #[serde(default)] checkpoint field — legacy checkpoints decode to an empty ring (round-trip + legacy-decode tests added).
  • Inertness is the invariant: record_result still receives the host's progress unchanged. An executor-level test asserts repeated identical-output completed calls do not newly trip no-progress.

Review fixes folded in (multi-agent review)

  • from_output is fail-open (digest failure → None, never fails the write).
  • A caller-level test drives a real run and asserts the digest lands in the ring (so the one production wiring line can't silently regress while inert).
  • Boundary note documents why the digest impl lives in ironclaw_turns.

Roadmap

Validation

Local gate was run partially under heavy machine load: cargo fmt clean; full-workspace --all-targets build was at 0 errors (compiling final binaries) when stopped to free the machine. Deferring the authoritative clippy --all-features + test gate to CI — please watch CI before merge.

🤖 Generated with Claude Code

serrrfirat and others added 2 commits June 16, 2026 22:57
…mpletion

When the runaway-loop safety guard fires (StopKind::NoProgressDetected), the
executor finalized a canned "I stopped because I was repeating the same step"
assistant reply and returned the run as Completed — a runtime control decision
leaking into the conversation as a fake successful turn. This hid real blockers
(auth, broken connector, empty search) and marked incomplete tasks as done.

The ExitStage NoProgressDetected arm now:
- keeps the #4837 final-answer-nudge path bit-for-bit: when the gate is enabled
  and the model synthesizes a real closing answer, complete with that answer
  (PinchBench path unchanged);
- otherwise writes the Final checkpoint and returns a typed
  failed_exit(LoopFailureKind::NoProgressDetected) instead of the canned reply.
  The product layer already maps "no_progress_detected" to deterministic copy,
  so the user sees an honest failure on every channel.

Deletes NO_PROGRESS_FALLBACK_REPLY and finalize_no_progress_fallback.

PR1 of the output-aware no-progress redesign (exit honesty). PR2 (content-digest
progress signal) and PR3 (failures off the no-progress axis) follow.

Wire behavior: gate-off no-progress runs now arrive as Failed{no_progress_detected}
instead of Completed with a canned reply.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…PR2)

Inert plumbing for output-aware no-progress detection (PR3 will consume it).
Adds a ContentDigest of each completed capability output so a later change can
treat "the output moved" as real progress. This PR does NOT change any
detection/stop behavior — it is observably inert.

- ContentDigest newtype (keyed-blake3, mirrors ArgsHash) in ironclaw_turns;
  normalize_for_hash moved there and re-exported from agent_loop::strategies so
  CapabilityCallSignature hashing is unchanged (no checkpoint-signature break).
- output_digest: Option<ContentDigest> on CapabilityResultMessage (serde default
  None = back-compat); computed host-side in the write_capability_result
  chokepoint alongside byte_len. Best-effort: a digest-compute failure degrades
  to None and never fails an otherwise-successful capability write.
- seen_capability_output_digests: BoundedRing<_, 64> on LoopExecutionState,
  populated at append_completed_capability_result but NOT consumed. Additive
  #[serde(default)] checkpoint field (legacy checkpoints decode to an empty
  ring; round-trip + legacy-decode tests added).
- record_result still receives the host's progress unchanged — detection is
  byte-identical (inertness test drives the executor and asserts no behavior
  change for repeated identical-output completed calls).

Review fixes folded in (multi-agent review): from_output is fail-open; a
caller-level test asserts the digest is recorded into the ring through a real
run; boundary note documents why the digest impl lives in ironclaw_turns.

PR2 of the no-progress redesign. PR1 (#4993) = honest typed failure;
PR3 = consume the digest (NoChange when output repeats) + take failures off the
no-progress axis. Benchmark-gated at PR3.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5000 June 16, 2026 22:56 Destroyed
@railway-app

railway-app Bot commented Jun 16, 2026 •

Copy link
Copy Markdown

🚅 Deployed to the ironclaw-pr-5000 environment in ironclaw-ci-preview

Service Status Web Updated (UTC)
ironclaw ✅ Success (View Logs) Web Jun 17, 2026 at 10:17 pm

@coderabbitai

coderabbitai Bot commented Jun 16, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 7d0c3ab6-4f3a-4a73-9717-4981bedce4ca

📥 Commits

Reviewing files that changed from the base of the PR and between 0606bf7 and 19a4d81.

📒 Files selected for processing (1)
  • crates/ironclaw_reborn/tests/loop_driver_host.rs

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added deterministic output_digest support for capability results (with output normalization for UUIDs, timestamps, and correlation keys).
    • Executor now records observed output digests into checkpointed run state.
  • Refactor

    • Updated capability result writing to return richer metadata, including an optional output digest.
    • Rewired hashing/normalization to use the shared digest utilities.
  • Tests

    • Updated fixtures/middleware/test harnesses for the new output_digest field.
    • Added coverage for checkpoint round-trips, backward-compatible decoding, digest recording, and repeated-identical digest behavior.

Walkthrough

Adds a ContentDigest newtype (Blake3/JCS, 8-byte truncated u64) computed from normalized JSON capability output. The digest is threaded through a new CapabilityWriteResult struct that replaces the (LoopResultRef, u64) tuple return of LoopCapabilityResultWriter::write_capability_result. Completed capability results carrying a digest are recorded as CapabilityOutputObservation entries in a new seen_capability_output_digests: BoundedRing<_, 64> field on LoopExecutionState. The local normalize_for_hash implementation is consolidated into ironclaw_turns::run_profile.

Changes

Capability Output Digest Feature

Layer / File(s) Summary
ContentDigest type, normalization, and hashing
crates/ironclaw_turns/Cargo.toml, crates/ironclaw_turns/src/run_profile/content_digest.rs, crates/ironclaw_turns/src/run_profile/mod.rs
New ContentDigest newtype and ContentDigestError implemented; pipeline is non-finite rejection → normalize_for_hash (UUID/timestamp/correlation-key scrubbing) → JCS canonicalization → Blake3 keyed hash → 8-byte truncation. blake3 and serde_jcs added as dependencies.
CapabilityResultMessage output_digest field
crates/ironclaw_turns/src/run_profile/host.rs
Optional output_digest: Option<ContentDigest> added to CapabilityResultMessage with #[serde(default, skip_serializing_if = "Option::is_none")] for backward-compatible checkpoint decode.
normalize_for_hash consolidated into ironclaw_turns
crates/ironclaw_agent_loop/src/strategies/progress.rs, crates/ironclaw_agent_loop/src/state/signature.rs
Local normalize_for_hash implementation (169 lines) removed from progress.rs and replaced with pub(crate) use ironclaw_turns::run_profile::normalize_for_hash. signature.rs import updated accordingly.
CapabilityWriteResult struct and trait signature
crates/ironclaw_loop_support/src/capability_port.rs, crates/ironclaw_loop_support/src/lib.rs
LoopCapabilityResultWriter::write_capability_result return type changed from (LoopResultRef, u64) to CapabilityWriteResult { result_ref, byte_len, output_digest }. Adds without_output_digest and from_output constructors; from_output degrades to None on digest failure.
CapabilityOutputObservation and LoopExecutionState field
crates/ironclaw_agent_loop/src/state/signature.rs, crates/ironclaw_agent_loop/src/state.rs
New CapabilityOutputObservation { signature, output_digest } struct added and re-exported. LoopExecutionState gains seen_capability_output_digests: BoundedRing<CapabilityOutputObservation, 64> with #[serde(default)]. Checkpoint round-trip and legacy-decode tests added.
Executor digest recording
crates/ironclaw_agent_loop/src/executor/capabilities.rs
append_completed_capability_result appends CapabilityOutputObservation to seen_capability_output_digests when output_digest is present. Synthesized messages for AwaitDependentRun/SpawnedChildRun paths set output_digest: None.
Test support: ScriptedCapabilityOutcome digest
crates/ironclaw_agent_loop/src/test_support/mod.rs
ScriptedCapabilityOutcome::Completed gains output_digest: Option<ContentDigest>. New completed_with_output_digest constructor added. All other completed* helpers set output_digest: None. Translation to CapabilityResultMessage forwards the field.
Production write_capability_result implementations
crates/ironclaw_reborn_composition/src/product_live_adapters.rs, crates/ironclaw_reborn_composition/src/runtime/local_dev.rs, .../local_dev/outbound_delivery.rs, .../local_dev/skill_activation.rs, .../production.rs, crates/ironclaw_loop_support/src/subagent_spawn_port.rs
All production LoopCapabilityResultWriter impls updated to return CapabilityWriteResult::from_output(...). SubagentSpawnCapabilityPort::finish_spawn switches from tuple destructuring to struct field access.
Test double and helper implementations
crates/ironclaw_loop_support/src/subagent_spawn_port/tests.rs, crates/ironclaw_loop_support/tests/host_capability_port_composition.rs, crates/ironclaw_reborn/tests/loop_driver_host.rs, crates/ironclaw_reborn_composition/tests/product_live_adapters.rs, tests/support/reborn/harness.rs
Test-only LoopCapabilityResultWriter impls (NoopResultWriter, FixedByteResultWriter, InMemoryCapabilityIo, UnusedResultWriter, RecordingCapabilityResultWriter) updated to return CapabilityWriteResult. Uses from_output for JSON capture, without_output_digest for fixed stubs.
Executor and safety net tests for digest recording
crates/ironclaw_agent_loop/src/executor/tests.rs, crates/ironclaw_agent_loop/tests/safety_nets.rs, crates/ironclaw_reborn/tests/loop_driver_host.rs
New test completed_output_digest_is_recorded_into_seen_capability_output_digests verifies end-to-end recording in checkpointed state. Safety net test asserts repeated identical digests do not alter progress-stop behavior (model calls = 5, warning prompt = 1). Loop driver test refactored to use tokio::join!.
Test fixture output_digest: None additions (broad)
crates/ironclaw_agent_loop/src/executor/..., crates/ironclaw_hooks/..., crates/ironclaw_loop_support/..., crates/ironclaw_product_workflow/..., crates/ironclaw_reborn/..., crates/ironclaw_reborn_composition/..., tests/support/reborn/harness.rs
All test stubs and fixture CapabilityResultMessage literals updated to set output_digest: None (23 executor test fixtures, cancellation, capability_helpers, middleware, loop_support, product_workflow, subagent, reborn, reborn_composition, harness). Import lists reformatted where needed.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Suggested reviewers

  • henrypark133

Poem

🔐 Blake3 chews the JSON whole,
UUIDs scrubbed, timestamps scroll,
Eight bytes tucked in a u64,
The ring records what came before.
No phantom progress shall prevail —
The digest guards the replay trail. 🦀

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed Title follows Conventional Commits style (feat scope: summary) with descriptive, specific content related to output digest plumbing.
Description check ✅ Passed Author description is comprehensive (objectives, validation status, roadmap, stacked PR note). Template sections unfilled, but substantive inline detail covers required aspects.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added scope: dependencies Dependency updates size: XL 500+ changed lines risk: low Changes to docs, tests, or low-risk modules contributor: core 20+ merged PRs labels Jun 16, 2026
serrrfirat and others added 2 commits June 17, 2026 15:22
…nto firat/no-progress-pr2-content-digest

# Conflicts:
#	crates/ironclaw_product_workflow/tests/inbound_turn_contract.rs
#	crates/ironclaw_reborn/tests/loop_driver_host.rs
#	crates/ironclaw_reborn_composition/tests/product_live_adapters.rs
#	tests/support/reborn/harness.rs
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5000 June 17, 2026 12:41 Destroyed
Base automatically changed from firat/elastic-driscoll-86596f to main June 17, 2026 21:15
…-content-digest

# Conflicts:
#	crates/ironclaw_agent_loop/src/executor/loop_exit.rs
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5000 June 17, 2026 21:24 Destroyed
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5000 June 17, 2026 22:00 Destroyed
@serrrfirat
serrrfirat merged commit a6f7395 into main Jun 17, 2026
68 checks passed
@serrrfirat
serrrfirat deleted the firat/no-progress-pr2-content-digest branch June 17, 2026 22:24
theredspoon pushed a commit to theredspoon/ironclaw that referenced this pull request Jun 21, 2026
…PR2) (nearai#5000)

* fix(agent-loop): no-progress stop fails honestly instead of faking completion

When the runaway-loop safety guard fires (StopKind::NoProgressDetected), the
executor finalized a canned "I stopped because I was repeating the same step"
assistant reply and returned the run as Completed — a runtime control decision
leaking into the conversation as a fake successful turn. This hid real blockers
(auth, broken connector, empty search) and marked incomplete tasks as done.

The ExitStage NoProgressDetected arm now:
- keeps the nearai#4837 final-answer-nudge path bit-for-bit: when the gate is enabled
  and the model synthesizes a real closing answer, complete with that answer
  (PinchBench path unchanged);
- otherwise writes the Final checkpoint and returns a typed
  failed_exit(LoopFailureKind::NoProgressDetected) instead of the canned reply.
  The product layer already maps "no_progress_detected" to deterministic copy,
  so the user sees an honest failure on every channel.

Deletes NO_PROGRESS_FALLBACK_REPLY and finalize_no_progress_fallback.

PR1 of the output-aware no-progress redesign (exit honesty). PR2 (content-digest
progress signal) and PR3 (failures off the no-progress axis) follow.

Wire behavior: gate-off no-progress runs now arrive as Failed{no_progress_detected}
instead of Completed with a canned reply.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(agent-loop): content-digest plumbing for output-aware progress (PR2)

Inert plumbing for output-aware no-progress detection (PR3 will consume it).
Adds a ContentDigest of each completed capability output so a later change can
treat "the output moved" as real progress. This PR does NOT change any
detection/stop behavior — it is observably inert.

- ContentDigest newtype (keyed-blake3, mirrors ArgsHash) in ironclaw_turns;
  normalize_for_hash moved there and re-exported from agent_loop::strategies so
  CapabilityCallSignature hashing is unchanged (no checkpoint-signature break).
- output_digest: Option<ContentDigest> on CapabilityResultMessage (serde default
  None = back-compat); computed host-side in the write_capability_result
  chokepoint alongside byte_len. Best-effort: a digest-compute failure degrades
  to None and never fails an otherwise-successful capability write.
- seen_capability_output_digests: BoundedRing<_, 64> on LoopExecutionState,
  populated at append_completed_capability_result but NOT consumed. Additive
  #[serde(default)] checkpoint field (legacy checkpoints decode to an empty
  ring; round-trip + legacy-decode tests added).
- record_result still receives the host's progress unchanged — detection is
  byte-identical (inertness test drives the executor and asserts no behavior
  change for repeated identical-output completed calls).

Review fixes folded in (multi-agent review): from_output is fail-open; a
caller-level test asserts the digest is recorded into the ring through a real
run; boundary note documents why the digest impl lives in ironclaw_turns.

PR2 of the no-progress redesign. PR1 (nearai#4993) = honest typed failure;
PR3 = consume the digest (NoChange when output repeats) + take failures off the
no-progress axis. Benchmark-gated at PR3.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(reborn): stabilize turn runner operator log capture

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

This branch was successfully deployed

No deployments
ironclaw-ci-preview / ironclaw-pr-5000 — 19a4d816 Deployed Jun 17, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: low Changes to docs, tests, or low-risk modules scope: dependencies Dependency updates size: XL 500+ changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant