fix(logs): preserve sparse-source breadcrumbs in nemoclaw <sandbox> logs

[laitingsheng]

Summary

nemoclaw <sandbox> logs previously dropped sparse-source breadcrumbs (e.g. the Telegram bridge watchdog message bridge did not start within 15s) when the OpenShell source emitted enough recent activity to dominate the global tail-clip. Each non-empty source now gets a floor of floor(maxLines / non-empty-source-count) of its most recent lines, so a quiet gateway log is no longer silently squeezed out by a chatty audit stream. Strict ≤ maxLines is preserved.

Related Issue

Fixes #5685

Changes

• src/lib/domain/sandbox/logs.ts — mergeTailLogLines rewritten with a per-source floor; remaining slots up to maxLines are filled by the most recent lines across all sources, then the selection is sorted chronologically. Extracted sortChronologically helper.
• src/lib/domain/sandbox/logs.test.ts — added unit cases for sparse-source preservation, untimestamped diagnostic inheritance from a preceding gateway timestamp, and the strict ≤ maxLines guarantee under two chatty sources.
• src/lib/actions/sandbox/logs.test.ts — added a behavioural case that mocks the OpenClaw gateway tail + OpenShell logs probe with a sparse-old gateway and dense-recent OpenShell, asserting both the watchdog breadcrumb and the boot line reach the merged stdout.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Verification

• PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub
• Git hooks passed during commit and push, or `npx prek run --from-ref main --to-ref HEAD` passes
• Targeted tests pass for changed behavior
• Full `npm test` passes (broad runtime changes only)
• Tests added or updated for new or changed behavior
• No secrets, API keys, or credentials committed
• Docs updated for user-facing behavior changes
• `npm run docs` builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: Tinson Lai tinsonl@nvidia.com

Summary by CodeRabbit

• New Features

  • Added gateway breadcrumb display in sandbox logs when output is dominated by other sources.

• Bug Fixes

  • Improved log merging to preserve sparse content from less-active sources.
  • Fixed timestamp inheritance for diagnostic lines across sources.

• Improvements

  • Enhanced log output balancing when merging multiple sources.
  • Optimized maximum line limits to prevent dominant sources from crowding sparse ones.

[coderabbitai]

📝 Walkthrough

Walkthrough

mergeTailLogLines is reworked from a global sort-and-slice approach to a two-phase quota-based algorithm: each non-empty source receives a reserved floor of floor(maxLines / sourceCount) recent lines, remaining slots are filled by newest-timestamp ranking, and a new sortChronologically helper provides deterministic final ordering. Three unit tests and one integration test cover the new behavior.

Quota-based tail merge for gateway log surfacing

Layer / File(s)	Summary
Quota-based mergeTailLogLines algorithm and sortChronologically helper src/lib/domain/sandbox/logs.ts	Replaces the prior flat accumulation and global slice with per-source ScoredLine lists, a floor(maxLines / nonEmptySourceCount) per-source reservation, a timestamp-ranked fill pass for remaining slots, and a new sortChronologically helper that orders by (timestamp, sourceIndex, lineIndex) for deterministic output.
Unit and integration tests for quota-aware merge src/lib/domain/sandbox/logs.test.ts, src/lib/actions/sandbox/logs.test.ts	Three mergeTailLogLines unit tests verify: sparse gateway events are not dropped when interleaved with a denser source; untimestamped diagnostic lines inherit the preceding timestamp; and two 500-line sources are capped to at most 100 non-empty lines. One captureLogsRun integration test asserts that the merged 200-line tail contains gateway breadcrumb text and exits with code 0.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• NVIDIA/NemoClaw#5197: Modifies the sandbox log composition path in showSandboxLogsWithDeps to conditionally include the OpenClaw gateway source, directly affecting the same merge pipeline that this PR fixes.

Poem

🐇 A bunny hopped through logs so wide,
Where gateway whispers got pushed aside.
"Each source deserves a seat!" I cried,
So I split the tail with quota-pride.
Now sparse and chatty live side by side. 🌿

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly and concisely describes the main change: fixing log preservation for sparse sources (breadcrumbs) in sandbox logs, matching the PR's core algorithm change.
Linked Issues check	✅ Passed	The PR fully addresses issue #5685 by implementing per-source floor allocation in mergeTailLogLines to prevent sparse gateway breadcrumbs from being squeezed out by chatty OpenShell logs.
Out of Scope Changes check	✅ Passed	All changes are scoped to logs aggregation functionality: the core algorithm fix in logs.ts, unit tests for the new logic, and integration test. No unrelated changes are present.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/sandbox-logs-merge-per-source-floor

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in the fix/sandbox-logs-mer... branch is 96%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.

File	main	fix/sandbox-logs-mer... 72e2ef4	+/-
nemoclaw/src/se...cret-scanner.ts	—	100%	—
nemoclaw/src/commands/slash.ts	—	100%	—
nemoclaw/src/li...bprocess-env.ts	—	100%	—
nemoclaw/src/bl...eprint/state.ts	—	98%	—
nemoclaw/src/onboard/config.ts	—	98%	—
nemoclaw/src/bl...int/snapshot.ts	—	97%	—
nemoclaw/src/bl...print/runner.ts	—	95%	—
nemoclaw/src/co...ration-state.ts	—	94%	—
nemoclaw/src/bl...ate-networks.ts	—	94%	—
nemoclaw/src/index.ts	—	94%	—

TypeScript / code-coverage/cli

The overall coverage in the fix/sandbox-logs-mer... branch is 47%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.

File	main	fix/sandbox-logs-mer... 72e2ef4	+/-
src/lib/state/o...oard-session.ts	—	91%	—
src/lib/inference/local.ts	—	76%	—
src/lib/sandbox/config.ts	—	72%	—
src/lib/actions...dbox/rebuild.ts	—	71%	—
src/lib/onboard/preflight.ts	—	64%	—
src/lib/actions...licy-channel.ts	—	60%	—
src/lib/state/sandbox.ts	—	55%	—
src/lib/onboard...er-gpu-patch.ts	—	50%	—
src/lib/policy/index.ts	—	49%	—
src/lib/onboard.ts	—	19%	—

---

_{Updated June 25, 2026 16:01 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.}

[coderabbitai]

🧹 Nitpick comments (1)

src/lib/domain/sandbox/logs.ts (1)

180-212: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

Quota algorithm correctly enforces the ≤ maxLines cap.

The bound holds: reserved.size ≤ nonEmptyCount * floor(maxLines / nonEmptyCount) ≤ maxLines, so remaining is never negative and the final set never exceeds maxLines. Identity-based Set<ScoredLine> and the chronological re-sort are sound.

One optional cleanup: perSource.flat() is recomputed on both the maxLines <= 0 path (Line 181) and the remaining-fill path (Line 199). Minor, but you could hoist it once if you want to avoid the duplicate traversal.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/domain/sandbox/logs.ts` around lines 180 - 212, The perSource.flat()
method is being called twice in this function, once on line 181 in the maxLines
<= 0 branch and again on line 199 when creating the candidates array, causing
unnecessary traversal of the same data. Extract perSource.flat() into a single
variable declared before the maxLines <= 0 conditional check, and then reuse
that variable in both the early return path and when filtering candidates for
the remaining quota allocation.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/domain/sandbox/logs.ts`:
- Around line 180-212: The perSource.flat() method is being called twice in this
function, once on line 181 in the maxLines <= 0 branch and again on line 199
when creating the candidates array, causing unnecessary traversal of the same
data. Extract perSource.flat() into a single variable declared before the
maxLines <= 0 conditional check, and then reuse that variable in both the early
return path and when filtering candidates for the remaining quota allocation.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4e3b06e7-b4e9-4efa-8435-ad84143ce46d

📥 Commits

Reviewing files that changed from the base of the PR and between 6fb9754 and 9460f56.

📒 Files selected for processing (3)

• src/lib/actions/sandbox/logs.test.ts
• src/lib/domain/sandbox/logs.test.ts
• src/lib/domain/sandbox/logs.ts

[github-actions]

E2E Advisor Recommendation

Required E2E: sandbox-operations-e2e
Optional E2E: full-e2e

Dispatch hint: sandbox-operations-e2e

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

• sandbox-operations-e2e (medium): This is the targeted existing live E2E for sandbox operations and includes nemoclaw <sandbox> logs and nemoclaw <sandbox> logs --follow checks against a real sandbox. The production change alters log tail merging for those commands, so a live sandbox validation is required.

Optional E2E

• full-e2e (medium): The full install/onboard/user-journey script also performs a basic nemoclaw logs retrieval check. It is useful additional confidence that the changed logs path still works in the common hosted onboarding flow, but the sandbox-operations E2E is the more targeted merge-blocking coverage.

New E2E recommendations

• sandbox logs and diagnostics (medium): Existing live E2E appears to assert that nemoclaw logs produces output, but not that sparse gateway diagnostics remain visible when OpenShell/audit output is much chattier. The new production behavior is specifically a fairness policy across log sources.
  • Suggested test: Add a sandbox-operations E2E case that writes identifiable sparse lines to the gateway log, generates a dense newer stream from another source, runs nemoclaw <sandbox> logs -n <N>, and asserts the sparse gateway breadcrumbs are retained in the merged output.

Dispatch hint

• Workflow: .github/workflows/nightly-e2e.yaml
• jobs input: sandbox-operations-e2e

[github-actions]

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: full-e2e-vitest
Optional Vitest E2E scenarios: None

Dispatch required Vitest E2E scenarios:

• gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=full-e2e-vitest

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required Vitest E2E scenarios

• full-e2e-vitest: The PR changes sandbox log tail merging used by nemoclaw <sandbox> logs. The wired full-e2e-vitest job runs a real OpenClaw sandbox flow and asserts that nemoclaw logs succeeds and produces output, making it the smallest dispatch that exercises this changed CLI surface through the Vitest E2E workflow.
  • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=full-e2e-vitest

Optional Vitest E2E scenarios

• None.

Relevant changed files

• src/lib/actions/sandbox/logs.test.ts
• src/lib/domain/sandbox/logs.test.ts
• src/lib/domain/sandbox/logs.ts

[github-actions]

PR Review Advisor — No blocking findings

Merge posture: No blocking advisor findings
Primary next action: Add or justify PRA-T1 and any related test follow-ups.
Open items: 0 required · 0 warnings · 0 suggestions · 8 test follow-ups
Since last review: 0 prior items resolved · 0 still apply · 0 new items found

Action checklist

• PRA-T1 Add or justify test follow-up: Runtime validation
• PRA-T2 Add or justify test follow-up: Runtime validation
• PRA-T3 Add or justify test follow-up: Runtime validation
• PRA-T4 Add or justify test follow-up: Acceptance clause
• PRA-T5 Add or justify test follow-up: Acceptance clause
• PRA-T6 Add or justify test follow-up: Acceptance clause
• PRA-T7 Add or justify test follow-up: Acceptance clause
• PRA-T8 Add or justify test follow-up: Acceptance clause

Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

• PRA-T1 Runtime validation — Add or identify a runtime/integration validation where /tmp/gateway.log contains an older Telegram 'bridge did not start within 15s' watchdog line, OpenShell emits at least --tail newer lines, and `nemoclaw <sandbox> logs --tail 200` includes the watchdog while returning no more than 200 non-empty lines.. The changed behavior has focused domain and action-level mocked coverage, but it affects a sandbox logs path where real OpenShell gateway capture, /tmp/gateway.log contents, and CLI stdout line limits can differ from mocks.
• PRA-T2 Runtime validation — Add a focused unit case for `mergeTailLogLines(["\n", "[1] real\n"], 1)` or the desired equivalent to confirm newline-only sources do not reserve blank tail slots if blank probe output is possible.. The changed behavior has focused domain and action-level mocked coverage, but it affects a sandbox logs path where real OpenShell gateway capture, /tmp/gateway.log contents, and CLI stdout line limits can differ from mocks.
• PRA-T3 Runtime validation — Extend the new action-level sparse gateway test to count non-empty stdout lines so the same mocked gateway/OpenShell scenario verifies both watchdog visibility and the strict tail cap.. The changed behavior has focused domain and action-level mocked coverage, but it affects a sandbox logs path where real OpenShell gateway capture, /tmp/gateway.log contents, and CLI stdout line limits can differ from mocks.
• PRA-T4 Acceptance clause — **NemoClaw version:** v0.0.66 — add test evidence or identify existing coverage. Issue environment metadata; this PR does not change versioning and the changed files do not provide runtime evidence for v0.0.66.
• PRA-T5 Acceptance clause — **Host:** DGX Spark — add test evidence or identify existing coverage. Issue environment metadata; the changed code in src/lib/domain/sandbox/logs.ts is host-independent TypeScript log selection logic and tests do not model DGX Spark.
• PRA-T6 Acceptance clause — **Sandbox name:** tg-watchdog — add test evidence or identify existing coverage. The action tests use sandbox name 'alpha', but src/lib/actions/sandbox/logs.ts passes the sandbox name through to OpenShell argument builders unchanged; no tg-watchdog-specific behavior is introduced.
• PRA-T7 Acceptance clause — **Policy presets:** telegram — add test evidence or identify existing coverage. The PR does not modify policy preset handling. It models the downstream Telegram watchdog log line produced when the preset/setup has already resulted in gateway output.
• PRA-T8 Acceptance clause — NemoClaw CLI installed and Docker running. — add test evidence or identify existing coverage. Existing action coverage exercises the Docker-not-down branch with mocked dependencies, and the Docker-down negative path remains covered, but no installed CLI plus real Docker runtime validation is included in the diff.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

[cjagwani]

Took a look at this. The fix actually solves the breadcrumb-drop from #5685, sparse sources like the Telegram bridge watchdog don't get squeezed out by a chatty OpenShell source anymore. Single-source case behaves the same as before. LGTM.