Skip to content

fix(dcode): harden Nemotron Ultra tool requests - #6494

Merged
ericksoa merged 88 commits into
mainfrom
fix/dcode-ultra-managed-compat/ericksoa
Jul 8, 2026
Merged

fix(dcode): harden Nemotron Ultra tool requests#6494
ericksoa merged 88 commits into
mainfrom
fix/dcode-ultra-managed-compat/ericksoa

Conversation

@ericksoa

@ericksoa ericksoa commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Harden the managed Nemotron 3 Ultra DCode aliases at two separate compatibility
boundaries: emit the model-specific nonempty-content template argument required
for reasoning-plus-tool-call requests, and reject the observed literal
[content] execute placeholder before it reaches shell dispatch.

#6431 is merged; this PR is based directly on current main. It leaves the
canonical NVIDIA profile and all unrelated models unchanged.

Source boundary and lifecycle

  • force_nonempty_content: the invalid empty-assistant-content state originates
    in the Ultra serving template. NemoClaw owns only the generated per-model
    request configuration. Remove the override after a reviewed serving/client
    update works for both managed Ultra IDs without it and the live DCode E2E passes.
  • Execute guard: the invalid state is a model-produced tool call whose complete
    execute.command is the literal [content] (ignoring ASCII whitespace around the token and
    brackets, and case). NemoClaw owns the managed alias middleware immediately before dispatch,
    not the model, serving template, or hash-locked canonical Deep Agents profile.
    Remove the guard only after those upstream paths no longer produce or convert
    the placeholder and the focused plus live tests pass with it deleted.

Changes

  • Generate force_nonempty_content = true only for the two managed Ultra model IDs.
  • Add a managed-alias-only sync/async middleware guard for the exact placeholder;
    concrete execute commands and other tools pass through unchanged.
  • Validate rejection through the pinned LangChain ToolMessage.content API in
    the image validator, fixture, and hosted E2E probe.
  • Preserve atomic/idempotent profile registration with the process-local
    registry as source of truth and a lock around multi-key registration.
  • Persist the credential-free observability enable bit across environment-less
    policy restarts; create/rebuild/clone paths pass an explicit 1 or 0.
  • Record invalid-state sources, regression evidence, and removal conditions in
    the dependency review.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: no CLI, setting, supported-model choice, or operator workflow changes; the internal dependency review is updated
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: pending maintainer review
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — combined Ultra, lifecycle, current-main, and acceptance suites passed 159/159; the post-hook launcher/acceptance subset passed 33/33
  • Applicable broad gate passed — final exact-head regular CI and hosted Deep Agents Code E2E are running
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Aaron Erickson aerickson@nvidia.com

Summary by CodeRabbit

  • New Features
    • Added model-specific “Ultra” configuration for selected OpenAI-compatible model IDs to enforce non-empty tool-call/reasoning content.
    • Added managed execute placeholder guard behavior and deeper end-to-end validation for Tavily policy remove/restore ordering and observability recovery.
  • Bug Fixes
    • Hardened managed execute handling to reject placeholder execute calls (including async/sync parity) while allowing concrete commands.
    • Improved managed alias registration for safer atomic idempotence and rollback.
  • Documentation
    • Updated compatibility/validation guidance and the dependency advisory review hash.
  • Tests
    • Expanded E2E/profile/config contract tests, including workflow host-dependency step ordering and guard dispatch assertions.

ericksoa added 30 commits July 7, 2026 18:50
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
…tra-profile-plugin

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
…tra-profile-plugin

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
…tra-profile-plugin

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
…tra-profile-plugin

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
…tra-profile-plugin

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: cjagwani <cjagwani@nvidia.com>
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ⚠️ Run cancelled — no signal

Run: 28981246261
Workflow ref: fix/dcode-ultra-managed-compat/ericksoa
Requested targets: ubuntu-repo-cloud-langchain-deepagents-code
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 0 passed, 0 failed, 1 cancelled, 0 skipped

Job Result
live ⚠️ cancelled

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ⚠️ Run cancelled — no signal

Run: 28981367043
Workflow ref: fix/dcode-ultra-managed-compat/ericksoa
Requested targets: ubuntu-repo-cloud-langchain-deepagents-code
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 0 passed, 0 failed, 1 cancelled, 0 skipped

Job Result
live ⚠️ cancelled

cjagwani and others added 3 commits July 8, 2026 23:00
Signed-off-by: cjagwani <cjagwani@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ❌ Some jobs failed

Run: 28981919658
Workflow ref: fix/dcode-ultra-managed-compat/ericksoa
Requested targets: ubuntu-repo-cloud-langchain-deepagents-code
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 0 passed, 1 failed, 0 cancelled, 0 skipped

Job Result
live ❌ failure

Failed jobs: live. Check run artifacts for logs.

ericksoa added 3 commits July 8, 2026 16:24
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Comment thread test/langchain-deepagents-code-proxy-launcher.test.ts Fixed
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All selected jobs passed

Run: 28983005052
Workflow ref: fix/dcode-ultra-managed-compat/ericksoa
Requested targets: ubuntu-repo-cloud-langchain-deepagents-code
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 1 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
live ✅ success

@cjagwani cjagwani added v0.0.79 and removed v0.0.78 labels Jul 8, 2026
Signed-off-by: cjagwani <cjagwani@nvidia.com>
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ⚠️ Run cancelled — no signal

Run: 28983472371
Workflow ref: fix/dcode-ultra-managed-compat/ericksoa
Requested targets: ubuntu-repo-cloud-langchain-deepagents-code
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 0 passed, 0 failed, 1 cancelled, 0 skipped

Job Result
live ⚠️ cancelled

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All selected jobs passed

Run: 28983514326
Workflow ref: fix/dcode-ultra-managed-compat/ericksoa
Requested targets: ubuntu-repo-cloud-langchain-deepagents-code
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 1 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
live ✅ success

@ericksoa
ericksoa merged commit 3f5133e into main Jul 8, 2026
121 checks passed
@ericksoa
ericksoa deleted the fix/dcode-ultra-managed-compat/ericksoa branch July 8, 2026 23:56
@cjagwani cjagwani mentioned this pull request Jul 9, 2026
21 tasks
cv pushed a commit that referenced this pull request Jul 9, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Prepares the user documentation for NemoClaw v0.0.78 by replacing the
unreleased section with release highlights and synchronizing the
affected inference, lifecycle, messaging, and CLI reference pages with
merged behavior.

## Changes

- Publish the v0.0.78 release-notes section with links to the most
specific user guides for each shipped behavior.
- Document authoritative Deep Agents route health, Nemotron Ultra
profile behavior, and Hermes compatible-endpoint context metadata.
- Document forced rebuild recovery after total backup failure and the
ownership-safe tunnel/full-stop behavior.
- Keep command examples and shared agent variants aligned with the
current OpenClaw, Hermes, and Deep Agents interfaces.

Source mapping:

- [#3787](#3787) ->
`docs/about/release-notes.mdx`: Record reliable workspace template
seeding during sandbox startup.
- [#4960](#4960) ->
`docs/about/release-notes.mdx`: Record safer detection of rewritten
OpenClaw gateway processes.
- [#5676](#5676) ->
`docs/about/release-notes.mdx`: Record warning-tolerant agent-list JSON
handling.
- [#5857](#5857) ->
`docs/about/release-notes.mdx`: Record synchronization of explicit
OpenClaw main-agent model state.
- [#5929](#5929) ->
`docs/about/release-notes.mdx`: Record copyable SSH port-forward
guidance for remote dashboards.
- [#6068](#6068) ->
`docs/about/release-notes.mdx`: Record custom-image plugin provenance
reconciliation.
- [#6116](#6116) ->
`docs/about/release-notes.mdx`: Record live-loopback dashboard-forward
recovery.
- [#6122](#6122) ->
`docs/about/release-notes.mdx`: Announce validated, round-trippable
policy YAML output.
- [#6211](#6211) ->
`docs/manage-sandboxes/lifecycle.mdx`, `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Explain the explicit no-backup `rebuild
--force` recovery boundary.
- [#6283](#6283) ->
`docs/about/release-notes.mdx`: Record Hermes WebUI port alignment.
- [#6293](#6293) ->
`docs/inference/switch-inference-providers.mdx`,
`docs/about/release-notes.mdx`: Document compatible-endpoint
context-length probing for Hermes.
- [#6320](#6320) ->
`docs/about/release-notes.mdx`: Record bounded gateway-recovery waits.
- [#6377](#6377) ->
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Explain
rebuild diagnostics and prepared MCP-destroy recovery.
- [#6412](#6412) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document authoritative agent-visible
inference route health.
- [#6421](#6421) ->
`docs/about/release-notes.mdx`: Record the longer quiet-pull window for
managed vLLM images.
- [#6431](#6431) ->
`docs/inference/model-capability-audit.mdx`,
`docs/about/release-notes.mdx`: Document the version-pinned Nemotron
Ultra profile plugin.
- [#6439](#6439) ->
`docs/about/release-notes.mdx`: Summarize the authenticated, pinned
credential-capture helper boundary.
- [#6450](#6450) ->
`docs/manage-sandboxes/messaging-channels.mdx`,
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Document
host-forward cleanup and ownership-safe gateway-port release.
- [#6474](#6474) ->
`docs/manage-sandboxes/messaging-channels.mdx`,
`docs/about/release-notes.mdx`: Record composable OpenClaw messaging
runtime loaders.
- [#6475](#6475) ->
`docs/about/release-notes.mdx`: Record removal of the unavailable Kimi
K2.6 production endpoint option.
- [#6480](#6480) ->
`docs/about/release-notes.mdx`: Record stderr routing for the plugin
registration banner.
- [#6481](#6481) ->
`docs/about/release-notes.mdx`: Record post-pull Ollama model discovery
checks.
- [#6482](#6482) ->
`docs/about/release-notes.mdx`: Record Ollama model warm-up after daemon
restart.
- [#6486](#6486) ->
`docs/about/release-notes.mdx`: Publish the opt-in, thread-scoped Deep
Agents auto-approval boundary.
- [#6490](#6490) ->
`docs/about/release-notes.mdx`: Record diagnostics for custom images
missing the managed runtime.
- [#6494](#6494) ->
`docs/inference/model-capability-audit.mdx`,
`docs/about/release-notes.mdx`: Document nonempty tool-call content
preservation and placeholder rejection.
- [#6497](#6497) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document isolated Deep Agents
route-probe output.
- [#6506](#6506) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document observability-preserving
managed route probes.
- [#6508](#6508) ->
`docs/about/release-notes.mdx`: Link the new extension taxonomy and
SDK-readiness reference from the release summary.

Release-source verification: GitHub reports all 29 cited source PRs as
merged with base `main`, and every merge commit is an ancestor of
`origin/main` at `17bf9a6a9688b3b1d69cf4b37d3f23110acb055e`. No
source-mapping mismatches were found.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [x] Doc only (includes code sample changes)

## Quality Gates

<!-- Check exactly one tests line and one docs line. Check other lines
when applicable. Add every requested justification or approval
reference. -->
- [ ] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [x] Tests not applicable — justification: Documentation-only
release-prep changes; `npm run docs` validates variants, routes, and
Fern content.
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

<!-- Check each applicable item only when supported by the requested
evidence. Run targeted tests once per relevant change set and rerun
after later edits or hook autofixes that can affect the tested behavior.
Do not rerun hook-covered checks. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: Tests
are not applicable to this documentation-only change set.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result:
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) — exited
0 with zero errors; Fern reported the existing unauthenticated
redirect-check and light-mode contrast warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

---------

Signed-off-by: cjagwani <cjagwani@nvidia.com>
cv added a commit that referenced this pull request Jul 12, 2026
<!-- markdownlint-disable MD041 -->
## Summary
Follow-up to #6431 for exact-head validation failures that completed
immediately before that PR merged. This isolates sequential DCode E2E
state, preserves observability across policy-only reloads, lets generic
OpenClaw cloud-onboard runs skip DCode-only TUI prerequisites, and
closes the remaining valid advisor requests without changing user-facing
behavior.

## Related Issue
Follow-up to #6431, which closed #6424. No new issue.

## Changes
- Remove the durable Tavily preset after check 09, arm cleanup on
`EXIT`, and fail unless managed DCode Python returns to the default
network denial.
- Move check 12's `expect`, Node, and timeout prerequisites after its
DCode sandbox guard. Generic `cloud-onboard` therefore skips correctly;
typed DCode jobs continue to require and install `expect`.
- Keep the credential-free observability enable marker in managed
`/sandbox/.deepagents` state for exec/login recovery. Current `main` now
owns the durable lifecycle: absent entrypoint env preserves the bit
across policy-only restarts, while create/rebuild/clone pass an explicit
authoritative `1` or `0`. Check 09 requires the host registry's DCode
`observabilityEnabled=true` intent and the exact persistent marker to
agree before and after Tavily policy mutation; it fails rather than
reconstructing intent from sandbox-writable state. Host-managed network
policy remains the egress boundary.
- Reject a pre-existing non-regular or symlink marker target with a
clear startup error, and use no-target-directory replacement so a raced
directory cannot absorb the enabled marker. Fixtures cover both enabled
and disabled startup.
- In live check 11, remove only `observability-otlp-local`, recreate the
exact marker as the sandbox user, and prove both the normally allowed
raw route and marker-triggered deterministic instrumentation produce
zero host captures. Restore only from an exactly inactive policy state
before the existing positive trace checks.
- Replace repeated credential-name limit literals with
`CREDENTIAL_NAME_PREFIX_MAX_LENGTH` and prove every Bash quantifier
remains equal to the canonical TypeScript context-pattern limit.
- Cover exact 128/129 context-prefix behavior, oversized OpenShell
placeholders, and Tavily cleanup-command failures through the real
wrapper/self-test boundaries.
- Preserve the merged #6494 Nemotron Ultra request hardening and its
observability create/snapshot lifecycle while resolving the overlapping
follow-up checks against current `main`.
- Keep standalone DCode Tavily persistence and rebuild behavior
unchanged.
- Avoid the rejected alternative of adding a privileged 47-line
apt/workflow boundary to the generic cloud job.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: no CLI, configuration,
model-ID, policy, or user-visible behavior changes
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — fresh exact-head review pending
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
none currently accepted

## Verification
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — after merging current `main`, focused
DCode integration suites pass 210/210 and focused E2E-support suites
pass 37/37. Bash syntax, ShellCheck, source-shape, test-size, build,
TypeScript typecheck, commit, and pre-push hooks pass
- [ ] Applicable broad gate passed — fresh exact-head CI, advisors,
focused DCode E2E, cloud-onboard, and credential-sanitization pending
- [ ] Quality Gates section completed with required justifications or
waivers — exact-head sensitive-path review pending
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

## Advisor dispositions
- Source-of-truth review: the invalid state is a managed DCode exec
losing host-selected observability because raw exec/login processes do
not inherit entrypoint environment and policy mutation reloads do not
re-run the entrypoint. Current `main` now owns `start.sh`
materialization plus `dcode-launcher.sh` recovery under
`/sandbox/.deepagents`; create/rebuild/clone carry the registry's
explicit `1`/`0` intent, and snapshot restore honors that authoritative
registry state so an earlier enabled snapshot does not override a later
opt-out. This PR adds fail-closed registry/marker agreement checks and
the no-policy export proof. Remove the bridge only when OpenShell both
propagates the bit to exec/login processes and preserves it through
policy reloads or re-runs the entrypoint.
- Sequential Tavily isolation is exercised by the required typed target
on one sandbox: check 09 proves denial immediately after cleanup; checks
10 and 11 do not mutate Tavily; check 12 rebuilds and then invokes check
06, which proves both managed and project-venv Python remain denied from
`api.tavily.com`. This gives immediate and post-rebuild denial evidence
without redundant probes after non-mutating checks.
- The marker integration test runs the transformed `start.sh` with
`NEMOCLAW_OBSERVABILITY=1`, verifies the
`/sandbox/.deepagents/.nemoclaw-observability-enabled` value and mode,
removes volatile runtime state, proves absent-env restart and launcher
recovery, and proves explicit `0` removal. Live check 11 verifies the
literal sandbox path.
- The `/sandbox/.deepagents` marker is intentionally untrusted
convenience state, not authorization. A sandbox user can request local
instrumentation by recreating the exact marker, but cannot grant OTLP
egress; the fixed exporter route remains subject to host-managed
`observability-otlp-local` policy. Live check 11 removes that host
policy, recreates the marker as the sandbox user, proves the exact raw
route is denied without capture, and runs real marker-triggered
deterministic instrumentation with zero captures before restoring the
policy. It also retains alternate host/path/method/port and
unmanaged-binary denials with policy active.
- Check 09 no longer repairs observability by re-running the stateful
entrypoint. The host registry is authoritative, the persistent marker is
derived convenience state, and disagreement fails closed both before and
after policy mutation. Merged #6506 prevents status/connect route probes
from clearing that marker by using the side-effect-free managed-exec
boundary.
- Startup rejects directory, symlink, and other non-regular marker
targets before either the enabled or disabled branch. Enabled
replacement treats the destination as a file, closing the
directory-descent edge case raised by the exact-head advisor.
- Current `main` includes #6506's side-effect-free managed-exec boundary
and #6494's durable observability lifecycle. Both are retained together
with this PR's host-registry agreement checks and no-policy proof; no
contributor work was replaced.
- The proxy env file remains intentionally volatile in `/tmp`: each
stateful entrypoint reconstructs it from root-owned image proxy inputs,
while the observability marker under `/sandbox/.deepagents` must survive
policy-only reloads that do not carry the sandbox-create environment.
Their different locations reflect different lifetime contracts, not
inconsistent authority.

## Exact-head validation plan
- Re-run automatic CI, CodeRabbit, GPT, and Nemotron advisors on
`0570b876d6e85bcb110823d5f8b6a5553d266f34`.
- Re-run `ubuntu-repo-cloud-langchain-deepagents-code` to prove Tavily
is denied after check 09 cleanup and remains denied across the check 12
rebuild.
- Re-run `cloud-onboard` to prove the non-DCode sandbox skips before
requiring `expect`.
- Re-run `credential-sanitization` for the named-limit parity change.
- Hold merge until tomorrow even if all gates pass earlier.

## Failure evidence addressed
- Focused DCode run
[28978277604](https://github.com/NVIDIA/NemoClaw/actions/runs/28978277604):
check 09 left durable Tavily state that check 12 correctly replayed
before the strict egress boundary.
- Cloud/credential run
[28978281322](https://github.com/NVIDIA/NemoClaw/actions/runs/28978281322):
generic OpenClaw cloud-onboard required `expect` before reaching its
DCode-only skip; credential-sanitization passed.
- Superseded follow-up run
[28980032707](https://github.com/NVIDIA/NemoClaw/actions/runs/28980032707):
Tavily cleanup and strict egress passed, then check 11 proved that
policy reload had cleared the runtime-functional `/tmp` observability
marker. The current head retains current `main`'s durable marker under
`/sandbox/.deepagents`.
- Superseded follow-up run
[28980032711](https://github.com/NVIDIA/NemoClaw/actions/runs/28980032711):
cloud-onboard and credential-sanitization both passed.
- Superseded follow-up run
[28981014655](https://github.com/NVIDIA/NemoClaw/actions/runs/28981014655):
after the marker moved to `/sandbox`, check 11 proved that Tavily policy
cleanup could still reconstruct runtime convenience state and remove it.
The current head uses current `main`'s absent-env-preserves lifecycle
and fails if the authoritative registry and derived marker disagree;
check 09 does not repair drift.
- Superseded follow-up run
[28981369440](https://github.com/NVIDIA/NemoClaw/actions/runs/28981369440):
cloud-onboard and credential-sanitization both passed on the
collaborator head; exact-head rerun pending.
- Superseded follow-up run
[28981690759](https://github.com/NVIDIA/NemoClaw/actions/runs/28981690759):
check 09 restored Tavily but sampled observability after policy-add had
already reloaded the sandbox, so check 11 found the marker absent. This
showed that sandbox marker sampling cannot be the authoritative source
of host intent.
- Superseded follow-up run
[28982642290](https://github.com/NVIDIA/NemoClaw/actions/runs/28982642290):
check 09 restored Tavily and check 10 passed, but check 11 found the
marker absent because a pre-#6506 status/connect route probe had already
invoked the stateful entrypoint without observability. Current `main`
supplies #6506's side-effect-free managed-exec probe and #6494's
explicit observability lifecycle, while this head compares the host
registry and persistent marker before and after policy mutation instead
of masking drift with a repair.
- Superseded exact-head focused run
[28983221993](https://github.com/NVIDIA/NemoClaw/actions/runs/28983221993)
passed every typed DCode check 03–12: check 09 reported 9/9, check 11
reported 11/11, and check 12 proved post-rebuild denial. Combined run
[28983222022](https://github.com/NVIDIA/NemoClaw/actions/runs/28983222022)
passed cloud-onboard and credential sanitization. These runs were
invalidated only to address the advisor's unsafe-target edge case and
add the stricter no-host-policy leak proof now running on the current
head.
- Superseded advisor run
[28983777162](https://github.com/NVIDIA/NemoClaw/actions/runs/28983777162)
rated the safety changes `merge_as_is` from GPT with no findings;
Nemotron agreed both substantive findings were resolved and requested
only the `/tmp` versus `/sandbox` lifetime cross-reference now present
on the current head.

---
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved validation for credential names and context placeholders,
including oversized names.
* Strengthened proxy environment file handling to prevent
destination-path issues.
  * Clarified and stabilized observability policy and marker handling.

* **Tests**
* Expanded end-to-end coverage for observability, Tavily access,
tracing, and auto-approval behavior.
* Standardized test environment paths and added boundary-case validation
for credential patterns.
  * Improved platform-specific launcher test reliability.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: cjagwani <cjagwani@nvidia.com>
Co-authored-by: cjagwani <cjagwani@nvidia.com>
Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
## Summary

Harden the managed Nemotron 3 Ultra DCode aliases at two separate
compatibility
boundaries: emit the model-specific nonempty-content template argument
required
for reasoning-plus-tool-call requests, and reject the observed literal
`[content]` execute placeholder before it reaches shell dispatch.

NVIDIA#6431 is merged; this PR is based directly on current `main`. It leaves
the
canonical NVIDIA profile and all unrelated models unchanged.

## Source boundary and lifecycle

- `force_nonempty_content`: the invalid empty-assistant-content state
originates
in the Ultra serving template. NemoClaw owns only the generated
per-model
request configuration. Remove the override after a reviewed
serving/client
update works for both managed Ultra IDs without it and the live DCode
E2E passes.
- Execute guard: the invalid state is a model-produced tool call whose
complete
`execute.command` is the literal `[content]` (ignoring ASCII whitespace
around the token and
brackets, and case). NemoClaw owns the managed alias middleware
immediately before dispatch,
not the model, serving template, or hash-locked canonical Deep Agents
profile.
Remove the guard only after those upstream paths no longer produce or
convert
  the placeholder and the focused plus live tests pass with it deleted.

## Changes

- Generate `force_nonempty_content = true` only for the two managed
Ultra model IDs.
- Add a managed-alias-only sync/async middleware guard for the exact
placeholder;
  concrete execute commands and other tools pass through unchanged.
- Validate rejection through the pinned LangChain `ToolMessage.content`
API in
  the image validator, fixture, and hosted E2E probe.
- Preserve atomic/idempotent profile registration with the process-local
  registry as source of truth and a lock around multi-key registration.
- Persist the credential-free observability enable bit across
environment-less
policy restarts; create/rebuild/clone paths pass an explicit `1` or `0`.
- Record invalid-state sources, regression evidence, and removal
conditions in
  the dependency review.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [x] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: no CLI, setting,
supported-model choice, or operator workflow changes; the internal
dependency review is updated
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification: pending maintainer
review
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — combined Ultra, lifecycle,
current-main, and acceptance suites passed 159/159; the post-hook
launcher/acceptance subset passed 33/33
- [ ] Applicable broad gate passed — final exact-head regular CI and
hosted Deep Agents Code E2E are running
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added model-specific “Ultra” configuration for selected
OpenAI-compatible model IDs to enforce non-empty tool-call/reasoning
content.
* Added managed execute placeholder guard behavior and deeper end-to-end
validation for Tavily policy remove/restore ordering and observability
recovery.
* **Bug Fixes**
* Hardened managed execute handling to reject placeholder `execute`
calls (including async/sync parity) while allowing concrete commands.
* Improved managed alias registration for safer atomic idempotence and
rollback.
* **Documentation**
* Updated compatibility/validation guidance and the dependency advisory
review hash.
* **Tests**
* Expanded E2E/profile/config contract tests, including workflow
host-dependency step ordering and guard dispatch assertions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: cjagwani <cjagwani@nvidia.com>
Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Co-authored-by: cjagwani <cjagwani@nvidia.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Prepares the user documentation for NemoClaw v0.0.78 by replacing the
unreleased section with release highlights and synchronizing the
affected inference, lifecycle, messaging, and CLI reference pages with
merged behavior.

## Changes

- Publish the v0.0.78 release-notes section with links to the most
specific user guides for each shipped behavior.
- Document authoritative Deep Agents route health, Nemotron Ultra
profile behavior, and Hermes compatible-endpoint context metadata.
- Document forced rebuild recovery after total backup failure and the
ownership-safe tunnel/full-stop behavior.
- Keep command examples and shared agent variants aligned with the
current OpenClaw, Hermes, and Deep Agents interfaces.

Source mapping:

- [NVIDIA#3787](NVIDIA#3787) ->
`docs/about/release-notes.mdx`: Record reliable workspace template
seeding during sandbox startup.
- [NVIDIA#4960](NVIDIA#4960) ->
`docs/about/release-notes.mdx`: Record safer detection of rewritten
OpenClaw gateway processes.
- [NVIDIA#5676](NVIDIA#5676) ->
`docs/about/release-notes.mdx`: Record warning-tolerant agent-list JSON
handling.
- [NVIDIA#5857](NVIDIA#5857) ->
`docs/about/release-notes.mdx`: Record synchronization of explicit
OpenClaw main-agent model state.
- [NVIDIA#5929](NVIDIA#5929) ->
`docs/about/release-notes.mdx`: Record copyable SSH port-forward
guidance for remote dashboards.
- [NVIDIA#6068](NVIDIA#6068) ->
`docs/about/release-notes.mdx`: Record custom-image plugin provenance
reconciliation.
- [NVIDIA#6116](NVIDIA#6116) ->
`docs/about/release-notes.mdx`: Record live-loopback dashboard-forward
recovery.
- [NVIDIA#6122](NVIDIA#6122) ->
`docs/about/release-notes.mdx`: Announce validated, round-trippable
policy YAML output.
- [NVIDIA#6211](NVIDIA#6211) ->
`docs/manage-sandboxes/lifecycle.mdx`, `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Explain the explicit no-backup `rebuild
--force` recovery boundary.
- [NVIDIA#6283](NVIDIA#6283) ->
`docs/about/release-notes.mdx`: Record Hermes WebUI port alignment.
- [NVIDIA#6293](NVIDIA#6293) ->
`docs/inference/switch-inference-providers.mdx`,
`docs/about/release-notes.mdx`: Document compatible-endpoint
context-length probing for Hermes.
- [NVIDIA#6320](NVIDIA#6320) ->
`docs/about/release-notes.mdx`: Record bounded gateway-recovery waits.
- [NVIDIA#6377](NVIDIA#6377) ->
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Explain
rebuild diagnostics and prepared MCP-destroy recovery.
- [NVIDIA#6412](NVIDIA#6412) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document authoritative agent-visible
inference route health.
- [NVIDIA#6421](NVIDIA#6421) ->
`docs/about/release-notes.mdx`: Record the longer quiet-pull window for
managed vLLM images.
- [NVIDIA#6431](NVIDIA#6431) ->
`docs/inference/model-capability-audit.mdx`,
`docs/about/release-notes.mdx`: Document the version-pinned Nemotron
Ultra profile plugin.
- [NVIDIA#6439](NVIDIA#6439) ->
`docs/about/release-notes.mdx`: Summarize the authenticated, pinned
credential-capture helper boundary.
- [NVIDIA#6450](NVIDIA#6450) ->
`docs/manage-sandboxes/messaging-channels.mdx`,
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Document
host-forward cleanup and ownership-safe gateway-port release.
- [NVIDIA#6474](NVIDIA#6474) ->
`docs/manage-sandboxes/messaging-channels.mdx`,
`docs/about/release-notes.mdx`: Record composable OpenClaw messaging
runtime loaders.
- [NVIDIA#6475](NVIDIA#6475) ->
`docs/about/release-notes.mdx`: Record removal of the unavailable Kimi
K2.6 production endpoint option.
- [NVIDIA#6480](NVIDIA#6480) ->
`docs/about/release-notes.mdx`: Record stderr routing for the plugin
registration banner.
- [NVIDIA#6481](NVIDIA#6481) ->
`docs/about/release-notes.mdx`: Record post-pull Ollama model discovery
checks.
- [NVIDIA#6482](NVIDIA#6482) ->
`docs/about/release-notes.mdx`: Record Ollama model warm-up after daemon
restart.
- [NVIDIA#6486](NVIDIA#6486) ->
`docs/about/release-notes.mdx`: Publish the opt-in, thread-scoped Deep
Agents auto-approval boundary.
- [NVIDIA#6490](NVIDIA#6490) ->
`docs/about/release-notes.mdx`: Record diagnostics for custom images
missing the managed runtime.
- [NVIDIA#6494](NVIDIA#6494) ->
`docs/inference/model-capability-audit.mdx`,
`docs/about/release-notes.mdx`: Document nonempty tool-call content
preservation and placeholder rejection.
- [NVIDIA#6497](NVIDIA#6497) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document isolated Deep Agents
route-probe output.
- [NVIDIA#6506](NVIDIA#6506) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document observability-preserving
managed route probes.
- [NVIDIA#6508](NVIDIA#6508) ->
`docs/about/release-notes.mdx`: Link the new extension taxonomy and
SDK-readiness reference from the release summary.

Release-source verification: GitHub reports all 29 cited source PRs as
merged with base `main`, and every merge commit is an ancestor of
`origin/main` at `17bf9a6a9688b3b1d69cf4b37d3f23110acb055e`. No
source-mapping mismatches were found.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [x] Doc only (includes code sample changes)

## Quality Gates

<!-- Check exactly one tests line and one docs line. Check other lines
when applicable. Add every requested justification or approval
reference. -->
- [ ] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [x] Tests not applicable — justification: Documentation-only
release-prep changes; `npm run docs` validates variants, routes, and
Fern content.
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

<!-- Check each applicable item only when supported by the requested
evidence. Run targeted tests once per relevant change set and rerun
after later edits or hook autofixes that can affect the tested behavior.
Do not rerun hook-covered checks. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: Tests
are not applicable to this documentation-only change set.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result:
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) — exited
0 with zero errors; Fern reported the existing unauthenticated
redirect-check and light-mode contrast warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

---------

Signed-off-by: cjagwani <cjagwani@nvidia.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
<!-- markdownlint-disable MD041 -->
## Summary
Follow-up to NVIDIA#6431 for exact-head validation failures that completed
immediately before that PR merged. This isolates sequential DCode E2E
state, preserves observability across policy-only reloads, lets generic
OpenClaw cloud-onboard runs skip DCode-only TUI prerequisites, and
closes the remaining valid advisor requests without changing user-facing
behavior.

## Related Issue
Follow-up to NVIDIA#6431, which closed NVIDIA#6424. No new issue.

## Changes
- Remove the durable Tavily preset after check 09, arm cleanup on
`EXIT`, and fail unless managed DCode Python returns to the default
network denial.
- Move check 12's `expect`, Node, and timeout prerequisites after its
DCode sandbox guard. Generic `cloud-onboard` therefore skips correctly;
typed DCode jobs continue to require and install `expect`.
- Keep the credential-free observability enable marker in managed
`/sandbox/.deepagents` state for exec/login recovery. Current `main` now
owns the durable lifecycle: absent entrypoint env preserves the bit
across policy-only restarts, while create/rebuild/clone pass an explicit
authoritative `1` or `0`. Check 09 requires the host registry's DCode
`observabilityEnabled=true` intent and the exact persistent marker to
agree before and after Tavily policy mutation; it fails rather than
reconstructing intent from sandbox-writable state. Host-managed network
policy remains the egress boundary.
- Reject a pre-existing non-regular or symlink marker target with a
clear startup error, and use no-target-directory replacement so a raced
directory cannot absorb the enabled marker. Fixtures cover both enabled
and disabled startup.
- In live check 11, remove only `observability-otlp-local`, recreate the
exact marker as the sandbox user, and prove both the normally allowed
raw route and marker-triggered deterministic instrumentation produce
zero host captures. Restore only from an exactly inactive policy state
before the existing positive trace checks.
- Replace repeated credential-name limit literals with
`CREDENTIAL_NAME_PREFIX_MAX_LENGTH` and prove every Bash quantifier
remains equal to the canonical TypeScript context-pattern limit.
- Cover exact 128/129 context-prefix behavior, oversized OpenShell
placeholders, and Tavily cleanup-command failures through the real
wrapper/self-test boundaries.
- Preserve the merged NVIDIA#6494 Nemotron Ultra request hardening and its
observability create/snapshot lifecycle while resolving the overlapping
follow-up checks against current `main`.
- Keep standalone DCode Tavily persistence and rebuild behavior
unchanged.
- Avoid the rejected alternative of adding a privileged 47-line
apt/workflow boundary to the generic cloud job.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: no CLI, configuration,
model-ID, policy, or user-visible behavior changes
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — fresh exact-head review pending
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
none currently accepted

## Verification
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — after merging current `main`, focused
DCode integration suites pass 210/210 and focused E2E-support suites
pass 37/37. Bash syntax, ShellCheck, source-shape, test-size, build,
TypeScript typecheck, commit, and pre-push hooks pass
- [ ] Applicable broad gate passed — fresh exact-head CI, advisors,
focused DCode E2E, cloud-onboard, and credential-sanitization pending
- [ ] Quality Gates section completed with required justifications or
waivers — exact-head sensitive-path review pending
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

## Advisor dispositions
- Source-of-truth review: the invalid state is a managed DCode exec
losing host-selected observability because raw exec/login processes do
not inherit entrypoint environment and policy mutation reloads do not
re-run the entrypoint. Current `main` now owns `start.sh`
materialization plus `dcode-launcher.sh` recovery under
`/sandbox/.deepagents`; create/rebuild/clone carry the registry's
explicit `1`/`0` intent, and snapshot restore honors that authoritative
registry state so an earlier enabled snapshot does not override a later
opt-out. This PR adds fail-closed registry/marker agreement checks and
the no-policy export proof. Remove the bridge only when OpenShell both
propagates the bit to exec/login processes and preserves it through
policy reloads or re-runs the entrypoint.
- Sequential Tavily isolation is exercised by the required typed target
on one sandbox: check 09 proves denial immediately after cleanup; checks
10 and 11 do not mutate Tavily; check 12 rebuilds and then invokes check
06, which proves both managed and project-venv Python remain denied from
`api.tavily.com`. This gives immediate and post-rebuild denial evidence
without redundant probes after non-mutating checks.
- The marker integration test runs the transformed `start.sh` with
`NEMOCLAW_OBSERVABILITY=1`, verifies the
`/sandbox/.deepagents/.nemoclaw-observability-enabled` value and mode,
removes volatile runtime state, proves absent-env restart and launcher
recovery, and proves explicit `0` removal. Live check 11 verifies the
literal sandbox path.
- The `/sandbox/.deepagents` marker is intentionally untrusted
convenience state, not authorization. A sandbox user can request local
instrumentation by recreating the exact marker, but cannot grant OTLP
egress; the fixed exporter route remains subject to host-managed
`observability-otlp-local` policy. Live check 11 removes that host
policy, recreates the marker as the sandbox user, proves the exact raw
route is denied without capture, and runs real marker-triggered
deterministic instrumentation with zero captures before restoring the
policy. It also retains alternate host/path/method/port and
unmanaged-binary denials with policy active.
- Check 09 no longer repairs observability by re-running the stateful
entrypoint. The host registry is authoritative, the persistent marker is
derived convenience state, and disagreement fails closed both before and
after policy mutation. Merged NVIDIA#6506 prevents status/connect route probes
from clearing that marker by using the side-effect-free managed-exec
boundary.
- Startup rejects directory, symlink, and other non-regular marker
targets before either the enabled or disabled branch. Enabled
replacement treats the destination as a file, closing the
directory-descent edge case raised by the exact-head advisor.
- Current `main` includes NVIDIA#6506's side-effect-free managed-exec boundary
and NVIDIA#6494's durable observability lifecycle. Both are retained together
with this PR's host-registry agreement checks and no-policy proof; no
contributor work was replaced.
- The proxy env file remains intentionally volatile in `/tmp`: each
stateful entrypoint reconstructs it from root-owned image proxy inputs,
while the observability marker under `/sandbox/.deepagents` must survive
policy-only reloads that do not carry the sandbox-create environment.
Their different locations reflect different lifetime contracts, not
inconsistent authority.

## Exact-head validation plan
- Re-run automatic CI, CodeRabbit, GPT, and Nemotron advisors on
`0570b876d6e85bcb110823d5f8b6a5553d266f34`.
- Re-run `ubuntu-repo-cloud-langchain-deepagents-code` to prove Tavily
is denied after check 09 cleanup and remains denied across the check 12
rebuild.
- Re-run `cloud-onboard` to prove the non-DCode sandbox skips before
requiring `expect`.
- Re-run `credential-sanitization` for the named-limit parity change.
- Hold merge until tomorrow even if all gates pass earlier.

## Failure evidence addressed
- Focused DCode run
[28978277604](https://github.com/NVIDIA/NemoClaw/actions/runs/28978277604):
check 09 left durable Tavily state that check 12 correctly replayed
before the strict egress boundary.
- Cloud/credential run
[28978281322](https://github.com/NVIDIA/NemoClaw/actions/runs/28978281322):
generic OpenClaw cloud-onboard required `expect` before reaching its
DCode-only skip; credential-sanitization passed.
- Superseded follow-up run
[28980032707](https://github.com/NVIDIA/NemoClaw/actions/runs/28980032707):
Tavily cleanup and strict egress passed, then check 11 proved that
policy reload had cleared the runtime-functional `/tmp` observability
marker. The current head retains current `main`'s durable marker under
`/sandbox/.deepagents`.
- Superseded follow-up run
[28980032711](https://github.com/NVIDIA/NemoClaw/actions/runs/28980032711):
cloud-onboard and credential-sanitization both passed.
- Superseded follow-up run
[28981014655](https://github.com/NVIDIA/NemoClaw/actions/runs/28981014655):
after the marker moved to `/sandbox`, check 11 proved that Tavily policy
cleanup could still reconstruct runtime convenience state and remove it.
The current head uses current `main`'s absent-env-preserves lifecycle
and fails if the authoritative registry and derived marker disagree;
check 09 does not repair drift.
- Superseded follow-up run
[28981369440](https://github.com/NVIDIA/NemoClaw/actions/runs/28981369440):
cloud-onboard and credential-sanitization both passed on the
collaborator head; exact-head rerun pending.
- Superseded follow-up run
[28981690759](https://github.com/NVIDIA/NemoClaw/actions/runs/28981690759):
check 09 restored Tavily but sampled observability after policy-add had
already reloaded the sandbox, so check 11 found the marker absent. This
showed that sandbox marker sampling cannot be the authoritative source
of host intent.
- Superseded follow-up run
[28982642290](https://github.com/NVIDIA/NemoClaw/actions/runs/28982642290):
check 09 restored Tavily and check 10 passed, but check 11 found the
marker absent because a pre-NVIDIA#6506 status/connect route probe had already
invoked the stateful entrypoint without observability. Current `main`
supplies NVIDIA#6506's side-effect-free managed-exec probe and NVIDIA#6494's
explicit observability lifecycle, while this head compares the host
registry and persistent marker before and after policy mutation instead
of masking drift with a repair.
- Superseded exact-head focused run
[28983221993](https://github.com/NVIDIA/NemoClaw/actions/runs/28983221993)
passed every typed DCode check 03–12: check 09 reported 9/9, check 11
reported 11/11, and check 12 proved post-rebuild denial. Combined run
[28983222022](https://github.com/NVIDIA/NemoClaw/actions/runs/28983222022)
passed cloud-onboard and credential sanitization. These runs were
invalidated only to address the advisor's unsafe-target edge case and
add the stricter no-host-policy leak proof now running on the current
head.
- Superseded advisor run
[28983777162](https://github.com/NVIDIA/NemoClaw/actions/runs/28983777162)
rated the safety changes `merge_as_is` from GPT with no findings;
Nemotron agreed both substantive findings were resolved and requested
only the `/tmp` versus `/sandbox` lifetime cross-reference now present
on the current head.

---
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved validation for credential names and context placeholders,
including oversized names.
* Strengthened proxy environment file handling to prevent
destination-path issues.
  * Clarified and stabilized observability policy and marker handling.

* **Tests**
* Expanded end-to-end coverage for observability, Tavily access,
tracing, and auto-approval behavior.
* Standardized test environment paths and added boundary-case validation
for credential patterns.
  * Improved platform-specific launcher test reliability.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: cjagwani <cjagwani@nvidia.com>
Co-authored-by: cjagwani <cjagwani@nvidia.com>
Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: inference Inference routing, serving, model selection, or outputs area: security Security controls, permissions, secrets, or hardening bug-fix PR fixes a bug or regression integration: dcode LangChain Deep Code integration behavior needs: review PR is conflict-free and awaiting maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants