Skip to content

fix(blueprint): allowlist safe run plan output - #3674

Merged
ericksoa merged 6 commits into
mainfrom
fix/3648-safe-plan-shapes
May 19, 2026
Merged

fix(blueprint): allowlist safe run plan output#3674
ericksoa merged 6 commits into
mainfrom
fix/3648-safe-plan-shapes

Conversation

@ericksoa

@ericksoa ericksoa commented May 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • remove credential_env from public blueprint plan output
  • persist plan.json through an explicit allowlist builder instead of ad-hoc credential exclusions
  • make status parse plan.json and re-render only safe allowlisted fields
  • add regression coverage for credential env names, defaults, secret values, token/auth fields, and future unallowlisted fields

Fixes #3648

Related contributor work

Validation

  • cd nemoclaw && npm test -- src/blueprint/runner.test.ts (91 passed)
  • cd nemoclaw && npm run build
  • cd nemoclaw && npm run check
  • git diff --check

Summary by CodeRabbit

  • Bug Fixes

    • Plan outputs and stored run plans no longer expose credential-related fields or secret values
  • Tests

    • Added comprehensive tests validating that sensitive credential information and secret values are not exposed in plan output, persisted plans, or status reports

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The head commit changed during the review from a5ec087 to 07c6939.

📝 Walkthrough

Walkthrough

RunPlan's public interface is tightened to exclude credential-bearing inference fields. New safe plan builders explicitly allowlist non-secret fields for public output, persistence, and status rendering. actionPlan, actionApply, and actionStatus are updated to use these builders instead of inline or ad-hoc credential exclusions. Comprehensive tests validate that no credential field names or secret values leak to stdout or persisted files.

Changes

Secret field sanitization in run plans

Layer / File(s) Summary
Type contract and safe plan builders
nemoclaw/src/blueprint/runner.ts
RunPlan.inference type removes credential_env and credential_default. New SafeInferencePlan, PersistedRunPlan, and StatusRunPlan types and builder functions (buildSafePublicRunPlan, buildPersistedRunPlan, buildStatusRunPlan) defensively construct allowlisted safe plans for each output context.
actionPlan safe public output
nemoclaw/src/blueprint/runner.ts, nemoclaw/src/blueprint/runner.test.ts
actionPlan now uses buildSafePublicRunPlan to construct its returned RunPlan, ensuring credential fields do not reach stdout. Test helper capturedJsonOutput filters and parses stdout; new actionPlan test asserts credential field names and secret values are absent from both the returned object and raw stdout when blueprint contains sensitive credentials.
actionApply safe persisted output
nemoclaw/src/blueprint/runner.ts, nemoclaw/src/blueprint/runner.test.ts
actionApply persists plan.json using buildPersistedRunPlan, replacing inline construction to ensure credential-bearing inference fields are omitted and persisted schema is standardized. Updated test validates exact allowlisted top-level keys, restricts inference to non-secret fields, and asserts persisted JSON does not contain credential/secret names or values.
actionStatus safe status output
nemoclaw/src/blueprint/runner.ts, nemoclaw/src/blueprint/runner.test.ts
actionStatus parses plan.json as unknown and sanitizes via buildStatusRunPlan to defensively re-render only allowlisted safe fields; failures fall back to "unknown" status. Test re-renders safe fields from sensitive plan.json and asserts no leaked tokens; corrupt plan.json case now asserts via parsed JSON status output.

Sequence Diagram

sequenceDiagram
  participant User
  participant actionPlan
  participant buildSafePublicRunPlan
  participant stdout
  participant actionApply
  participant buildPersistedRunPlan
  participant plan.json
  participant actionStatus
  participant buildStatusRunPlan
  
  User->>actionPlan: nemoclaw plan
  actionPlan->>buildSafePublicRunPlan: inferenceCfg, routerCfg, ...
  buildSafePublicRunPlan-->>actionPlan: RunPlan (no credentials)
  actionPlan->>stdout: JSON.stringify(safe plan)
  
  User->>actionApply: nemoclaw apply
  actionApply->>buildPersistedRunPlan: plan, inferenceCfg, ...
  buildPersistedRunPlan-->>actionApply: PersistedRunPlan (allowlisted fields)
  actionApply->>plan.json: writeFileSync(safe persisted plan)
  
  User->>actionStatus: nemoclaw status
  actionStatus->>plan.json: readFileSync()
  plan.json-->>actionStatus: raw JSON (unknown)
  actionStatus->>buildStatusRunPlan: unknown object
  buildStatusRunPlan-->>actionStatus: StatusRunPlan (safe fields only)
  actionStatus->>stdout: JSON.stringify(safe status)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested labels

fix

Poem

🐰 Secrets tucked away, no more exposed,
Allowlists guard what once lay close,
Three safe builders tame the flow,
Public, persisted, status—all aglow!
Credentials stay hidden, as they should be,
A rabbit's fix for security! 🔐

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main objective: implementing an allowlist for safe run plan output to prevent credential leakage.
Linked Issues check ✅ Passed The PR fully addresses issue #3648: removes credential_env from actionPlan output, implements explicit allowlist builders for persisted/status plan rendering, and adds comprehensive regression tests for credential/token/secret leakage.
Out of Scope Changes check ✅ Passed All changes are scoped to addressing issue #3648: sanitizing run plan output and adding related tests; no unrelated modifications detected.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/3648-safe-plan-shapes
⚔️ Resolve merge conflicts
  • Resolve merge conflict in branch fix/3648-safe-plan-shapes

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: test-e2e-sandbox, credential-sanitization-e2e, inference-routing-e2e
Optional E2E: network-policy-e2e, ubuntu-repo-cloud-openclaw

Dispatch hint: credential-sanitization-e2e,inference-routing-e2e

Auto-dispatched E2E: credential-sanitization-e2e, inference-routing-e2e via nightly-e2e.yaml at 07c6939df0d67529a732e03ebf570dd661bd5922nightly run

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • test-e2e-sandbox (medium): Most direct existing E2E coverage for this file: test/e2e-test.sh validates the compiled blueprint runner inside the sandbox image, including plan RUN_ID/progress, apply progress, and persisted run state. This should block because runner.ts changed runtime plan/apply/status behavior.
  • credential-sanitization-e2e (high): The PR changes secret redaction boundaries for blueprint runner stdout and persisted state. Existing credential sanitization E2E is the closest live-stack guard for preventing credential material from leaking into migration/runtime artifacts.
  • inference-routing-e2e (medium): Blueprint apply configures OpenShell providers, credentials, endpoints, and inference routes. Run this to confirm credential isolation and provider/routing behavior still works after the safe inference plan refactor.

Optional E2E

  • network-policy-e2e (high): Optional confidence for the policy_additions part of run-state/status allowlisting. The policy merge/apply path was not the primary change, but actionApply still persists policy additions and actionStatus now validates/re-renders them.
  • ubuntu-repo-cloud-openclaw (high): Optional scenario-level smoke/inference/credentials coverage for the default OpenClaw user flow after changes in blueprint-related runtime code.

New E2E recommendations

  • blueprint-runner-secret-redaction (high): Existing unit tests now cover plan/apply/status redaction, but no dedicated E2E exercises the compiled blueprint runner with a fake or live OpenShell and asserts stdout plus ~/.nemoclaw/state/runs//plan.json never contain credential_env, credential_default, token, authorization, or secret values.
    • Suggested test: Add a blueprint-runner-redaction-e2e job/script that invokes compiled dist/blueprint/runner.js plan/apply/status against a controlled blueprint and verifies public output and persisted state use only the safe allowlisted schema.
  • blueprint-runner-status-corrupt-state (medium): The PR changes actionStatus to return unknown status for corrupt/non-object plan.json, but existing E2E coverage only smoke-tests apply persistence. A live-stack regression could leave status output unusable or leak old fields from legacy plan files.
    • Suggested test: Add an E2E step to create synthetic run-state directories with corrupt and legacy-sensitive plan.json files, run blueprint status --run-id, and assert safe JSON output without secret fields.

Dispatch hint

  • Workflow: nightly-e2e.yaml
  • jobs input: credential-sanitization-e2e,inference-routing-e2e

@ericksoa ericksoa added the fix label May 18, 2026
@ericksoa ericksoa self-assigned this May 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26009769049
Target ref: 2ab1d7119f81bbd2f76344c1554ef7940d47b0af
Workflow ref: main
Requested jobs: credential-sanitization-e2e,inference-routing-e2e,sandbox-operations-e2e
Summary: 2 passed, 0 failed, 0 skipped

Job Result
credential-sanitization-e2e ✅ success
inference-routing-e2e ✅ success
sandbox-operations-e2e ⚠️ cancelled

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26010189361
Target ref: 67a6a807e1b5ca9fcf184760108a197463bb40db
Workflow ref: main
Requested jobs: credential-sanitization-e2e,inference-routing-e2e
Summary: 2 passed, 0 failed, 0 skipped

Job Result
credential-sanitization-e2e ✅ success
inference-routing-e2e ✅ success

@ericksoa ericksoa added v0.0.45 and removed v0.0.45 labels May 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26072002555
Target ref: 64bc2a72f2d058cf523b74d5b419ea2a94974e32
Workflow ref: main
Requested jobs: credential-sanitization-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job Result
credential-sanitization-e2e ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26103843685
Target ref: 96079f7ca1c662106ea56033cd4adff86125be4f
Workflow ref: main
Requested jobs: credential-sanitization-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job Result
credential-sanitization-e2e ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26116463793
Target ref: 07c6939df0d67529a732e03ebf570dd661bd5922
Workflow ref: main
Requested jobs: credential-sanitization-e2e,inference-routing-e2e
Summary: 2 passed, 0 failed, 0 skipped

Job Result
credential-sanitization-e2e ✅ success
inference-routing-e2e ✅ success

@cjagwani cjagwani left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE.

Same direction as #3649 (explicit allowlist) but executed more thoroughly:

  • Tightens RunPlan.inference type itself (nemoclaw/src/blueprint/runner.ts:461-466) — credential_env is removed from the public interface, so it can't reappear by accident.
  • Single source of truth via four builders (buildSafeInferencePlan, buildSafePublicRunPlan, buildPersistedRunPlan, buildStatusRunPlan) — no duplicate allowlists.
  • actionStatus recursively allowlists nested sandbox.* and router.* fields and falls back to {status: "unknown"} on malformed plan.json. Catches credentials in plans written by older versions.
  • +179 lines of regression tests covering credential names, env-var values, future-secret fields (token, authorization), persisted schema shape, and the corrupt-JSON path.
  • credential-sanitization-e2e PASSED in nightly (run 26103843685). Self-hosted sandbox/gateway/non-root smoke all green.

@ericksoa
ericksoa merged commit e51c63c into main May 19, 2026
24 checks passed
@ericksoa
ericksoa deleted the fix/3648-safe-plan-shapes branch May 19, 2026 18:28
@wscurran wscurran added area: cli Command line interface, flags, terminal UX, or output bug-fix PR fixes a bug or regression and removed priority: high labels Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cli Command line interface, flags, terminal UX, or output bug-fix PR fixes a bug or regression VDR Linked to VDR finding

Projects

None yet

Development

Successfully merging this pull request may close these issues.

actionPlan logs credential_env to stdout; actionStatus logs raw plan.json without field allowlist

3 participants