Skip to content

test: add e2e inference smoke test - #125

Closed
ericksoa wants to merge 2 commits into
mainfrom
test/e2e-hello-jensen
Closed

test: add e2e inference smoke test#125
ericksoa wants to merge 2 commits into
mainfrom
test/e2e-hello-jensen

Conversation

@ericksoa

@ericksoa ericksoa commented Mar 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Adds test/e2e-inference.sh — a full end-to-end smoke test that goes from zero to LLM response through the entire NemoClaw stack
  • Exercises: gateway start → provider create → inference route → sandbox build → sandbox create → SSH into sandbox → curl inference.local → validate LLM response
  • Uses Ollama as the inference provider (configurable via OLLAMA_MODEL env var)
  • Includes automatic cleanup via EXIT trap and a workaround for the .dockerignore issue (bug: .dockerignore /dist excludes nemoclaw/dist/ in openshell builds #126)

Prerequisites

  • Docker running
  • openshell CLI installed
  • Ollama running on localhost:11434 with a model pulled (default: nemotron-mini)

Test plan

  • Verified locally — 11/11 assertions pass
  • Ran twice consecutively to confirm cleanup + idempotency
  • No secrets or hardcoded paths in the script
  • Integrate into CI pipeline (requires Ollama + openshell in CI environment)

🤖 Generated with Claude Code

@ericksoa @claude
test: add e2e inference smoke test (hello Jensen)
aa5a815 
End-to-end test that exercises the full NemoClaw stack:
gateway → sandbox → provider → inference. Sends a prompt
through inference.local and validates the LLM responds.

Requires Ollama running locally with a model pulled.
Includes a workaround for the .dockerignore /dist issue
(see issue to follow).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ericksoa ericksoa mentioned this pull request Mar 17, 2026
Closed
@ericksoa @claude
test: reference issue #126 for .dockerignore workaround
b710376 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ericksoa

ericksoa commented Mar 17, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #226, which includes a more complete E2E test covering install → onboard → inference with both live and mock modes.

@ericksoa ericksoa closed this Mar 17, 2026
@cv cv mentioned this pull request May 17, 2026
Closed
6 tasks
@wscurran wscurran added the chore Build, CI, dependency, or tooling maintenance label Jun 8, 2026
cjagwani added a commit that referenced this pull request Jun 23, 2026
@cjagwani
fix(platform): clear advisor PRA-3, PRA-4, PRA-5, PRA-6 on #5345
0acacdd 
Advisor rerun on 9b271be turned up three required + one warning
that the DAC addition surfaced or that the matrix had not yet hardened.

- PRA-3 (REQUIRED). The Deep Agents Code matrix row documented the
  onboarding command as `--agent deepagents-code`, but `resolveAgentName`
  in `src/lib/agent/defs.ts:586-590` rejects flags not returned by
  `listAgents()`, which scans `agents/*` directory names. The real
  runtime ID is `langchain-deepagents-code` (see
  `agents/langchain-deepagents-code/manifest.yaml:9` and the existing
  quickstart at `docs/get-started/quickstart-langchain-deepagents-code.mdx`).
  Update the matrix note to use the canonical ID so the page no longer
  ships a non-working onboarding command.
- PRA-4 (REQUIRED). The Out-of-scope row said "LangChain, AutoGen,
  CrewAI, or other agent harnesses ... Only OpenClaw and Hermes are
  integrated", but the Agents table now lists LangChain Deep Agents
  Code as Experimental. Scope the row to "Other LangChain, AutoGen,
  CrewAI, or non-listed agent harnesses" and call out Deep Agents
  Code as the integrated exception so the matrix gives exactly one
  status per agent.
- PRA-5 (REQUIRED). docs/inference/set-up-sub-agent.mdx passed
  `$NVIDIA_API_KEY` as a positional argv to the demo patch script,
  and instructed users to write the resulting config to
  `/tmp/openclaw.updated.json` and `/tmp/auth-profiles.json` with
  no umask, no mode-0600 protection, and no cleanup. On shared or
  multi-user hosts this leaks the provider key through process
  listings and world-readable temp files. Switch to: `umask 077` +
  `mktemp -d` for the working directory, `trap 'rm -rf' EXIT` for
  cleanup, pass `NVIDIA_API_KEY` through the environment to both
  helpers, and use `envsubst` for the auth-profile template so the
  key never appears in argv.
- PRA-6. `_escape_cell` handled pipes, newlines, and HTML control
  characters but left raw `{` and `}` intact. MDX treats those as
  expression delimiters, so a future matrix note containing a JSON
  snippet or destructuring pattern would render as JSX evaluation.
  Encode `{` and `}` as `&#123;` and `&#125;` so the rendered glyph
  is preserved while the parse hazard is closed. Add a focused test
  for the new behavior.

Regenerated docs/reference/platform-support.mdx and the .agents/
skills/ + skills/ trees.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@cv
cv deleted the test/e2e-hello-jensen branch June 28, 2026 00:24
@jyaunches jyaunches mentioned this pull request Aug 21, 2026
Open
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

chore Build, CI, dependency, or tooling maintenance

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ericksoa @wscurran