Skip to content
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/guides/dev/cli-internals.md
Original file line number Diff line number Diff line change
Expand Up @@ -518,6 +518,7 @@ Vendoring commit messages use title + body (upload and stale delete). `github st
```go
SandboxWorkspace = "/sandbox/workspace"
SandboxClaudeConfig = "/sandbox/claude-config"
SandboxPiConfig = "/sandbox/pi-config"
```

For sandbox workspace layout, agent rule layering, and security scanning
Expand Down
47 changes: 24 additions & 23 deletions docs/runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,25 @@ When adding a runtime, fill in the security matrix below and register it in `run
|---------|---------|-----------|
| `claude` | Production agent runs via Claude Code | Required |
| `opencode` | OpenCode agent runs (stub — not yet functional; resolved by `runtime.Resolve()` but not in `ValidRuntimes()` until implemented) | Required |
| `pi` | pi agent runs (stub — not yet functional; resolved by `runtime.Resolve()` but not in `ValidRuntimes()` until implemented; #6464) | Required |
| `dummy` | Behaviour tests — scripted ops in real sandbox | None |

## Security feature matrix

| Feature | Where it runs | Claude Code | OpenCode (stub) | Notes for future runtimes |
|---------|---------------|-------------|-----------------|---------------------------|
| **Host-side context injection scan** (DeBERTa / LLM Guard, unicode, SSRF patterns on repo context files) | Host + sandbox `scan context` | ✓ | N/A — stub | Requires sandbox image with ML models; harness `security.host_scanners` |
| **Host-side runtime content scan** (agent def, SKILL.md, plugin JSON before upload) | Host (`scanRuntimeContent`) | ✓ | N/A — stub | Uses `security.InputPipeline()`; not part of `Runtime` interface — runner responsibility |
| **Tirith** (Bash command scanning) | Sandbox PreToolUse hook | ✓ (loaded via `--settings`, #6358) | N/A — stub | `tirith_check.py`; harness `security.sandbox_hooks.tirith`; fails open on missing binary/timeout unless `TIRITH_REQUIRED=1` |
| **SSRF pre-tool** | Sandbox PreToolUse hook | ✓ (e2e-guarded by `hooks-loaded.feature`) | N/A — stub | `ssrf_pretool.py`; default on |
| **Canary token detection** | Sandbox Pre/PostToolUse hooks | pre ✓; post-tool field mismatch (#6357) | N/A — stub | `canary_pretool.py` / `canary_posttool.py`; both inert unless `FULLSEND_CANARY_TOKEN` is set |
| **Secret redaction** | Sandbox PostToolUse hook | wired; not effective under Claude Code (#6357) | N/A — stub | `secret_redact_posttool.py` |
| **Unicode normalization** | Sandbox PostToolUse hook | wired; not effective under Claude Code (#6357) | N/A — stub | `unicode_posttool.py` |
| **Context suppression** | Sandbox PostToolUse hook | wired; not effective under Claude Code (#6357) | N/A — stub | `context_suppress_posttool.py` |
| **Tool allowlist** | Sandbox PreToolUse hook | opt-in; ✓ when enabled | N/A — stub | `tool_allowlist_pretool.py`; requires `FULLSEND_TOOL_ALLOWLIST` (fail-closed when unset) |
| **Prompt injection (DeBERTa)** | Host Path A + sandbox Path B | ✓ | N/A — stub | Same scanner stack as context files when enabled in harness |
| **Sandbox tool hooks wiring** | `SandboxHooksBootstrap` type assert in `Bootstrap` | ✓ scripts at `claude-config/hooks/`, wiring at `claude-config/hooks.json` via `--settings` (#6358) | ✗ — `Bootstrap` is a stub; must wire `security.HookPlan` via OpenCode plugin hooks | Hook scripts and wiring plan are runtime-neutral (see [Sandbox hook contract](#sandbox-hook-contract)); a runtime that ignores `SandboxHooksBootstrap` installs **no** sandbox tool hooks — say so explicitly here |
| **Transcript / debug artifacts** | `TranscriptHandler` (+ optional `DebugLogNamer`) | ✓ (stream-json, `claude-debug.log`) | No-op — see #1935 | Format-specific; not shared across runtimes. Debug-log filename defaults to `agent-debug.log` unless the runtime implements `DebugLogNamer` |
| Feature | Where it runs | Claude Code | OpenCode (stub) | pi (stub) | Notes for future runtimes |
|---------|---------------|-------------|-----------------|-----------|---------------------------|
| **Host-side context injection scan** (DeBERTa / LLM Guard, unicode, SSRF patterns on repo context files) | Host + sandbox `scan context` | ✓ | N/A — stub | N/A — stub | Requires sandbox image with ML models; harness `security.host_scanners` |
| **Host-side runtime content scan** (agent def, SKILL.md, plugin JSON before upload) | Host (`scanRuntimeContent`) | ✓ | N/A — stub | N/A — stub | Uses `security.InputPipeline()`; not part of `Runtime` interface — runner responsibility |
| **Tirith** (Bash command scanning) | Sandbox PreToolUse hook | ✓ (loaded via `--settings`, #6358) | N/A — stub | N/A — stub | `tirith_check.py`; harness `security.sandbox_hooks.tirith`; fails open on missing binary/timeout unless `TIRITH_REQUIRED=1` |
| **SSRF pre-tool** | Sandbox PreToolUse hook | ✓ (e2e-guarded by `hooks-loaded.feature`) | N/A — stub | N/A — stub | `ssrf_pretool.py`; default on |
| **Canary token detection** | Sandbox Pre/PostToolUse hooks | pre ✓; post-tool field mismatch (#6357) | N/A — stub | N/A — stub | `canary_pretool.py` / `canary_posttool.py`; both inert unless `FULLSEND_CANARY_TOKEN` is set |
| **Secret redaction** | Sandbox PostToolUse hook | wired; not effective under Claude Code (#6357) | N/A — stub | N/A — stub | `secret_redact_posttool.py` |
| **Unicode normalization** | Sandbox PostToolUse hook | wired; not effective under Claude Code (#6357) | N/A — stub | N/A — stub | `unicode_posttool.py` |
| **Context suppression** | Sandbox PostToolUse hook | wired; not effective under Claude Code (#6357) | N/A — stub | N/A — stub | `context_suppress_posttool.py` |
| **Tool allowlist** | Sandbox PreToolUse hook | opt-in; ✓ when enabled | N/A — stub | N/A — stub | `tool_allowlist_pretool.py`; requires `FULLSEND_TOOL_ALLOWLIST` (fail-closed when unset) |
| **Prompt injection (DeBERTa)** | Host Path A + sandbox Path B | ✓ | N/A — stub | N/A — stub | Same scanner stack as context files when enabled in harness |
| **Sandbox tool hooks wiring** | `SandboxHooksBootstrap` type assert in `Bootstrap` | ✓ scripts at `claude-config/hooks/`, wiring at `claude-config/hooks.json` via `--settings` (#6358) | ✗ — `Bootstrap` is a stub; must wire `security.HookPlan` via OpenCode plugin hooks | ✗ — `Bootstrap` is a stub; must wire `security.HookPlan` via pi's `tool_call`/`tool_result` extension hooks | Hook scripts and wiring plan are runtime-neutral (see [Sandbox hook contract](#sandbox-hook-contract)); a runtime that ignores `SandboxHooksBootstrap` installs **no** sandbox tool hooks — say so explicitly here |
| **Transcript / debug artifacts** | `TranscriptHandler` (+ optional `DebugLogNamer`) | ✓ (stream-json, `claude-debug.log`) | No-op — see #1935 | N/A — stub | Format-specific; not shared across runtimes. Debug-log filename defaults to `agent-debug.log` unless the runtime implements `DebugLogNamer` |

### Fail modes

Expand Down Expand Up @@ -63,15 +64,15 @@ The hook scripts in `internal/security/hooks/*.py` are plain programs with no Cl

Harness keys are runtime-neutral in the YAML but each runtime owns their translation. Claude Code passes them through unchanged; other runtimes must document their mapping here (this is also an acceptance criterion in #6319).

| Harness key | Claude Code | OpenCode (stub) | Dummy | Notes for new runtimes |
|-------------|-------------|-----------------|-------|------------------------|
| `model` | `--model` (identity; aliases like `opus` resolved by the CLI) | — | ignored | `validModelName` is `^[a-zA-Z0-9_.@-]+$` — no `/`. Runtimes with `provider/model` ids need an alias table or a follow-up regex change |
| `effort` | `--effort` (`low\|medium\|high\|xhigh\|max`, #6218) | — | ignored | Map to the runtime's reasoning knob or reject with a clear error |
| `plugins` | Claude plugin marketplace layout (`bootstrapPlugins`) | — | ignored | Claude-specific format; warn and skip if unsupported |
| Agent frontmatter `tools:` (`Bash(gh,jq)` syntax, ADR 0027) | Native Claude permission syntax | — | ignored | Enforce via `--tools`/allowlist plus a hook adapter; Claude tool names differ in case from most runtimes (#608) |
| `skills` | `CLAUDE_CONFIG_DIR/skills/` | — | ignored | Agent Skills spec (`SKILL.md`) is portable; destination is `rt.ConfigDir() + "/skills"` (also used by the runtime fetch service) |
| `security.sandbox_hooks` | `SandboxHooksBootstrap` → hooks.json via `--settings` | ✗ (stub) | ignored | See [Sandbox hook contract](#sandbox-hook-contract) |
| `--debug` (CLI flag) | `--debug-file`, artifact `claude-debug.log` | — | no-op | Implement `DebugLogNamer` to name the artifact |
| Harness key | Claude Code | OpenCode (stub) | pi (stub) | Dummy | Notes for new runtimes |
|-------------|-------------|-----------------|-----------|-------|------------------------|
| `model` | `--model` (identity; aliases like `opus` resolved by the CLI) | — | — | ignored | `validModelName` is `^[a-zA-Z0-9_.@-]+$` — no `/`. Runtimes with `provider/model` ids need an alias table or a follow-up regex change |
| `effort` | `--effort` (`low\|medium\|high\|xhigh\|max`, #6218) | — | — | ignored | Map to the runtime's reasoning knob or reject with a clear error |
| `plugins` | Claude plugin marketplace layout (`bootstrapPlugins`) | — | — | ignored | Claude-specific format; warn and skip if unsupported |
| Agent frontmatter `tools:` (`Bash(gh,jq)` syntax, ADR 0027) | Native Claude permission syntax | — | — | ignored | Enforce via `--tools`/allowlist plus a hook adapter; Claude tool names differ in case from most runtimes (#608) |
| `skills` | `CLAUDE_CONFIG_DIR/skills/` | — | — | ignored | Agent Skills spec (`SKILL.md`) is portable; destination is `rt.ConfigDir() + "/skills"` (also used by the runtime fetch service) |
| `security.sandbox_hooks` | `SandboxHooksBootstrap` → hooks.json via `--settings` | ✗ (stub) | ✗ (stub) | ignored | See [Sandbox hook contract](#sandbox-hook-contract) |
| `--debug` (CLI flag) | `--debug-file`, artifact `claude-debug.log` | — | — | no-op | Implement `DebugLogNamer` to name the artifact |

## Sandbox workspace layout

Expand Down
29 changes: 29 additions & 0 deletions images/sandbox/Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
#
# Pre-installs all runtime dependencies needed by any fullsend agent:
# - Claude Code (npm, version-pinned via CLAUDE_CODE_VERSION ARG)
# - pi (npm, version-pinned via PI_VERSION ARG; stub runtime, not yet
# user-selectable — see #6464)
# - jq (JSON parsing in agent scripts and skills)
# - gitleaks (secret scanner, SHA256-verified)
# - acli (Atlassian CLI for Jira issue queries)
Expand Down Expand Up @@ -52,6 +54,33 @@ RUN apt-get update \
ARG CLAUDE_CODE_VERSION=2.1.234
RUN npm install -g @anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}

# ---------------------------------------------------------------------------
# Install pi (earendil-works/pi) — pinned version, same supply-chain rationale
# as Claude Code above. --ignore-scripts is the upstream-recommended install
# (the package ships an npm-shrinkwrap.json; no postinstall needed). Stub
# runtime for now — see #6464; the runtime is not user-selectable until
# implemented, but the binary ships so fixture recording and the upcoming
# Bootstrap/Run work run against a pinned, reviewed version.
#
# To update: bump PI_VERSION; renovate tracks this ARG via a customManagers
# regex entry in renovate.json. pi releases ~weekly minors and has changed
# its --mode json wire shape within a minor before (0.84.0) — the pi stream
# parser fixtures must be re-verified when this pin moves.
ARG PI_VERSION=0.84.2
RUN node -e "process.exit(process.versions.node.split('.').map(Number).reduce((a,b,i)=>a!==0?a:(b-[22,19,0][i]),0)>=0?0:1)" \
|| (echo "pi ${PI_VERSION} requires Node >=22.19.0, base image has $(node --version)" >&2 && exit 1) \
&& npm install -g --ignore-scripts @earendil-works/pi-coding-agent@${PI_VERSION}

# pi is present in every sandbox regardless of runtime selection (PiRuntime
# is a stub, not yet reachable via config — see internal/runtime/pi.go), so
# an agent invoking the `pi` binary ad-hoc via Bash would otherwise get none
# of PiRuntime.EnvExports()'s safety defaults. Bake them at image level too;
# PI_CODING_AGENT_DIR must match sandbox.SandboxPiConfig (internal/sandbox/sandbox.go).
ENV PI_CODING_AGENT_DIR="/sandbox/pi-config" \
PI_CODING_AGENT_SESSION_DIR="/sandbox/pi-config/sessions" \
PI_OFFLINE="1" \
PI_SKIP_VERSION_CHECK="1"

# ---------------------------------------------------------------------------
# ONNX Runtime + ProtectAI DeBERTa-v3 model for ML-based prompt injection
# scanning. The fullsend binary loads the model directly via Go (hugot +
Expand Down
17 changes: 13 additions & 4 deletions internal/config/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,7 @@ func TestValidRuntimes(t *testing.T) {
assert.Contains(t, runtimes, "claude")
assert.Contains(t, runtimes, "dummy")
assert.NotContains(t, runtimes, "opencode", "opencode is resolved via runtime.Resolve() but not user-selectable until implemented")
assert.NotContains(t, runtimes, "pi", "pi is resolved via runtime.Resolve() but not user-selectable until implemented")
Comment thread
waynesun09 marked this conversation as resolved.
}

func TestOrgConfigValidateRuntime(t *testing.T) {
Expand All @@ -421,11 +422,14 @@ func TestOrgConfigValidateRuntime(t *testing.T) {
}
require.NoError(t, cfg.Validate())

// opencode is resolvable via runtime.Resolve() but not in ValidRuntimes(),
// so config validation must reject it until the runtime is implemented.
// opencode/pi are resolvable via runtime.Resolve() but not in ValidRuntimes(),
// so config validation must reject them until the runtime is implemented.
cfg.Defaults.Runtime = "opencode"
require.Error(t, cfg.Validate())

cfg.Defaults.Runtime = "pi"
require.Error(t, cfg.Validate())

cfg.Defaults.Runtime = "invalid"
require.Error(t, cfg.Validate())
}
Expand Down Expand Up @@ -675,13 +679,18 @@ func TestPerRepoConfigValidate_Runtime(t *testing.T) {
}
assert.NoError(t, cfg.Validate())

// opencode is resolvable via runtime.Resolve() but not in ValidRuntimes(),
// so config validation must reject it until the runtime is implemented.
// opencode/pi are resolvable via runtime.Resolve() but not in ValidRuntimes(),
// so config validation must reject them until the runtime is implemented.
cfg.Runtime = "opencode"
err := cfg.Validate()
assert.Error(t, err)
assert.Contains(t, err.Error(), "invalid runtime")

cfg.Runtime = "pi"
err = cfg.Validate()
assert.Error(t, err)
assert.Contains(t, err.Error(), "invalid runtime")

cfg.Runtime = "invalid"
err = cfg.Validate()
assert.Error(t, err)
Expand Down
2 changes: 2 additions & 0 deletions internal/runtime/capabilities_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ func TestDebugLogNameFor(t *testing.T) {
t.Parallel()
assert.Equal(t, "claude-debug.log", DebugLogNameFor(ClaudeRuntime{}))
assert.Equal(t, DefaultDebugLogName, DebugLogNameFor(OpenCodeRuntime{}))
assert.Equal(t, DefaultDebugLogName, DebugLogNameFor(PiRuntime{}))
assert.Equal(t, DefaultDebugLogName, DebugLogNameFor(DummyRuntime{}))
assert.Equal(t, "other.log", DebugLogNameFor(namedDebugLog{name: "other.log"}))
// An empty name falls back to the default rather than producing "".
Expand All @@ -37,5 +38,6 @@ func TestWantsClaudeMDBridge(t *testing.T) {
t.Parallel()
assert.True(t, WantsClaudeMDBridge(ClaudeRuntime{}))
assert.False(t, WantsClaudeMDBridge(OpenCodeRuntime{}))
assert.False(t, WantsClaudeMDBridge(PiRuntime{}))
assert.False(t, WantsClaudeMDBridge(DummyRuntime{}))
}
91 changes: 91 additions & 0 deletions internal/runtime/pi.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
package runtime

import (
"context"
"fmt"
"io"
"time"

"github.com/fullsend-ai/fullsend/internal/sandbox"
"github.com/fullsend-ai/fullsend/internal/ui"
)

// PiRuntime is a stub implementation of the Runtime and TranscriptHandler
// interfaces for the pi agent runtime (earendil-works/pi, CLI `pi`). All
// methods are no-ops or return not-implemented errors. Subsequent PRs will
// fill in stream parsing (`pi --mode json`, per docs/json.md — confirm at
// implementation time whether `--print` is required alongside it), bootstrap,
// run execution, and transcript extraction. Tracked in #6464.
type PiRuntime struct{}

func (PiRuntime) Name() string { return "pi" }

// System returns the OTEL GenAI gen_ai.system value. Like OpenCode, pi is
// multi-provider (Anthropic, OpenAI, Google, ...), so the system is the
// runtime itself rather than a single model vendor (same precedent as
// OpenCodeRuntime; the per-message `provider` field in pi's event stream may
// allow capturing the actual vendor in a future PR).
func (PiRuntime) System() string { return "pi" }

// ConfigDir returns the pi config directory inside the sandbox. It is
// exported to the agent process as PI_CODING_AGENT_DIR (see EnvExports) and
// lives outside the agent-writable workspace so the target repo cannot
// rewrite the runtime's settings, extensions, or skills.
func (PiRuntime) ConfigDir() string { return sandbox.SandboxPiConfig }

func (PiRuntime) WorkspaceDir() string { return sandbox.SandboxWorkspace }

// EnvExports pins pi's config and session locations to runner-owned paths
// and disables all startup network traffic (update checks, package update
// checks, telemetry). PI_OFFLINE does not affect the inference call itself.
// Var names/semantics per earendil-works/pi docs/environment-variables.md
// (PI_CODING_AGENT_DIR, PI_CODING_AGENT_SESSION_DIR, PI_OFFLINE,
// PI_SKIP_VERSION_CHECK) — re-verify against that doc when PI_VERSION moves.
func (PiRuntime) EnvExports() []string {
return []string{
Comment thread
waynesun09 marked this conversation as resolved.
"export PI_CODING_AGENT_DIR=" + sandbox.SandboxPiConfig,
"export PI_CODING_AGENT_SESSION_DIR=" + sandbox.SandboxPiConfig + "/sessions",
"export PI_OFFLINE=1",
"export PI_SKIP_VERSION_CHECK=1",
}
}

func (PiRuntime) Bootstrap(_ BootstrapInput) error {
return fmt.Errorf("pi runtime is not yet implemented")
}

func (PiRuntime) Run(_ context.Context, _ RunParams, _ *ui.Printer, _ time.Time, _ *RunMetrics) (int, error) {
return -1, fmt.Errorf("pi runtime is not yet implemented")
}

func (PiRuntime) ClearIterationArtifacts(_ string) error { return nil }

// TranscriptHandler stub methods — return not-implemented errors for extract
// methods (to avoid silent success claims in CI logs) and no-ops for parse
// methods (which correctly indicate "nothing found"). pi writes JSONL session
// files under PI_CODING_AGENT_SESSION_DIR, so extraction will follow the
// Claude find-and-download shape rather than OpenCode's export path (#6464).

func (PiRuntime) ExtractTranscripts(_, _, _ string) error {
return fmt.Errorf("pi transcript extraction not implemented (see #6464)")
}

func (PiRuntime) ExtractDebugLog(_, _, _ string) error {
return fmt.Errorf("pi debug log extraction not implemented (see #6464)")
}

func (PiRuntime) ParseTranscriptErrors(_ string) []TranscriptError { return nil }

func (PiRuntime) ParseTranscriptFile(_ string) (TranscriptError, bool) {
return TranscriptError{}, false
}

func (PiRuntime) EmitTranscriptErrors(w io.Writer, summaries []TranscriptError) {
emitTranscriptErrors(w, summaries)
}

// Compile-time interface assertions.
var (
_ Runtime = PiRuntime{}
_ TranscriptHandler = PiRuntime{}
)
Loading
Loading