Skip to content
Open
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
210c597
feat(acp): complete harness v2 hard cutover
sozercan Aug 4, 2026
28fa351
chore: merge main into acp
sozercan Aug 5, 2026
001936e
feat(acp): add OpenCode runtime
sozercan Aug 4, 2026
7ce964e
fix(acp): reject ignored OpenCode model controls
sozercan Aug 4, 2026
c5afefe
fix(e2e): pass OpenCode bash policy pointer
sozercan Aug 4, 2026
7ab9c9d
fix(acp): reject OpenCode agent system prompts
sozercan Aug 4, 2026
b72d09d
fix(acp): close OpenCode authorization gaps
sozercan Aug 4, 2026
09154b9
fix(acp): reject ignored context limits
sozercan Aug 4, 2026
f2f4fa7
fix(acp): close remaining review gaps
sozercan Aug 4, 2026
5f15c4f
fix(acp): reject unsupported model limits
sozercan Aug 4, 2026
6fc5b98
fix(acp): replace musl ripgrep artifact
sozercan Aug 4, 2026
2666b48
fix(acp): close final OpenCode review gaps
sozercan Aug 4, 2026
0ce4504
fix(acp): reject OpenCode provider refs during cutover
sozercan Aug 4, 2026
55cbdbe
fix(acp): align token authorization with runtime policy
sozercan Aug 4, 2026
86fdd61
fix(acp): require OpenCode recreation at cutover
sozercan Aug 4, 2026
8a11955
fix(acp): align delegated transaction authorization
sozercan Aug 4, 2026
061c282
fix(acp): make OpenCode authorization identity authoritative
sozercan Aug 4, 2026
0d3a003
fix(acp): reject fractional OpenCode limits
sozercan Aug 5, 2026
f062252
fix(acp): align effective Bash and read-only policies
sozercan Aug 5, 2026
83a9b51
fix(tools): preserve nested OpenCode model IDs
sozercan Aug 5, 2026
b41e470
fix(acp): admit OpenCode monitor readers
sozercan Aug 5, 2026
6bcae58
fix(acp): translate OpenAI output limits
sozercan Aug 5, 2026
7e9a4af
fix(tools): preserve qualified OpenCode object IDs
sozercan Aug 5, 2026
77bdc4d
fix(monitor): align ACP review task validation
sozercan Aug 5, 2026
443f174
feat: add isolated harness controller modes (#349)
sozercan Aug 8, 2026
e5379b7
fix: harden harness v2 recovery and security workflows
sozercan Aug 11, 2026
11f01ad
fix(acp): preserve terminal session projection identity
sozercan Aug 12, 2026
9db58f0
fix(workspace): allow protected skills alias
sozercan Aug 12, 2026
2e41031
fix(acp): harden runtime and workspace boundaries (#360)
sozercan Aug 13, 2026
020fcf1
refactor: remove dead code and consolidate duplicated validation, scr…
sozercan Aug 13, 2026
2ff8122
Merge remote-tracking branch 'origin/main' into acp
sozercan Aug 13, 2026
2d1cfe7
fix: address PR review threads on settlement durability, chart config…
sozercan Aug 13, 2026
68a30ea
fix(ci): repair newly wired script-suite failures
sozercan Aug 13, 2026
97755a7
fix: address second-round review findings on authority binding and in…
sozercan Aug 13, 2026
db39451
fix(cliwrapper): stop Wait from racing child output collection
sozercan Aug 13, 2026
4143cd7
fix: address third-round review findings on effect leases, tool prefl…
sozercan Aug 14, 2026
2bb9486
fix(ci): trigger coexistence-smoke on any script change
sozercan Aug 14, 2026
a5632fe
fix: address fourth-round review findings on projection identity, too…
sozercan Aug 14, 2026
e568795
fix: address fifth-round review findings on tool URL validation and e…
sozercan Aug 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
137 changes: 128 additions & 9 deletions .agents/skills/agent-sandbox-deploy/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,9 +108,8 @@ test -x "$kindctl"
"$orka_kind_deploy"
```

The harness-wrapper image must be present for the separate plain-agent model
smoke. The model-free direct workspace-adapter smoke bypasses the Task/harness
path.
The digest-pinned ACP runtime images must be present for the separate plain-agent
model smoke. The model-free direct workspace-adapter smoke bypasses the Task-to-RuntimeSession path.

3. **Install agent-sandbox** by driving the canonical script against the kindctl
kubeconfig. Export `KUBECONFIG` from kindctl so the script's `kubectl` calls
Expand Down Expand Up @@ -208,12 +207,132 @@ test -x "$kindctl"
wget -qO- http://127.0.0.1:1337/readyz
```

If you only need model-free confidence, run the CI parity script from the
`Model-free CI parity` section of `references/validate.md`. It validates
installation/configuration plus the direct workspace-adapter lifecycle
(SandboxClaim readiness, router exec, delete, retained reuse, and claim
cleanup);
only the full workspace-backed agent Task path remains gated.
If you only need model-free confidence, run the CI parity script below. It
validates installation/configuration only while workspace-backed agent Tasks
remain gated; it is not a claim/readiness/exec/cleanup smoke.

## Validate

> **Current boundary:** this skill validates the upstream agent-sandbox
> provider directly. Orka ACP RuntimeSessions do not yet map to sandbox claims;
> execution-workspace-backed agent Tasks remain expected-future evidence. The
> removed v1 harness-wrapper path must not be reintroduced. Validate plain
> Codex/Claude ACP Tasks with `scripts/live-acp-runtime-e2e.sh`.

Do **not** use an execution-workspace agent Task as the success criterion yet.
Validate the two currently wired paths separately:

- **Model path through ACP** (requires the optional `AGENTIC=1` step and
vekil ready): run a plain agent Task with no `execution.workspace` and wait
for it to succeed.

```bash
"$kindctl" kubectl -n demo-magic apply -f - <<'YAML'
apiVersion: core.orka.ai/v1alpha1
kind: Agent
metadata:
name: sandbox-codex-agent
namespace: demo-magic
spec:
runtime:
type: codex
defaultMaxTurns: 1
defaultAllowBash: true
model:
name: gpt-5.5
secretRef:
name: sandbox-model-key
---
apiVersion: core.orka.ai/v1alpha1
kind: Task
metadata:
name: orka-live-model-smoke
namespace: demo-magic
spec:
type: agent
agentRef:
name: sandbox-codex-agent
agentRuntime:
maxTurns: 1
timeout: 10m0s
prompt: "Reply exactly: ORKA_LIVE_MODEL_OK"
YAML

"$kindctl" kubectl -n demo-magic \
wait --for=jsonpath='{.status.phase}'=Succeeded task/orka-live-model-smoke --timeout=10m
```

- **Installation/configuration parity**: run the model-free CI parity script
below when you want a self-contained cluster bring-up with fake model
credentials. It verifies the install/config path, but it does **not** exercise
claim → ready → exec → cleanup through the direct adapter.

If you need to demonstrate the intended API shape before RuntimeSession-backed workspace support lands, run it only as an **expected-failure** check and wait for the gate
instead of `Succeeded`:

```bash
"$kindctl" kubectl apply -f - <<'YAML'
apiVersion: core.orka.ai/v1alpha1
kind: Agent
metadata:
name: sandbox-codex-agent
namespace: demo-magic
spec:
runtime:
type: codex
defaultMaxTurns: 1
defaultAllowBash: true
model:
name: gpt-5.5
secretRef:
name: sandbox-model-key
---
apiVersion: core.orka.ai/v1alpha1
kind: Task
metadata:
name: orka-live-sandbox-smoke
namespace: demo-magic
spec:
type: agent
agentRef:
name: sandbox-codex-agent
agentRuntime:
maxTurns: 1
timeout: 10m0s
execution:
workspace:
enabled: true
templateRef:
name: orka-live-template
reusePolicy: none
cleanupPolicy: delete
prompt: "Reply exactly: ORKA_LIVE_SANDBOX_OK"
YAML

"$kindctl" kubectl -n demo-magic \
wait --for=jsonpath='{.status.executionWorkspace.reason}'=WorkspaceValidationFailed \
task/orka-live-sandbox-smoke --timeout=2m
```

Once ACP RuntimeSessions map agent Tasks to execution workspaces, the expected-failure
check can become the live success smoke. At that point, a successful sandbox
wrapper log should include the claimed workspace name, e.g. `completed in
sandbox workspace sandbox-claim-...`. Orka Task status does **not** expose
sandbox claim/exec/cleanup state — read worker logs and upstream agent-sandbox
resources for lifecycle detail.

### Model-free CI parity

`scripts/live-agent-sandbox-e2e.sh` (run by the `Live Agent Sandbox E2E`
workflow) stands up a clean kind cluster with fake model credentials and **no
model access**. The script exercises the direct workspace adapter (claim, readiness, router exec, delete, retained reuse, and cleanup) but deliberately skips the unsupported full ACP Task-to-workspace path:

```bash
bash scripts/live-agent-sandbox-e2e.sh
```

That script owns its own cluster lifecycle; do not run it against a kindctl
cluster you want to keep.

## Guardrails

Expand Down
27 changes: 5 additions & 22 deletions .agents/skills/agent-sandbox-deploy/references/validate.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,31 +2,14 @@

Validation steps for `$agent-sandbox-deploy`. Read after the standard workflow completes.

> **Known gate (verified live 2026-06): valid enabled provider-based agent
> workspace requests are rejected during execution planning by the current
> service-backed harness runtime.** After workspace validation/resolution, the
> provider-based request documented below fails with
> `status.executionWorkspace.reason=WorkspaceValidationFailed` and message
> `execution workspace is not supported by harness runtime yet`. The gate is in
> `internal/controller/agent_execution_plan.go` (`planAgentExecution`), not a
> misconfiguration. The agent CLI runtimes now
> run through the long-lived `agent-harness-wrapper` service, and the
> Task→sandbox-workspace path for agents is not wired through it yet. A **plain**
> agent Task (no `execution.workspace`) runs fine through the harness + model
> proxy, so use that to confirm the model path. The model-free e2e confirms
> installation/configuration and exercises the direct workspace adapter through
> SandboxClaim readiness, router exec, delete, retained release/reuse, and final
> claim cleanup. It skips only the full execution-workspace Task smoke while the
> harness gate is present. Treat the execution-workspace YAML in the optional
> expected-failure check as the intended future Task API once the harness wires
> workspaces.
> **Known gate:** Orka ACP RuntimeSessions do not yet map to agent-sandbox claims. The direct adapter lifecycle is supported for local validation, but a `Task.spec.execution.workspace` agent Task must still fail closed with `WorkspaceValidationFailed`. Plain Codex/Claude Tasks run through controller-owned ACP RuntimePools and validate the model path separately.

Do **not** use an execution-workspace agent Task as the success criterion yet.
Validate the three current surfaces separately: installation/configuration,
direct workspace-adapter lifecycle, and the model path through a plain agent
Task.

- **Model path through the harness** (requires the optional `AGENTIC=1` step and
- **Model path through ACP** (requires the optional `AGENTIC=1` step and
vekil ready): run a plain agent Task with no `execution.workspace` and wait
for it to succeed.

Expand Down Expand Up @@ -73,7 +56,7 @@ YAML
flags and confirms rollout, then exercises claim → ready → router exec →
delete and retained release/reuse → claim cleanup through
`AgentSandboxExecutor`. It skips only the full Orka agent Task
workspace path while the harness gate is present.
workspace path while the ACP workspace-dispatch gate is present.

If you need to demonstrate the intended API shape before harness workspace
support lands, run it only as an **expected-failure** check and wait for the gate
Expand Down Expand Up @@ -123,7 +106,7 @@ YAML
task/orka-live-sandbox-smoke --timeout=2m
```

Once the harness wires agent Tasks to execution workspaces, the expected-failure
Once ACP RuntimeSessions map agent Tasks to execution workspaces, the expected-failure
check can become the live success smoke. At that point, a successful sandbox
wrapper log should include the claimed workspace name, e.g. `completed in
sandbox workspace sandbox-claim-...`. Orka Task status does **not** expose
Expand All @@ -140,7 +123,7 @@ smoke that creates SandboxClaims, waits for readiness, executes through the
router, deletes one claim, retains and reuses another, and performs final claim
cleanup. It skips only the full Orka agent Task workspace smoke, so it proves the
provider-adapter path but not Task-to-workspace controller routing, Task status/
result wiring, harness execution, or model access:
result wiring, ACP Task execution, or model access:

```bash
bash scripts/live-agent-sandbox-e2e.sh
Expand Down
98 changes: 83 additions & 15 deletions .agents/skills/agent-substrate-deploy/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,7 @@ the CI-proven `scripts/agent-substrate-e2e.sh`. **Drive the installer in place;
do not copy either script into the skill.** They pin the Substrate revision
(`SUBSTRATE_REF`, default `b80031d260959b1fc5c6f61e3099fe2a6d368af1`) and own the
heavy lifting: clone Substrate at the pinned ref, create the kind cluster + local
registry, deploy the `ate-system` control plane, build/push the controller,
agent-harness-wrapper, workspace-agent, MCP server, and tool-client images;
registry, deploy the `ate-system` control plane, build/push the controller, workspace-agent, MCP server, and tool-client images;
publish the Substrate `ateom-gvisor` image; create a `WorkerPool` + gVisor
`ActorTemplate`, initialize the RustFS snapshot bucket, and deploy Orka
wired with `--substrate-*`. Re-pin by overriding `SUBSTRATE_REF`, not by editing
Expand Down Expand Up @@ -120,17 +119,9 @@ is a larger task; confirm scope before attempting it.
kind export kubeconfig --name "${cluster}" --kubeconfig "${KUBECONFIG}"
DEMO_CLUSTER_REUSE=reuse bash hack/demos/cluster/install-substrate.sh

# The base e2e creates codex-substrate-ci without model env and patches the
# service-backed harness wrapper to use a fake Codex CLI. Patch the Agent with
# the model Secret from the agentic layer and remove the fake CLI override
# before using a plain agent Task as model-validation evidence.
kubectl --context "$ctx" -n default patch agent codex-substrate-ci --type=merge \
-p "$(jq -cn \
--arg ref substrate-model-key \
--arg model gpt-5.5 \
'{spec:{model:{name:$model},secretRef:{name:$ref}}}')"
kubectl --context "$ctx" -n orka-system set env deployment/orka-agent-harness-wrapper CODEX_CLI_PATH-
kubectl --context "$ctx" -n orka-system rollout status deployment/orka-agent-harness-wrapper --timeout=5m
# This installer validates only the direct Substrate and MCP paths. The
# pre-cutover AGENTIC/model layer is retired. Validate Codex or Claude ACP
# RuntimePools separately with scripts/live-acp-runtime-e2e.sh.
```

> **Login race (verified live 2026-06): disarm vekil's liveness probe before
Expand Down Expand Up @@ -180,8 +171,85 @@ is a larger task; confirm scope before attempting it.
```

For a model-free validation, stay on `AGENTIC=0` and rely on the built-in
smoke exercises documented in `references/validate.md` instead of standing
up vekil.
smoke exercises (next section) instead of standing up vekil.

## Validate

> **Current boundary:** this skill validates direct Substrate Actor and MCP
> behavior only. Orka ACP RuntimeSessions do not yet map to Substrate Actors;
> execution-workspace-backed agent Tasks remain future integration evidence,
> not a success criterion. The removed v1 harness-wrapper path must not be
> reintroduced. Validate plain Codex/Claude ACP Tasks with
> `scripts/live-acp-runtime-e2e.sh` instead.

The installer leaves a fully wired cluster. During standup it smoke-tests direct
actor create/resume/exec/suspend/delete and Substrate-backed MCP tool lifecycle.
It does **not** currently smoke-test retained workspace reuse for Orka agent
Tasks because ACP RuntimeSession-to-Actor dispatch is not yet wired.

If you skipped the kubeconfig export in the workflow above, do it before any
manual `kubectl` commands — the e2e standup uses an isolated kubeconfig and does
**not** leave `kind-<KIND_CLUSTER>` in your default one. Keep using the scoped
`KUBECONFIG` in that shell:

```bash
cluster="${KIND_CLUSTER:-orka-agent-substrate-e2e}"
ctx="kind-${cluster}"
export KUBECONFIG="$(mktemp -t orka-substrate-kubeconfig.XXXXXX)"
kind export kubeconfig --name "${cluster}" --kubeconfig "${KUBECONFIG}"
```

To drive an Orka Task yourself (intended shape; currently gated as noted above):

```bash
cluster="${KIND_CLUSTER:-orka-agent-substrate-e2e}"
ctx="kind-${cluster}"
export KUBECONFIG="$(mktemp -t orka-substrate-kubeconfig.XXXXXX)"
kind export kubeconfig --name "${cluster}" --kubeconfig "${KUBECONFIG}"
kubectl --context "$ctx" -n default apply -f - <<'YAML'
apiVersion: core.orka.ai/v1alpha1
kind: Task
metadata:
name: substrate-smoke
namespace: default
spec:
type: agent
agentRef:
name: codex-substrate-ci
prompt: "Run make test and summarize the result."
sessionRef:
name: substrate-demo
create: true
execution:
workspace:
enabled: true
provider: substrate
templateRef:
name: orka-codex-ci
namespace: ate-demo
reusePolicy: session
cleanupPolicy: retain
YAML

kubectl --context "$ctx" -n default get task substrate-smoke -o yaml
```

Check the provider-neutral workspace lifecycle in
`status.executionWorkspace` (`phase`, `placement`, `density`, `resumeLatency`).
Status is intentionally sanitized — it must not expose actor IDs, snapshot URIs,
worker pod IPs, daemon URLs, or tokens.

### CI parity

`scripts/agent-substrate-e2e.sh` (the `Agent Substrate E2E` workflow) runs the
same path end-to-end and is secret-free. Run it directly when you want a clean,
self-contained validation with its own cluster lifecycle:

```bash
PATH="$(go env GOPATH)/bin:$PATH" SUBSTRATE_E2E_EXTENDED=1 bash scripts/agent-substrate-e2e.sh
```

Set `KEEP_CLUSTER=1` to inspect the cluster after a failure.

## Guardrails

Expand Down
23 changes: 2 additions & 21 deletions .agents/skills/agent-substrate-deploy/references/validate.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,31 +2,12 @@

Validation steps for `$agent-substrate-deploy`. Read after the standard workflow completes.

> **Known gate (verified live 2026-06): valid enabled provider-based agent
> workspace requests are rejected during execution planning by the current
> service-backed harness runtime.** After workspace validation/resolution, a
> `provider: substrate` (or `agent-sandbox`) request fails with
> `status.executionWorkspace.reason=WorkspaceValidationFailed` and message
> `execution workspace is not supported by harness runtime yet`. The gate is in
> `internal/controller/agent_execution_plan.go` (`planAgentExecution`), not a
> misconfiguration — the agent CLI runtimes now
> run through the long-lived `agent-harness-wrapper` service, and the
> Task→workspace path for agents is not wired through it yet. The bundled e2e
> reflects this: it prints `Skipping agent Task execution-workspace checks:
> harness-wrapper runtime is service-backed`. The bundled e2e validates the
> **direct** Substrate path (actor create/resume/router/daemon exec/suspend/delete)
> plus Substrate-backed MCP tool create/reconcile/cleanup. It does not run a plain
> agent Task. After clearing the fake `CODEX_CLI_PATH` override in standard
> workflow step 4 (`Add the model proxy (vekil) — pause for the human`) of
> `../SKILL.md`, use a **plain** agent Task (no `execution.workspace`) to validate
> the harness + model proxy separately. Treat the Task YAML below as the intended
> workspace API once the harness wires workspaces; until then, validate the
> workspace provider via the e2e's direct-actor exercises.
> **Known gate:** Orka ACP RuntimeSessions do not yet map to Substrate Actors. The bundled E2E validates direct Actor create/resume/exec/suspend/delete plus Substrate-backed MCP lifecycle; a provider-backed agent Task remains expected-failure evidence. Validate plain Codex/Claude ACP Tasks separately with `scripts/live-acp-runtime-e2e.sh`.

The installer leaves a fully wired cluster. During standup it smoke-tests direct
actor create/resume/exec/suspend/delete and Substrate-backed MCP tool lifecycle.
It does **not** currently smoke-test retained workspace reuse for Orka agent
Tasks because those execution-workspace checks are skipped by the harness gate.
Tasks because those execution-workspace checks are skipped by the ACP workspace-dispatch gate.

If you skipped standard workflow step 3 (`Export kubeconfig for follow-up
kubectl commands`) in `../SKILL.md`, do it before any
Expand Down
9 changes: 9 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@
# Re-include Go module files
!go.mod
!go.sum
!LICENSE
!NOTICE.md

# Re-include script-local Go helpers used by live E2E Docker builds
!scripts/
Expand All @@ -55,3 +57,10 @@ ui/tsconfig.tsbuildinfo
# Re-include pre-built UI assets for embed
!internal/uiembed/dist/
!internal/uiembed/dist/**

# Re-include the checksum-pinned Codex ACP source patch used by its image build
!workers/acp/images/codex/patch-agent-mode.mjs

# Re-include pinned OpenCode runtime policy and notice inputs.
!workers/acp/images/opencode/AGENTS.md
!workers/acp/images/opencode/NOTICE.md
Loading
Loading