From 9f8922c19e97aa6b01dc2bbfd3db85675a9e6593 Mon Sep 17 00:00:00 2001 From: Aakanksha Duggal Date: Wed, 6 May 2026 17:37:30 -0400 Subject: [PATCH 1/5] feat: Add containerized OpenClaw agent Kustomize-based deployment of OpenClaw on OpenShift with vLLM model serving, OAuth SSO, and restricted-v2 SCC compliance. Includes base manifests, example overlay, Codex Harness sidecar overlay, and docs for installer deployment, troubleshooting, and model compatibility. Tested on OpenShift 4.19 (ROSA) with vLLM model serving. Co-Authored-By: Claude Opus 4.6 --- AGENTS.md | 4 +- agents/openclaw/openclaw_agent/README.md | 106 +++++ .../openclaw_agent/docs/codex-harness.md | 191 ++++++++ .../docs/installer-deployment.md | 433 ++++++++++++++++++ .../docs/model-compatibility.md | 60 +++ .../openclaw_agent/docs/troubleshooting.md | 328 +++++++++++++ .../openclaw_agent/manifests/01-secret.yaml | 11 + .../manifests/02-configmap.yaml | 55 +++ .../openclaw_agent/manifests/03-pvc.yaml | 15 + .../manifests/04-deployment.yaml | 81 ++++ .../openclaw_agent/manifests/05-service.yaml | 15 + .../openclaw_agent/manifests/06-route.yaml | 18 + .../manifests/kustomization.yaml | 14 + .../overlays/codex-harness/codex-secret.yaml | 12 + .../codex-harness/configmap-patch.yaml | 53 +++ .../codex-harness/deployment-patch.yaml | 56 +++ .../overlays/codex-harness/kustomization.yaml | 19 + .../overlays/example/configmap-patch.yaml | 53 +++ .../overlays/example/kustomization.yaml | 24 + 19 files changed, 1547 insertions(+), 1 deletion(-) create mode 100644 agents/openclaw/openclaw_agent/README.md create mode 100644 agents/openclaw/openclaw_agent/docs/codex-harness.md create mode 100644 agents/openclaw/openclaw_agent/docs/installer-deployment.md create mode 100644 agents/openclaw/openclaw_agent/docs/model-compatibility.md create mode 100644 agents/openclaw/openclaw_agent/docs/troubleshooting.md create mode 100644 agents/openclaw/openclaw_agent/manifests/01-secret.yaml create mode 100644 agents/openclaw/openclaw_agent/manifests/02-configmap.yaml create mode 100644 agents/openclaw/openclaw_agent/manifests/03-pvc.yaml create mode 100644 agents/openclaw/openclaw_agent/manifests/04-deployment.yaml create mode 100644 agents/openclaw/openclaw_agent/manifests/05-service.yaml create mode 100644 agents/openclaw/openclaw_agent/manifests/06-route.yaml create mode 100644 agents/openclaw/openclaw_agent/manifests/kustomization.yaml create mode 100644 agents/openclaw/openclaw_agent/overlays/codex-harness/codex-secret.yaml create mode 100644 agents/openclaw/openclaw_agent/overlays/codex-harness/configmap-patch.yaml create mode 100644 agents/openclaw/openclaw_agent/overlays/codex-harness/deployment-patch.yaml create mode 100644 agents/openclaw/openclaw_agent/overlays/codex-harness/kustomization.yaml create mode 100644 agents/openclaw/openclaw_agent/overlays/example/configmap-patch.yaml create mode 100644 agents/openclaw/openclaw_agent/overlays/example/kustomization.yaml diff --git a/AGENTS.md b/AGENTS.md index 44b5d985..d1eff5da 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -49,12 +49,14 @@ make dry-run # preview Helm manifests ## Non-standard agents -Two agents diverge significantly from the standard pattern: +Several agents diverge significantly from the standard pattern: **langflow/simple_tool_calling_agent** - Podman Compose flow-import deployment, not standalone FastAPI. No Dockerfile, pyproject.toml, main.py, src/, or tests/. Different Makefile targets (`init`, `ollama`, `run`, `stop`, `clean`). Uses infra-only env vars (PostgreSQL, Langfuse, Ollama) -- not `API_KEY`/`BASE_URL`/`MODEL_ID`. **a2a/langgraph_crewai_agent** - Uses `python:3.12-slim` (not UBI9), PYTHONPATH `/app` (not `/opt/app-root/src`), `charts/a2a-langgraph-crewai/` chart, `template.env` (not `.env.example`), `entrypoint.sh` (not `main.py`), Starlette (not FastAPI). No tests/ directory. +**openclaw/openclaw_agent** - Kustomize-based deployment of OpenClaw on OpenShift. Uses pre-built image (`ghcr.io/openclaw/openclaw:latest`), not a Dockerfile. Kustomize overlays for customization (model endpoint, storage class, Codex Harness sidecar). No Makefile, no FastAPI, no src/. Port 18789 (gateway). + ## Common gotchas - `agent.yaml` is read at runtime by Makefile -- if malformed, all make targets break diff --git a/agents/openclaw/openclaw_agent/README.md b/agents/openclaw/openclaw_agent/README.md new file mode 100644 index 00000000..1443a39d --- /dev/null +++ b/agents/openclaw/openclaw_agent/README.md @@ -0,0 +1,106 @@ +# OpenClaw on OpenShift + +> Tested: 2026-04-13 on OpenShift 4.19 (ROSA) with vLLM model serving + +Deploy [OpenClaw](https://github.com/openclaw/openclaw) on Red Hat OpenShift with vLLM model serving, OAuth SSO, and production-grade security — no cluster-admin required. + +## Prerequisites + +- OpenShift 4.17+ with namespace-scoped access (`oc login`) +- A model serving endpoint (vLLM, KServe, or external API) +- Block storage class (gp3-csi, managed-csi, thin-csi) — not NFS + +## Quick Start + +### Option A: Using claw-installer (recommended) + +The [claw-installer](https://github.com/sallyom/claw-installer) handles OAuth proxy, Routes, ServiceAccounts, and lifecycle management automatically. + +```bash +git clone https://github.com/sallyom/claw-installer.git +cd claw-installer +npm install && npm run build && npm run dev +# Open http://localhost:3000, select OpenShift, fill in the form +``` + +See [docs/installer-deployment.md](docs/installer-deployment.md) for the full walkthrough, validation, and rollback procedures. + +### Option B: Kustomize + +```bash +oc new-project my-openclaw + +# Copy and customize the example overlay +cp -r overlays/example overlays/my-env +# Edit overlays/my-env/configmap-patch.yaml with your vLLM endpoint +# Edit overlays/my-env/kustomization.yaml with your namespace and storage class + +oc apply -k overlays/my-env +``` + +### Option C: Direct YAML apply + +```bash +oc new-project my-openclaw + +# Edit manifests/02-configmap.yaml with your model endpoint +# Edit manifests/01-secret.yaml with your gateway token + +oc apply -k manifests/ +``` + +## Architecture + +```text + +-----------------------+ + | OpenShift Route | + | (TLS edge) | + +-----------+-----------+ + | + v + +------------------+------------------+ + | Pod | + | +---------------+ +-------------+ | + | | oauth-proxy | | gateway | | + | | port 8443 +->| port 18789 | | + | | (OpenShift | | (loopback) | | + | | OAuth SSO) | | | | + | +---------------+ +------+------+ | + | | | + | +------+------+ | + | | PVC (5Gi) | | + | +-------------+ | + +--------------------------------------+ + | + | OpenAI-compatible API + v + +---------------------------+ + | vLLM / KServe / API | + +---------------------------+ +``` + +## Customization + +| What | Where | +|------|-------| +| Model endpoint URL | `overlays//configmap-patch.yaml` | +| Storage class | `overlays//kustomization.yaml` (patch) | +| Namespace | `overlays//kustomization.yaml` (`namespace:` field) | +| Gateway token | `manifests/01-secret.yaml` | +| Resource limits | Patch `manifests/04-deployment.yaml` | +| Codex Harness | `overlays/codex-harness/` (adds Codex app-server sidecar) | + +## Docs + +| Document | Description | +|----------|-------------| +| [docs/installer-deployment.md](docs/installer-deployment.md) | Full deployment guide: prerequisites, validation, rollback, appendix | +| [docs/troubleshooting.md](docs/troubleshooting.md) | Common issues and fixes (route 503, model override, heartbeat, config clobber) | +| [docs/model-compatibility.md](docs/model-compatibility.md) | Model testing results for agentic tool-calling | +| [docs/codex-harness.md](docs/codex-harness.md) | Codex Harness plugin — sidecar deployment, mixed models, guardian approvals | + +## Related Projects + +- [claw-installer](https://github.com/sallyom/claw-installer) — Web-based deployment tool with OpenShift plugin +- [openclaw-on-openshift](https://github.com/aakankshaduggal/openclaw-on-openshift) — Source repo with full docs +- [OpenClaw](https://github.com/openclaw/openclaw) — Upstream project diff --git a/agents/openclaw/openclaw_agent/docs/codex-harness.md b/agents/openclaw/openclaw_agent/docs/codex-harness.md new file mode 100644 index 00000000..5d89acfb --- /dev/null +++ b/agents/openclaw/openclaw_agent/docs/codex-harness.md @@ -0,0 +1,191 @@ +# Codex Harness on OpenShift + +The [Codex Harness](https://docs.openclaw.ai/plugins/codex-harness) is a bundled OpenClaw plugin that routes agent turns through the Codex app-server instead of OpenClaw's built-in PI harness. This gives you access to Codex-native features (model discovery, thread resumption, compaction) while OpenClaw handles chat channels, session files, tools, and media delivery. + +## How It Works + +``` +┌──────────────────────────────────────────┐ +│ Pod │ +│ │ +│ ┌─────────────┐ ┌────────────────┐ │ +│ │ gateway │ ws │ codex │ │ +│ │ port 18789 ├────►│ app-server │ │ +│ │ │ │ port 39175 │ │ +│ └─────────────┘ └───────┬────────┘ │ +│ │ │ +│ │ HTTPS │ +│ ▼ │ +│ ┌──────────────┐ │ +│ │ OpenAI API │ │ +│ └──────────────┘ │ +└──────────────────────────────────────────┘ +``` + +- The **gateway** receives user messages and delegates agent turns to the Codex app-server via WebSocket (`ws://localhost:39175`) +- The **Codex app-server** runs as a sidecar in the same pod, handling the LLM interaction with OpenAI's API +- Model references like `codex/gpt-5.4` route through the harness; `openai/gpt-5.4` bypasses it entirely + +## When to Use This + +| Use case | Provider | +|----------|----------| +| Direct OpenAI API access | `openai/gpt-5.4` — no harness, OpenClaw PI runtime | +| Codex-managed sessions with thread resumption | `codex/gpt-5.4` — Codex harness | +| Self-hosted models via vLLM | `openai-compat/my-model` — no harness | + +The Codex harness is useful when you want: +- Native Codex thread management and compaction +- Guardian-reviewed tool approvals +- Model discovery through Codex's catalog +- Mixed deployments where some agents use Codex and others use vLLM + +## Important: Sidecar Image Availability + +> **Status (2026-04-21):** The Codex app-server container image (`ghcr.io/openai/codex-app-server:latest`) is **not publicly accessible** — pulling it returns `403 Forbidden`. Until OpenAI publishes this image, the sidecar deployment described below will not work. +> +> **Workaround:** Use the direct OpenAI API instead of the Codex harness. Set the model to `openai/gpt-5.4` with `api: "openai-responses"` in your provider config, and provide your `OPENAI_API_KEY` via a Secret. This gives you GPT-5.x access without the sidecar, though you lose Codex-specific features (thread management, guardian approvals). + +## Deploy on OpenShift + +### Prerequisites + +- OpenAI API key with Codex access +- Codex app-server image (`ghcr.io/openai/codex-app-server:latest`, version 0.118.0+) — see [availability note](#important-sidecar-image-availability) above + +### Using the Kustomize overlay + +```bash +# Copy the overlay +cp -r overlays/codex-harness overlays/my-codex + +# Edit secrets with your API key +vim overlays/my-codex/codex-secret.yaml + +# Edit namespace +vim overlays/my-codex/kustomization.yaml + +# Deploy +oc apply -k overlays/my-codex +``` + +### What the overlay adds + +On top of the base manifests, the Codex overlay: + +1. **Adds a sidecar container** — `codex-app-server` listening on port 39175 +2. **Configures WebSocket transport** — gateway connects to `ws://localhost:39175` (pod-local, no network hop) +3. **Adds secrets** — `OPENAI_API_KEY` for Codex auth, `CODEX_APP_SERVER_TOKEN` for WebSocket auth between containers +4. **Updates the ConfigMap** — enables the `codex` plugin, sets `codex/gpt-5.4` as the model, configures the embedded harness + +### Resource impact + +| Container | CPU request | Memory request | Memory limit | +|-----------|-------------|----------------|--------------| +| gateway | 250m | 256Mi | 1Gi | +| codex-app-server | 250m | 256Mi | 1Gi | + +The Codex app-server is lightweight — it proxies requests to OpenAI's API, so compute happens remotely. + +## Configuration Reference + +### Model references + +| Reference | What happens | +|-----------|-------------| +| `codex/gpt-5.4` | Routes through Codex app-server harness | +| `openai/gpt-5.4` | Direct OpenAI API via OpenClaw's PI runtime | +| `openai-codex/gpt-5.4` | OpenAI with OAuth, no harness | + +### Plugin config fields + +| Field | Default | Purpose | +|-------|---------|---------| +| `transport` | `stdio` | `stdio` spawns locally; `websocket` connects to remote | +| `url` | unset | WebSocket endpoint (e.g., `ws://localhost:39175`) | +| `authToken` | unset | Bearer token for WebSocket auth | +| `requestTimeoutMs` | `60000` | Control-plane call timeout | +| `approvalPolicy` | `never` | Tool approval policy (`never`, `on-request`) | +| `sandbox` | `workspace-write` | Sandbox mode for Codex | +| `approvalsReviewer` | `user` | Use `guardian_subagent` for automated review | + +### Guardian-reviewed approvals + +For production deployments where you want automated tool approval review: + +```json +{ + "plugins": { + "entries": { + "codex": { + "enabled": true, + "config": { + "appServer": { + "transport": "websocket", + "url": "ws://localhost:39175", + "approvalPolicy": "on-request", + "approvalsReviewer": "guardian_subagent", + "sandbox": "workspace-write" + } + } + } + } + } +} +``` + +## Mixed Deployments + +You can run both Codex and vLLM models in the same OpenClaw instance using `runtime: "auto"`: + +```json +{ + "agents": { + "defaults": { + "embeddedHarness": { + "runtime": "auto" + } + }, + "list": [ + { + "id": "coding-agent", + "model": { "primary": "codex/gpt-5.4" } + }, + { + "id": "general-agent", + "model": { "primary": "openai-compat/gpt-oss-20b" } + } + ] + } +} +``` + +The `coding-agent` routes through the Codex harness; the `general-agent` uses vLLM directly. + +## Troubleshooting + +### Codex sidecar not starting + +Check the sidecar logs: +```bash +oc logs deployment/openclaw -c codex-app-server -n +``` + +Common issues: +- Missing `OPENAI_API_KEY` — check the `codex-secrets` secret +- Image pull failure — `ghcr.io/openai/codex-app-server:latest` is [not publicly available](#important-sidecar-image-availability) (returns 403). Use the direct OpenAI API fallback instead. + +### Gateway can't connect to Codex + +If you see WebSocket connection errors in the gateway logs: +```bash +oc logs deployment/openclaw -c gateway -n | grep -i codex +``` + +- Ensure the sidecar is ready (`oc get pods` shows 2/2 or 3/3) +- Check that `url` in the config matches `ws://localhost:39175` +- Verify `CODEX_APP_SERVER_TOKEN` matches in both containers + +### Protocol version mismatch + +If you see "protocol rejection during handshake" errors, the Codex app-server is older than 0.118.0. Update the image tag. diff --git a/agents/openclaw/openclaw_agent/docs/installer-deployment.md b/agents/openclaw/openclaw_agent/docs/installer-deployment.md new file mode 100644 index 00000000..75a5a9e4 --- /dev/null +++ b/agents/openclaw/openclaw_agent/docs/installer-deployment.md @@ -0,0 +1,433 @@ +# Deploying OpenClaw on OpenShift with openclaw-installer + +> Tested: 2026-04-13 on OpenShift 4.19 (ROSA) with vLLM model serving + +Step-by-step guide for deploying OpenClaw on an OpenShift cluster using the [openclaw-installer](https://github.com/sallyom/claw-installer). This is the recommended deployment method — it handles OAuth proxy, ServiceAccounts, Routes, and lifecycle management automatically. + +## Table of Contents + +- [Prerequisites](#prerequisites) +- [Architecture Overview](#architecture-overview) +- [Deployment Steps](#deployment-steps) +- [Validation](#validation) +- [Instance Management](#instance-management) +- [Rollback Instructions](#rollback-instructions) +- [Appendix](#appendix) + +## Prerequisites + +### Environment Requirements + +- **OpenShift Version:** 4.17+ (tested on 4.19.18) +- **Access Level:** Namespace-scoped — no cluster-admin needed +- **CLI Tools:** `oc` (OpenShift CLI), `node` (Node.js 22+), `git` +- **Storage:** Block storage class (gp3-csi, managed-csi, thin-csi) — avoid NFS (SQLite requires POSIX file locking) +- **Model Endpoint:** vLLM, KServe, or external API (OpenAI, Anthropic, Vertex AI) + +### Verify Prerequisites + +```bash +# Verify oc CLI +oc version +``` + +Expected output: +``` +Client Version: 4.17.0 +... +``` + +```bash +# Verify cluster access +oc whoami +``` + +Expected output: +``` + +``` + +```bash +# Verify Node.js +node --version +``` + +Expected output: +``` +v22.x.x +``` + +```bash +# Check available storage classes +oc get storageclass +``` + +Expected output (look for a block storage class): +``` +NAME PROVISIONER RECLAIMPOLICY ... +gp3-csi ebs.csi.aws.com Delete ... +``` + +## Architecture Overview + +``` + +-----------------------+ + | OpenShift Route | + | (TLS edge) | + +-----------+-----------+ + | + v + +------------------+------------------+ + | Pod | + | +---------------+ +-------------+ | + | | oauth-proxy | | gateway | | + | | port 8443 +->| port 18789 | | + | | (OpenShift | | (loopback) | | + | | OAuth SSO) | | | | + | +---------------+ +------+------+ | + | | | + | +------+------+ | + | | PVC (10Gi) | | + | +-------------+ | + +--------------------------------------+ + | + | OpenAI-compatible API + v + +---------------------------+ + | vLLM / KServe / API | + +---------------------------+ +``` + +## Deployment Steps + +### Step 1: Create your namespace + +```bash +oc login --token= --server=https://api.:443 +oc new-project -openclaw +``` + +Expected output: +``` +Now using project "-openclaw" on server "https://api.:443". +``` + +### Step 2: Start the installer + +```bash +git clone https://github.com/sallyom/claw-installer.git +cd openclaw-installer +npm install && npm run build && npm run dev +``` + +Expected output: +``` +> openclaw-installer@x.x.x dev +> next dev + + ▲ Next.js x.x.x + - Local: http://localhost:3000 +``` + +Open `http://localhost:3000`. The installer auto-detects your OpenShift cluster and loads the OpenShift provider plugin. + +### Step 3: Fill in the deploy form + +| Field | Value | Notes | +|-------|-------|-------| +| **Agent name** | your choice | e.g., `my-agent` | +| **Project / Namespace** | your namespace | e.g., `my-openclaw` | +| **Image** | `ghcr.io/openclaw/openclaw:latest` | | +| **Provider** | Self-hosted (vLLM) | or Anthropic/OpenAI/Vertex AI if using cloud | +| **Model endpoint** | your vLLM URL | e.g., `https://vllm-20b-gpt-oss.apps.rosa./v1` | + +Click **Deploy**. The installer streams logs as it creates each resource. + +Expected output (installer log panel): +``` +Creating ServiceAccount... ✓ +Creating Secrets... ✓ +Creating ConfigMap... ✓ +Creating PVC... ✓ +Creating Deployment... ✓ +Creating Service... ✓ +Creating Route... ✓ +Deployment complete. +``` + +### Step 4: Verify pod startup + +```bash +oc get pods -n +``` + +Expected output: +``` +NAME READY STATUS RESTARTS AGE +openclaw-xxxxxxxxxx-xxxxx 2/2 Running 0 60s +``` + +> **Note:** `2/2` indicates both the oauth-proxy and gateway containers are running. If you see `1/2` or `CrashLoopBackOff`, check [troubleshooting.md](troubleshooting.md). + +### Step 5: Fix the Route (if needed) + +Open the Route URL: + +```bash +oc get route openclaw -n -o jsonpath='{.spec.host}' +``` + +Navigate to `https://` in your browser. If you see "Application is not available", the TLS termination is misconfigured. See [troubleshooting.md](troubleshooting.md#route-returns-application-is-not-available-503). + +**Verification:** +```bash +curl -s -o /dev/null -w "%{http_code}" -k https://$(oc get route openclaw -n -o jsonpath='{.spec.host}') +``` + +Expected output: +``` +403 +``` + +> A `403` is correct — it means the OAuth proxy is running and will redirect you to OpenShift SSO. A `503` means the Route is misconfigured. + +### Step 6: Approve device pairing + +After SSO login, the Control UI requires a one-time device pairing. Get the request ID from the UI prompt, then: + +```bash +oc exec deployment/openclaw -n -c gateway -- \ + openclaw devices approve +``` + +Expected output: +``` +Device approved: +``` + +Or use the **Open** action from the installer's **Instances** tab — it opens with the token pre-filled and may auto-pair. + +### Step 7: Verify model configuration + +Check the gateway logs to confirm the correct model is loaded: + +```bash +oc logs deployment/openclaw -c gateway -n | grep "agent model" +``` + +Expected output: +``` +[gateway] agent model: openai-compat/gpt-oss-20b +``` + +If it shows a different model (e.g., `anthropic/claude-sonnet-4-6`), the gateway auto-detected a provider and overrode your config. See [troubleshooting.md](troubleshooting.md#gateway-uses-wrong-model--no-api-key-errors). + +## Validation + +After completing the deployment steps, validate each component: + +### 1. Pod Health + +```bash +oc get pods -n +``` + +Expected: All pods show `2/2 Running` status with zero restarts. + +### 2. Route Accessibility + +```bash +curl -s -o /dev/null -w "%{http_code}" -k \ + https://$(oc get route openclaw -n -o jsonpath='{.spec.host}') +``` + +Expected: `403` (OAuth redirect gate). + +### 3. Gateway Logs + +```bash +oc logs deployment/openclaw -c gateway -n | grep -E "(ready|agent model|heartbeat)" +``` + +Expected output: +``` +[gateway] agent model: openai-compat/gpt-oss-20b +[gateway] ready (4 plugins: ...; 3.xs) +[heartbeat] disabled +``` + +### 4. OAuth Proxy Logs + +```bash +oc logs deployment/openclaw -c oauth-proxy -n --tail=5 +``` + +Expected: No error lines. Should show listening messages. + +### 5. PVC Bound + +```bash +oc get pvc -n +``` + +Expected: +``` +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE +openclaw-home-pvc Bound ... 10Gi RWO gp3-csi 5m +``` + +### 6. End-to-End Test + +After device pairing, send a test message in the Control UI. The agent should respond using the configured model. If you see "No API key found" errors, the model provider isn't configured correctly — see [troubleshooting.md](troubleshooting.md). + +## What the installer creates + +| Resource | Name | Purpose | +|----------|------|---------| +| ServiceAccount | `openclaw-oauth-proxy` | SA for OAuth with redirect annotation | +| Secret | `openclaw-oauth-config` | OAuth client-secret + cookie secret | +| Secret | `openclaw-secrets` | Gateway token + provider API keys | +| ConfigMap | `openclaw-config` | `openclaw.json` gateway configuration | +| ConfigMap | `openclaw-agent` | Agent workspace files (HEARTBEAT.md, SOUL.md, etc.) | +| PVC | `openclaw-home-pvc` | 10Gi persistent state (SQLite, agent sessions, logs) | +| Service | `openclaw` | ClusterIP: gateway (18789) + oauth-ui (8443) | +| Route | `openclaw` | TLS edge-terminated route to oauth-proxy | +| Deployment | `openclaw` | Init container + oauth-proxy sidecar + gateway | + +## Instance Management + +From the installer's **Instances** tab: + +| Action | Effect | +|--------|--------| +| **Open** | Opens Route URL with token pre-filled | +| **Re-deploy** | Syncs local agent workspace files + restarts pod | +| **Stop** | Scales replicas to 0 (preserves PVC data) | +| **Start** | Scales replicas back to 1 | +| **Approve Pairing** | Approves pending device pairing requests | + +## Rollback Instructions + +### Complete Cleanup + +To completely remove the OpenClaw deployment and all associated resources: + +```bash +# Delete the entire namespace (removes all resources including PVC data) +oc delete project +``` + +> **Warning:** This deletes all data, including agent sessions and SQLite state on the PVC. + +### Partial Rollback + +To remove specific components while preserving others: + +```bash +# Stop the deployment (preserves PVC and config) +oc scale deployment/openclaw --replicas=0 -n + +# Remove deployment only (preserves PVC, secrets, config) +oc delete deployment openclaw -n + +# Remove route only (pod keeps running, just not externally accessible) +oc delete route openclaw -n +``` + +### Config Rollback + +If a ConfigMap change broke the gateway: + +```bash +# Check deployment revision history +oc rollout history deployment/openclaw -n + +# Roll back to previous revision +oc rollout undo deployment/openclaw -n +``` + +### PVC Data Reset + +If the on-disk config is corrupted and the gateway won't start: + +```bash +# Scale down, delete PVC, redeploy +oc scale deployment/openclaw --replicas=0 -n +oc delete pvc openclaw-home-pvc -n + +# Re-deploy via installer or re-apply manifests +# The init container will reinitialize the config from the ConfigMap +oc scale deployment/openclaw --replicas=1 -n +``` + +## Appendix + +### A. Configuration Parameters + +| Parameter | Location | Default | Description | +|-----------|----------|---------|-------------| +| `agents.defaults.model.primary` | ConfigMap | varies | Model reference (e.g., `openai-compat/gpt-oss-20b`) | +| `agents.defaults.heartbeat.every` | ConfigMap | `30m` | Heartbeat interval (`0m` to disable) | +| `gateway.bind` | ConfigMap | `loopback` | Bind address (`loopback` recommended with OAuth proxy) | +| `gateway.controlUi.allowedOrigins` | ConfigMap | `[]` | Route URL for CORS — must match your Route host | +| `models.providers.*.baseUrl` | ConfigMap | none | Model serving endpoint URL | +| `tools.deny` | ConfigMap | `[]` | Tool deny list (e.g., `["group:web", "browser"]`) | +| `OPENCLAW_GATEWAY_TOKEN` | Secret | generated | Gateway auth token (auto-generated by installer) | +| `OPENCLAW_HOME` | Env var | `/data/.openclaw` | Data directory on PVC | + +### B. Image Sources + +| Component | Registry | Image | +|-----------|----------|-------| +| Gateway | GitHub Container Registry | `ghcr.io/openclaw/openclaw:latest` | +| OAuth Proxy | Red Hat Registry | `registry.redhat.io/openshift4/ose-oauth-proxy:latest` | +| Init Container | Same as gateway | `ghcr.io/openclaw/openclaw:latest` | + +### C. Network Ports + +| Component | Port | Protocol | Purpose | +|-----------|------|----------|---------| +| Gateway | 18789 | HTTP | API server + Control UI WebSocket | +| OAuth Proxy | 8443 | HTTPS | OAuth-gated frontend | +| Browser Control | 18791 | HTTP | Internal browser automation (loopback only) | + +### D. Resource Requirements + +| Container | CPU Request | Memory Request | Memory Limit | +|-----------|-------------|----------------|--------------| +| init-config | — | — | 128Mi | +| gateway | 250m | 256Mi | 1Gi | +| oauth-proxy | 100m | 64Mi | 256Mi | + +### E. Test Environment Specifications + +| Property | Value | +|----------|-------| +| **OpenShift Version** | 4.19.18 | +| **Platform** | AWS (ROSA) | +| **Worker Nodes** | 4 nodes | +| **Storage Class** | gp3-csi (EBS) | +| **Model Endpoint** | vLLM (gpt-oss-20b) on GPU node | +| **Node OS** | Red Hat Enterprise Linux CoreOS | +| **Kernel** | 5.14.0-570.x | + +### F. Known Limitations + +| Limitation | Impact | Workaround | +|------------|--------|------------| +| **Single replica only** | No HA — Recreate strategy required because SQLite does not support concurrent writers | Use PostgreSQL-backed model server for multi-replica; or accept single-replica for PoC | +| **NFS incompatible** | SQLite requires POSIX file locking; NFS advisory locks are unreliable | Use block storage (gp3-csi, managed-csi, thin-csi) | +| **Config auto-override** | Gateway may overwrite ConfigMap settings with auto-detected providers on first start | Verify model in logs after every rollout; patch ConfigMap if overridden | +| **Heartbeat on by default** | Fires every 30 min, consumes model API tokens even when idle | Set `agents.defaults.heartbeat.every: "0m"` in ConfigMap | +| **PVC data lost on delete** | Deleting the PVC removes all agent sessions, SQLite state, and logs | Back up PVC data before deletion; consider `Retain` reclaim policy | +| **Device pairing required** | Every new browser session needs a one-time device approval via CLI | Use installer's **Open** action, or approve via `oc exec` | + +### G. References + +| Resource | URL | +|----------|-----| +| OpenClaw Upstream | https://github.com/openclaw/openclaw | +| openclaw-installer | https://github.com/sallyom/claw-installer | +| OpenClaw Docs | https://docs.openclaw.ai | +| OpenShift Docs | https://docs.openshift.com | +| Kustomize | https://kustomize.io | diff --git a/agents/openclaw/openclaw_agent/docs/model-compatibility.md b/agents/openclaw/openclaw_agent/docs/model-compatibility.md new file mode 100644 index 00000000..80d8c468 --- /dev/null +++ b/agents/openclaw/openclaw_agent/docs/model-compatibility.md @@ -0,0 +1,60 @@ +# Model Compatibility for OpenClaw Tool-Calling + +> Tested: 2026-04-13 + +OpenClaw relies on models that support structured tool-calling (function calling). Not all models work — some emit raw JSON instead of using the native tool API, and some hallucinate tool results when calls fail silently. + +## Test Results + +| Model | Provider | Tool-Calling | Notes | +|-------|----------|:------------:|-------| +| **gpt-oss-20b** | vLLM (cluster) | Working | `--tool-call-parser openai --enable-auto-tool-choice` flags required | +| **qwen2.5:7b** | Ollama (local) | Working | Best local option for 32GB machines (~8GB RAM) | +| **qwen3.5:27b** | Ollama (local) | Working | Tools work, but 41GB RAM usage makes it impractical locally | +| **llama3.2:3b** | Ollama (local) | Partial | Executes tools but gets confused by OpenClaw's internal system prompts | +| **phi4-mini:3.8b** | Ollama (local) | Broken | Emits raw JSON in response body instead of native tool calls | +| **codex/gpt-5.4** | Codex Harness | Expected | Routes through Codex app-server; requires sidecar deployment (see [codex-harness.md](codex-harness.md)) | + +## Critical Warning: Hallucinated Tool Results + +When phi4-mini's tool calls failed silently, the model **fabricated an entire file listing** — realistic-looking paths that did not exist on the machine, presented confidently as real data. + +This is the most dangerous failure mode for an agentic system: the user has no way to know the results are fake without independently verifying every output. + +## Provider Routing + +OpenClaw supports multiple model providers simultaneously. The model reference prefix determines the routing: + +| Prefix | Route | Harness | +|--------|-------|---------| +| `openai-compat/` | Self-hosted vLLM endpoint | OpenClaw PI runtime | +| `openai/` | Direct OpenAI API | OpenClaw PI runtime | +| `codex/` | Codex app-server (sidecar) | Codex harness | +| `anthropic/` | Anthropic API | OpenClaw PI runtime | +| `ollama/` | Local Ollama instance | OpenClaw PI runtime | + +Mixed deployments can use `runtime: "auto"` to let OpenClaw select the harness based on model prefix. + +## Recommendations + +- **On OpenShift (self-hosted):** Use gpt-oss-20b or larger via vLLM with tool-calling flags enabled +- **On OpenShift (Codex):** Use `codex/gpt-5.4` with the Codex Harness sidecar for thread management and guardian approvals +- **On OpenShift (Google):** Use `google/gemini-2.5-pro` via Google AI Studio with `GEMINI_API_KEY` (Vertex AI SA auth is not yet supported natively — see [vertex-ai-provider.md](vertex-ai-provider.md#known-limitation-vertex-ai-auth)) +- **Local testing:** Use qwen2.5:7b via Ollama with a 32k context window +- **Always verify:** Test tool-calling with a known-answer query (e.g., "list files on my Desktop") before trusting agent output + +## Valid API Types + +When configuring `models.providers.*.api` in the ConfigMap, use one of: + +| API Type | Use Case | +|----------|----------| +| `openai-completions` | vLLM, LM Studio, any OpenAI-compatible endpoint | +| `openai-responses` | Direct OpenAI API | +| `openai-codex-responses` | Codex app-server | +| `anthropic-messages` | Anthropic API or Anthropic on Vertex AI | +| `google-generative-ai` | Google Gemini (AI Studio or Vertex AI) | +| `ollama` | Local Ollama instance | +| `azure-openai-responses` | Azure OpenAI | +| `bedrock-converse-stream` | AWS Bedrock | +| `github-copilot` | GitHub Copilot | diff --git a/agents/openclaw/openclaw_agent/docs/troubleshooting.md b/agents/openclaw/openclaw_agent/docs/troubleshooting.md new file mode 100644 index 00000000..34d0f554 --- /dev/null +++ b/agents/openclaw/openclaw_agent/docs/troubleshooting.md @@ -0,0 +1,328 @@ +# Troubleshooting + +Common issues and solutions for OpenClaw on OpenShift. Issues are listed from most to least frequently encountered. + +## Table of Contents + +- [Route returns "Application is not available" (503)](#route-returns-application-is-not-available-503) +- [Gateway uses wrong model / "No API key" errors](#gateway-uses-wrong-model--no-api-key-errors) +- [Heartbeat flooding the chat UI](#heartbeat-flooding-the-chat-ui) +- [Device pairing required after SSO login](#device-pairing-required-after-sso-login) +- [Pod stuck in CrashLoopBackOff](#pod-stuck-in-crashloopbackoff) +- [Config clobbered on restart](#config-clobbered-on-restart) +- [Device pairing rate limiter](#device-pairing-rate-limiter) +- [Diagnostic commands](#diagnostic-commands) + +--- + +## Route returns "Application is not available" (503) + +**Severity:** Blocking — users cannot access the UI. + +**Cause:** The oauth-proxy uses `--http-address` (plain HTTP) by default. If the Route has `tls.termination: reencrypt`, the router expects TLS on the backend and the connection fails silently. + +**Symptoms:** +``` +Browser: "Application is not available" +curl returns: 503 +``` + +**Fix:** Recreate the Route with `edge` termination: + +```bash +oc delete route openclaw -n + +cat <<'EOF' | oc apply -f - +apiVersion: route.openshift.io/v1 +kind: Route +metadata: + name: openclaw + namespace: + labels: + app: openclaw + annotations: + haproxy.router.openshift.io/timeout: 30m +spec: + to: + kind: Service + name: openclaw + weight: 100 + port: + targetPort: oauth-ui + tls: + termination: edge + insecureEdgeTerminationPolicy: Redirect +EOF +``` + +**Verification:** +```bash +curl -s -o /dev/null -w "%{http_code}" -k https:// +``` + +Expected output: +``` +403 +``` + +> A `403` is correct — it means the OAuth proxy is running and will redirect to OpenShift SSO. A `503` means the Route is still misconfigured. + +**Prevention:** When deploying via the installer, verify the Route TLS mode immediately after deployment: +```bash +oc get route openclaw -n -o jsonpath='{.spec.tls.termination}' +# Should show: edge +``` + +--- + +## Gateway uses wrong model / "No API key" errors + +**Severity:** Blocking — agent cannot respond to messages. + +**Cause:** OpenClaw auto-generates its config on first start. If the gateway detects a provider (e.g., Anthropic) it will override the ConfigMap settings with its own defaults. This means the ConfigMap says vLLM but the gateway is actually trying to use Anthropic. + +**Symptoms in logs:** +``` +No API key found for provider "anthropic" +model fallback decision: decision=candidate_failed requested=anthropic/claude-sonnet-4-6 +``` + +**Symptoms in UI:** The model selector in the Control UI shows a model name you did not configure (e.g., `gpt-oss-20b` instead of `gemini-2.5-pro`). + +**Fix:** Patch the ConfigMap with the correct model provider config, then restart: + +```bash +# Export current config +oc get configmap openclaw-config -n \ + -o jsonpath='{.data.openclaw\.json}' > /tmp/openclaw-config.json + +# Edit /tmp/openclaw-config.json: +# - Set agents.defaults.model.primary to "openai-compat/gpt-oss-20b" +# - Add models.providers.openai-compat with your vLLM endpoint + +# Apply and restart +oc create configmap openclaw-config \ + --from-file=openclaw.json=/tmp/openclaw-config.json \ + -n --dry-run=client -o yaml | oc apply -f - +oc rollout restart deployment/openclaw -n +``` + +**Verification:** +```bash +oc logs deployment/openclaw -c gateway -n | grep "agent model" +``` + +Expected output: +``` +[gateway] agent model: openai-compat/gpt-oss-20b +``` + +**Prevention:** Always check the gateway logs after every rollout to confirm the model matches your intent. + +--- + +## Heartbeat flooding the chat UI + +**Severity:** Moderate — consumes API tokens and clutters the chat history. + +**Cause:** OpenClaw's heartbeat scheduler fires every 30 minutes by default, sending `HEARTBEAT_OK` messages to the chat. If the model provider is misconfigured, each heartbeat triggers an error instead, producing dozens of error messages per day. + +**Symptoms in UI:** Repeated "HEARTBEAT_OK" messages or "Agent failed before reply" errors every 30 minutes. + +**Symptoms in logs:** +``` +[heartbeat] started +``` + +**Fix:** Set `heartbeat.every` to `"0m"` in the ConfigMap: + +```bash +# Export config +oc get configmap openclaw-config -n \ + -o jsonpath='{.data.openclaw\.json}' > /tmp/openclaw-config.json + +# Add to agents.defaults: +# "heartbeat": { "every": "0m" } +# Also add to each agent in agents.list: +# "heartbeat": { "every": "0m" } + +# Apply and restart +oc create configmap openclaw-config \ + --from-file=openclaw.json=/tmp/openclaw-config.json \ + -n --dry-run=client -o yaml | oc apply -f - +oc rollout restart deployment/openclaw -n +``` + +**Verification:** +```bash +oc logs deployment/openclaw -c gateway -n | grep heartbeat +``` + +Expected output: +``` +[heartbeat] disabled +``` + +**Reference:** See [OpenClaw heartbeat docs](https://docs.openclaw.ai/gateway/heartbeat) for advanced options (`activeHours`, `target`, `lightContext`). + +--- + +## Device pairing required after SSO login + +**Severity:** Low — one-time setup per browser. + +**Cause:** The Control UI browser session needs a one-time device approval, even after OpenShift OAuth login. This is an OpenClaw security feature, not an OpenShift issue. + +**Fix:** +```bash +# Get the request ID from the UI prompt, then: +oc exec deployment/openclaw -n -c gateway -- \ + openclaw devices approve +``` + +Expected output: +``` +Device approved: +``` + +Alternatively, use the **Open** action from the installer's **Instances** tab — it opens with the token pre-filled and may auto-pair. + +--- + +## Pod stuck in CrashLoopBackOff + +**Severity:** Blocking — gateway is down. + +**Cause:** OpenClaw auto-generates config at startup. If the config on the PVC conflicts with the ConfigMap, the gateway detects a change, overwrites the file, and triggers a process restart that kills PID 1. + +**Diagnostic:** +```bash +oc describe pod -l app=openclaw -n | grep -A 5 "Last State" +oc logs deployment/openclaw -c gateway -n --previous +``` + +Look for: `Config overwrite` or `Config observe anomaly` messages. + +**Fix:** Delete the PVC data and redeploy: +```bash +oc scale deployment/openclaw --replicas=0 -n +oc delete pvc openclaw-home-pvc -n +# Redeploy via installer or re-apply manifests +oc scale deployment/openclaw --replicas=1 -n +``` + +> **Warning:** This deletes all agent sessions and chat history. Back up the PVC first if needed: +> ```bash +> oc cp :/data/.openclaw /tmp/openclaw-backup -c gateway -n +> ``` + +--- + +## Config clobbered on restart + +**Severity:** Moderate — settings revert unexpectedly. + +**Cause:** OpenClaw's init container copies `openclaw.json` from the ConfigMap to the PVC on every pod start. However, the gateway also writes back to the same file at runtime (model discovery, plugin state). On the next restart, the init container overwrites these runtime changes with the original ConfigMap version. This produces `.clobbered.*` backup files on the PVC. + +**Symptoms:** +```bash +oc exec deployment/openclaw -c gateway -n -- ls /home/node/.openclaw/ +# Shows: openclaw.json.clobbered.2026-04-15T14-45-55-777Z +``` + +**Fix:** Update the ConfigMap to include any runtime changes you want to preserve: +```bash +# Export the running config (not the ConfigMap) +oc exec deployment/openclaw -c gateway -n -- \ + cat /home/node/.openclaw/openclaw.json > /tmp/openclaw-config.json + +# Review and apply +oc create configmap openclaw-config \ + --from-file=openclaw.json=/tmp/openclaw-config.json \ + -n --dry-run=client -o yaml | oc apply -f - +``` + +--- + +## Device Pairing Rate Limiter + +**Severity:** Blocking — browser cannot connect even with valid credentials. + +**Cause:** When a browser repeatedly sends a stale device token (e.g., after PVC data was deleted and device pairings were cleared), the gateway's in-memory rate limiter kicks in after several failed attempts. Once rate-limited, all connection attempts from that client are rejected — even after fixing the underlying token issue. + +**Symptoms in logs:** +``` +unauthorized ... reason=device_token_mismatch +unauthorized ... reason=rate_limited +``` + +**Symptoms in browser:** "Too many failed authentication attempts (retry later)" or the connection silently fails. + +**Fix (both steps required):** + +1. **Clear browser site data** for the OpenClaw route URL — this removes the cached stale device token: + - Chrome: Click lock icon in URL bar → "Site settings" → "Clear data" + - Or: DevTools (F12) → Application → Storage → "Clear site data" + +2. **Restart the pod** to reset the in-memory rate limiter: + ```bash + oc rollout restart deployment/openclaw -n + ``` + +3. **Refresh the browser** — you should now see a fresh device pairing prompt. Approve it: + ```bash + oc exec deployment/openclaw -c gateway -n -- \ + openclaw devices approve + ``` + +**Verification:** The browser connects and shows the chat UI without errors. + +**Prevention:** When clearing device pairings (`openclaw devices clear`), always clear browser site data at the same time to prevent the stale token retry loop. + +--- + +## Diagnostic Commands + +```bash +# Pod status +oc get pods -n + +# Gateway logs (tail) +oc logs deployment/openclaw -c gateway -n --tail=50 + +# Gateway logs (full) +oc logs deployment/openclaw -c gateway -n + +# OAuth proxy logs +oc logs deployment/openclaw -c oauth-proxy -n + +# Previous pod logs (after crash) +oc logs deployment/openclaw -c gateway -n --previous + +# Check route config +oc get route openclaw -n -o jsonpath='{.spec.tls.termination}' + +# Check what model is active +oc logs deployment/openclaw -c gateway -n | grep "agent model" + +# Check ConfigMap vs on-disk config +oc get configmap openclaw-config -n \ + -o jsonpath='{.data.openclaw\.json}' | python3 -m json.tool + +# Check on-disk config (what gateway is actually using) +oc exec deployment/openclaw -c gateway -n -- \ + cat /home/node/.openclaw/openclaw.json | python3 -m json.tool + +# List clobbered config backups +oc exec deployment/openclaw -c gateway -n -- \ + ls -la /home/node/.openclaw/openclaw.json.clobbered.* 2>/dev/null + +# Check PVC status +oc get pvc -n + +# Check deployment events +oc describe deployment openclaw -n | tail -20 + +# Exec into the gateway +oc exec -it deployment/openclaw -c gateway -n -- sh +``` diff --git a/agents/openclaw/openclaw_agent/manifests/01-secret.yaml b/agents/openclaw/openclaw_agent/manifests/01-secret.yaml new file mode 100644 index 00000000..4ac66e8e --- /dev/null +++ b/agents/openclaw/openclaw_agent/manifests/01-secret.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: Secret +metadata: + name: openclaw-secrets + labels: + app: openclaw +type: Opaque +stringData: + # Replace with your own token + OPENCLAW_GATEWAY_TOKEN: "CHANGE-ME" + VLLM_API_KEY: "not-needed" diff --git a/agents/openclaw/openclaw_agent/manifests/02-configmap.yaml b/agents/openclaw/openclaw_agent/manifests/02-configmap.yaml new file mode 100644 index 00000000..7489f56f --- /dev/null +++ b/agents/openclaw/openclaw_agent/manifests/02-configmap.yaml @@ -0,0 +1,55 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: openclaw-config + labels: + app: openclaw +data: + openclaw.json: | + { + "gateway": { + "port": 18789, + "bind": "0.0.0.0", + "mode": "local", + "auth": { + "mode": "token" + } + }, + "channels": {}, + "agents": { + "defaults": { + "heartbeat": {}, + "model": "openai-compat/gpt-oss-20b", + "sandbox": { + "mode": "off" + } + } + }, + "models": { + "mode": "replace", + "providers": { + "openai-compat": { + "baseUrl": "https://YOUR-VLLM-ENDPOINT/v1", + "api": "openai-completions", + "apiKey": "not-needed", + "models": [ + { + "id": "gpt-oss-20b", + "name": "gpt-oss-20b", + "reasoning": false, + "input": ["text"], + "contextWindow": 32768, + "maxTokens": 4096 + } + ] + } + } + }, + "plugins": {}, + "tools": { + "deny": [ + "group:web", + "browser" + ] + } + } diff --git a/agents/openclaw/openclaw_agent/manifests/03-pvc.yaml b/agents/openclaw/openclaw_agent/manifests/03-pvc.yaml new file mode 100644 index 00000000..ed3e7b7d --- /dev/null +++ b/agents/openclaw/openclaw_agent/manifests/03-pvc.yaml @@ -0,0 +1,15 @@ +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: openclaw-state + labels: + app: openclaw +spec: + accessModes: + - ReadWriteOnce + # Change to your cluster's block storage class (e.g., gp3-csi, managed-csi, thin-csi) + # Avoid NFS — OpenClaw uses SQLite which requires POSIX file locking + storageClassName: gp3-csi + resources: + requests: + storage: 5Gi diff --git a/agents/openclaw/openclaw_agent/manifests/04-deployment.yaml b/agents/openclaw/openclaw_agent/manifests/04-deployment.yaml new file mode 100644 index 00000000..f0f3808c --- /dev/null +++ b/agents/openclaw/openclaw_agent/manifests/04-deployment.yaml @@ -0,0 +1,81 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: openclaw + labels: + app: openclaw +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app: openclaw + template: + metadata: + labels: + app: openclaw + spec: + initContainers: + - name: init-config + image: ghcr.io/openclaw/openclaw:latest + command: + - sh + - -c + - | + cp /config/openclaw.json /data/.openclaw/openclaw.json + chgrp -R 0 /data/.openclaw + chmod -R g=u /data/.openclaw + echo "Config initialized" + volumeMounts: + - name: config + mountPath: /config + - name: data + mountPath: /data/.openclaw + resources: + limits: + memory: 128Mi + cpu: 100m + containers: + - name: gateway + image: ghcr.io/openclaw/openclaw:latest + ports: + - containerPort: 18789 + protocol: TCP + env: + - name: OPENCLAW_GATEWAY_TOKEN + valueFrom: + secretKeyRef: + name: openclaw-secrets + key: OPENCLAW_GATEWAY_TOKEN + - name: OPENCLAW_HOME + value: /data/.openclaw + volumeMounts: + - name: data + mountPath: /data/.openclaw + resources: + requests: + memory: 256Mi + cpu: 250m + limits: + memory: 1Gi + cpu: "1" + livenessProbe: + httpGet: + path: / + port: 18789 + initialDelaySeconds: 15 + periodSeconds: 30 + readinessProbe: + httpGet: + path: / + port: 18789 + initialDelaySeconds: 10 + periodSeconds: 10 + volumes: + - name: config + configMap: + name: openclaw-config + - name: data + persistentVolumeClaim: + claimName: openclaw-state diff --git a/agents/openclaw/openclaw_agent/manifests/05-service.yaml b/agents/openclaw/openclaw_agent/manifests/05-service.yaml new file mode 100644 index 00000000..7c8faf07 --- /dev/null +++ b/agents/openclaw/openclaw_agent/manifests/05-service.yaml @@ -0,0 +1,15 @@ +apiVersion: v1 +kind: Service +metadata: + name: openclaw + labels: + app: openclaw +spec: + selector: + app: openclaw + ports: + - name: gateway + port: 18789 + targetPort: 18789 + protocol: TCP + type: ClusterIP diff --git a/agents/openclaw/openclaw_agent/manifests/06-route.yaml b/agents/openclaw/openclaw_agent/manifests/06-route.yaml new file mode 100644 index 00000000..71c4ead1 --- /dev/null +++ b/agents/openclaw/openclaw_agent/manifests/06-route.yaml @@ -0,0 +1,18 @@ +apiVersion: route.openshift.io/v1 +kind: Route +metadata: + name: openclaw + labels: + app: openclaw + annotations: + haproxy.router.openshift.io/timeout: 30m +spec: + to: + kind: Service + name: openclaw + weight: 100 + port: + targetPort: gateway + tls: + termination: edge + insecureEdgeTerminationPolicy: Redirect diff --git a/agents/openclaw/openclaw_agent/manifests/kustomization.yaml b/agents/openclaw/openclaw_agent/manifests/kustomization.yaml new file mode 100644 index 00000000..abc6201c --- /dev/null +++ b/agents/openclaw/openclaw_agent/manifests/kustomization.yaml @@ -0,0 +1,14 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +resources: + - 01-secret.yaml + - 02-configmap.yaml + - 03-pvc.yaml + - 04-deployment.yaml + - 05-service.yaml + - 06-route.yaml + +commonLabels: + app.kubernetes.io/name: openclaw + app.kubernetes.io/part-of: openclaw-on-openshift diff --git a/agents/openclaw/openclaw_agent/overlays/codex-harness/codex-secret.yaml b/agents/openclaw/openclaw_agent/overlays/codex-harness/codex-secret.yaml new file mode 100644 index 00000000..3f7cb3d1 --- /dev/null +++ b/agents/openclaw/openclaw_agent/overlays/codex-harness/codex-secret.yaml @@ -0,0 +1,12 @@ +apiVersion: v1 +kind: Secret +metadata: + name: codex-secrets + labels: + app: openclaw +type: Opaque +stringData: + # OpenAI API key for Codex app-server authentication + OPENAI_API_KEY: "CHANGE-ME" + # Token for WebSocket auth between gateway and Codex app-server sidecar + CODEX_APP_SERVER_TOKEN: "CHANGE-ME" diff --git a/agents/openclaw/openclaw_agent/overlays/codex-harness/configmap-patch.yaml b/agents/openclaw/openclaw_agent/overlays/codex-harness/configmap-patch.yaml new file mode 100644 index 00000000..4166a7ea --- /dev/null +++ b/agents/openclaw/openclaw_agent/overlays/codex-harness/configmap-patch.yaml @@ -0,0 +1,53 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: openclaw-config +data: + openclaw.json: | + { + "gateway": { + "port": 18789, + "bind": "0.0.0.0", + "mode": "local", + "auth": { + "mode": "token" + } + }, + "channels": {}, + "agents": { + "defaults": { + "heartbeat": {}, + "model": "codex/gpt-5.4", + "embeddedHarness": { + "runtime": "codex", + "fallback": "none" + }, + "sandbox": { + "mode": "off" + } + } + }, + "plugins": { + "entries": { + "codex": { + "enabled": true, + "config": { + "appServer": { + "transport": "websocket", + "url": "ws://localhost:39175", + "authToken": "${CODEX_APP_SERVER_TOKEN}", + "requestTimeoutMs": 60000, + "approvalPolicy": "never", + "sandbox": "workspace-write" + } + } + } + } + }, + "tools": { + "deny": [ + "group:web", + "browser" + ] + } + } diff --git a/agents/openclaw/openclaw_agent/overlays/codex-harness/deployment-patch.yaml b/agents/openclaw/openclaw_agent/overlays/codex-harness/deployment-patch.yaml new file mode 100644 index 00000000..7a255605 --- /dev/null +++ b/agents/openclaw/openclaw_agent/overlays/codex-harness/deployment-patch.yaml @@ -0,0 +1,56 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: openclaw +spec: + template: + spec: + containers: + # Patch the gateway container to add Codex env vars + - name: gateway + env: + - name: OPENCLAW_GATEWAY_TOKEN + valueFrom: + secretKeyRef: + name: openclaw-secrets + key: OPENCLAW_GATEWAY_TOKEN + - name: OPENCLAW_HOME + value: /data/.openclaw + - name: CODEX_APP_SERVER_TOKEN + valueFrom: + secretKeyRef: + name: codex-secrets + key: CODEX_APP_SERVER_TOKEN + # Add Codex app-server as a sidecar + - name: codex-app-server + image: ghcr.io/openai/codex-app-server:latest + args: + - "app-server" + - "--listen" + - "ws://0.0.0.0:39175" + ports: + - containerPort: 39175 + protocol: TCP + env: + - name: OPENAI_API_KEY + valueFrom: + secretKeyRef: + name: codex-secrets + key: OPENAI_API_KEY + - name: CODEX_APP_SERVER_TOKEN + valueFrom: + secretKeyRef: + name: codex-secrets + key: CODEX_APP_SERVER_TOKEN + resources: + requests: + memory: 256Mi + cpu: 250m + limits: + memory: 1Gi + cpu: "1" + readinessProbe: + tcpSocket: + port: 39175 + initialDelaySeconds: 5 + periodSeconds: 10 diff --git a/agents/openclaw/openclaw_agent/overlays/codex-harness/kustomization.yaml b/agents/openclaw/openclaw_agent/overlays/codex-harness/kustomization.yaml new file mode 100644 index 00000000..e70075a5 --- /dev/null +++ b/agents/openclaw/openclaw_agent/overlays/codex-harness/kustomization.yaml @@ -0,0 +1,19 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +resources: + - ../../manifests + - codex-secret.yaml + +namespace: my-openclaw + +patches: + - target: + kind: ConfigMap + name: openclaw-config + path: configmap-patch.yaml + + - target: + kind: Deployment + name: openclaw + path: deployment-patch.yaml diff --git a/agents/openclaw/openclaw_agent/overlays/example/configmap-patch.yaml b/agents/openclaw/openclaw_agent/overlays/example/configmap-patch.yaml new file mode 100644 index 00000000..3c102bcf --- /dev/null +++ b/agents/openclaw/openclaw_agent/overlays/example/configmap-patch.yaml @@ -0,0 +1,53 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: openclaw-config +data: + openclaw.json: | + { + "gateway": { + "port": 18789, + "bind": "0.0.0.0", + "mode": "local", + "auth": { + "mode": "token" + } + }, + "channels": {}, + "agents": { + "defaults": { + "heartbeat": {}, + "model": "openai-compat/my-model", + "sandbox": { + "mode": "off" + } + } + }, + "models": { + "mode": "replace", + "providers": { + "openai-compat": { + "baseUrl": "https://my-vllm-endpoint.apps.my-cluster.example.com/v1", + "api": "openai-completions", + "apiKey": "not-needed", + "models": [ + { + "id": "my-model", + "name": "my-model", + "reasoning": false, + "input": ["text"], + "contextWindow": 32768, + "maxTokens": 4096 + } + ] + } + } + }, + "plugins": {}, + "tools": { + "deny": [ + "group:web", + "browser" + ] + } + } diff --git a/agents/openclaw/openclaw_agent/overlays/example/kustomization.yaml b/agents/openclaw/openclaw_agent/overlays/example/kustomization.yaml new file mode 100644 index 00000000..40da6a1b --- /dev/null +++ b/agents/openclaw/openclaw_agent/overlays/example/kustomization.yaml @@ -0,0 +1,24 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +# Point to the base manifests +resources: + - ../../manifests + +# Set your namespace +namespace: my-openclaw + +# Patch the ConfigMap with your vLLM endpoint +patches: + - target: + kind: ConfigMap + name: openclaw-config + path: configmap-patch.yaml + + - target: + kind: PersistentVolumeClaim + name: openclaw-state + patch: | + - op: replace + path: /spec/storageClassName + value: managed-csi From 45429a153ac2acc1b46f0dfa34f1c54b80c43a76 Mon Sep 17 00:00:00 2001 From: Aakanksha Duggal Date: Wed, 6 May 2026 18:06:10 -0400 Subject: [PATCH 2/5] fix: resolve markdownlint errors in OpenClaw docs Add language specifiers to fenced code blocks, blank lines around fences and lists, and wrap bare URLs in markdown link syntax. Co-Authored-By: Claude Opus 4.6 --- .../openclaw_agent/docs/codex-harness.md | 6 ++- .../docs/installer-deployment.md | 52 ++++++++++++------- .../openclaw_agent/docs/troubleshooting.md | 34 +++++++++--- 3 files changed, 64 insertions(+), 28 deletions(-) diff --git a/agents/openclaw/openclaw_agent/docs/codex-harness.md b/agents/openclaw/openclaw_agent/docs/codex-harness.md index 5d89acfb..093e91b2 100644 --- a/agents/openclaw/openclaw_agent/docs/codex-harness.md +++ b/agents/openclaw/openclaw_agent/docs/codex-harness.md @@ -4,7 +4,7 @@ The [Codex Harness](https://docs.openclaw.ai/plugins/codex-harness) is a bundled ## How It Works -``` +```text ┌──────────────────────────────────────────┐ │ Pod │ │ │ @@ -35,6 +35,7 @@ The [Codex Harness](https://docs.openclaw.ai/plugins/codex-harness) is a bundled | Self-hosted models via vLLM | `openai-compat/my-model` — no harness | The Codex harness is useful when you want: + - Native Codex thread management and compaction - Guardian-reviewed tool approvals - Model discovery through Codex's catalog @@ -167,17 +168,20 @@ The `coding-agent` routes through the Codex harness; the `general-agent` uses vL ### Codex sidecar not starting Check the sidecar logs: + ```bash oc logs deployment/openclaw -c codex-app-server -n ``` Common issues: + - Missing `OPENAI_API_KEY` — check the `codex-secrets` secret - Image pull failure — `ghcr.io/openai/codex-app-server:latest` is [not publicly available](#important-sidecar-image-availability) (returns 403). Use the direct OpenAI API fallback instead. ### Gateway can't connect to Codex If you see WebSocket connection errors in the gateway logs: + ```bash oc logs deployment/openclaw -c gateway -n | grep -i codex ``` diff --git a/agents/openclaw/openclaw_agent/docs/installer-deployment.md b/agents/openclaw/openclaw_agent/docs/installer-deployment.md index 75a5a9e4..c6b1027c 100644 --- a/agents/openclaw/openclaw_agent/docs/installer-deployment.md +++ b/agents/openclaw/openclaw_agent/docs/installer-deployment.md @@ -32,7 +32,8 @@ oc version ``` Expected output: -``` + +```text Client Version: 4.17.0 ... ``` @@ -43,7 +44,8 @@ oc whoami ``` Expected output: -``` + +```text ``` @@ -53,7 +55,8 @@ node --version ``` Expected output: -``` + +```text v22.x.x ``` @@ -63,14 +66,15 @@ oc get storageclass ``` Expected output (look for a block storage class): -``` + +```text NAME PROVISIONER RECLAIMPOLICY ... gp3-csi ebs.csi.aws.com Delete ... ``` ## Architecture Overview -``` +```text +-----------------------+ | OpenShift Route | | (TLS edge) | @@ -108,7 +112,8 @@ oc new-project -openclaw ``` Expected output: -``` + +```text Now using project "-openclaw" on server "https://api.:443". ``` @@ -121,7 +126,8 @@ npm install && npm run build && npm run dev ``` Expected output: -``` + +```text > openclaw-installer@x.x.x dev > next dev @@ -144,7 +150,8 @@ Open `http://localhost:3000`. The installer auto-detects your OpenShift cluster Click **Deploy**. The installer streams logs as it creates each resource. Expected output (installer log panel): -``` + +```text Creating ServiceAccount... ✓ Creating Secrets... ✓ Creating ConfigMap... ✓ @@ -162,7 +169,8 @@ oc get pods -n ``` Expected output: -``` + +```text NAME READY STATUS RESTARTS AGE openclaw-xxxxxxxxxx-xxxxx 2/2 Running 0 60s ``` @@ -180,12 +188,14 @@ oc get route openclaw -n -o jsonpath='{.spec.host}' Navigate to `https://` in your browser. If you see "Application is not available", the TLS termination is misconfigured. See [troubleshooting.md](troubleshooting.md#route-returns-application-is-not-available-503). **Verification:** + ```bash curl -s -o /dev/null -w "%{http_code}" -k https://$(oc get route openclaw -n -o jsonpath='{.spec.host}') ``` Expected output: -``` + +```text 403 ``` @@ -201,7 +211,8 @@ oc exec deployment/openclaw -n -c gateway -- \ ``` Expected output: -``` + +```text Device approved: ``` @@ -216,7 +227,8 @@ oc logs deployment/openclaw -c gateway -n | grep "agent model" ``` Expected output: -``` + +```text [gateway] agent model: openai-compat/gpt-oss-20b ``` @@ -250,7 +262,8 @@ oc logs deployment/openclaw -c gateway -n | grep -E "(ready|agent mo ``` Expected output: -``` + +```text [gateway] agent model: openai-compat/gpt-oss-20b [gateway] ready (4 plugins: ...; 3.xs) [heartbeat] disabled @@ -271,7 +284,8 @@ oc get pvc -n ``` Expected: -``` + +```text NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE openclaw-home-pvc Bound ... 10Gi RWO gp3-csi 5m ``` @@ -426,8 +440,8 @@ oc scale deployment/openclaw --replicas=1 -n | Resource | URL | |----------|-----| -| OpenClaw Upstream | https://github.com/openclaw/openclaw | -| openclaw-installer | https://github.com/sallyom/claw-installer | -| OpenClaw Docs | https://docs.openclaw.ai | -| OpenShift Docs | https://docs.openshift.com | -| Kustomize | https://kustomize.io | +| OpenClaw Upstream | | +| openclaw-installer | | +| OpenClaw Docs | | +| OpenShift Docs | | +| Kustomize | | diff --git a/agents/openclaw/openclaw_agent/docs/troubleshooting.md b/agents/openclaw/openclaw_agent/docs/troubleshooting.md index 34d0f554..f41bdd20 100644 --- a/agents/openclaw/openclaw_agent/docs/troubleshooting.md +++ b/agents/openclaw/openclaw_agent/docs/troubleshooting.md @@ -22,7 +22,8 @@ Common issues and solutions for OpenClaw on OpenShift. Issues are listed from mo **Cause:** The oauth-proxy uses `--http-address` (plain HTTP) by default. If the Route has `tls.termination: reencrypt`, the router expects TLS on the backend and the connection fails silently. **Symptoms:** -``` + +```text Browser: "Application is not available" curl returns: 503 ``` @@ -56,18 +57,21 @@ EOF ``` **Verification:** + ```bash curl -s -o /dev/null -w "%{http_code}" -k https:// ``` Expected output: -``` + +```text 403 ``` > A `403` is correct — it means the OAuth proxy is running and will redirect to OpenShift SSO. A `503` means the Route is still misconfigured. **Prevention:** When deploying via the installer, verify the Route TLS mode immediately after deployment: + ```bash oc get route openclaw -n -o jsonpath='{.spec.tls.termination}' # Should show: edge @@ -82,7 +86,8 @@ oc get route openclaw -n -o jsonpath='{.spec.tls.termination}' **Cause:** OpenClaw auto-generates its config on first start. If the gateway detects a provider (e.g., Anthropic) it will override the ConfigMap settings with its own defaults. This means the ConfigMap says vLLM but the gateway is actually trying to use Anthropic. **Symptoms in logs:** -``` + +```text No API key found for provider "anthropic" model fallback decision: decision=candidate_failed requested=anthropic/claude-sonnet-4-6 ``` @@ -108,12 +113,14 @@ oc rollout restart deployment/openclaw -n ``` **Verification:** + ```bash oc logs deployment/openclaw -c gateway -n | grep "agent model" ``` Expected output: -``` + +```text [gateway] agent model: openai-compat/gpt-oss-20b ``` @@ -130,7 +137,8 @@ Expected output: **Symptoms in UI:** Repeated "HEARTBEAT_OK" messages or "Agent failed before reply" errors every 30 minutes. **Symptoms in logs:** -``` + +```text [heartbeat] started ``` @@ -154,12 +162,14 @@ oc rollout restart deployment/openclaw -n ``` **Verification:** + ```bash oc logs deployment/openclaw -c gateway -n | grep heartbeat ``` Expected output: -``` + +```text [heartbeat] disabled ``` @@ -174,6 +184,7 @@ Expected output: **Cause:** The Control UI browser session needs a one-time device approval, even after OpenShift OAuth login. This is an OpenClaw security feature, not an OpenShift issue. **Fix:** + ```bash # Get the request ID from the UI prompt, then: oc exec deployment/openclaw -n -c gateway -- \ @@ -181,7 +192,8 @@ oc exec deployment/openclaw -n -c gateway -- \ ``` Expected output: -``` + +```text Device approved: ``` @@ -196,6 +208,7 @@ Alternatively, use the **Open** action from the installer's **Instances** tab **Cause:** OpenClaw auto-generates config at startup. If the config on the PVC conflicts with the ConfigMap, the gateway detects a change, overwrites the file, and triggers a process restart that kills PID 1. **Diagnostic:** + ```bash oc describe pod -l app=openclaw -n | grep -A 5 "Last State" oc logs deployment/openclaw -c gateway -n --previous @@ -204,6 +217,7 @@ oc logs deployment/openclaw -c gateway -n --previous Look for: `Config overwrite` or `Config observe anomaly` messages. **Fix:** Delete the PVC data and redeploy: + ```bash oc scale deployment/openclaw --replicas=0 -n oc delete pvc openclaw-home-pvc -n @@ -212,6 +226,7 @@ oc scale deployment/openclaw --replicas=1 -n ``` > **Warning:** This deletes all agent sessions and chat history. Back up the PVC first if needed: +> > ```bash > oc cp :/data/.openclaw /tmp/openclaw-backup -c gateway -n > ``` @@ -225,12 +240,14 @@ oc scale deployment/openclaw --replicas=1 -n **Cause:** OpenClaw's init container copies `openclaw.json` from the ConfigMap to the PVC on every pod start. However, the gateway also writes back to the same file at runtime (model discovery, plugin state). On the next restart, the init container overwrites these runtime changes with the original ConfigMap version. This produces `.clobbered.*` backup files on the PVC. **Symptoms:** + ```bash oc exec deployment/openclaw -c gateway -n -- ls /home/node/.openclaw/ # Shows: openclaw.json.clobbered.2026-04-15T14-45-55-777Z ``` **Fix:** Update the ConfigMap to include any runtime changes you want to preserve: + ```bash # Export the running config (not the ConfigMap) oc exec deployment/openclaw -c gateway -n -- \ @@ -251,7 +268,8 @@ oc create configmap openclaw-config \ **Cause:** When a browser repeatedly sends a stale device token (e.g., after PVC data was deleted and device pairings were cleared), the gateway's in-memory rate limiter kicks in after several failed attempts. Once rate-limited, all connection attempts from that client are rejected — even after fixing the underlying token issue. **Symptoms in logs:** -``` + +```text unauthorized ... reason=device_token_mismatch unauthorized ... reason=rate_limited ``` From 92496f24c8eebdbf70a64f18f046520c724990f4 Mon Sep 17 00:00:00 2001 From: Aakanksha Duggal Date: Wed, 6 May 2026 18:15:23 -0400 Subject: [PATCH 3/5] fix: address CodeRabbit review feedback on OpenClaw agent - Add pod and container securityContext (runAsNonRoot, drop ALL capabilities, disallow privilege escalation) - Fix PVC name mismatch: troubleshooting.md now references openclaw-state (matching manifests) with note about installer variant - Fix markdownlint errors in troubleshooting.md (blank lines before fenced code blocks in numbered lists) Co-Authored-By: Claude Opus 4.6 --- .../openclaw/openclaw_agent/docs/troubleshooting.md | 5 ++++- .../openclaw_agent/manifests/04-deployment.yaml | 12 ++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/agents/openclaw/openclaw_agent/docs/troubleshooting.md b/agents/openclaw/openclaw_agent/docs/troubleshooting.md index f41bdd20..047272d7 100644 --- a/agents/openclaw/openclaw_agent/docs/troubleshooting.md +++ b/agents/openclaw/openclaw_agent/docs/troubleshooting.md @@ -220,7 +220,8 @@ Look for: `Config overwrite` or `Config observe anomaly` messages. ```bash oc scale deployment/openclaw --replicas=0 -n -oc delete pvc openclaw-home-pvc -n +oc delete pvc openclaw-state -n +# If deployed via claw-installer, the PVC is named openclaw-home-pvc instead # Redeploy via installer or re-apply manifests oc scale deployment/openclaw --replicas=1 -n ``` @@ -283,11 +284,13 @@ unauthorized ... reason=rate_limited - Or: DevTools (F12) → Application → Storage → "Clear site data" 2. **Restart the pod** to reset the in-memory rate limiter: + ```bash oc rollout restart deployment/openclaw -n ``` 3. **Refresh the browser** — you should now see a fresh device pairing prompt. Approve it: + ```bash oc exec deployment/openclaw -c gateway -n -- \ openclaw devices approve diff --git a/agents/openclaw/openclaw_agent/manifests/04-deployment.yaml b/agents/openclaw/openclaw_agent/manifests/04-deployment.yaml index f0f3808c..00195f56 100644 --- a/agents/openclaw/openclaw_agent/manifests/04-deployment.yaml +++ b/agents/openclaw/openclaw_agent/manifests/04-deployment.yaml @@ -16,6 +16,8 @@ spec: labels: app: openclaw spec: + securityContext: + runAsNonRoot: true initContainers: - name: init-config image: ghcr.io/openclaw/openclaw:latest @@ -36,6 +38,11 @@ spec: limits: memory: 128Mi cpu: 100m + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL containers: - name: gateway image: ghcr.io/openclaw/openclaw:latest @@ -72,6 +79,11 @@ spec: port: 18789 initialDelaySeconds: 10 periodSeconds: 10 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL volumes: - name: config configMap: From e21d547bc7d8f2ad50a6d78f82489c5e1d9c498d Mon Sep 17 00:00:00 2001 From: Aakanksha Duggal Date: Wed, 6 May 2026 18:33:58 -0400 Subject: [PATCH 4/5] refactor: split out Codex Harness into follow-up PR Remove codex-harness overlay and docs from this PR to keep scope focused on core OpenClaw deployment. Codex Harness will be added in a follow-up PR. Co-Authored-By: Claude Opus 4.6 --- AGENTS.md | 2 +- agents/openclaw/openclaw_agent/README.md | 2 - .../openclaw_agent/docs/codex-harness.md | 195 ------------------ .../overlays/codex-harness/codex-secret.yaml | 12 -- .../codex-harness/configmap-patch.yaml | 53 ----- .../codex-harness/deployment-patch.yaml | 56 ----- .../overlays/codex-harness/kustomization.yaml | 19 -- 7 files changed, 1 insertion(+), 338 deletions(-) delete mode 100644 agents/openclaw/openclaw_agent/docs/codex-harness.md delete mode 100644 agents/openclaw/openclaw_agent/overlays/codex-harness/codex-secret.yaml delete mode 100644 agents/openclaw/openclaw_agent/overlays/codex-harness/configmap-patch.yaml delete mode 100644 agents/openclaw/openclaw_agent/overlays/codex-harness/deployment-patch.yaml delete mode 100644 agents/openclaw/openclaw_agent/overlays/codex-harness/kustomization.yaml diff --git a/AGENTS.md b/AGENTS.md index d1eff5da..4771c68c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,7 +55,7 @@ Several agents diverge significantly from the standard pattern: **a2a/langgraph_crewai_agent** - Uses `python:3.12-slim` (not UBI9), PYTHONPATH `/app` (not `/opt/app-root/src`), `charts/a2a-langgraph-crewai/` chart, `template.env` (not `.env.example`), `entrypoint.sh` (not `main.py`), Starlette (not FastAPI). No tests/ directory. -**openclaw/openclaw_agent** - Kustomize-based deployment of OpenClaw on OpenShift. Uses pre-built image (`ghcr.io/openclaw/openclaw:latest`), not a Dockerfile. Kustomize overlays for customization (model endpoint, storage class, Codex Harness sidecar). No Makefile, no FastAPI, no src/. Port 18789 (gateway). +**openclaw/openclaw_agent** - Kustomize-based deployment of OpenClaw on OpenShift. Uses pre-built image (`ghcr.io/openclaw/openclaw:latest`), not a Dockerfile. Kustomize overlays for customization (model endpoint, storage class). No Makefile, no FastAPI, no src/. Port 18789 (gateway). ## Common gotchas diff --git a/agents/openclaw/openclaw_agent/README.md b/agents/openclaw/openclaw_agent/README.md index 1443a39d..523e7bb1 100644 --- a/agents/openclaw/openclaw_agent/README.md +++ b/agents/openclaw/openclaw_agent/README.md @@ -88,7 +88,6 @@ oc apply -k manifests/ | Namespace | `overlays//kustomization.yaml` (`namespace:` field) | | Gateway token | `manifests/01-secret.yaml` | | Resource limits | Patch `manifests/04-deployment.yaml` | -| Codex Harness | `overlays/codex-harness/` (adds Codex app-server sidecar) | ## Docs @@ -97,7 +96,6 @@ oc apply -k manifests/ | [docs/installer-deployment.md](docs/installer-deployment.md) | Full deployment guide: prerequisites, validation, rollback, appendix | | [docs/troubleshooting.md](docs/troubleshooting.md) | Common issues and fixes (route 503, model override, heartbeat, config clobber) | | [docs/model-compatibility.md](docs/model-compatibility.md) | Model testing results for agentic tool-calling | -| [docs/codex-harness.md](docs/codex-harness.md) | Codex Harness plugin — sidecar deployment, mixed models, guardian approvals | ## Related Projects diff --git a/agents/openclaw/openclaw_agent/docs/codex-harness.md b/agents/openclaw/openclaw_agent/docs/codex-harness.md deleted file mode 100644 index 093e91b2..00000000 --- a/agents/openclaw/openclaw_agent/docs/codex-harness.md +++ /dev/null @@ -1,195 +0,0 @@ -# Codex Harness on OpenShift - -The [Codex Harness](https://docs.openclaw.ai/plugins/codex-harness) is a bundled OpenClaw plugin that routes agent turns through the Codex app-server instead of OpenClaw's built-in PI harness. This gives you access to Codex-native features (model discovery, thread resumption, compaction) while OpenClaw handles chat channels, session files, tools, and media delivery. - -## How It Works - -```text -┌──────────────────────────────────────────┐ -│ Pod │ -│ │ -│ ┌─────────────┐ ┌────────────────┐ │ -│ │ gateway │ ws │ codex │ │ -│ │ port 18789 ├────►│ app-server │ │ -│ │ │ │ port 39175 │ │ -│ └─────────────┘ └───────┬────────┘ │ -│ │ │ -│ │ HTTPS │ -│ ▼ │ -│ ┌──────────────┐ │ -│ │ OpenAI API │ │ -│ └──────────────┘ │ -└──────────────────────────────────────────┘ -``` - -- The **gateway** receives user messages and delegates agent turns to the Codex app-server via WebSocket (`ws://localhost:39175`) -- The **Codex app-server** runs as a sidecar in the same pod, handling the LLM interaction with OpenAI's API -- Model references like `codex/gpt-5.4` route through the harness; `openai/gpt-5.4` bypasses it entirely - -## When to Use This - -| Use case | Provider | -|----------|----------| -| Direct OpenAI API access | `openai/gpt-5.4` — no harness, OpenClaw PI runtime | -| Codex-managed sessions with thread resumption | `codex/gpt-5.4` — Codex harness | -| Self-hosted models via vLLM | `openai-compat/my-model` — no harness | - -The Codex harness is useful when you want: - -- Native Codex thread management and compaction -- Guardian-reviewed tool approvals -- Model discovery through Codex's catalog -- Mixed deployments where some agents use Codex and others use vLLM - -## Important: Sidecar Image Availability - -> **Status (2026-04-21):** The Codex app-server container image (`ghcr.io/openai/codex-app-server:latest`) is **not publicly accessible** — pulling it returns `403 Forbidden`. Until OpenAI publishes this image, the sidecar deployment described below will not work. -> -> **Workaround:** Use the direct OpenAI API instead of the Codex harness. Set the model to `openai/gpt-5.4` with `api: "openai-responses"` in your provider config, and provide your `OPENAI_API_KEY` via a Secret. This gives you GPT-5.x access without the sidecar, though you lose Codex-specific features (thread management, guardian approvals). - -## Deploy on OpenShift - -### Prerequisites - -- OpenAI API key with Codex access -- Codex app-server image (`ghcr.io/openai/codex-app-server:latest`, version 0.118.0+) — see [availability note](#important-sidecar-image-availability) above - -### Using the Kustomize overlay - -```bash -# Copy the overlay -cp -r overlays/codex-harness overlays/my-codex - -# Edit secrets with your API key -vim overlays/my-codex/codex-secret.yaml - -# Edit namespace -vim overlays/my-codex/kustomization.yaml - -# Deploy -oc apply -k overlays/my-codex -``` - -### What the overlay adds - -On top of the base manifests, the Codex overlay: - -1. **Adds a sidecar container** — `codex-app-server` listening on port 39175 -2. **Configures WebSocket transport** — gateway connects to `ws://localhost:39175` (pod-local, no network hop) -3. **Adds secrets** — `OPENAI_API_KEY` for Codex auth, `CODEX_APP_SERVER_TOKEN` for WebSocket auth between containers -4. **Updates the ConfigMap** — enables the `codex` plugin, sets `codex/gpt-5.4` as the model, configures the embedded harness - -### Resource impact - -| Container | CPU request | Memory request | Memory limit | -|-----------|-------------|----------------|--------------| -| gateway | 250m | 256Mi | 1Gi | -| codex-app-server | 250m | 256Mi | 1Gi | - -The Codex app-server is lightweight — it proxies requests to OpenAI's API, so compute happens remotely. - -## Configuration Reference - -### Model references - -| Reference | What happens | -|-----------|-------------| -| `codex/gpt-5.4` | Routes through Codex app-server harness | -| `openai/gpt-5.4` | Direct OpenAI API via OpenClaw's PI runtime | -| `openai-codex/gpt-5.4` | OpenAI with OAuth, no harness | - -### Plugin config fields - -| Field | Default | Purpose | -|-------|---------|---------| -| `transport` | `stdio` | `stdio` spawns locally; `websocket` connects to remote | -| `url` | unset | WebSocket endpoint (e.g., `ws://localhost:39175`) | -| `authToken` | unset | Bearer token for WebSocket auth | -| `requestTimeoutMs` | `60000` | Control-plane call timeout | -| `approvalPolicy` | `never` | Tool approval policy (`never`, `on-request`) | -| `sandbox` | `workspace-write` | Sandbox mode for Codex | -| `approvalsReviewer` | `user` | Use `guardian_subagent` for automated review | - -### Guardian-reviewed approvals - -For production deployments where you want automated tool approval review: - -```json -{ - "plugins": { - "entries": { - "codex": { - "enabled": true, - "config": { - "appServer": { - "transport": "websocket", - "url": "ws://localhost:39175", - "approvalPolicy": "on-request", - "approvalsReviewer": "guardian_subagent", - "sandbox": "workspace-write" - } - } - } - } - } -} -``` - -## Mixed Deployments - -You can run both Codex and vLLM models in the same OpenClaw instance using `runtime: "auto"`: - -```json -{ - "agents": { - "defaults": { - "embeddedHarness": { - "runtime": "auto" - } - }, - "list": [ - { - "id": "coding-agent", - "model": { "primary": "codex/gpt-5.4" } - }, - { - "id": "general-agent", - "model": { "primary": "openai-compat/gpt-oss-20b" } - } - ] - } -} -``` - -The `coding-agent` routes through the Codex harness; the `general-agent` uses vLLM directly. - -## Troubleshooting - -### Codex sidecar not starting - -Check the sidecar logs: - -```bash -oc logs deployment/openclaw -c codex-app-server -n -``` - -Common issues: - -- Missing `OPENAI_API_KEY` — check the `codex-secrets` secret -- Image pull failure — `ghcr.io/openai/codex-app-server:latest` is [not publicly available](#important-sidecar-image-availability) (returns 403). Use the direct OpenAI API fallback instead. - -### Gateway can't connect to Codex - -If you see WebSocket connection errors in the gateway logs: - -```bash -oc logs deployment/openclaw -c gateway -n | grep -i codex -``` - -- Ensure the sidecar is ready (`oc get pods` shows 2/2 or 3/3) -- Check that `url` in the config matches `ws://localhost:39175` -- Verify `CODEX_APP_SERVER_TOKEN` matches in both containers - -### Protocol version mismatch - -If you see "protocol rejection during handshake" errors, the Codex app-server is older than 0.118.0. Update the image tag. diff --git a/agents/openclaw/openclaw_agent/overlays/codex-harness/codex-secret.yaml b/agents/openclaw/openclaw_agent/overlays/codex-harness/codex-secret.yaml deleted file mode 100644 index 3f7cb3d1..00000000 --- a/agents/openclaw/openclaw_agent/overlays/codex-harness/codex-secret.yaml +++ /dev/null @@ -1,12 +0,0 @@ -apiVersion: v1 -kind: Secret -metadata: - name: codex-secrets - labels: - app: openclaw -type: Opaque -stringData: - # OpenAI API key for Codex app-server authentication - OPENAI_API_KEY: "CHANGE-ME" - # Token for WebSocket auth between gateway and Codex app-server sidecar - CODEX_APP_SERVER_TOKEN: "CHANGE-ME" diff --git a/agents/openclaw/openclaw_agent/overlays/codex-harness/configmap-patch.yaml b/agents/openclaw/openclaw_agent/overlays/codex-harness/configmap-patch.yaml deleted file mode 100644 index 4166a7ea..00000000 --- a/agents/openclaw/openclaw_agent/overlays/codex-harness/configmap-patch.yaml +++ /dev/null @@ -1,53 +0,0 @@ -apiVersion: v1 -kind: ConfigMap -metadata: - name: openclaw-config -data: - openclaw.json: | - { - "gateway": { - "port": 18789, - "bind": "0.0.0.0", - "mode": "local", - "auth": { - "mode": "token" - } - }, - "channels": {}, - "agents": { - "defaults": { - "heartbeat": {}, - "model": "codex/gpt-5.4", - "embeddedHarness": { - "runtime": "codex", - "fallback": "none" - }, - "sandbox": { - "mode": "off" - } - } - }, - "plugins": { - "entries": { - "codex": { - "enabled": true, - "config": { - "appServer": { - "transport": "websocket", - "url": "ws://localhost:39175", - "authToken": "${CODEX_APP_SERVER_TOKEN}", - "requestTimeoutMs": 60000, - "approvalPolicy": "never", - "sandbox": "workspace-write" - } - } - } - } - }, - "tools": { - "deny": [ - "group:web", - "browser" - ] - } - } diff --git a/agents/openclaw/openclaw_agent/overlays/codex-harness/deployment-patch.yaml b/agents/openclaw/openclaw_agent/overlays/codex-harness/deployment-patch.yaml deleted file mode 100644 index 7a255605..00000000 --- a/agents/openclaw/openclaw_agent/overlays/codex-harness/deployment-patch.yaml +++ /dev/null @@ -1,56 +0,0 @@ -apiVersion: apps/v1 -kind: Deployment -metadata: - name: openclaw -spec: - template: - spec: - containers: - # Patch the gateway container to add Codex env vars - - name: gateway - env: - - name: OPENCLAW_GATEWAY_TOKEN - valueFrom: - secretKeyRef: - name: openclaw-secrets - key: OPENCLAW_GATEWAY_TOKEN - - name: OPENCLAW_HOME - value: /data/.openclaw - - name: CODEX_APP_SERVER_TOKEN - valueFrom: - secretKeyRef: - name: codex-secrets - key: CODEX_APP_SERVER_TOKEN - # Add Codex app-server as a sidecar - - name: codex-app-server - image: ghcr.io/openai/codex-app-server:latest - args: - - "app-server" - - "--listen" - - "ws://0.0.0.0:39175" - ports: - - containerPort: 39175 - protocol: TCP - env: - - name: OPENAI_API_KEY - valueFrom: - secretKeyRef: - name: codex-secrets - key: OPENAI_API_KEY - - name: CODEX_APP_SERVER_TOKEN - valueFrom: - secretKeyRef: - name: codex-secrets - key: CODEX_APP_SERVER_TOKEN - resources: - requests: - memory: 256Mi - cpu: 250m - limits: - memory: 1Gi - cpu: "1" - readinessProbe: - tcpSocket: - port: 39175 - initialDelaySeconds: 5 - periodSeconds: 10 diff --git a/agents/openclaw/openclaw_agent/overlays/codex-harness/kustomization.yaml b/agents/openclaw/openclaw_agent/overlays/codex-harness/kustomization.yaml deleted file mode 100644 index e70075a5..00000000 --- a/agents/openclaw/openclaw_agent/overlays/codex-harness/kustomization.yaml +++ /dev/null @@ -1,19 +0,0 @@ -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization - -resources: - - ../../manifests - - codex-secret.yaml - -namespace: my-openclaw - -patches: - - target: - kind: ConfigMap - name: openclaw-config - path: configmap-patch.yaml - - - target: - kind: Deployment - name: openclaw - path: deployment-patch.yaml From ea4446ad425b1fb9641ce8698726bd07e3136b59 Mon Sep 17 00:00:00 2001 From: Aakanksha Duggal Date: Tue, 19 May 2026 09:50:13 -0400 Subject: [PATCH 5/5] refactor: move openclaw_agent to deployment dir per repo restructure Aligns with agreed repo structure: platform-based agents use agents//deployment/ for deployment manifests. Co-Authored-By: Claude Opus 4.6 --- AGENTS.md | 2 +- agents/openclaw/{openclaw_agent => deployment}/README.md | 0 .../{openclaw_agent => deployment}/docs/installer-deployment.md | 0 .../{openclaw_agent => deployment}/docs/model-compatibility.md | 0 .../{openclaw_agent => deployment}/docs/troubleshooting.md | 0 .../{openclaw_agent => deployment}/manifests/01-secret.yaml | 0 .../{openclaw_agent => deployment}/manifests/02-configmap.yaml | 0 .../{openclaw_agent => deployment}/manifests/03-pvc.yaml | 0 .../{openclaw_agent => deployment}/manifests/04-deployment.yaml | 0 .../{openclaw_agent => deployment}/manifests/05-service.yaml | 0 .../{openclaw_agent => deployment}/manifests/06-route.yaml | 0 .../{openclaw_agent => deployment}/manifests/kustomization.yaml | 0 .../overlays/example/configmap-patch.yaml | 0 .../overlays/example/kustomization.yaml | 0 14 files changed, 1 insertion(+), 1 deletion(-) rename agents/openclaw/{openclaw_agent => deployment}/README.md (100%) rename agents/openclaw/{openclaw_agent => deployment}/docs/installer-deployment.md (100%) rename agents/openclaw/{openclaw_agent => deployment}/docs/model-compatibility.md (100%) rename agents/openclaw/{openclaw_agent => deployment}/docs/troubleshooting.md (100%) rename agents/openclaw/{openclaw_agent => deployment}/manifests/01-secret.yaml (100%) rename agents/openclaw/{openclaw_agent => deployment}/manifests/02-configmap.yaml (100%) rename agents/openclaw/{openclaw_agent => deployment}/manifests/03-pvc.yaml (100%) rename agents/openclaw/{openclaw_agent => deployment}/manifests/04-deployment.yaml (100%) rename agents/openclaw/{openclaw_agent => deployment}/manifests/05-service.yaml (100%) rename agents/openclaw/{openclaw_agent => deployment}/manifests/06-route.yaml (100%) rename agents/openclaw/{openclaw_agent => deployment}/manifests/kustomization.yaml (100%) rename agents/openclaw/{openclaw_agent => deployment}/overlays/example/configmap-patch.yaml (100%) rename agents/openclaw/{openclaw_agent => deployment}/overlays/example/kustomization.yaml (100%) diff --git a/AGENTS.md b/AGENTS.md index 4771c68c..76cecb14 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,7 +55,7 @@ Several agents diverge significantly from the standard pattern: **a2a/langgraph_crewai_agent** - Uses `python:3.12-slim` (not UBI9), PYTHONPATH `/app` (not `/opt/app-root/src`), `charts/a2a-langgraph-crewai/` chart, `template.env` (not `.env.example`), `entrypoint.sh` (not `main.py`), Starlette (not FastAPI). No tests/ directory. -**openclaw/openclaw_agent** - Kustomize-based deployment of OpenClaw on OpenShift. Uses pre-built image (`ghcr.io/openclaw/openclaw:latest`), not a Dockerfile. Kustomize overlays for customization (model endpoint, storage class). No Makefile, no FastAPI, no src/. Port 18789 (gateway). +**openclaw/deployment** - Kustomize-based deployment of OpenClaw on OpenShift. Uses pre-built image (`ghcr.io/openclaw/openclaw:latest`), not a Dockerfile. Kustomize overlays for customization (model endpoint, storage class). No Makefile, no FastAPI, no src/. Port 18789 (gateway). ## Common gotchas diff --git a/agents/openclaw/openclaw_agent/README.md b/agents/openclaw/deployment/README.md similarity index 100% rename from agents/openclaw/openclaw_agent/README.md rename to agents/openclaw/deployment/README.md diff --git a/agents/openclaw/openclaw_agent/docs/installer-deployment.md b/agents/openclaw/deployment/docs/installer-deployment.md similarity index 100% rename from agents/openclaw/openclaw_agent/docs/installer-deployment.md rename to agents/openclaw/deployment/docs/installer-deployment.md diff --git a/agents/openclaw/openclaw_agent/docs/model-compatibility.md b/agents/openclaw/deployment/docs/model-compatibility.md similarity index 100% rename from agents/openclaw/openclaw_agent/docs/model-compatibility.md rename to agents/openclaw/deployment/docs/model-compatibility.md diff --git a/agents/openclaw/openclaw_agent/docs/troubleshooting.md b/agents/openclaw/deployment/docs/troubleshooting.md similarity index 100% rename from agents/openclaw/openclaw_agent/docs/troubleshooting.md rename to agents/openclaw/deployment/docs/troubleshooting.md diff --git a/agents/openclaw/openclaw_agent/manifests/01-secret.yaml b/agents/openclaw/deployment/manifests/01-secret.yaml similarity index 100% rename from agents/openclaw/openclaw_agent/manifests/01-secret.yaml rename to agents/openclaw/deployment/manifests/01-secret.yaml diff --git a/agents/openclaw/openclaw_agent/manifests/02-configmap.yaml b/agents/openclaw/deployment/manifests/02-configmap.yaml similarity index 100% rename from agents/openclaw/openclaw_agent/manifests/02-configmap.yaml rename to agents/openclaw/deployment/manifests/02-configmap.yaml diff --git a/agents/openclaw/openclaw_agent/manifests/03-pvc.yaml b/agents/openclaw/deployment/manifests/03-pvc.yaml similarity index 100% rename from agents/openclaw/openclaw_agent/manifests/03-pvc.yaml rename to agents/openclaw/deployment/manifests/03-pvc.yaml diff --git a/agents/openclaw/openclaw_agent/manifests/04-deployment.yaml b/agents/openclaw/deployment/manifests/04-deployment.yaml similarity index 100% rename from agents/openclaw/openclaw_agent/manifests/04-deployment.yaml rename to agents/openclaw/deployment/manifests/04-deployment.yaml diff --git a/agents/openclaw/openclaw_agent/manifests/05-service.yaml b/agents/openclaw/deployment/manifests/05-service.yaml similarity index 100% rename from agents/openclaw/openclaw_agent/manifests/05-service.yaml rename to agents/openclaw/deployment/manifests/05-service.yaml diff --git a/agents/openclaw/openclaw_agent/manifests/06-route.yaml b/agents/openclaw/deployment/manifests/06-route.yaml similarity index 100% rename from agents/openclaw/openclaw_agent/manifests/06-route.yaml rename to agents/openclaw/deployment/manifests/06-route.yaml diff --git a/agents/openclaw/openclaw_agent/manifests/kustomization.yaml b/agents/openclaw/deployment/manifests/kustomization.yaml similarity index 100% rename from agents/openclaw/openclaw_agent/manifests/kustomization.yaml rename to agents/openclaw/deployment/manifests/kustomization.yaml diff --git a/agents/openclaw/openclaw_agent/overlays/example/configmap-patch.yaml b/agents/openclaw/deployment/overlays/example/configmap-patch.yaml similarity index 100% rename from agents/openclaw/openclaw_agent/overlays/example/configmap-patch.yaml rename to agents/openclaw/deployment/overlays/example/configmap-patch.yaml diff --git a/agents/openclaw/openclaw_agent/overlays/example/kustomization.yaml b/agents/openclaw/deployment/overlays/example/kustomization.yaml similarity index 100% rename from agents/openclaw/openclaw_agent/overlays/example/kustomization.yaml rename to agents/openclaw/deployment/overlays/example/kustomization.yaml