Add fullsend run agent command and experiment for test it

experiments/runner-hello-world/.fullsend/agents/hello-world.md

@@ -0,0 +1,14 @@
+---
+name: hello-world
+description: A minimal agent that runs a tool and summarizes the repository code.
+skills:
+  - hello-world-summary
+tools: Bash(hello-world-bin)
+model: sonnet
+---
+
+You are a minimal test agent. Your job is to:
+
+1. Run the `hello-world-bin` tool — this writes `output/hello-world.md`
+2. Explore the repository in the current path
+3. Use the `hello-world-summary` skill to write a summary of the repository to `output/summary.md`

experiments/runner-hello-world/.fullsend/env/gcp-vertex.env

@@ -0,0 +1,4 @@
+export CLAUDE_CODE_USE_VERTEX=1
+export ANTHROPIC_VERTEX_PROJECT_ID=${ANTHROPIC_VERTEX_PROJECT_ID}
+export CLOUD_ML_REGION=${CLOUD_ML_REGION}
+export GOOGLE_APPLICATION_CREDENTIALS=/tmp/workspace/.gcp-credentials.json

experiments/runner-hello-world/.fullsend/harness/hello-world.yaml

@@ -0,0 +1,24 @@
+# harness/hello-world.yaml
+agent: agents/hello-world.md
+model: sonnet
+image: quay.io/manonru/fullsend-exp:latest
+policy: policies/hello-world.yaml
+
+host_files:
+  - src: env/gcp-vertex.env
+    dest: /tmp/workspace/.env.d/gcp-vertex.env
+    expand: true
+  - src: ${GOOGLE_APPLICATION_CREDENTIALS}
+    dest: /tmp/workspace/.gcp-credentials.json
+
+skills:
+  - skills/hello-world-summary
+
+validation_loop:
+  script: scripts/validate-output.sh
+  max_iterations: 3
+
+runner_env:
+  VALIDATION_EXPECTED_FAILURES: "1"
+
+timeout_minutes: 5

experiments/runner-hello-world/.fullsend/policies/hello-world.yaml

@@ -0,0 +1,28 @@
+version: 1
+
+# Minimal policy for the hello-world agent.
+# Only allows outbound access to Google Cloud APIs (Vertex AI authentication).
+
+filesystem_policy:
+  include_workdir: true
+  read_only: [/usr, /lib, /proc, /dev/urandom, /app, /etc, /var/log]
+  read_write: [/sandbox, /tmp, /dev/null]
+landlock:
+  compatibility: best_effort
+process:
+  run_as_user: sandbox
+  run_as_group: sandbox
+
+network_policies:
+  vertex_ai:
+    name: vertex-ai
+    endpoints:
+      - host: "*.googleapis.com"
+        port: 443
+        protocol: tcp
+        enforcement: enforce
+        access: allow
+    binaries:
+      - path: "**/curl"
+      - path: "**/claude"
+      - path: "**/node"

experiments/runner-hello-world/.fullsend/scripts/validate-output.sh

@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+HELLO_FILE="output/hello-world.md"
+SUMMARY_FILE="output/summary.md"
+
+# Counter-based failure for testing retry logic.
+# VALIDATION_EXPECTED_FAILURES controls how many times to fail before passing.
+# Counter file lives in FULLSEND_RUN_DIR so it persists across iterations.
+EXPECTED_FAILURES="${VALIDATION_EXPECTED_FAILURES:-0}"
+COUNTER_FILE="${FULLSEND_RUN_DIR:-.}/.validation-counter"
+
+if [ "$EXPECTED_FAILURES" -gt 0 ]; then
+  COUNT=0
+  if [ -f "$COUNTER_FILE" ]; then
+    COUNT=$(cat "$COUNTER_FILE")
+  fi
+  COUNT=$((COUNT + 1))
+  echo "$COUNT" > "$COUNTER_FILE"
+
+  if [ "$COUNT" -le "$EXPECTED_FAILURES" ]; then
+    echo "FAIL: deliberate failure $COUNT of $EXPECTED_FAILURES (testing retry)"
+    exit 1
+  fi
+fi
+
+# Validate hello-world.md exists and contains expected output.
+if [ ! -f "$HELLO_FILE" ]; then
+  echo "FAIL: $HELLO_FILE not found"
+  exit 1
+fi
+
+if ! grep -q "Hello world from repo" "$HELLO_FILE"; then
+  echo "FAIL: $HELLO_FILE missing expected 'Hello world from repo' line"
+  exit 1
+fi
+
+# Validate summary.md exists.
+if [ ! -f "$SUMMARY_FILE" ]; then
+  echo "FAIL: $SUMMARY_FILE not found"
+  exit 1
+fi
+
+echo "PASS: output validated"
+exit 0

experiments/runner-hello-world/.fullsend/skills/hello-world-summary/SKILL.md

@@ -0,0 +1,15 @@
+---
+name: hello-world-summary
+description: Explore the repository and produce a summary of its code
+allowed-tools: Read, Glob, Grep
+---
+
+Explore the repository in the current directory and produce a summary.
+
+The summary should include:
+- The repository name
+- A brief description of what the repository contains
+- The main languages and frameworks used
+- A list of the top-level directories and their purpose
+
+Write the summary to `$FULLSEND_OUTPUT_DIR/summary.md` (the `FULLSEND_OUTPUT_DIR` environment variable points to the output directory).

experiments/runner-hello-world/HOW_TO.md

@@ -0,0 +1,134 @@
+# How to run the experiment
+
+## Two-repo model
+
+This experiment uses a two-repo setup that mirrors the production layout:
+
+- **`.fullsend` repo** (`test-fullsend/.fullsend`): Contains the harness definition, agents, skills, env files, policies, scripts, and the GitHub Actions workflow. This is where the workflow runs.
+- **Target repo** (`test-fullsend/test-repo`): The codebase the agent analyzes. Checked out by the workflow and passed to the CLI via `--target-repo`.
+
+The `run-experiment.sh` script syncs the `.fullsend/` directory and workflow to the `.fullsend` repo, then triggers the workflow with the target repo as an input.
+
+## Requirements
+
+### Local (to run the experiment)
+
+- **Go toolchain** (1.23+)
+- **gh CLI** authenticated with access to the fullsend fork and both repos in the target org
+- **podman** (for building and pushing the container image)
+- **rsync**
+- A local clone of the `.fullsend` repo (see below)
+
+### Repos
+
+The default setup uses the `test-fullsend` org with two repos:
+
+- `test-fullsend/.fullsend` — harness and workflow (secrets configured here)
+- `test-fullsend/test-repo` — target codebase for the agent
+
+The `.fullsend` repo needs **GitHub secrets** configured (see [Setting up GCP secrets](#setting-up-gcp-secrets) below) and a **GitHub release** on your fullsend fork (used to distribute the binary to the runner).
+
+### Setting up GCP secrets
+
+The experiment uses Claude Code via Vertex AI, which requires a GCP project with the Vertex AI API enabled and a service account key.
+
+**If you already use Claude Code via Vertex AI locally** (i.e. `ANTHROPIC_VERTEX_PROJECT_ID` and `CLOUD_ML_REGION` are set in your environment), you already have a GCP project with the Vertex AI API enabled — skip step 1 and reuse your project ID and region for the secrets in step 4:
+
+```bash
+gh secret set GCP_PROJECT --repo your-org/.fullsend --body "${ANTHROPIC_VERTEX_PROJECT_ID}"
+gh secret set GCP_REGION --repo your-org/.fullsend --body "${CLOUD_ML_REGION}"
+```
+
+You still need a service account key for CI (steps 2-4), since the workflow can't use interactive authentication like `gcloud auth application-default login`.
+
+**If you need to set up Vertex AI from scratch**, follow all steps below:
+
+1. **Create or select a GCP project** with the [Vertex AI API](https://console.cloud.google.com/apis/library/aiplatform.googleapis.com) enabled.
+
+2. **Create a service account** with the `Vertex AI User` role:
+   ```bash
+   gcloud iam service-accounts create fullsend-runner \
+     --display-name="Fullsend Runner" \
+     --project=${ANTHROPIC_VERTEX_PROJECT_ID}
+
+   gcloud projects add-iam-policy-binding ${ANTHROPIC_VERTEX_PROJECT_ID} \
+     --member="serviceAccount:fullsend-runner@${ANTHROPIC_VERTEX_PROJECT_ID}.iam.gserviceaccount.com" \
+     --role="roles/aiplatform.user" \
+     --condition=None
+   ```
+
+3. **Create and download a JSON key**:
+   ```bash
+   gcloud iam service-accounts keys create /tmp/sa-key.json \
+     --iam-account=fullsend-runner@${ANTHROPIC_VERTEX_PROJECT_ID}.iam.gserviceaccount.com
+   ```
+
+4. **Set the secrets on the `.fullsend` repo** (where the workflow runs):
+   ```bash
+   gh secret set GCP_SA_KEY --repo your-org/.fullsend < /tmp/sa-key.json
+   gh secret set GCP_PROJECT --repo your-org/.fullsend --body "${ANTHROPIC_VERTEX_PROJECT_ID}"
+   gh secret set GCP_REGION --repo your-org/.fullsend --body "${CLOUD_ML_REGION}"
+   ```
+
+5. **Delete the local key file** (it's now stored as a GitHub secret):
+   ```bash
+   rm /tmp/sa-key.json
+   ```
+
+Available regions for Claude on Vertex AI include `us-east5`, `europe-west1`, and `asia-southeast1`. Check the [Vertex AI documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#regions) for the latest list.
+
+### GitHub Actions runner
+
+The workflow installs these automatically:
+
+- **fullsend** binary (from a GitHub release)
+- **OpenShell** CLI
+
+Claude Code and experiment tool binaries are pre-installed in the container image (`quay.io/manonru/fullsend-exp`), which the sandbox is created from via `--from`.
+
+## Quick start (using defaults)
+
+```bash
+# Clone the .fullsend repo (one-time setup)
+git clone git@github.com:test-fullsend/.fullsend.git /tmp/dot-fullsend
+
+# Run the experiment (builds image, pushes to quay.io, syncs, triggers workflow)
+./experiments/runner-hello-world/experiment/run-experiment.sh
+```
+
+The script will print the workflow run URL. You can watch it with:
+
+```bash
+gh run watch <RUN_ID> --repo test-fullsend/.fullsend
+```
+
+## Using a different org
+
+To run against your own org, edit the variables at the top of `experiment/run-experiment.sh`:
+
+```bash
+FULLSEND_REPO="/tmp/your-dot-fullsend"    # Local clone of your .fullsend repo
+RELEASE_REPO="your-user/fullsend"         # Where to upload the fullsend binary
+RELEASE_TAG="runner-hello-world-dev"       # Release tag name
+WORKFLOW_REPO="your-org/.fullsend"         # Where to trigger the workflow
+WORKFLOW_FILE="hello-world.yml"            # Workflow file name
+TARGET_REPO="your-org/your-target-repo"    # Target repo for the agent
+IMAGE_REPO="quay.io/your-user/your-image"  # Container image registry
+```
+
+Then update the workflow file (`experiment/workflow/hello-world.yml`) to point the fullsend install step at your release:
+
+```yaml
+- name: Install fullsend
+  run: |
+    curl -LsSf https://github.com/your-user/fullsend/releases/download/runner-hello-world-dev/fullsend_dev_linux_amd64.tar.gz -o /tmp/fullsend.tar.gz
+    sudo tar xzf /tmp/fullsend.tar.gz -C /usr/local/bin/
+```
+
+Steps:
+
+1. Create two repos in your org: `.fullsend` and a target repo with some content
+2. Create a GitHub release on your fullsend fork: `gh release create runner-hello-world-dev --repo your-user/fullsend --title "Dev" --notes "Dev build"`
+3. Set the required secrets on the `.fullsend` repo (see [Setting up GCP secrets](#setting-up-gcp-secrets))
+4. Clone the `.fullsend` repo locally: `git clone git@github.com:your-org/.fullsend.git /tmp/your-dot-fullsend`
+5. Run `./experiments/runner-hello-world/experiment/run-experiment.sh`

experiments/runner-hello-world/HOW_TO_LOCAL.md

@@ -0,0 +1,122 @@
+# How to run `fullsend run` locally
+
+This guide explains how to run the `fullsend run` CLI command on your local machine against a target repository.
+
+## Prerequisites
+
+- **Go toolchain** (1.23+)
+- **OpenShell** installed and in PATH
+- **Docker** (required by OpenShell — Podman is not supported)
+- **GCP credentials** for Vertex AI (if using the hello-world experiment)
+
+## Install Docker
+
+OpenShell requires Docker. Podman is not supported. Install Docker CE and make sure the daemon is running:
+
+```bash
+# Fedora
+sudo dnf config-manager addrepo --from-repofile=https://download.docker.com/linux/fedora/docker-ce.repo
+sudo dnf install -y docker-ce docker-ce-cli containerd.io
+
+# Start the daemon
+sudo systemctl start docker
+sudo systemctl enable docker
+
+# Allow your user to run docker without sudo
+sudo usermod -aG docker $USER
+newgrp docker
+
+# Verify
+docker info
+```
+
+## Install OpenShell
+
+```bash
+curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/v0.0.30/install.sh | sh
+openshell --version
+```
+
+## Build fullsend
+
+From the repository root:
+
+```bash
+go build -o ~/.local/bin/fullsend ./cmd/fullsend/
+fullsend --version
+```
+
+## Required environment variables
+
+The hello-world experiment requires these environment variables on the host:
+
+| Variable | Purpose |
+|----------|---------|
+| `ANTHROPIC_VERTEX_PROJECT_ID` | GCP project with Vertex AI API enabled |
+| `CLOUD_ML_REGION` | GCP region (e.g. `us-east5`, `europe-west1`) |
+| `GOOGLE_APPLICATION_CREDENTIALS` | Path to GCP service account key JSON |
+
+If you already use Claude Code via Vertex AI locally, `ANTHROPIC_VERTEX_PROJECT_ID` and `CLOUD_ML_REGION` are likely already set. For `GOOGLE_APPLICATION_CREDENTIALS`, you can use a service account key file or the Application Default Credentials from gcloud:
+
+```bash
+gcloud auth application-default login
+export GOOGLE_APPLICATION_CREDENTIALS=~/.config/gcloud/application_default_credentials.json
+```
+
+## Pre-pull the sandbox image
+
+The first sandbox creation will time out if the image hasn't been pulled yet. Pull it in advance:
+
+```bash
+docker pull quay.io/manonru/fullsend-exp:latest
+```
+
+## Running the agent
+
+```bash
+fullsend run hello-world \
+  --fullsend-dir /path/to/experiments/runner-hello-world/.fullsend \
+  --target-repo /path/to/your-repo \
+  --output-dir /tmp/fullsend-output
+```
+
+### CLI flags
+
+| Flag | Required | Default | Purpose |
+|------|----------|---------|---------|
+| `--fullsend-dir` | yes | | Base directory containing the `.fullsend` layout |
+| `--target-repo` | yes | | Path to the target repository |
+| `--output-dir` | no | `/tmp/fullsend` | Base directory for run output (per-invocation subdirectories are created under it) |
+
+## Inspecting results
+
+After a run, output is organized under the `--output-dir` directory:
+
+```
+/tmp/fullsend-output/
+  agent-hello-world-<pid>-<timestamp>/
+    iteration-1/
+      output/          # Files the agent produced (hello-world.md, summary.md)
+      transcripts/     # Claude transcript .jsonl files
+    iteration-2/       # Only if validation retried
+      output/
+      transcripts/
+```
+
+The CLI prints the full run directory path at the end:
+
+```
+Run directory    /tmp/fullsend-output/agent-hello-world-12345-1713200000
+Agent exit code  0
+Agent runs       1
+Validation       passed
+```
+
+## Troubleshooting
+
+- **"openshell not found in PATH"**: Install OpenShell (see above).
+- **"sandbox not ready after 1m0s"**: The sandbox image hasn't been pulled yet. Run `docker pull quay.io/manonru/fullsend-exp:latest` first (see above).
+- **"Docker socket exists but the daemon is not responding"**: Run `sudo systemctl start docker`.
+- **"permission denied while trying to connect to the Docker daemon"**: Your user is not in the `docker` group. Run `sudo usermod -aG docker $USER` and start a new shell session.
+- **"host variable X is not set"**: A required environment variable is missing. Check the table above.
+- **Validation failures**: Check `iteration-N/output/` for the agent's output files. The validation script (`scripts/validate-output.sh`) runs on the host and checks for expected files.