Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/guides/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Guides for org administrators who install, configure, and manage fullsend.
Guides for developers working in repositories where fullsend is active.

- [Bugfix workflow](user/bugfix-workflow.md) — End-to-end guide to how fullsend handles a bug report from issue to merge
- [Running agents locally](user/running-agents-locally.md) — Run fullsend agents on your machine using released binaries (macOS + Linux)

## Development

Expand Down
247 changes: 247 additions & 0 deletions docs/guides/user/running-agents-locally.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,247 @@
# Running agents locally

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor — deferred] This guide documents FULLSEND_SANDBOX_ARCH and cross-platform binary resolution behavior that only exists in PR #662's branch. This creates a merge ordering dependency — please merge after #662 lands. Also consider linking issue #457 ("Support local execution of individual agents for pre-adoption evaluation") in the PR description since it directly describes this use case.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged — this PR should merge after #662 lands. The cross-platform binary resolution and FULLSEND_SANDBOX_ARCH are documented based on that PR's implementation.


This guide walks through running fullsend agents on your machine using released binaries. No Go toolchain or source checkout is needed. Both macOS and Linux are supported with Podman as the container runtime.

> For building fullsend from source or contributing to the CLI, see [Local development](../dev/local-dev.md).

## Prerequisites

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate] OpenShell is listed as a prerequisite but the guide never explains what it is, where to get it, or how to install it. The troubleshooting section mentions openshell version but a new user encountering OpenShell for the first time would be stuck. Please add a brief install step or link to OpenShell's installation docs.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — added a dedicated "Step 2: Install OpenShell" section with uv tool install instructions, version verification, and gateway binary download from GitHub releases with a direct link.


| Requirement | macOS | Linux |
|-------------|-------|-------|
| Container runtime | Podman Desktop with a running machine | Podman |
| [OpenShell](https://github.com/NVIDIA/OpenShell) | 0.0.38 | 0.0.38 |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] documentation-consistency

OpenShell version listed as 0.0.38, while docs/guides/dev/local-dev.md lists 0.0.37-dev+. Version discrepancy between the two guides could confuse users.

Suggested fix: Align both guides to the current recommended OpenShell version.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This guide targets OpenShell 0.0.38 (the version PR #919 migrates to). The local-dev.md reference to 0.0.37-dev+ is stale and should be updated separately when #919 merges.

| GCP credentials | Service account key (`Vertex AI User` role) | Same |
| GitHub PAT | `repo` scope for the target org | Same |

> **No Go toolchain required.** On macOS, the CLI automatically downloads a Linux binary for the sandbox. See [Cross-platform binary resolution](#cross-platform-binary-resolution) below.

## 1. Download the fullsend CLI

Download the latest release from [GitHub Releases](https://github.com/fullsend-ai/fullsend/releases). Pick the archive matching your platform:

| Platform | Archive |
|----------|---------|
| macOS (Apple Silicon) | `fullsend_{version}_darwin_arm64.tar.gz` |
| macOS (Intel) | `fullsend_{version}_darwin_amd64.tar.gz` |
| Linux (x86_64) | `fullsend_{version}_linux_amd64.tar.gz` |
| Linux (arm64) | `fullsend_{version}_linux_arm64.tar.gz` |

Extract and move to a directory in your PATH:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate — deferred] Consider adding a step to verify the downloaded binary checksum or signature. This is a security best practice for software distribution. GoReleaser can generate checksums files automatically — if the release includes a checksums.txt, add a verification step:

# Verify the checksum (optional but recommended)
shasum -a 256 -c checksums.txt 2>&1 | grep fullsend_0.4.0_darwin_arm64.tar.gz

Not blocking for this PR, but worth considering for a future update.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged — will add checksum verification in a follow-up once we confirm GoReleaser is generating checksums.txt in the release artifacts.


```bash
tar xzf fullsend_0.4.0_darwin_arm64.tar.gz
sudo mv fullsend_0.4.0_darwin_arm64/fullsend /usr/local/bin/

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move it to $HOME/.local/bin instead.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — updated to $HOME/.local/bin/ for both the fullsend binary and the openshell-gateway binary. No sudo needed.

```

Verify the installation:

```bash
fullsend version
```

## 2. Install OpenShell

[OpenShell](https://github.com/NVIDIA/OpenShell) provides the sandbox runtime. Install the CLI and download the gateway binary:

```bash
# Install the CLI (requires uv — https://docs.astral.sh/uv/)
uv tool install openshell==0.0.38

# Verify
openshell --version
```

Download the gateway binary from the [OpenShell releases](https://github.com/NVIDIA/OpenShell/releases/tag/v0.0.38) page. Pick the archive matching your platform and extract it:

```bash
# Example for macOS (Apple Silicon)
curl -fsSL https://github.com/NVIDIA/OpenShell/releases/download/v0.0.38/openshell-gateway-aarch64-apple-darwin.tar.gz \
-o /tmp/openshell-gateway.tar.gz
tar xzf /tmp/openshell-gateway.tar.gz -C /usr/local/bin/
```

## 3. Clone the .fullsend config directory

Each organization has a `.fullsend` config repo containing harness definitions, agent prompts, policies, scripts, and skills. Clone it to a local path:

```bash
gh repo clone <org>/.fullsend /tmp/fullsend-dot
```

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate — deferred] Nit: /tmp/fullsend.env is world-readable on most systems, which means tokens and credential paths are accessible to any local user. Consider suggesting ~/.config/fullsend/env as the example path and adding a chmod 600 note. Not blocking, but worth addressing in a follow-up.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged — will address in a follow-up. The sa-key.json already has a chmod 600 step, but the env file path could be improved.


## 4. Create an env file

Env files contain secrets and must never be committed. Keep your env file outside the repo (e.g. `/tmp/fullsend.env`).

### Core variables (all agents)

```bash
ANTHROPIC_VERTEX_PROJECT_ID=<gcp-project-with-vertex-ai>
CLOUD_ML_REGION=global
GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa-key.json
GH_TOKEN=<github-pat>
```

### Per-agent variables

Each agent requires additional variables via its harness `runner_env`. Add the ones needed for the agent you want to run:

| Variable | Agent(s) | Description |
|----------|----------|-------------|
| `GITHUB_ISSUE_URL` | triage | Full URL of the GitHub issue to triage |
| `REPO_FULL_NAME` | triage, code, review, fix | `owner/repo` of the target repository |
| `ISSUE_NUMBER` | code | Issue number the code agent should implement |
| `TARGET_BRANCH` | code, fix | Branch to base work on (e.g. `main`) |
| `PUSH_TOKEN` | code, fix | GitHub token with push access for the post-script |
| `PR_NUMBER` | review, fix | Pull request number to review or fix |
| `GITHUB_PR_URL` | review | Full URL of the pull request to review |
| `REVIEW_TOKEN` | review | GitHub token for posting review comments |

See the harness definitions in `harness/*.yaml` within your `.fullsend` config directory for the complete list per agent.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] correctness

REPO_FULL_NAME agent list (code, review, fix) differs from local-dev.md (triage, code, review, fix). Verify whether triage requires this variable and align the two guides.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This guide is correct — verified against the harness definitions. Triage's runner_env does NOT include REPO_FULL_NAME; it only has GITHUB_ISSUE_URL and GH_TOKEN. The repo is derived from the issue URL. local-dev.md should be updated to match.

### arm64 image override

On arm64 hosts (Apple Silicon, Graviton), add:

```bash
FULLSEND_SANDBOX_IMAGE=ghcr.io/fullsend-ai/fullsend-sandbox:dev
```

The default `:latest` tag is amd64-only. The `:dev` tag is a multi-arch image that includes arm64 support. On amd64 hosts this line is not needed.

## 5. Start the OpenShell gateway

The fullsend CLI requires a running OpenShell gateway — it will not start one automatically. Start the gateway with the Podman driver:

```bash
openshell-gateway \
--bind-address 127.0.0.1 \
--drivers podman \
--disable-tls \
--db-url "sqlite:/tmp/gateway.db?mode=rwc" &
```

Wait for the health check to pass, then register the gateway:

```bash
# Wait for gateway to start
for i in $(seq 1 15); do
curl -sf http://127.0.0.1:8080/healthz >/dev/null 2>&1 && break
sleep 2
done

# Register and select the local gateway
openshell gateway add http://127.0.0.1:8080 --local --name local
openshell gateway select local
```

> On macOS, make sure the Podman machine is running (`podman machine start`) before starting the gateway.

## 6. Run an agent

```bash
fullsend run <agent> \
--fullsend-dir /tmp/fullsend-dot \
--target-repo /path/to/repo \
--env-file /tmp/fullsend.env
```

The `--env-file` flag loads environment variables before the harness is parsed. It is repeatable — later files override earlier ones.

### Example: triage an issue

```bash
fullsend run triage \
--fullsend-dir /tmp/fullsend-dot \
--target-repo /path/to/repo \
--env-file /tmp/fullsend.env
```

### Example: review a pull request

```bash
fullsend run review \
--fullsend-dir /tmp/fullsend-dot \
--target-repo /path/to/repo \
--env-file /tmp/fullsend.env
```

### Example: implement an issue

```bash
fullsend run code \
--fullsend-dir /tmp/fullsend-dot \
--target-repo /path/to/repo \
--env-file /tmp/fullsend.env
```

### Example: fix review feedback on a PR

```bash
fullsend run fix \
--fullsend-dir /tmp/fullsend-dot \
--target-repo /path/to/repo \
--env-file /tmp/fullsend.env
```
Comment thread
waynesun09 marked this conversation as resolved.
Outdated

## Testing without side effects

To run agent inference without the post-script (which posts PR comments, pushes branches, or creates PRs), use `--no-post-script`:

```bash
fullsend run review \
--fullsend-dir /tmp/fullsend-dot \
--target-repo /path/to/repo \
--env-file /tmp/fullsend.env \
--no-post-script
```

The agent runs normally inside the sandbox, but the post-script is skipped. This means `PUSH_TOKEN` and `REVIEW_TOKEN` are not needed in your env file — only the core variables (`GH_TOKEN`, GCP credentials) are required.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really true?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes — verified against the code. PUSH_TOKEN and REVIEW_TOKEN are in the harness runner_env, which is only consumed by the post-script runner. With --no-post-script, the post-script is skipped entirely, so those variables are never read. Only GH_TOKEN (used by sandbox env files for repo cloning) and GCP credentials are needed.


## Cross-platform binary resolution

The pre-agent security scan runs `fullsend scan context` inside the Linux sandbox. This requires a Linux binary, even when your host is macOS.

On **Linux**, the CLI copies its own executable into the sandbox — no extra steps needed.

On **macOS**, the CLI automatically obtains a Linux binary using the following priority:

| Priority | Strategy | When it applies |
|----------|----------|-----------------|
| 1 | `--fullsend-binary <path>` | Always used if provided — skips auto-resolution |
| 2 | Download from GitHub Release | CLI version matches a release (e.g. `0.4.0`) |
| 3 | Cross-compile from source | Go toolchain available, run from module root |
| 4 | Download latest release | Fallback when cross-compilation unavailable |

When using a released binary (this guide's workflow), **strategy 2 applies automatically** — the CLI downloads the matching Linux release. No Go toolchain or manual steps needed.

## Platform notes

### macOS

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[important] The prerequisites say Podman is required with OpenShell 0.0.37-dev+, but this section casually references Docker as if it works ("if using Docker instead of Podman..."). This sends mixed signals — a user with only Docker would think it's supported based on this note, but the prerequisites say otherwise.

Please either:

  1. Remove the Docker reference and state explicitly that Podman is the only supported runtime, or
  2. If Docker genuinely works, list it as an alternative in the prerequisites table.

Also, the guide never explains why OpenShell 0.0.37-dev+ is required (it's the version that adds Podman support). A sentence explaining this would help users understand whether they can use an older version with Docker.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — the Docker reference was removed in an earlier update. The guide now exclusively references Podman as the container runtime (prerequisites table, gateway setup, platform notes, troubleshooting). No Docker mentions remain.


- **Podman machine**: ensure the Podman machine is running (`podman machine start`) before invoking fullsend. The CLI does not start it automatically.
- **Podman host-gateway**: if sandbox creation fails with `unable to replace "host-gateway"`, set `host_containers_internal_ip = "192.168.127.254"` under `[containers]` in `~/.config/containers/containers.conf` and restart the Podman machine.
- **Architecture mismatch**: if your sandbox image uses a different CPU architecture than the host (e.g. amd64 image on an arm64 Mac via QEMU emulation), set `FULLSEND_SANDBOX_ARCH=amd64` so the CLI downloads the correct binary. This is not needed in the typical setup where the Podman VM matches the host arch.

### Linux

- **Rootless Podman**: Podman runs rootless by default. Ensure your user has subuids/subgids configured (`grep $USER /etc/subuid`). If not, run `sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 $USER && podman system migrate`.
- **SELinux**: on Fedora/RHEL, bind-mounted volumes may need the `:z` suffix for standalone `podman run`. OpenShell handles this automatically.

## Troubleshooting

**Sandbox creation fails immediately**
- Check that `podman machine start` has been run (macOS only)
- Verify OpenShell is installed: `openshell --version`
- Verify the gateway is running: `openshell gateway list`

**`Syntax error: "(" unexpected` inside sandbox**
- The macOS Mach-O binary was injected instead of a Linux ELF. Update to fullsend 0.4.0+ which auto-resolves the correct binary, or provide one explicitly with `--fullsend-binary`

**Agent fails with missing environment variable**
- Check your env file contains all variables listed in the agent's harness YAML (`harness/<agent>.yaml` in the `.fullsend` config directory)

**arm64 sandbox image pull fails**
- The default `:latest` tag is amd64-only. Add `FULLSEND_SANDBOX_IMAGE=ghcr.io/fullsend-ai/fullsend-sandbox:dev` to your env file

**`unable to replace "host-gateway"` on macOS**
- Set `host_containers_internal_ip = "192.168.127.254"` under `[containers]` in `~/.config/containers/containers.conf` and restart the Podman machine

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor — deferred] Consider adding a "Cleanup" or "Maintenance" section covering:

  • How to remove stopped sandbox containers (podman ps -a, podman rm)
  • When/how to update the local .fullsend config directory clone
  • How to clean up temporary artifacts

Users following this guide might accumulate containers or wonder about keeping their config up-to-date. Not urgent, but would round out the guide.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged — cleanup/maintenance section is a good addition for a follow-up. OpenShell may already handle container cleanup, but documenting podman ps -a / podman rm and config repo updates would help.

Loading