docs(#6485): add pi runtime local-run guide section - #6508
Conversation
Add a new "Run a minimal agent on the pi runtime" section to the running-agents-locally guide. The section documents the working directory layout, config.yaml requirements, harness composition (host_files, OpenShell profiles/providers for Vertex egress), run command, expected output, artifacts, runtime knobs, security hooks, platform-specific notes, and common troubleshooting scenarios. Key points from real-world verification: - A bare config.yaml + harness/ is not sufficient; the harness must include host_files for GCP credentials and reference OpenShell egress profiles for Vertex - config.yaml must both register the agent and set defaults.runtime: pi - --env-file sets runner env only; sandbox env comes from the harness via env.sandbox and host_files - --debug is an optional-value flag requiring = syntax - A stale cached :latest image fails at pi preflight, not with a clear "image is old" message Also adds a cross-reference from docs/runtimes.md Pi-specific known constraints section to the new guide section. Closes #6485
Site previewPreview: https://57928e39-site.fullsend-ai.workers.dev Commit: |
|
🤖 Finished Review · ✅ Success · Started 12:51 AM UTC · Completed 1:06 AM UTC Commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
ReviewFindingsLow
Previous runReviewFindingsLow
Previous run (2)ReviewFindingsMedium
Low
|
![fullsend-ai-review[bot]](https://avatars.githubusercontent.com/in/3478501?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
fullsend-ai-review
Bot
added
the
requires-manual-review
The section promised minimal contents for `policies/`, `profiles/` and `providers/` and then never gave them, so a reader following it without an agents clone stops at the first missing file. All three are short; they are now included, with the reason each matters — the policy's read_only prefixes (why the pi extension moved out of /opt, #6504), the egress allowlist (without it pi reports model-not-found rather than a network error), and the provider binding. Also: drop `--keep-sandbox` from the first command a reader runs and explain it separately, since kept sandboxes are not cleaned up; and give the example agent the `name:`/`description:` frontmatter the fleet agents carry. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
waynesun09
left a comment
waynesun09
left a comment
There was a problem hiding this comment.
Reviewed against the run this documents — I am the one who worked the recipe out, so I checked the guide's claims rather than its prose.
Verified correct: both tools: forms parse (piToolSpecs accepts a comma string or a YAML list), model: in agent frontmatter is real (the fleet's triage.md uses it), analyze-transcript.py summary <file> is a valid invocation, the env/gcp-vertex.env contents line up with the host_files destination (GOOGLE_APPLICATION_CREDENTIALS=/tmp/.gcp-credentials.json matches where the key is copied), and all three troubleshooting entries are real failures I hit in that order.
Four things I pushed on top (26f7b0ae, fast-forward):
- The section promised minimal contents for
policies/,profiles/andproviders/and then never gave them — a reader without an agents clone stops at the first missing file. All three are short, so they are now inline, each with the reason it matters: the policy'sread_onlyprefixes (which is why the pi extension had to move out of/opt, #6504), the egress allowlist (without it pi reports model-not-found rather than a network error — a misleading symptom worth naming), and the provider binding. --keep-sandboxwas in the first command a reader runs. Kept sandboxes are not cleaned up, so that teaches a leak; moved to a note after the success output.- The example agent frontmatter had no
name:/description:, unlike every fleet agent. Optional in the parser, but the example is what people copy. Closes #6485→Refs #6485. The issue also asks for macOS verification, which has not happened. The Linux half is verified end to end; closing the issue on a Linux-only result would bury the remaining half.
Approving with those applied.
waynesun09
left a comment
waynesun09
left a comment
There was a problem hiding this comment.
Reviewed against the run this documents. Verified: both tools: forms parse (piToolSpecs takes a comma string or a YAML list), model: in agent frontmatter is real, analyze-transcript.py summary <file> is valid, the env/gcp-vertex.env values line up with the host_files destination, and all three troubleshooting entries are failures I hit in that order.
Four fixes pushed on top (26f7b0ae, fast-forward):
- The section promised minimal contents for
policies/,profiles/andproviders/and never gave them — a reader without an agents clone stops at the first missing file. All three are short, so they are inline now, each with why it matters: the policy'sread_onlyprefixes (why the pi extension moved out of/opt, #6504), the egress allowlist (without it pi reports model-not-found rather than a network error), and the provider binding. --keep-sandboxwas in the first command a reader runs; kept sandboxes are not cleaned up, so that teaches a leak. Moved to a note after the success output.- The example agent frontmatter lacked
name:/description:unlike every fleet agent — optional in the parser, but examples are what people copy. Closes #6485→Refs #6485: the issue also asks for macOS verification, which has not happened. Closing on a Linux-only result would bury the remaining half.
|
🤖 Review · Commit: |
…g id
The review is right that "registering only the harness file causes
resolveAgentSource to fail" reads as though a missing `defaults.runtime`
is what fails, which it is not. The two requirements fail in different
places, and the difference matters to anyone debugging:
- no `agents:` entry — `resolveAgentSource` fails outright with
`no config and agents-repo fallback unavailable` (observed)
- no `defaults.runtime: pi` — the run *succeeds* on the default claude
runtime and pi is never started; the tell is `runtime: selected "claude"`
The second is the more dangerous one to leave undocumented, since nothing
errors.
Also drops `{#troubleshooting-pi}`: no other file under docs/ uses a custom
heading id, so the heading is renamed to produce the auto-slug instead.
Assisted-by: Claude
Signed-off-by: Wayne Sun <gsun@redhat.com>
|
🤖 Finished Review · ✅ Success · Started 1:47 AM UTC · Completed 2:04 AM UTC Commit: |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
fullsend-ai-review
Bot
added
ready-for-merge
The pi walkthrough keeps its platform notes inline so the section reads start to finish, but a reader who goes to '## Platform notes' for platform-specific detail would not find them. Adds a pointer there rather than splitting the pi section. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
waynesun09
left a comment
waynesun09
left a comment
There was a problem hiding this comment.
Re-approving after the review round: 8800b3bd corrects the two config failure modes and drops the custom heading id, 226a48b0 adds the pointer from ## Platform notes. All three review threads resolved.
Validating the guide's own recipe on macOS now (arm64, macOS 26.5.2) — following it as written, from a clean directory, which is the real test of whether the supporting-file contents are complete. Will report before this merges.
|
🤖 Review · Commit: |
Validated on darwin/arm64 (macOS 26.5.2): sandbox creation, pi bootstrap and preflight, Vertex extension load and model-id translation all work. The run stopped at the inference call because the GCP project is VPN-restricted from that host. Documents the symptom, because it is misleading: 'Could not load the default credentials' names credentials, but the same text appears when the token endpoint is unreachable. Turns: 1 with zero tokens distinguishes 'pi started and the call never completed' from a genuine credential problem. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
Vertex project access from that host is environment-specific and not something the guide should carry. The macOS note keeps only what is verified there: sandbox creation, pi bootstrap and preflight, Vertex extension load, and model-id translation. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
|
🤖 Review · Commit: |
|
🤖 Review · Commit: |
Re-validated by rebuilding the example from this section's own code blocks and running it: all seven files extract cleanly and the run reaches 3 turns, so the layout and file contents are complete and correct. Two things the section got wrong for a reader following it literally: - The artifact tree promised security/findings.jsonl, which a clean run does not create — the directory is empty unless a hook reports something. It also omitted the per-run sandbox-named subdirectory, so the paths shown did not exist as written, and left out output.jsonl and run-telemetry.jsonl. - The analyze-transcript path was missing that same subdirectory, and the command showed no output, so a reader could not tell whether it worked. Adds the real output and names the other two useful subcommands. The sample run output is refreshed from the validation run. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
|
🤖 Review · Commit: |
The harness declares role: triage but the example wrote
{"target_branch": "main"}, which is not a triage result — the schema
requires action/reasoning/comment and forbids extra keys. Harmless under
--no-post-script, which this example passes, but it teaches a shape that
fails the moment anyone drops the flag or reuses the harness.
The agent now writes a schema-valid triage result, with a line explaining
why it looks like that. Re-validated end to end: the example rebuilt from
these code blocks runs to 2 turns and the file it produces has exactly the
three required keys and no extras.
Assisted-by: Claude
Signed-off-by: Wayne Sun <gsun@redhat.com>
|
🤖 Finished Review · ✅ Success · Started 2:35 AM UTC · Completed 2:47 AM UTC Commit: |
|
🤖 Finished Retro · ✅ Success · Started 2:51 AM UTC · Completed 3:02 AM UTC Commit: |
Retro: PR #6508 — pi runtime local-run guideWorkflow: Issue #6485 → triage (3 runs) → code agent (run Agents repo: What went well
What could go better
Existing issue evidence
Proposals filed
|
1 participant
Summary
Adds a new "Run a minimal agent on the pi runtime" section to
docs/guides/user/running-agents-locally.md, documenting how to run a minimal pi agent locally without cloning the fleet repo. Also adds a cross-reference fromdocs/runtimes.mdPi-specific known constraints section.Changes
docs/guides/user/running-agents-locally.md— new section covering:PI_VERSION, GCP credentials)--fullsend-dirdirectory layout with all required files (config.yaml, harness, agent definition, policies, profiles, providers, env)config.yaml, harness YAML, agent.md, and env file with explanationsmetrics.json, session transcripts,pi-debug.log)FULLSEND_PI_MODEL,FULLSEND_PI_PROVIDER,FULLSEND_PI_BASH_ALLOWLIST)docs/runtimes.md— adds a callout linking to the new guide section from the Pi-specific known constraints headingTesting
lycheelink check passes (all internal links resolve, including fragment anchors)lint-docs-linkspasses (no docs links escape the repo root)Refs #6485 — deliberately not
Closes.Linux: verified end to end — a real
fullsend runon Fedora reached 2 turns on Vertex, wroteagent-result.json, produced a pi transcript with tool calls, and recorded"runtime": "pi"inmetrics.json.macOS: verified on
darwin/arm64(macOS 26.5.2) through sandbox creation, pi bootstrap and preflight, Vertex extension load from/usr/local/share/pi-extensions, and model-id translation. The inference call was not exercised from that host — Vertex project access there is environment-specific — so the macOS half of #6485 stays open and the issue stays open with it.One fix the macOS attempt produced: the example
env/gcp-vertex.envused bareKEY=valuelines. The sandbox sources.env.d/*.envwith a plain.and noset -a(internal/cli/run.go), so those become shell variables pi never inherits — present in the sandbox, no effect. The fleet's own env file usesexportfor exactly this reason. Fixed, along with the*.envglob requirement and the sandbox-vs-runner meaning ofGOOGLE_APPLICATION_CREDENTIALS.Post-script verification
agent/6485-pi-runtime-local-guide)9a184a9ee309d767f0c12131b158f15f9d89fa57..HEAD)