Skip to content

feat(#935): add pi runtime plumbing to fleet harnesses - #965

Merged
waynesun09 merged 2 commits into
mainfrom
agent/935-pi-runtime-fleet-plumbing
Aug 23, 2026
Merged

feat(#935): add pi runtime plumbing to fleet harnesses#965
waynesun09 merged 2 commits into
mainfrom
agent/935-pi-runtime-fleet-plumbing

Conversation

@fullsend-ai-coder

@fullsend-ai-coder fullsend-ai-coder Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

C1 plumbing for the opt-in pi runtime (#935). Claude Code stays the fleet default; nothing here changes defaults.runtime.

  • Binary allowlists — add **/pi next to **/claude in profiles/fullsend-vertex-ai.yaml and every inline policy under policies/github/, policies/gitlab/, policies/jira/ (10 sites). npm-global pi is a node shebang, so **/node already covered it; the entry keeps the rows symmetrical and is needed if a standalone binary is ever installed.

  • Protected paths — add .pi/ next to .claude/ in harness/review.yaml (runner + sandbox blocks), the scripts/post-review-test.sh fixture, skills/pr-review/SKILL.md (governance list + classification glob) and agents/fix.md. Behaviour change: repos that already have a .pi/ directory will now see PRs touching it refuse auto-approval, exactly like .claude/ / .cursor/ (root-anchored prefix match in post-review.src.sh).

  • Image digest pins — bump all seven harness image: lines to the current :latest index digests, built from fullsend main by sandbox-images.yml (no release needed; the latest release v0.36.0 predates pi):

    • ghcr.io/fullsend-ai/fullsend-sandbox@sha256:9af3fba87d193bc58ae9da6c916fcc9689176eeee9247b563bc14d8acb3cc734 (triage, prioritize, retro, scribe)
    • ghcr.io/fullsend-ai/fullsend-code@sha256:9743bc7b6e451e0bcea25ae4a67e0c040c296f1fee04c08988ae80c53fafcfe6 (code, fix, review)

    Provenance: both are v0.37.0, org.opencontainers.image.revision=1f027be5 (2026-08-23); skopeo inspect shows PI_CODING_AGENT_DIR, PI_CODING_AGENT_SESSION_DIR, PI_OFFLINE, PI_SKIP_VERSION_CHECK, PI_TELEMETRY in the image env (pi 0.84.2 + vendored pi-anthropic-vertex 0.1.13 per images/sandbox/Containerfile). Diffing the Containerfiles between the old pins (revision f1da2bb) and these: the only non-pi delta is Claude Code 2.1.220 → 2.1.234; base image, Go, gitsign, lychee, gopls and ProtectAI pins are unchanged.

  • Env fileenv/gcp-vertex.env needs no change (CLOUD_ML_REGION is already exported; the vendored Vertex provider reads it first, so no GOOGLE_CLOUD_LOCATION alias).

  • Testsscripts/post-review-test.sh gains two cases against the default list: .pi/settings.json is protected; docs/.pi/notes.md is not (root-anchored prefix).

Test plan

  • bash scripts/post-review-test.sh — all pass (the harness-vs-fixture drift test runs in CI where yq is installed; script-test job)
  • All 17 changed YAML files parse; hack/lint-agent-docs passes
  • Functional tests on the new images (Functional Tests workflow, triage/review/code/fix/retro matrix)
  • C2 pilot (runtime: pi on a disposable repo) and per-agent docs — follow-ups tracked on Enable fleet agents on the pi runtime #935, not this PR

Refs #935 (C1 only — the C2 pilot and docs checkboxes stay open on the tracker; do not auto-close).

Post-script verification

  • Branch is not main/master (agent/935-pi-runtime-fleet-plumbing)
  • Secret scan passed (gitleaks — a0a2c25fac3c99c4d2614f8a110f422c019cfd1d..HEAD)
  • PR body secret scan: N/A (commit body path)

@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner August 23, 2026 12:51
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Aug 23, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 23, 2026

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 12:53 PM UTC · Ended 1:03 PM UTC

Commit: 252f28a · View workflow run →

@waynesun09
waynesun09 force-pushed the agent/935-pi-runtime-fleet-plumbing branch from 252f28a to 7a828ac Compare August 23, 2026 13:03
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 23, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 1:05 PM UTC · Completed 1:20 PM UTC

Commit: 7a828ac · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 23, 2026

Copy link
Copy Markdown

Review

Findings

Medium

  • [protected-path] All 20 protected files in this PR fall under governance paths: agents/fix.md, harness/{code,fix,prioritize,retro,review,scribe,triage}.yaml, policies/github/{prioritize,review}.yaml, policies/gitlab/{code,fix,prioritize,retro,review,triage}.yaml, policies/jira/triage.yaml, profiles/fullsend-vertex-ai.yaml, scripts/post-review-test.sh, skills/pr-review/SKILL.md. The PR links to issue Enable fleet agents on the pi runtime #935 and explains the rationale for each change category (binary allowlists, protected paths, image digest pins). Human approval is always required for protected-path changes, regardless of context.

Low

  • [permission-expansion] policies/github/review.yaml:39**/pi binary allowlist entry added to vertex_ai network policy across 10 policy/profile files, mirroring existing **/claude and **/node patterns. Correctly scoped to inference endpoints only (not forge-specific API policies). Since the pi runtime uses a node shebang, **/node already grants equivalent network access; the incremental permission expansion is negligible.
  • [permission-expansion] harness/review.yaml:52.pi/ added to REVIEW_PROTECTED_PATHS in both runner and sandbox environments, consistent with existing .claude/ and .cursor/ entries. Security-positive change expanding protection scope.
Previous run

Review

Findings

Medium

  • [protected-path] All 20 changed files fall under protected paths: agents/fix.md, harness/{code,fix,prioritize,retro,review,scribe,triage}.yaml, policies/github/{prioritize,review}.yaml, policies/gitlab/{code,fix,prioritize,retro,review,triage}.yaml, policies/jira/triage.yaml, profiles/fullsend-vertex-ai.yaml, scripts/post-review-test.sh, skills/pr-review/SKILL.md. The PR links to issue Enable fleet agents on the pi runtime #935 and explains the rationale for each change category (binary allowlists, protected paths, image digest pins). Human approval is always required for protected-path changes, regardless of context.

Low

  • [permission-expansion] policies/github/review.yaml:39**/pi binary allowlist entry added across 10 policy/profile files, mirroring existing **/claude and **/node patterns. Issue Enable fleet agents on the pi runtime #935 authorizes this expansion. Since npm-global pi runs via node shebang, **/node already grants equivalent network access; the incremental permission expansion is negligible.
  • [permission-expansion] harness/review.yaml:52.pi/ added to REVIEW_PROTECTED_PATHS in both runner and sandbox environments. Security-positive change expanding protection scope, consistent with .claude/ and .cursor/.
Previous run (2)

Review

Findings

Medium

  • [protected-path] All 20 changed files fall under protected paths: agents/fix.md, harness/{code,fix,prioritize,retro,review,scribe,triage}.yaml, policies/github/{prioritize,review}.yaml, policies/gitlab/{code,fix,prioritize,retro,review,triage}.yaml, policies/jira/triage.yaml, profiles/fullsend-vertex-ai.yaml, scripts/post-review-test.sh, skills/pr-review/SKILL.md. The PR links to issue Enable fleet agents on the pi runtime #935 and explains the rationale for each change category (binary allowlists, protected paths, image digest pins). Human approval is always required for protected-path changes, regardless of context.

Low

  • [permission-expansion] policies/github/review.yaml:39**/pi binary allowlist entry added across 10 policy/profile files, mirroring existing **/claude and **/node patterns. Since npm-global pi runs via node shebang, **/node already grants equivalent network access; the incremental permission expansion is negligible.
  • [permission-expansion] harness/review.yaml:52.pi/ added to REVIEW_PROTECTED_PATHS in both runner and sandbox environments. Security-positive change expanding protection scope, consistent with .claude/ and .cursor/.

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Aug 23, 2026
C1 plumbing for the opt-in pi runtime (issue #935):

- Binary allowlists: add `**/pi` next to `**/claude` in
  profiles/fullsend-vertex-ai.yaml and all inline policies
  under policies/github/, policies/gitlab/, and policies/jira/
- Protected paths: add `.pi/` next to `.claude/` in
  harness/review.yaml (both runner and sandbox blocks),
  scripts/post-review-test.sh (test fixture), skills/pr-review/
  SKILL.md (governance paths list and classification glob),
  and agents/fix.md (protected paths list). Repos that already
  carry a `.pi/` directory will now see changes under it refuse
  auto-approval, like `.claude/`.
- Image digest pins: bump all seven harness image: lines to the
  current `:latest` index digests (v0.37.0, fullsend revision
  1f027be, built from main by sandbox-images.yml — no release
  needed): fullsend-sandbox sha256:9af3fba8... (triage,
  prioritize, retro, scribe) and fullsend-code sha256:9743bc7b...
  (code, fix, review). Both images carry the pi install
  (PI_CODING_AGENT_DIR/PI_* in the image env); the only other
  delta vs the previous pins is Claude Code 2.1.220 -> 2.1.234.
- Env file: env/gcp-vertex.env needs no change (CLOUD_ML_REGION
  is already exported; no GOOGLE_CLOUD_LOCATION alias required)
- post-review-test.sh: cover `.pi/settings.json` against the
  default list (protected) and `docs/.pi/` (not a root prefix)

Refs #935 (C1 plumbing; the C2 pilot and docs stay open on the tracker)

Assisted-by: Claude (code, fix), Grok (review)
Signed-off-by: Wayne Sun <gsun@redhat.com>
@waynesun09
waynesun09 force-pushed the agent/935-pi-runtime-fleet-plumbing branch from 7a828ac to 90f19f8 Compare August 23, 2026 13:22
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 23, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 1:24 PM UTC · Completed 1:38 PM UTC

Commit: 90f19f8 · View workflow run →

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed via review-squad (Claude + Grok, two rounds) and fix-and-verify; CI green incl. functional tests on the new images; protected-path changes are the intended C1 scope of #935. Approving.

@waynesun09
waynesun09 added this pull request to the merge queue Aug 23, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 23, 2026
@waynesun09
waynesun09 added this pull request to the merge queue Aug 23, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 23, 2026
@waynesun09
waynesun09 added this pull request to the merge queue Aug 23, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 23, 2026
Merge-queue functional tests flaked twice on functional-tests (triage),
each time a different eval case exceeding max_turns: 30 (observed
17-31 turns across runs on the current sandbox image, median ~24).
The budget sits inside normal variance rather than signaling a real
regression, so raise the ceiling instead of retrying blind.

Signed-off-by: Wayne Sun <gsun@redhat.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 23, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 3:10 PM UTC · Completed 3:24 PM UTC

Commit: ccd5c9f · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

Comment thread policies/github/review.yaml
Comment thread harness/review.yaml

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving after folding in the eval max_turns fix (30→35) directly on this branch — the flake blocking merge-queue was eval budget, not real regression; all checks now green including functional-tests (triage).

@waynesun09
waynesun09 added this pull request to the merge queue Aug 23, 2026
Merged via the queue into main with commit 888a02e Aug 23, 2026
42 checks passed
@waynesun09
waynesun09 deleted the agent/935-pi-runtime-fleet-plumbing branch August 23, 2026 15:39
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 23, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 3:40 PM UTC · Completed 3:52 PM UTC

Commit: ccd5c9f · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #965 — pi runtime plumbing

Timeline. Issue #935 was triaged on 2026-08-22 (2 runs, blockers identified). On 2026-08-23, waynesun09 re-triaged after blockers resolved, then triggered /fs-code. The code agent completed in ~9 minutes, producing a clean 28-file PR covering binary allowlists (**/pi), protected paths (.pi/), image digest pins (7 harnesses), and 2 new test cases. The review agent ran 3 times (one per commit push, ~16 min each), correctly flagging the changes as low/medium-severity permission-expansion and protected-path modifications. waynesun09 acknowledged the review findings as intentional and approved after external review-squad verification. The merge queue dequeued the PR twice due to triage eval cases exceeding max_turns: 30; waynesun09 pushed a second commit bumping the budget to 35 across all 8 eval cases. Merged at 15:39 UTC.

What went well.

  • Code agent delivered accurate, complete C1 scope in a single pass (~9 min). No rework needed on the feature code.
  • Review agent findings were appropriate: correctly identified permission expansion as low-risk and protected-path additions as security-positive. No false positives that wasted human time.
  • The cancelled review run (due to force-push) was handled correctly by GitHub's concurrency controls.

Existing issue coverage (no new proposals needed).

  • Redundant review runs: Three full review runs (~48 min total compute) for essentially the same diff. This is extensively tracked on fullsend-ai/fullsend — issues #963 (skip dispatch when SHA already reviewed), #1356 (skip re-review when diff unchanged), #1422 (dedup rapid rebases), and #4069 (reduce dispatch frequency during rapid iterations). This PR provides additional evidence: the second commit (ccd5c9f) only changed eval annotation files (no feature code), yet triggered a full ~15 min review that produced identical findings to the previous run.

One proposal filed for standardizing eval turn-budget headroom to reduce merge-queue flakes.

Proposals filed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-review Triggers review agent dispatch requires-manual-review Review requires human judgment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant