Skip to content

feat(#6689)!: run GPT on pi through OpenAI Workload Identity Federation - #6695

Merged
waynesun09 merged 8 commits into
mainfrom
agent/6689-openai-wif-provider
Aug 28, 2026
Merged

feat(#6689)!: run GPT on pi through OpenAI Workload Identity Federation#6695
waynesun09 merged 8 commits into
mainfrom
agent/6689-openai-wif-provider

Conversation

@fullsend-ai-coder

@fullsend-ai-coder fullsend-ai-coder Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Secretless GPT on the pi runtime (#6689, ADR 0092), end to end: fullsend run obtains a short-lived OpenAI access token itself, keeps it in a run-scoped OpenShell provider that it refreshes and removes, and the sandbox only ever sees the gateway's placeholder. Credential-delivery tier 2 (ADR 0025) — no OpenAI credential, no GitHub OIDC token and no credential file inside the sandbox.

Breaking change (feat(#6689)!): ${OPENAI_API_KEY} now expands to the empty string wherever a run reads a value from the runner environment — harness YAML, env.sandbox, provider definitions — and is stripped from pre/post scripts (the same treatment as the other runner-only variables). A harness that copied the key into the sandbox itself must declare providers: [openai] and let the run-scoped provider carry it.

Seven commits, one per plan item:

  1. Exchange, pi gate, scaffold (internal/inference/openaiwif, internal/runtime/pi_run.go, profiles/fullsend-openai.yaml, providers/openai.yaml, ADR 0092) — OpenAI Workload Identity Federation: GitHub OIDC assertion for the configured audience → POST https://auth.openai.com/oauth/token (JSON body, subject_token_type jwt, token_type validated as Bearer, HTTPS-only, GitHub OIDC host only, strict body bound, lifetime capped at 1 h, whitespace in the token rejected) → opaque access_token + expires_in + scope. On pi: the placeholder the sandbox environment carries for OPENAI_API_KEY is written into pi's auth.json (PiOpenAIAuthSeed) before the agent-writable .env is sourced, and pi starts without --api-key — pi's AuthStorage re-reads that file on every revision change and resolves the key per request, which is what lets a running iteration follow a refresh (see "Verified against OpenShell 0.0.115"). The pi binary is pinned read-only before .env (readonly FULLSEND_PI_BIN) and launched by path; a config-dir integrity guard (exit 98 when models.json exists, or auth.json is anything but pi's own {} or exactly the seeded placeholder entry; any \u escape rejected) runs before .env and again after it behind unset -f test command grep tr sed printf pi, with PI_CODING_AGENT_DIR and the offline switches re-exported in between; OPENAI_BASE_URL/AZURE_OPENAI_API_KEY/OPENAI_API_KEY are unset after .env.
  2. Run-scoped provider wiring (internal/cli/run_openai.go) — the fullsend-openai profile is imported from the scaffold embedded in the binary on every run (its id is reserved: a workspace or URL-resolved profile with that id is refused) and confirmed present via provider list-profiles -o json; a bare providers: [openai] with no providers/openai.yaml on disk falls back to the embedded definition (OpenAI only); credential source resolution (WIF when the three ids are set — runner variables, or the committed inference.openai block, never merged; a partial trio is an error; a token whose mapping grants more than api.model.request/api.model.read is refused; OPENAI_API_KEY from the runner environment for local runs, with a CI warning); exact-value redaction + ::add-mask::; openai-<sandbox suffix> created empty through a no-expansion path (sandbox.EnsureProviderLiteral) and then given its value and --credential-expires-at in one update; OPENAI_API_KEY and the FULLSEND_OPENAI_* ids join the expansion deny list, which now also covers provider definitions (sandbox.DenyExpansionKeys). Egress preflight: before the agent starts, the run reads the sandbox's effective policy (stdout only, fails closed when unreadable) and refuses to continue when any rule admits api.openai.com:443 without L7 inspection (no protocol, protocol: tcp, tls: skip, a host-only rule, or an allow_uninspected_credentials opt-in, which does not make the credential injectable either) — since OpenShell 0.0.110 the proxy will not inject a credential over such a route, and the base image's default policy ships exactly that in its codex rule; the fleet's policies/base.yaml has no network rules, so with it the provider's inspected entry is the only route. Cleanup deletes the provider (waiting up to a minute for the gateway to release a just-deleted sandbox's reference); under --keep-sandbox the credential is expired in place instead.
  3. Refresh — a per-provider refresher re-exchanges a fresh assertion before expiry (margin capped at half the token lifetime). Once the sandbox is up it first reads the placeholder pi currently holds (bounded retries; an unknown baseline fails the attempt), updates the provider's value and expiry in one call, then waits — on a context independent of the token's remaining lifetime (≤ 90 s; ~20 s measured) — for the sandbox to hand new processes the new placeholder, re-seeds pi's auth.json through sandbox exec and verifies the file names the new generation (one repeat covers an iteration seed racing it). A settle timeout fails the attempt rather than re-seeding the old placeholder. A static key has its expiry pushed out and is re-seeded the same way, because an expiry-only update is a new generation too and the generation pi holds keeps the expiry it was built with. Bounded retries, then the run fails visibly; stopped before cleanup.
  4. config.yaml identifiers (internal/config, fullsend github setup --openai-audience --openai-identity-provider-id --openai-service-account-id) — the three ids are recorded in the committed inference.openai block the same way the Vertex project and provider are (all-three-or-none flags; each id layers independently). They are identifiers, not secrets: a token is issued only to a caller whose GitHub OIDC claims match the mapping, and pull-request events read the config from the base branch (ADR 0033). Within config.yaml the three layer field by field (a base preset can carry the org's audience and provider id, each repository its service account). The FULLSEND_OPENAI_* repository variables remain the opt-out and win when set; on a machine without a GitHub OIDC endpoint a set OPENAI_API_KEY wins over the committed block, so local runs of a repository that committed it still work.
  5. Workflow plumbing — the three variables reach the runner through env: on every fullsend run step (thirteen steps, pinned by TestOpenAIVariableForwarding); providers/ is now in every LAYERED_DIRS list (pre-existing docs(#5670): add fixture authoring section to behaviour-testing guide #5673 gap), profiles/ deliberately not (TestLayeredDirsMatchWorkspacePreparation).
  6. Gated behaviour scenariofeatures/runtime/pi-openai.feature (@requires:capability:runtime-pi-openai, carries policy: policies/base.yaml), not declared in the Makefile until an OpenAI organization is mapped to the pool repositories.
  7. Docs — user-facing operator guide docs/guides/infrastructure/openai-workload-identity.md (console steps, claims check, mapping on repository + ref — fullsend runs agents from seven workflow files, so a single workflow_ref assertion cannot cover them — incl. which runs it trusts, fullsend github setup --openai-* or variables, model selection, local runs incl. the policy requirement, troubleshooting), docs/runtimes/pi.md, running-agents-locally, docs/cli/run.md / github.md, layered-config-reference, contributing/architecture cross-references.

Verified against OpenShell 0.0.115 (2026-08-27)

main bumped the pin from 0.0.83 to 0.0.115 while this PR was in review; every OpenShell behaviour the design relies on was re-verified against the 0.0.115 source and a live local gateway, and two of them had changed:

  • The placeholder a process holds does not follow a rotation (OpenShell retains up to eight generations before a placeholder falls back to the current credential; the eleven quick updates were coalesced by the supervisor's ~10 s poll). On a hand-made sandbox: the environment placeholder is the gateway namespace (openshell:resolve:env, a colon, then v<opaque>_OPENAI_API_KEY) and resolves (200); the canonical, unrevisioned form (OPENAI_API_KEY in that namespace) returns 500 credential_unavailable for this endpoint-bound credential (secrets.rs resolve_placeholder, "provider-shaped aliases carry no credential identity"); after provider update to an invalid value the old placeholder still returned 200 with the old token and the new placeholder 401 — and still after eleven further updates — while a fresh sandbox exec had the new placeholder within ~20 s; expiring the credential in place made every generation fail closed. This retires the round-2 "current alias" hand-off and is why pi now reads the credential from a runner-seeded auth.json that is re-seeded after each refresh. It also contradicts the 0.0.115 providers-v2 docs ("the proxy resolves existing placeholders against current credentials … without restarting the process"), which describe gateway-managed refresh handles, not static/external credentials; recorded in the ADR.
  • Credentialed endpoints must be L7-inspected (policy.rs find_uninspected_credentialed_endpoint, supervisor: "credentialed endpoint requires L7 inspection; raw tunnel is not explicitly allowed"). A harness without policy: inherits the base image's default policy, whose codex rule allows api.openai.com:443 as an L4 tunnel for /usr/bin/node, and pi got connection errors; making that entry protocol: rest in the live sandbox gave 200, and the fleet's policies/base.yaml (no network rules) works as-is. Hence the egress preflight and the policy requirement in the docs.
  • pi's side of the hand-off — AuthStorage re-reading auth.json on a file-revision change and prepareRequest resolving auth per request — is verified in the pi 0.84.3 source (auth-storage.ts, packages/ai/src/auth/resolve.ts, model-runtime.ts; auth.json outranks the environment and only --api-key outranks it) and exercised across iteration starts locally; a rotation inside one running iteration has not been observed live (two attempts to drive pi's RPC mode inside a bare sandbox stalled before its first request) — the first WIF run is that check, and a failure would be visible as a 401 after the old token's expiry. Recorded in the ADR.
  • OpenShell 0.0.110+ resets any model request whose body contains the contiguous placeholder prefix (openshell:resolve:env + :), treating it as credential-bearing traffic — reproduced on the local gateway with a plain POST /v1/responses body, and it is why this PR's own dispatch / Review job failed on every head (the review agent's request carried the diff; the sandbox log shows POST request body credential traffic denied for aiplatform.googleapis.com). main contains no such literal; this PR now builds the prefix from two parts in source and tests and never writes it contiguously in docs or in this description. Verified against the v0.0.115 source (substring guard in secrets.rs contains_raw_reserved_marker, applied to every inspected REST endpoint once the sandbox holds any static credential — hence credential-less Vertex being hit): intended fail-closed design, with the bare-prefix false positive tracked upstream as bug(supervisor): literal and foreign-provider placeholders in conversation history block inference NVIDIA/OpenShell#2904; the only opt-out is allow_uninspected_credentials: true on the model endpoint, adopted for the two default model profiles (fullsend-openai, fullsend-vertex-ai; verified: a body carrying the prefix gets 200 while the bearer header is still injected and GET /v1/models stays 403). The fleet's own agents resolve their Vertex profile from fullsend-ai/agents, which needs the same one-line change to unblock fullsend's review agent on such content — tracked in OpenShell 0.0.110+ resets model requests whose body mentions the placeholder prefix; fullsend agents cannot read text containing it #6716.
  • Unchanged and confirmed: empty-map-only credential check on create (--credential KEY= accepted), provider update --credential-expires-at KEY=RFC3339, ! Provider <name> not found on delete, FAILED_PRECONDITION delete while attached, list-profiles -o json (array with id), static credentials endpoint-bound (which closes the ADR-0025 residual this PR inherited).

Static-key path end to end on this branch against the local 0.0.115 gateway with fullsend-sandbox:latest (pi 0.84.3) and openai/gpt-5.6-luna, custom providers: [openai] + policy: policies/base.yaml harness: embedded profile imported, run-scoped provider created empty then stored with its expiry, sandbox attached, egress preflight passed, pi read the placeholder from the seeded auth.json, POST /v1/responses allowed through the inspected provider entry, tool call through the hook adapter, agent-result.json written, tokens counted, exit 0; provider deleted at the end (expired in place under --keep-sandbox). The WIF path awaits an OpenAI organization.

Testing

  • go test ./internal/inference/openaiwif/ — exchange happy path, JSON body, audience encoding, non-200s, oversized/non-JSON bodies, timeout, missing config, non-Bearer rejected, lifetime cap, whitespace, OIDC host, secrets never in errors.
  • go test ./internal/cli/ — credential resolution (variables, config block, partial sources, static), run-scoped naming, ensureOpenAIProvider against a recording openshell stub, reserved profile id, embedded definition fallback (OpenAI only), egress preflight (uninspectedEndpointRules, host globs, unreadable policy = warning), re-seed after refresh (polls until the placeholder changes, seeds once), cleanup (delete / detach-race retry / expire-in-place), refresh loop, runAgent-level provider lifecycle with and without --keep-sandbox, github setup flags (all-or-none, overlay).
  • go test ./internal/config/inference.openai layering, setter round-trip, zero value removes the block.
  • go test ./internal/runtime/ — OpenAI gate; PiOpenAIAuthSeed under a real sh (shape, atomic rename, refusals, .env cannot reach the file); the config-dir guard (empty auth.json passes, seeded entry passes pretty-printed or not, real key / other provider / extra entry / escaped key fail, models.json fails, shadowed builtins defeated by unset -f); no --api-key; ordering around .env.
  • go test ./internal/sandbox/ — literal create/update args, DenyExpansionKeys, expiry, delete, profileListed (JSON and table), StripPolicyHeader.
  • go test ./internal/scaffold/ — variable forwarding on every run step, layered dirs.
  • Patch coverage ≥ 80% on every touched function; make lint clean.

Out of scope

  • Fleet-side copies of the profile/provider and the pilot harness — Enable fleet agents on the pi runtime agents#935.
  • fullsend repos install support for OpenAI (ValidProviders() stays vertex-only).
  • GitLab CI ID tokens — the exchange is GitHub Actions only; GitLab runs use OPENAI_API_KEY.
  • An upstream OpenShell report on the static-credential rotation behaviour vs its docs (follow-up).

Closes #6689

@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner August 27, 2026 19:04
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Aug 27, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 27, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 7:06 PM UTC · Completed 7:26 PM UTC

Commit: a793035 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $11.81

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown

Site preview

Preview: https://c22307fd-site.fullsend-ai.workers.dev

Commit: 149e6673098369b9bec321aa08c577af1d21ef02

@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

@fullsend-ai-review fullsend-ai-review Bot added the risk/moderate PR risk: moderate label Aug 27, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 27, 2026

Copy link
Copy Markdown

Risk Assessment: elevated (3/5)

Details

Elevated risk driven by large size (52 files, 5399 lines), 8 protected .github/ paths modified, security-critical modifications to credential handling and redaction infrastructure, and the security-labeled WIF domain. Partially offset by majority of added lines in new files with no regression surface, excellent issue specification, strong issue-to-PR alignment, no dependency changes, bot author, and familiar pattern mirroring existing provider gates.

Previous run

Risk Assessment: elevated (3/5)

Details

Elevated risk driven by triple growth in size (2988 lines, 36 files), 7 protected .github/workflows/ paths now modified (previously untouched), security-critical modifications to sandbox.go (credential handling) and redactor.go (token masking), and the security-labeled domain (WIF token exchange, credential isolation). Partially offset by majority of added lines in new files with no regression surface, excellent issue specification with 8 acceptance criteria, strong issue-to-PR alignment, no dependency changes, and familiar pattern mirroring existing xai-vertex provider gate.

Previous run (2)

Risk Assessment: moderate (2/5)

Details

Moderate risk driven primarily by large change size (995 lines, 13 files) offset by additive nature (new package, scaffold files), no protected paths or security-sensitive files touched, bot author, no CI/dependency changes, and strong issue-to-PR alignment.

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 27, 2026

Copy link
Copy Markdown

Review

Findings

Medium

  • [protected-path] .github/workflows/reusable-{code,dispatch,fix,prioritize,retro,review,triage}.yml, .github/actions/prepare-workspace/action.yml — This PR modifies 8 files under .github/, a protected path. The changes add FULLSEND_OPENAI_AUDIENCE, FULLSEND_OPENAI_IDENTITY_PROVIDER_ID, and FULLSEND_OPENAI_SERVICE_ACCOUNT_ID variable forwarding to the agent env: blocks across all reusable stage workflows and inline dispatch jobs, and add providers to the LAYERED_DIRS list. The PR links to pi runtime: secretless GPT via OpenAI Workload Identity Federation — runner-side exchange, run-scoped OpenShell provider, pi openai gate #6689 and explains the rationale (OpenAI WIF requires these non-secret identifiers in the runner environment; vars resolution inside reusable workflows needs explicit env: entries). Human approval is always required for protected-path changes.

  • [breaking-env-contract] internal/cli/run.goOPENAI_API_KEY is now added to oidcDenyKeys, which means ${OPENAI_API_KEY} in harness YAML, env.sandbox, and provider definitions expands to an empty string instead of the real key. This is the declared breaking change (! in the PR title). Downstream repos using ${OPENAI_API_KEY} expansion must migrate to the provider-based path (providers: [openai]). Ensure this is documented in the release notes.

  • [scope_boundary] internal/runtime/pi_run.go — The linked issue's acceptance criterion 5 specified pass --api-key "$OPENAI_API_KEY" for the pi command. The implementation deliberately does NOT pass --api-key, instead seeding auth.json and letting pi re-read it per request. The ADR explains why (--api-key outranks auth.json in pi's resolution order and would pin the iteration to the placeholder it launched with, preventing credential refresh within a running iteration). The maintainer authorized this design deviation.

Low

  • [credential-leakage] internal/inference/openaiwif/openaiwif.go — Prior finding resolved: the HTTP client now sets CheckRedirect to http.ErrUseLastResponse, correctly refusing to follow redirects. Defense-in-depth is in place.

  • [data-exposure] internal/inference/openaiwif/openaiwif.go — Error wrapping in fetchAssertion may include the full request URL via Go's url.Error type, which contains the audience as a query parameter. The audience is a non-secret repository variable, and TestExchange_ErrorsNeverContainSecrets explicitly verifies the runner token does not appear in errors. Informational only.

  • [credential-refresh-race] internal/cli/run_openai.go — During credential refresh, there is a documented window where a pi iteration starting concurrently could seed auth.json from its own exec environment (carrying the old placeholder). reseedOpenAIAuth mitigates by verifying the file content after seeding and repeating if needed. Documented as accepted residual in ADR 0092.

  • [race-condition] internal/security/redactor.goresetRuntimeSecrets() is test-only and unexported. While it acquires runtimeSecretsMu, if a future test adds t.Parallel() alongside tests calling RegisterRuntimeSecret, the global slice could be cleared during a concurrent Scan. Currently safe — the test file does not use t.Parallel() for the test that calls it.

  • [race-condition] internal/runtime/pi_run.go — TOCTOU window between the piOpenAIConfigGuard check and pi actually starting. A concurrent process could write models.json or modify auth.json after the guard passes. Acknowledged in ADR 0092 as an accepted residual; the two-pass guard with unset -f test command grep is well-designed defense-in-depth.

  • [edge-case] internal/runtime/pi_run.gopiOpenAIConfigGuard uses a whole-file regex match on auth.json (after whitespace removal via tr -d) and additionally rejects \u00 JSON escapes. The false-positive direction is safe (fails closed).

  • [missing-doc] docs/guides/infrastructure/layered-config-reference.md — The Code defaults reference table does not include rows for the three new inference.openai.* fields (audience, identity_provider_id, service_account_id). The Merge semantics table was updated but the defaults table was not.

  • [missing-doc] docs/guides/infrastructure/layered-config-reference.md — Prose says "all four subfields fall through to the parent layer" but there are now seven subfields with the openai addition.

  • [naming-convention] internal/cli/run_openai.goshellQuoteArg is functionally identical to shellQuote in internal/sandbox/sandbox.go and internal/runtime/. The codebase convention is shellQuote; the new name breaks that convention without distinguishing semantics.

  • [pattern-inconsistency] internal/scaffold/fullsend-repo/profiles/fullsend-openai.yaml — Endpoint definition omits the access field that every other scaffold profile declares. The omission is intentional and documented in a comment (explicit rules: provide narrower permissions than an access: preset).

  • [error-handling-idiom] internal/sandbox/sandbox.goDeleteProvider wraps errors with %w (enabling errors.Is/errors.As) while the sibling Delete function uses %s. The new code follows better practice; the inconsistency is cosmetic.

Previous run

Review

Findings

Medium

  • [protected-path] .github/workflows/reusable-{code,dispatch,fix,prioritize,retro,review,triage}.yml — This PR modifies 7 files under .github/workflows/, a protected path. The changes add FULLSEND_OPENAI_AUDIENCE, FULLSEND_OPENAI_IDENTITY_PROVIDER_ID, and FULLSEND_OPENAI_SERVICE_ACCOUNT_ID variable forwarding to the agent env: blocks across all reusable stage workflows and inline dispatch jobs. The PR links to pi runtime: secretless GPT via OpenAI Workload Identity Federation — runner-side exchange, run-scoped OpenShell provider, pi openai gate #6689 and explains the rationale (OpenAI WIF requires these non-secret identifiers in the runner environment; vars resolution inside reusable workflows needs explicit env: entries). Human approval is always required for protected-path changes.

Low

  • [credential-leakage] internal/inference/openaiwif/openaiwif.go — The HTTP client used for both the OIDC assertion fetch and the OpenAI token exchange follows Go's default redirect policy (up to 10 hops). While the endpoints are trusted first-party URLs with HTTPS enforcement via requireSecureURL, and Go strips the Authorization header on cross-host redirects, setting CheckRedirect to disable redirects would be defense-in-depth since neither endpoint should redirect on success.

  • [data-exposure] internal/inference/openaiwif/openaiwif.go — Error wrapping in fetchAssertion may include the full request URL via Go's url.Error type, which contains the audience as a query parameter. The audience is a non-secret repository variable, and TestExchange_ErrorsNeverContainSecrets explicitly verifies the runner token does not appear in errors. Informational only.

  • [race-condition] internal/security/redactor.goresetRuntimeSecrets() is test-only and unexported. While it acquires runtimeSecretsMu, if a future test adds t.Parallel() alongside tests calling RegisterRuntimeSecret, the global slice could be cleared during a concurrent Scan. Currently safe — the test file does not use t.Parallel() for the test that calls it.

  • [race-condition] internal/runtime/pi_run.go — TOCTOU window between the piOpenAIConfigGuard check and pi actually starting. A concurrent process could write models.json or modify auth.json after the guard passes. Acknowledged in ADR 0092 as an accepted residual; the two-pass guard with unset -f test command grep is well-designed defense-in-depth.

  • [edge-case] internal/runtime/pi_run.gopiOpenAIConfigGuard uses command -p grep -q '"openai"' for a substring match on auth.json. Could false-positive if the file contains "openai" in an unexpected context. In practice, pi's auth.json uses provider names as top-level keys, and the false-positive direction is safe (fails closed).

  • [error-handling-idiom] internal/sandbox/sandbox.goDeleteProvider wraps errors with %w (enabling errors.Is/errors.As) while the sibling Delete function uses %s. The new code follows better practice; the inconsistency is cosmetic.

  • [pattern-inconsistency] internal/scaffold/fullsend-repo/profiles/fullsend-openai.yaml — Endpoint definition omits the access field that every other scaffold profile declares (fullsend-vertex-ai.yaml, fullsend-github.yaml, etc. all specify access: read-only or access: read-write).

  • [missing-doc] docs/guides/README.md — The Infrastructure section of the guide index does not list the new OpenAI Workload Identity guide (docs/guides/infrastructure/openai-workload-identity.md). Every other infrastructure guide has an entry.

Previous run (2)

Review

Findings

High

  • [stale-doc] docs/contributing/runtime-implementation.md:255 — The "Other clouds" bullet states no non-Vertex provider is wired into Run’s alias table, credential hygiene, or the runner’s OIDC refresh, and that the egress profile "allows only Anthropic + Google hosts." This is factually incorrect after this PR: OpenAI is now wired into the pi runtime gate, credential hygiene, and runner-side OIDC exchange, and the egress profile includes api.openai.com.
    Remediation: Update to acknowledge OpenAI WIF and narrow "not yet wired" to amazon-bedrock and azure-openai-responses only.

Medium

Low

  • [credential-exposure-via-os.ExpandEnv] internal/sandbox/sandbox.go:460buildProviderArgs and buildProviderUpdateArgs unconditionally call os.ExpandEnv on credential values. Currently safe (empty credential), but the ADR describes a future integration where the opaque token may contain $. A bypass mechanism will be needed before the integration PR lands.

  • [OIDC-URL-assumption] internal/inference/openaiwif/openaiwif.go:152fetchAssertion constructs the URL with &audience=... assuming oidcURL already contains ?. True for GitHub Actions but not validated. Using url.Parse would be more robust.

  • [misleading-label] PR title — feat prefix for a partial implementation that doesn’t yet deliver user-exercisable capability. Per COMMITS.md, internal infrastructure may be better labeled refactor.

  • [test-inadequate] internal/inference/openaiwif/openaiwif_test.go:377secretToken constant declared but never used in TestExchange_ErrorsNeverContainSecrets. Looks like an incomplete assertion.

  • [naming-convention] internal/runtime/pi_run.go:34piOpenAIProviderName uses a Name suffix inconsistent with piDefaultProvider and piXaiVertexProvider.

  • [pattern-inconsistency] internal/scaffold/fullsend-repo/profiles/fullsend-openai.yaml:16 — Endpoint entry omits access field present in all other profiles in this directory.

  • [stale-doc] docs/guides/dev/cli-internals.md:702 — Inference package table doesn’t include the new internal/inference/openaiwif/ package.

  • [data-exposure] internal/inference/openaiwif/openaiwif.go:162 — Error wrapping in fetchAssertion may include the full request URL via Go’s HTTP client error. The audience is not secret, so risk is minimal.

  • [error-handling-idiom] internal/sandbox/sandbox.go:857DeleteProvider uses %w to wrap errors while sibling Delete uses %s. The new code is better practice but introduces visible inconsistency.

  • [pattern-inconsistency] internal/scaffold/fullsend-repo/providers/openai.yaml:6 — Credentials key uses OPENAI_API_KEY while other providers use noop _NOOP_* placeholders. Semantically correct but diverges from sibling pattern.


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR

fullsend-ai-review[bot]

This comment was marked as outdated.

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ❌ Failure (post-script /home/runner/work/fullsend/fullsend/.fullsend/.fullsend-cache/resources/sha256/416e3495dbe6b3d0ece160ca879d45f30595f26b71c301116c148872b408d019/scripts/post-fix.sh failed: exit status 1) · Started 7:28 PM UTC · Completed 7:42 PM UTC

Commit: a793035 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.22

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 27, 2026

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 7:29 PM UTC · Ended 8:00 PM UTC

Commit: 58644c9 · View workflow run →

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

⚠️ Post-fix script failed — Push rejected (exit code 1)

The fix agent completed, but the post-fix script failed before finishing.

Workflow run: https://github.com/fullsend-ai/fullsend/actions/runs/33108479027

Details:
To https://github.com/fullsend-ai/fullsend.git
! [rejected] agent/6689-openai-wif-provider -> agent/6689-openai-wif-provider (fetch first)
error: failed to push some refs to 'https://github.com/fullsend-ai/fullsend.git'
hint: Updates were rejected because the remote contains work that you do not
hint: have locally. This is usually caused by another repository pushing to
hint: the same ref. If you want to integrate the remote changes, use
hint: 'git pull' before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
To https://github.com/fullsend-ai/fullsend.git
! [rejected] agent/6689-openai-wif-provider -> agent/6689-openai-wif-provider (stale info)
error: failed to push some refs to 'https://github.com/fullsend-ai/fullsend.git'
Please check the workflow logs for full details and retry with /fs-fix if appropriate.

@waynesun09 waynesun09 added the fullsend-no-fix Skip bot-triggered fix agent runs label Aug 27, 2026
@waynesun09

Copy link
Copy Markdown
Member

/fs-fix-stop

@github-actions

Copy link
Copy Markdown

Fix agent disabled for this PR. Remove the fullsend-no-fix label or use /fs-fix to re-engage.

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 27, 2026

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 8:02 PM UTC · Ended 8:06 PM UTC

Commit: 4d38de7 · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 27, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 8:08 PM UTC · Completed 8:27 PM UTC

Commit: 5fb76bf · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $13.86

@fullsend-ai-review fullsend-ai-review Bot removed the risk/moderate PR risk: moderate label Aug 27, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 28, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 2:01 AM UTC · Completed 2:44 AM UTC

Commit: 4967308 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: The following review comments could not be posted on the diff (GitHub returned 422) and are included here instead:

  • docs/guides/infrastructure/layered-config-reference.md (file-level): Line 325 · [low] missing-doc

Code defaults reference table does not include rows for inference.openai.audience, inference.openai.identity_provider_id, inference.openai.service_account_id.

Suggested fix: Add three rows with empty string defaults to match the Merge semantics table.

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

Comment thread internal/runtime/pi_run.go
Comment thread internal/inference/openaiwif/openaiwif.go
Comment thread internal/inference/openaiwif/openaiwif.go
Comment thread internal/cli/run_openai.go
Comment thread docs/guides/infrastructure/layered-config-reference.md
Comment thread internal/cli/run_openai.go Outdated
A WIF access token lives at most an hour and OpenAI issues no refresh
token, so `fullsend run` now keeps a refresher per run-scoped provider:
shortly before the recorded expiry (margin plus jitter, bounded retries)
it requests a fresh GitHub assertion, re-exchanges it, hot-updates the
provider through the no-expansion path (sandbox.UpdateProviderLiteral)
and records the new expiry; a static OPENAI_API_KEY only has its
provider expiry pushed out on the same schedule. When the retries are
exhausted the loop stops and says so — the recorded expiry then makes
the gateway fail closed instead of the run silently outliving its
credential. The refresher is stopped before the deferred cleanup runs,
so a late refresh can never resurrect an expired credential.

This is the OpenAI half of #6464 Track D; the Vertex OIDC file refresher
is left as it is.

Assisted-by: Claude
Signed-off-by: Wayne Sun <gsun@redhat.com>
The three FULLSEND_OPENAI_* repository variables (audience, identity
provider id, service account id) reach the runner through an explicit
`env:` entry on the `fullsend run` step of each reusable stage workflow
and each inline stage job in reusable-dispatch.yml, the way the OTEL
variables do. They are identifiers, not secrets: the `vars` context
resolves against the caller repository inside a reusable workflow, so no
forwarding hop or shim change is needed. Unset variables leave the openai
provider unusable in that repo, which is the opt-in the pilot wants.
TestOpenAIVariableForwarding pins every run step.

Assisted-by: Claude
Signed-off-by: Wayne Sun <gsun@redhat.com>
The same shape as features/runtime/pi.feature — a minimal tool-using
agent, asserting the pi runtime, a tool call in the session transcript
and token metrics — on openai/gpt-5.6-luna with the openai provider and
no host files, so a successful run proves the secretless credential path
end to end. Gated on runtime-pi-openai, which the Makefile deliberately
does not declare until an OpenAI organization is mapped to the pool
repositories.

Assisted-by: Claude
Signed-off-by: Wayne Sun <gsun@redhat.com>
A user-facing infrastructure guide for GPT on pi without a stored key:
what to create in the OpenAI console (organization-level provider, a
claims check, the per-repository mapping with api.model.request), the
three repository variables, how to put an agent on a GPT model, running
locally with an env file, how it stays safe in plain terms, and an
action-oriented troubleshooting table. The pi runtime page and the
local-run guide point at it; the docs sidebar lists it.

Assisted-by: Claude
Signed-off-by: Wayne Sun <gsun@redhat.com>
… placeholder prefix

OpenShell 0.0.110+ (PR #2493) resets any inspected request body that
contains the literal placeholder prefix, whether or not a credential
follows it — a substring test, applied to every inspected REST endpoint
once the sandbox holds any static credential (NVIDIA/OpenShell#2904).
An agent that has read a file, diff or issue mentioning the prefix could
therefore not call the model: fullsend's own review agent failed on
every head of this PR with ECONNRESET.

Model request bodies never legitimately carry a placeholder, so the two
default model profiles now set allow_uninspected_credentials on their
endpoints. Verified on a 0.0.115 gateway: a body carrying the prefix
gets 200, the bearer header is still injected, and the method/path rules
still apply (GET /v1/models stays 403). The fleet's own Vertex profile
lives in fullsend-ai/agents and needs the same flag.

Assisted-by: Claude (fix), Codex (review), Grok (review)
Signed-off-by: Wayne Sun <gsun@redhat.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 28, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 3:01 AM UTC · Completed 3:42 AM UTC

Commit: 149e667 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $18.15

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving after six review-squad rounds (Claude, Codex, Grok) plus fullsend's own review agent: every MEDIUM+ finding is fixed or explicitly accepted in ADR 0092, all review threads are resolved, required checks (behaviour, e2e) and the rest of CI are green on 149e667, and the static-key path was verified end to end on the pinned OpenShell 0.0.115 (three clean smoke runs on the final design).

The red dispatch / Review job is not a code issue: the review agent reconstructs the placeholder prefix while reasoning about it and OpenShell 0.0.110+ resets that request body (#6716, upstream NVIDIA/OpenShell#2904); the fleet's Vertex profile in fullsend-ai/agents needs the same allow_uninspected_credentials opt-out this PR applies to the default profiles.

Still gated on external access: the live WIF exchange and a mid-iteration rotation (first mapped repository is that check; failure mode and fallback recorded in the ADR).

@waynesun09
waynesun09 added this pull request to the merge queue Aug 28, 2026
Merged via the queue into main with commit 2ca4eb8 Aug 28, 2026
172 of 173 checks passed
@waynesun09
waynesun09 deleted the agent/6689-openai-wif-provider branch August 28, 2026 03:57
waynesun09 pushed a commit to fullsend-ai/agents that referenced this pull request Aug 28, 2026
Add allow_uninspected_credentials: true to the *.googleapis.com
endpoint in profiles/fullsend-vertex-ai.yaml. OpenShell 0.0.110+
resets any inspected request whose body contains the literal
gateway-placeholder prefix (NVIDIA/OpenShell#2904), which causes
ECONNRESET failures when agents quote the placeholder namespace
in model request bodies. This flag skips the request-body
placeholder scan while preserving TLS termination, bearer header
injection, and method/path rule enforcement.

The scaffold copies were already patched in
fullsend-ai/fullsend#6695; this is the fleet harness copy loaded
from this repository.

Closes #1080

Signed-off-by: Wayne Sun <gsun@redhat.com>
Assisted-by: Claude (code), Codex (review), Grok (review)
@cgwalters

Copy link
Copy Markdown
Contributor

re-seeds pi's auth.json through sandbox exec

That's ugly, why isn't just live updating the provider viable? This smells like a pi bug or design flaw that we can't just rely on substitution at the HTTP layer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fullsend-no-fix Skip bot-triggered fix agent runs ready-for-review Triggers review agent dispatch requires-manual-review Review requires human judgment risk/elevated PR risk: elevated

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pi runtime: secretless GPT via OpenAI Workload Identity Federation — runner-side exchange, run-scoped OpenShell provider, pi openai gate

2 participants