Skip to content

Add inference provider credential provisioning (Vertex AI) - #215

Merged
ralphbean merged 5 commits into
mainfrom
feature/vertex-inference-provisioning
Apr 17, 2026
Merged

Add inference provider credential provisioning (Vertex AI)#215
ralphbean merged 5 commits into
mainfrom
feature/vertex-inference-provisioning

Conversation

@ralphbean

@ralphbean ralphbean commented Apr 9, 2026

Copy link
Copy Markdown
Member

Summary

  • Adds InferenceLayer to the install stack (position 4: after secrets, before dispatch-token) that provisions inference provider credentials as repo-level secrets on .fullsend
  • Implements Vertex AI provider with three modes: (1) create SA + key, (2) verify existing SA + create key, (3) use pre-made key JSON directly
  • Abstracts behind inference.Provider interface in internal/inference/ for future provider support
  • Adds inference: { provider: vertex } config section to config.yaml
  • CLI flags: --gcp-project, --gcp-service-account, --gcp-credentials-file
  • Updates normative specs (ADR 0011 config schema, ADR 0014 credential surface) and ADR 0006 layer ordering

Test plan

  • 9 unit tests for internal/inference/vertex/ (all 3 modes + error cases)
  • 9 unit tests for internal/layers/inference.go (install, analyze, nil provider, errors)
  • 7 unit tests for internal/config/ (inference validation, parse, marshal)
  • All existing tests pass
  • E2E tests updated to use E2E_HALFSEND_VERTEX_KEY env var (mode 3) when available, skip gracefully otherwise
  • Wire E2E_HALFSEND_VERTEX_KEY into CI workflow to enable e2e inference testing

Closes #152

🤖 Generated with Claude Code

@ralphbean
ralphbean marked this pull request as ready for review April 9, 2026 21:20

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flagging a security concern around raw GCP API error body leakage — see inline comment on gcp.go. The same pattern appears at lines 76, 98, and 116.

Comment thread internal/inference/vertex/gcp.go Outdated

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GOOGLE_APPLICATION_CREDENTIALS secret naming creates a consumption trap

Severity: Medium — not broken today, predictable bug later

The Vertex provider stores raw SA key JSON in a GitHub secret named GOOGLE_APPLICATION_CREDENTIALS:

// internal/inference/vertex/vertex.go
SecretCredentials = "GOOGLE_APPLICATION_CREDENTIALS"

This name collides with the well-known GCP SDK environment variable of the same name, but they expect different value types:

Namespace What GOOGLE_APPLICATION_CREDENTIALS holds
GitHub Actions secret (what this PR creates) Raw JSON content ({"type":"service_account",...})
GCP SDK env var (what the runtime needs) A file path like /tmp/creds.json

Why this matters

The agent dispatch workflow is currently a stub, so nothing breaks today. But when someone writes the real consumption step, the identical name in both namespaces invites a natural one-liner:

env:
  GOOGLE_APPLICATION_CREDENTIALS: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}

The GCP SDK will try to open a file at path {"type":"service_account",...} → silent auth failure.

The correct pattern requires an intermediate write-to-file step:

- run: |
    echo "$GCP_CREDS" > /tmp/gcp_credentials.json
    echo "GOOGLE_APPLICATION_CREDENTIALS=/tmp/gcp_credentials.json" >> "$GITHUB_ENV"
  env:
    GCP_CREDS: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}

Existing repo precedent

The experiments directory already uses a clearer name that avoids this confusion:

# experiments/agent-scoped-tools-triage/README.md
echo "$GCP_SA_KEY_JSON" > /tmp/gcp_credentials.json
echo "GOOGLE_APPLICATION_CREDENTIALS=/tmp/gcp_credentials.json" >> "$GITHUB_ENV"

Recommendation

Rename the secret constant to something that signals "this is JSON content, not a file path":

SecretCredentials = "FULLSEND_GCP_SA_KEY_JSON"

This follows the FULLSEND_ prefix convention already used for other secrets (FULLSEND_DISPATCH_TOKEN, FULLSEND_<ROLE>_APP_PRIVATE_KEY) and makes the write-to-file step obvious to the workflow author.

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multi-agent review: 5 High-severity issues requiring changes

Five high-severity issues identified across three independent review agents (security, architecture, Gemini-style). All five are in new code introduced by this PR.

Summary of findings:

# File Issue
1 gcp.go:21-33 PATH-hijackable gcloud subprocess — use GCP Go SDK instead
2 gcp.go:46-62 No HTTP client timeout — http.DefaultClient blocks indefinitely
3 gcp.go:66-79 URL injection — unescaped projectID/saName in API URL paths
4 vertex.go:78-100 SA key accumulation — every Provision() creates a new key (10 key limit), violating ADR 0006 idempotency
5 admin.go:118-137 --gcp-service-account / --gcp-credentials-file silently ignored without --gcp-project

See also prior review comments on this PR regarding GCP API error body leaks and GOOGLE_APPLICATION_CREDENTIALS secret naming.

Full review identified 38 total issues (5 High, 16 Medium, 17 Low) across security, correctness, architecture, and code quality. The inline comments above cover the 5 High items that should be addressed before merge.

Comment thread internal/inference/vertex/gcp.go
Comment thread internal/inference/vertex/gcp.go
Comment thread internal/inference/vertex/gcp.go Outdated
Comment thread internal/inference/vertex/vertex.go
Comment thread internal/cli/admin.go

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up from Gemini 3.1 Pro review: config overwrite on re-install is a data loss bug. See inline comment.

Comment thread internal/cli/admin.go

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the implementation against all 7 high-severity issues from the multi-agent review. All are resolved:

  1. Raw GCP API error body leakage (gcp.go:114) — extractGCPErrorMessage() now parses only error.message from GCP responses across all three methods. Raw body no longer leaked.

  2. PATH-hijackable gcloud subprocess (gcp.go:46) — accessToken() now uses golang.org/x/oauth2/google.FindDefaultCredentials(). No subprocess, no $PATH dependency.

  3. No HTTP client timeout / http.DefaultClient (gcp.go:92) — NewLiveGCPClient() creates a dedicated &http.Client{Timeout: 30 * time.Second}. Global client no longer used.

  4. URL injection via unescaped path parameters (gcp.go) — Input validated with gcpIDPattern and saEmailPattern regexes, and all path parameters go through url.PathEscape().

  5. SA key accumulation / idempotency contract (vertex.go:106) — InferenceLayer.Install() checks whether all expected secrets already exist in GitHub before calling Provision(). If present, provisioning is skipped and no new GCP SA key is created.

  6. --gcp-service-account silently ignored without --gcp-project (admin.go:145) — Explicit early error when --gcp-service-account or --gcp-credentials-file are set without --gcp-project.

  7. Re-install without --gcp-project silently erases inference config (admin.go:141) — loadExistingInferenceProvider() reads the existing inference.provider from config.yaml in .fullsend before building the layer stack, preserving the existing config when no GCP flags are provided.

Per ADR convention, leaving resolution of the inline threads to the PR owner.

@ralphbean
ralphbean requested a review from waynesun09 April 16, 2026 02:24

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: 5 Medium-Severity Findings

All 7 previously reported high-severity issues have been resolved — nice work. Five medium-severity items remain across naming convention, input validation, code safety, test coverage, and credential handling. See inline comments.

Comment thread internal/inference/vertex/vertex.go Outdated
Comment thread internal/cli/admin.go Outdated
Comment thread internal/cli/admin.go Outdated
Comment thread internal/inference/vertex/gcp.go
Comment thread internal/inference/vertex/vertex.go Outdated

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional: 5 Low-Severity Findings (non-blocking)

Comment thread internal/cli/admin.go
Comment thread internal/inference/vertex/gcp.go Outdated
Comment thread internal/cli/admin.go
Comment thread internal/inference/vertex/vertex.go
Comment thread internal/inference/vertex/gcp.go Outdated

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All 5 medium-severity and 5 low-severity findings from the previous review round are resolved (or explicitly declined by the PR owner for the type naming item).

Two independent review agents (Gemini code review + security-focused review) found no critical or high-severity issues in the latest commit. Remaining observations are non-blocking:

  • TOCTOU race in idempotency check (Medium) — acceptable for a manual admin CLI tool
  • Credential string copies survive zeroing (Medium) — Go language limitation, CLI exits after use
  • Project ID not validated in Mode 3 (Medium) — could cause confusing runtime failures but not a security issue; consider adding gcpIDPattern validation before the Mode 3 early return in a follow-up

Security posture is solid: ADC replaces gcloud subprocess, URL path parameters escaped and validated, GCP error messages sanitized, io.LimitReader bounded, credential file validated and symlink-protected, memory zeroed where possible. Supply chain additions (golang.org/x/oauth2, cloud.google.com/go/compute/metadata) are official Google/Go libraries.

Test coverage is thorough: 28 unit tests + e2e across all 3 provisioning modes, error cases, nil client, idempotency, and 409 Conflict handling.

ralphbean and others added 4 commits April 17, 2026 08:28
Adds an InferenceLayer to the install stack that provisions and stores
inference provider credentials as repo secrets on .fullsend. Supports
three modes: create SA + key, verify existing SA + key, or use a
pre-made key directly. Coded behind an abstract inference.Provider
interface to support future providers.

Closes #152

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace gcloud subprocess with Go SDK (golang.org/x/oauth2/google)
  to eliminate PATH-hijack risk
- Add dedicated HTTP client with 30s timeout instead of http.DefaultClient
- Escape user-supplied URL path parameters and validate GCP naming rules
- Extract only error.message from GCP API responses to prevent metadata leakage
- Rename secret from GOOGLE_APPLICATION_CREDENTIALS to FULLSEND_GCP_SA_KEY_JSON
  to avoid collision with the GCP SDK env var (which expects a file path)
- Add idempotency check: skip provisioning if secrets already exist,
  preventing SA key accumulation against GCP's 10-key limit
- Error if --gcp-service-account or --gcp-credentials-file set without
  --gcp-project instead of silently ignoring them
- Preserve existing inference config on re-install when GCP flags omitted

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Guard against nil gcpAPI in Provision() for modes 1/2, returning a
  clear error instead of panicking if called without a GCP client
- Add saEmailPattern validation in CreateServiceAccountKey to prevent
  URL injection via crafted email parameters
- Add tests for nil GCP client in all three modes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rename GCP_PROJECT_ID to FULLSEND_GCP_PROJECT_ID for prefix consistency
- Validate credential file JSON structure before upload
- Add NewAnalyzeOnly() constructor to replace fragile sentinel
- Add 409 Conflict idempotency test for SA creation
- Use []byte for credential JSON and zero after use
- Add symlink protection on credential file path
- Use json.Marshal instead of fmt.Sprintf for SA payload
- Deduplicate runAnalyze by reusing loadExistingInferenceProvider
- Bound HTTP response body reads with io.LimitReader

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 17, 2026

Copy link
Copy Markdown

Site preview

Preview: https://7ba5f381-site.fullsend-ai.workers.dev

Commit: 825b2eacf3ea7176dc1c5ddc608b8bc3fad50c28

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fullsend admin install: provision and store Google Vertex API credentials

2 participants