diff --git a/AGENTS.md b/AGENTS.md index 3c13297c09..1b55a565ea 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,7 +4,11 @@ See [CLAUDE.md](CLAUDE.md) for project rules and design decisions. ## Commit messages -You **must** read and follow [COMMITS.md](COMMITS.md) when writing or reviewing commit messages. Getting the prefix right is not optional — GoReleaser uses it to build release notes. +Use [Conventional Commits](https://www.conventionalcommits.org/) format for every commit. The allowed types are: `feat`, `fix`, `refactor`, `docs`, `test`, `chore`, `ci`, `perf`, `build`. See [CONTRIBUTING.md](CONTRIBUTING.md#commit-messages) for the full specification. + +This is not optional — GoReleaser parses commit prefixes to build release notes. A missing or wrong prefix produces incorrect changelogs. + +When reviewing PRs, check that commit messages and PR titles follow this format. Flag violations as a required change — they are not cosmetic. ## Forge abstraction diff --git a/CLAUDE.md b/CLAUDE.md index 6b521bfc3a..41936412ac 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,7 +11,7 @@ Fullsend is a platform for fully autonomous agentic development for GitHub-hoste - Keep core problem documents organization-agnostic. Organization-specific details belong in `docs/problems/applied//`. - The target audience is any contributor community considering autonomous agents — keep language accessible, avoid presuming solutions. - Always run `make lint` before submitting changes and fix any failures. -- You **must** read and follow [COMMITS.md](COMMITS.md) when writing or reviewing commit messages. Getting the prefix right is not optional — GoReleaser uses it to build release notes. +- Use [Conventional Commits](https://www.conventionalcommits.org/) for all commit messages. See [CONTRIBUTING.md](CONTRIBUTING.md#commit-messages) for the full specification. This is critical — GoReleaser uses commit prefixes to generate release notes. - Never commit secrets (tokens, API keys, PEM keys, gcloud credentials) or sensitive data (GCP project names, service account identifiers, Model Armor template names, internal hostnames). Use environment variables with no defaults for sensitive values. ## Go code diff --git a/COMMITS.md b/COMMITS.md deleted file mode 100644 index 183d86711d..0000000000 --- a/COMMITS.md +++ /dev/null @@ -1,73 +0,0 @@ -# Commit Messages - -This project uses [Conventional Commits](https://www.conventionalcommits.org/). Every commit on `main` feeds the auto-generated release notes (via GoReleaser), so getting the prefix right matters. You **must** consult this file when writing or reviewing commit messages. - -## Format - -``` -(): - - - - -``` - -## Types - -| Type | Purpose | Appears in release notes? | -|---|---|---| -| `feat` | New user-facing functionality | Yes — under **Features** | -| `fix` | Bug fix visible to users | Yes — under **Bug Fixes** | -| `refactor` | Code restructuring (no behavior change) | Yes — under **Refactoring** | -| `docs` | Documentation only | No | -| `test` | Adding or updating tests | No | -| `chore` | Maintenance (CI, deps, tooling) | No | -| `ci` | CI/CD pipeline changes | No | -| `perf` | Performance improvement | Yes — under **Others** | -| `build` | Build system or dependency changes | No | - -## `feat` is for end users - -The `feat` prefix populates the **Features** section of our release notes. End users read that list to decide whether to upgrade. Reserve `feat` for changes an end user would recognize as new capability: - -- A new CLI command or flag they can invoke -- A new behavior they interact with (e.g., the agent now comments on their PR with a new kind of analysis) -- A new integration or platform they can target - -**`feat` is wrong for:** - -- Restructuring internals (extracting a sub-agent, splitting a package) → `refactor` -- Adding internal packages, helpers, or abstractions that don't change user-visible behavior → `refactor` -- Upgrading a dependency or vendored tool version → `chore` -- Tightening internal heuristics, adjusting prompts, or tuning agent behavior that users don't directly control → `refactor` or `fix` depending on whether it corrects a defect -- Addressing review feedback on an existing PR → `fix` or `refactor`, not `feat` - -Apply the same discipline to `fix` — bumping a dependency version is `chore`, not `fix`, unless it corrects a user-visible bug. Removing a trailing blank line is `chore`, not `fix`. - -**When in doubt, prefer `refactor` or `chore` over `feat` or `fix`.** A change miscategorized as `refactor` is harmless — it shows up in a lower section of the release notes. A change miscategorized as `feat` erodes the signal of the Features list. - -## Scope - -The parenthesized scope is optional but encouraged. Use it to identify the subsystem: `feat(appsetup)`, `fix(mint)`, `docs(adr)`, `chore(ci)`. When fixing a specific issue, prefer the issue number as scope: `fix(#123): ...`. - -## Breaking changes - -Append `!` after the type/scope to flag a breaking change: `feat(cli)!: rename --gcp flags to --inference`. Include a `BREAKING CHANGE:` trailer in the body explaining migration steps. - -## Examples - -``` -feat(review-agent): add outcome labels to post-review.sh - -fix(#933): use .yaml extension for shim workflow path - -refactor(#1797): extract challenger pass into dedicated sub-agent - -chore(sandbox): bump gopls from 0.18.1 to 0.22.0 - -docs: add mint URL stability note to installation guide -``` - -## Reviewing commit messages - -When reviewing PRs, check that commit messages and PR titles use the correct type prefix. Flag violations as a required change — they are not cosmetic. Pay particular attention to `feat` — challenge it if the change is not user-facing. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3dc5d6b20b..fe476ae86e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -4,7 +4,55 @@ Thank you for your interest in contributing! This document covers the social nor ## Commit messages -This project uses [Conventional Commits](https://www.conventionalcommits.org/). See [COMMITS.md](COMMITS.md) for the full specification, type selection rules, and examples. +This project uses [Conventional Commits](https://www.conventionalcommits.org/). Every commit on `main` feeds the auto-generated release notes (via GoReleaser), so getting the format right matters. + +### Format + +``` +(): + + + + +``` + +### Types + +| Type | Purpose | Appears in release notes? | +|---|---|---| +| `feat` | New functionality | Yes — under **Features** | +| `fix` | Bug fix | Yes — under **Bug Fixes** | +| `refactor` | Code restructuring (no behavior change) | Yes — under **Refactoring** | +| `docs` | Documentation only | No | +| `test` | Adding or updating tests | No | +| `chore` | Maintenance (CI, deps, tooling) | No | +| `ci` | CI/CD pipeline changes | No | +| `perf` | Performance improvement | Yes — under **Others** | +| `build` | Build system or dependency changes | No | + +### Scope + +The parenthesized scope is optional but encouraged. Use it to identify the subsystem: `feat(appsetup)`, `fix(mint)`, `docs(adr)`, `chore(ci)`. When fixing a specific issue, prefer the issue number as scope: `fix(#123): ...`. + +### Breaking changes + +Append `!` after the type/scope to flag a breaking change: `feat(cli)!: rename --gcp flags to --inference`. Include a `BREAKING CHANGE:` trailer in the body explaining migration steps. Breaking changes trigger a major version bump. + +### Examples + +``` +feat(review-agent): add outcome labels to post-review.sh + +fix(#933): use .yaml extension for shim workflow path + +docs: add mint URL stability note to installation guide + +chore(ci): update goreleaser to v2 +``` + +### Why this matters + +GoReleaser groups changelog entries by type prefix (see `.goreleaser.yml`). Commits without a recognized prefix land under "Others". Commits prefixed `docs:`, `test:`, `chore:`, `ci:`, or `build:` are excluded from release notes entirely. A wrong prefix means the change shows up in the wrong section — or not at all. ## DCO (Developer Certificate of Origin) diff --git a/docs/ADRs/0038-universal-harness-access.md b/docs/ADRs/0038-universal-harness-access.md index eee65ad57b..eedd0d3f70 100644 --- a/docs/ADRs/0038-universal-harness-access.md +++ b/docs/ADRs/0038-universal-harness-access.md @@ -364,4 +364,9 @@ The proposed model follows the GitHub Actions approach: URL-based references wit ## Implementation Plan -See `docs/plans/universal-harness-access.md` for full implementation details, security analysis, and migration path. See `docs/plans/universal-harness-access-phase1.md` for the phased PR breakdown (Phase 1 MVP), `docs/plans/universal-harness-access-phase2.md` for Phase 2 (transitive dependency resolution), `docs/plans/universal-harness-access-phase3.md` for Phase 3 (lock files), and `docs/plans/universal-harness-access-phase4.md` for Phase 4 (runtime dependency loading). +See `docs/plans/universal-harness-access.md` for full implementation details, security analysis, and migration path. Per-phase PR breakdowns: + +- `docs/plans/universal-harness-access-phase1.md` — Phase 1: URL detection, fetch, cache, schema, resolver, CLI (complete) +- `docs/plans/universal-harness-access-phase2.md` — Phase 2: Transitive dependency resolution (complete) +- `docs/plans/universal-harness-access-phase3.md` — Phase 3: Lock files (complete) +- `docs/plans/universal-harness-access-phase4.md` — Phase 4: Runtime dependency loading (in progress) diff --git a/docs/ADRs/0045-forge-portable-harness-schema.md b/docs/ADRs/0045-forge-portable-harness-schema.md index 090a221c12..a8a6260b1e 100644 --- a/docs/ADRs/0045-forge-portable-harness-schema.md +++ b/docs/ADRs/0045-forge-portable-harness-schema.md @@ -265,6 +265,8 @@ compatibility. | `sandbox_timeout_seconds` | Sandbox-level timeout, not forge-specific | | `security` | Security scanning is forge-agnostic | | `allowed_remote_resources` | URL allowlist for resource fetching (ADR 0038) | +| `allow_runtime_fetch` | Opt-in for runtime dependency loading (ADR 0038) | +| `max_runtime_fetches` | Rate limit for runtime fetches (ADR 0038) | | `description` | Documentation, no runtime effect | | `role` | Agent identity is forge-agnostic | | `slug` | Kept top-level; per-forge slug differences handled via `base` composition or a future `forge..slug` extension — see trade-off note below | diff --git a/docs/guides/dev/cli-internals.md b/docs/guides/dev/cli-internals.md index c964086fc8..a8580e0ef6 100644 --- a/docs/guides/dev/cli-internals.md +++ b/docs/guides/dev/cli-internals.md @@ -235,19 +235,7 @@ Install: process 1→7 (forward) Uninstall: process 7→1 (reverse) ``` -Per-repo mode does not use the layer stack — it runs the same phases inline in `runPerRepoInstall()` and `runGitHubSetupPerRepo()` since there's no need for composable uninstall ordering with a single repo. Binary vendoring (when `--vendor-fullsend-binary` is set) and stale binary cleanup are handled inline or via shared helpers; per-org mode uses `VendorBinaryLayer`. - -### Binary acquisition (`internal/binary`) - -Linux binary resolution for `fullsend run` and vendoring lives in `internal/binary`: - -| Function | Policy | -|----------|--------| -| `ResolveForRun` | Release download (released CLI only) → cross-compile → latest release | -| `ResolveForVendor` | Cross-compile → matching release (released CLI only) → fail (no latest) | -| `ResolveExplicit` | Validate linux/{arch} ELF for `--fullsend-binary` | - -Vendoring commit messages use title + body (upload and stale delete). `admin analyze` reports stale vendored binaries at `bin/fullsend` or `.fullsend/bin/fullsend` without install-intent flags. +Per-repo mode does not use the layer stack — it runs the same phases inline in `runPerRepoInstall()` since there's no need for composable uninstall ordering with a single repo. Binary vendoring (when `--vendor-fullsend-binary` is set) and stale binary cleanup are handled inline rather than through `VendorBinaryLayer`. --- diff --git a/docs/guides/getting-started/github-setup.md b/docs/guides/getting-started/github-setup.md index a973d0a81c..7163e80aad 100644 --- a/docs/guides/getting-started/github-setup.md +++ b/docs/guides/getting-started/github-setup.md @@ -12,7 +12,7 @@ For the all-in-one setup that provisions both GCP and GitHub in a single command - **GitHub CLI** (`gh`) authenticated — the installer runs a preflight check and tells you which scopes are missing. When prompted, run the `gh auth refresh -s ` command it suggests. - **fullsend CLI** — download the latest binary from [GitHub Releases](https://github.com/fullsend-ai/fullsend/releases) - **From your Mint service provider admin** (currently GCP-managed; other providers planned): - - Token mint URL (`--mint-url`) — the HTTPS endpoint of the deployed mint Cloud Function. If you are using the fullsend hosted mint, the URL is `https://fullsend-mint-gljhbkcloq-uc.a.run.app` (see [Hosted mint](../infrastructure/mint-administration.md#hosted-mint)). + - Token mint URL (`--mint-url`) — the HTTPS endpoint of the deployed mint Cloud Function - **From your Inference provider admin** (currently GCP Agent Platform, formerly Vertex AI; other providers planned): - GCP project ID (`--inference-project`) — the project where Agent Platform is enabled (e.g., `my-gcp-project`) - WIF provider resource name (`--inference-wif-provider`) — the full resource path, e.g., `projects/123456789/locations/global/workloadIdentityPools/fullsend-inference/providers/github-oidc` (note: the leading number is the GCP **project number**, not the project ID string; your GCP admin can find it with `gcloud projects describe --format='value(projectNumber)'`) @@ -118,16 +118,9 @@ fullsend github setup acme-corp \ | `--app-set` | No | `fullsend-ai` | App set name prefix for GitHub Apps | | `--enroll-all` | No | `false` | Enroll all repositories without prompting (per-org only) | | `--enroll-none` | No | `false` | Skip enrollment without prompting (per-org only) | -| `--vendor-fullsend-binary` | No | `false` | Resolve and upload a linux/amd64 fullsend binary for CI (see [Vendoring the CLI binary](#vendoring-the-cli-binary)) | -| `--fullsend-binary` | No | | Path to a Linux fullsend binary when vendoring (skips auto-resolution) | +| `--vendor-fullsend-binary` | No | `false` | Build and upload the fullsend binary to the config repo for local dev testing (e.g., macOS with a Podman Linux VM) | | `--dry-run` | No | `false` | Preview changes without making them | -### Vendoring the CLI binary - -Same policy as [admin install](installation.md#vendoring-the-cli-binary): `--fullsend-binary` → checkout cross-compile → matching release (released CLI only) → fail. Per-repo setup now wires vendoring and stale-binary cleanup when the flag is off. - -`fullsend admin analyze ` reports when a stale vendored binary is present (no install-intent flags on analyze). - ## Per-repo setup Per-repo mode bootstraps a single repository with a `.fullsend/` directory, shim workflow, and repo-level secrets: diff --git a/docs/guides/getting-started/installation.md b/docs/guides/getting-started/installation.md index 35e0aa6015..d2b7671dc9 100644 --- a/docs/guides/getting-started/installation.md +++ b/docs/guides/getting-started/installation.md @@ -256,8 +256,7 @@ The installer automatically provisions [Workload Identity Federation (WIF)](http | `--skip-mint-check` | `false` | Skip mint validation, GCP provisioning, and app setup; requires `--mint-url` | | `--enroll-all` | `false` | Enroll all repositories without prompting (per-org only) | | `--enroll-none` | `false` | Skip repository enrollment without prompting (per-org only) | -| `--vendor-fullsend-binary` | `false` | Resolve and upload a linux/amd64 fullsend binary for CI (see [Vendoring the CLI binary](#vendoring-the-cli-binary)) | -| `--fullsend-binary` | | Path to a Linux fullsend binary to upload when `--vendor-fullsend-binary` is set (skips auto-resolution) | +| `--vendor-fullsend-binary` | `false` | Cross-compile and vendor the fullsend binary for development iteration | The `--skip-mint-check` flag bypasses all mint validation, GCP provisioning, and app setup. It requires `--mint-url` to be set and only validates that the URL uses HTTPS. This is useful when the mint infrastructure is managed externally or you want to skip GCP API calls entirely. @@ -267,25 +266,6 @@ The installer automatically detects when the deployed mint function is up-to-dat A single token mint can serve multiple GitHub organizations. See [Mint service administration — Multi-org setup](../infrastructure/mint-administration.md#multi-org-setup) for the complete multi-org workflow. -### Vendoring the CLI binary - -Use `--vendor-fullsend-binary` to upload a linux/amd64 `fullsend` binary into the config repo (`bin/fullsend`) or per-repo path (`.fullsend/bin/fullsend`). CI workflows prefer this file over downloading from GitHub releases. - -When the flag is set, the binary is resolved in this order: - -1. **`--fullsend-binary `** — upload that file (validated as linux/amd64 ELF) -2. **Checkout build** — cross-compile from the fullsend module root (`go env GOMOD`), stamped `{version}-vendored` -3. **Release fetch** — only if step 2 is unavailable **and** the running CLI is a released version (e.g. `0.4.0`); downloads the matching GitHub release (no `-vendored` suffix) -4. **Fail** — dev CLI outside a checkout fails with a clear error (no “latest release” fallback) - -When the flag is **off**, any existing vendored binary is removed so CI uses released versions. - -**Notes:** - -- Vendoring the CLI alone does not air-gap the full pipeline (OpenShell, gateway, sandbox image, upstream scaffold still download at runtime). -- Release fallback requires network access at install time; CI consumes the uploaded file. -- Works from any directory inside the module checkout (module root discovery via `GOMOD`). - ### Merge enrollment PRs If you chose to enroll repositories during install, the installer dispatches a workflow that creates an enrollment PR in each enrolled repo. These PRs add a shim workflow (`.github/workflows/fullsend.yaml`) that wires events to the agent pipeline. diff --git a/docs/guides/infrastructure/mint-administration.md b/docs/guides/infrastructure/mint-administration.md index 6a2cb5f688..01f0a96407 100644 --- a/docs/guides/infrastructure/mint-administration.md +++ b/docs/guides/infrastructure/mint-administration.md @@ -2,17 +2,7 @@ This guide covers deploying and managing the fullsend token mint Cloud Function. The mint is the OIDC token exchange service that lets GitHub Actions workflows authenticate as GitHub Apps — it is infrastructure that serves all enrolled organizations and repositories. -> **This guide is for platform operators** who deploy, manage, or troubleshoot the token mint Cloud Function. If you are an end user setting up fullsend for your organization, see [Installing fullsend](../getting-started/installation.md) instead — the mint is typically deployed once by a platform operator, and organizations are enrolled as needed. - -## Hosted mint - -The fullsend team operates a public hosted mint service. If your organization is enrolled, you can use it directly without deploying your own: - -``` -https://fullsend-mint-gljhbkcloq-uc.a.run.app -``` - -Pass this URL as `--mint-url` when running `fullsend admin install`, or set the `FULLSEND_MINT_URL` repository/org variable in GitHub. If you are using the hosted mint, the rest of this guide (deploying, enrolling, troubleshooting) is handled by the fullsend team — you do not need to manage mint infrastructure yourself. +> **This guide is for platform operators** who deploy, manage, or troubleshoot the token mint Cloud Function. If you are an end user setting up fullsend for your organization, see [Installing fullsend](../getting-started/installation.md) instead — the mint is typically deployed once by a platform operator, and organizations are enrolled as needed. Work is in progress to offer a hosted public mint service, which will further reduce the need for per-org mint administration. ## Prerequisites diff --git a/docs/guides/user/running-agents-locally.md b/docs/guides/user/running-agents-locally.md index 52fcfcf306..ecc592cced 100644 --- a/docs/guides/user/running-agents-locally.md +++ b/docs/guides/user/running-agents-locally.md @@ -303,7 +303,7 @@ to the server (gateway). It is likely that you need to bind the gateway to `0.0. - Check that it's registered: `openshell gateway list` **`Syntax error: "(" unexpected` inside sandbox** -- The macOS Mach-O binary was injected instead of a Linux ELF. Update to fullsend 0.4.0+ which auto-resolves the correct binary, or provide one explicitly with `--fullsend-binary` +- The macOS Mach-O binary was injected instead of a Linux ELF. The CLI auto-resolves the correct binary (download or cross-compile). If auto-resolution fails, provide one explicitly with `--fullsend-binary` **Agent fails with missing environment variable** - Check your env file contains all variables listed in the agent's harness YAML (`harness/{agent}.yaml` in the `.fullsend` config directory) diff --git a/docs/plans/universal-harness-access-phase1.md b/docs/plans/universal-harness-access-phase1.md index 7890810247..2b5a213804 100644 --- a/docs/plans/universal-harness-access-phase1.md +++ b/docs/plans/universal-harness-access-phase1.md @@ -198,18 +198,14 @@ allowed_remote_resources: ## Future Phases (high-level) -### Phase 2: Transitive dependency resolution (2-3 PRs) -- Parse `dependencies:` field from SKILL.md YAML frontmatter (read from resolved skill directory, whether local or cached from forge) -- Recursive resolution with cycle detection (visited set), depth limit (10), breadth limit (50) -- Relative URL resolution for URL-fetched resources (RFC 3986 base URL semantics) - -### Phase 3: Lock files (2 PRs) -- `internal/lock/` package: LockFile struct, parse/generate/write -- `fullsend lock ` CLI subcommand; prefer lock file entries in resolver - -### Phase 4: Runtime dependency loading (2 PRs) -- `allow_runtime_fetch` + `max_runtime_fetches` harness fields -- `fullsend-fetch-skill` binary in sandbox, Unix socket to runner, rate limiting +### Phase 2: Transitive dependency resolution (3 PRs) — COMPLETE +See `docs/plans/universal-harness-access-phase2.md` for detailed plan. + +### Phase 3: Lock files (2 PRs) — COMPLETE +See `docs/plans/universal-harness-access-phase3.md` for detailed plan. + +### Phase 4: Runtime dependency loading (3 PRs) +See `docs/plans/universal-harness-access-phase4.md` for detailed plan. --- diff --git a/docs/plans/universal-harness-access-phase2.md b/docs/plans/universal-harness-access-phase2.md index 0dd65af7e1..e09ce2996f 100644 --- a/docs/plans/universal-harness-access-phase2.md +++ b/docs/plans/universal-harness-access-phase2.md @@ -440,12 +440,10 @@ After PR 3 merges, verify Phase 2 end-to-end: --- -## Future Phases (unchanged from Phase 1 plan) +## Future Phases -### Phase 3: Lock files (2 PRs) -- `internal/lock/` package: LockFile struct, parse/generate/write -- `fullsend lock ` CLI subcommand; prefer lock file entries in resolver +### Phase 3: Lock files (2 PRs) — COMPLETE +See `docs/plans/universal-harness-access-phase3.md` for detailed plan. -### Phase 4: Runtime dependency loading (2 PRs) -- `allow_runtime_fetch` + `max_runtime_fetches` harness fields -- `fullsend-fetch-skill` binary in sandbox, Unix socket to runner, rate limiting +### Phase 4: Runtime dependency loading (3 PRs) +See `docs/plans/universal-harness-access-phase4.md` for detailed plan. diff --git a/docs/plans/universal-harness-access-phase4.md b/docs/plans/universal-harness-access-phase4.md index 3533570825..5a09d33531 100644 --- a/docs/plans/universal-harness-access-phase4.md +++ b/docs/plans/universal-harness-access-phase4.md @@ -44,12 +44,12 @@ A `fullsend-fetch-skill` binary available inside the sandbox. When the agent run ### Implementation steps -#### PR 1: Runner-side fetch service -- Unix socket listener in the runner process -- Request/response protocol: URL -> local path or error -- Rate limiting enforcement -- Forge API integration for skill directory fetching (reuses Phase 1 forge client) -- Audit logging with `fetch_type: "runtime"` +#### PR 1: Harness schema and ResolveSkillURL (this PR) +- Add `allow_runtime_fetch` and `max_runtime_fetches` to harness schema with validation +- Export `ResolveSkillURL` in `internal/resolve/` for single-URL runtime skill resolution +- Uses forge API for directory-based skill fetching (same model as static resolution) +- Audit logging with `fetch_type: "runtime"` to distinguish from static resolution +- No transitive resolution for runtime-fetched skills (leaf nodes only) #### PR 2: In-sandbox fetch binary - `fullsend-fetch-skill` binary compiled and uploaded to sandbox during bootstrap @@ -57,10 +57,12 @@ A `fullsend-fetch-skill` binary available inside the sandbox. When the agent run - Reports errors to stderr, success path to stdout - Returns the sandbox-local skill directory path (not a single file path) -#### PR 3: Harness schema and CLI integration -- Add `allow_runtime_fetch` and `max_runtime_fetches` to harness schema -- Validation: reject runtime fetch fields if `allowed_remote_resources` is empty +#### PR 3: Runner-side fetch service and CLI wiring +- Unix socket listener in the runner process +- Request/response protocol: URL -> local path or error +- Rate limiting enforcement (uses `MaxRuntimeFetches` / `DefaultMaxRuntimeFetches`) - Socket setup in sandbox provisioning +- Wire `ResolveSkillURL` into the socket handler ## Verification diff --git a/docs/plans/universal-harness-access.md b/docs/plans/universal-harness-access.md index 669bcf7668..c45e573006 100644 --- a/docs/plans/universal-harness-access.md +++ b/docs/plans/universal-harness-access.md @@ -317,9 +317,9 @@ Resolution algorithm: **Implementation:** New package `internal/resolve/` provides `ResolveHarness(ctx, h *harness.Harness, opts ResolveOpts) ([]Dependency, error)`. -### Runtime Dependency Loading (Future) +### Runtime Dependency Loading (In Progress — Phase 4) -The current design requires all dependencies to be declared in the harness. A future enhancement would allow agents to discover and load resources at runtime: +The current design requires all dependencies to be declared in the harness. Phase 4 adds runtime dependency loading, allowing agents to discover and load resources during execution: ```markdown # Agent encounters unfamiliar code @@ -340,7 +340,7 @@ This requires: - Fetch requests are rate-limited (max 10 per agent run) - Anomalous fetch patterns trigger alerts -**Status:** Not implemented in initial design. Tracked in a future issue. +**Status:** In progress. Schema fields (`allow_runtime_fetch`, `max_runtime_fetches`) and `ResolveSkillURL` are implemented. See `docs/plans/universal-harness-access-phase4.md` for the full plan. ### Access Policy Model @@ -367,7 +367,7 @@ The runner enforces: - Transitive dependencies must also match an allowed prefix - No runtime fetches are allowed (agent cannot fetch new resources during execution) -**Phase 2: Runtime fetch with policy (future)** +**Phase 2: Runtime fetch with policy (in progress — Phase 4)** The harness declares allowed prefixes, and the agent can fetch resources at runtime if they match: @@ -1420,14 +1420,14 @@ harnesses: - If harness YAML references change but lock file is stale, warn: "harness/code.yaml has changed since lock file was generated. Run `fullsend lock harness/code.yaml` to update." - `fullsend lock --update` re-resolves all dependencies and updates lock file -### Phase 4: Runtime dependency loading +### Phase 4: Runtime dependency loading (in progress) -- Implement `fullsend-fetch-skill` binary for sandbox use -- Add `allow_runtime_fetch: true` flag to harness schema -- Enforce runtime fetches against `allowed_remote_resources` -- Audit log all runtime fetches +- ~~Add `allow_runtime_fetch: true` flag to harness schema~~ (done — PR 1) +- ~~Add `ResolveSkillURL` for runtime fetch resolution~~ (done — PR 1) +- Implement `fullsend-fetch-skill` script for sandbox use (PR 2) +- Wire runtime fetch handler goroutine into `fullsend run` (PR 3) -**Deliverable:** Agents can fetch skills mid-run if the harness allows it +**Deliverable:** Agents can fetch skills mid-run if the harness allows it. See `docs/plans/universal-harness-access-phase4.md` for the detailed plan. ## Testing Strategy diff --git a/e2e/admin/admin_test.go b/e2e/admin/admin_test.go index 948832d44d..6022346c65 100644 --- a/e2e/admin/admin_test.go +++ b/e2e/admin/admin_test.go @@ -24,7 +24,6 @@ import ( "github.com/fullsend-ai/fullsend/internal/config" "github.com/fullsend-ai/fullsend/internal/forge" gh "github.com/fullsend-ai/fullsend/internal/forge/github" - "github.com/fullsend-ai/fullsend/internal/layers" ) // e2eEnv holds the shared state for an e2e test run. @@ -652,33 +651,3 @@ func runUnenrollmentTest(t *testing.T, env *e2eEnv) { require.True(t, forge.IsNotFound(err), "shim should be removed from %s after unenrollment", testRepo) t.Log("Verified shim is gone") } - -// TestVendorFromSubdirectory verifies that --vendor-fullsend-binary cross-compiles -// when the CLI is run from a subdirectory inside the module (GOMOD discovery). -func TestVendorFromSubdirectory(t *testing.T) { - env := setupE2ETest(t) - ctx := context.Background() - - subdir := filepath.Join(moduleRoot(t), "internal", "cli") - installArgs := []string{ - "admin", "install", env.org, - "--skip-app-setup", - "--skip-mint-check", - "--mint-url", env.cfg.mintURL, - "--app-set", e2eAppSet, - "--enroll-none", - "--vendor-fullsend-binary", - } - runCLIFromDir(t, env.binary, env.token, subdir, installArgs...) - - _, err := env.client.GetFileContent(ctx, env.org, forge.ConfigRepoName, layers.VendoredBinaryPath) - require.NoError(t, err, "vendored binary should exist at %s", layers.VendoredBinaryPath) - - registerRepoCleanup(t, env.client, env.org, forge.ConfigRepoName) - - runCLI(t, env.binary, env.token, - "admin", "uninstall", env.org, - "--yolo", - "--app-set", e2eAppSet, - ) -} diff --git a/e2e/admin/testutil.go b/e2e/admin/testutil.go index b19d46330b..d4e3bdbf8e 100644 --- a/e2e/admin/testutil.go +++ b/e2e/admin/testutil.go @@ -260,19 +260,19 @@ func buildCLIBinary(t *testing.T) string { } // runCLI executes the fullsend CLI with the given args, passing GITHUB_TOKEN. -// By default the working directory is the module root. Use runCLIFromDir to -// run from a subdirectory (GOMOD discovery makes this work for vendoring). +// The working directory is set to the module root so that --vendor-fullsend-binary +// can find ./cmd/fullsend/ (same as a user running from the repo root). func runCLI(t *testing.T, binary, token string, args ...string) string { - return runCLIFromDir(t, binary, token, moduleRoot(t), args...) -} - -// runCLIFromDir runs the CLI with cwd set to dir. -func runCLIFromDir(t *testing.T, binary, token, dir string, args ...string) string { t.Helper() - t.Logf("[cli] fullsend %s (cwd=%s)", strings.Join(args, " "), dir) + t.Logf("[cli] fullsend %s", strings.Join(args, " ")) + + modRoot, err := exec.Command("go", "list", "-m", "-f", "{{.Dir}}").Output() + if err != nil { + t.Fatalf("finding module root for runCLI: %v", err) + } cmd := exec.Command(binary, args...) - cmd.Dir = dir + cmd.Dir = strings.TrimSpace(string(modRoot)) cmd.Env = append(os.Environ(), "GITHUB_TOKEN="+token, "CI=true") out, runErr := cmd.CombinedOutput() output := string(out) @@ -283,15 +283,6 @@ func runCLIFromDir(t *testing.T, binary, token, dir string, args ...string) stri return output } -func moduleRoot(t *testing.T) string { - t.Helper() - modRoot, err := exec.Command("go", "list", "-m", "-f", "{{.Dir}}").Output() - if err != nil { - t.Fatalf("finding module root: %v", err) - } - return strings.TrimSpace(string(modRoot)) -} - // retryOnNotFound retries an operation up to maxAttempts times with linear // backoff when it returns a not-found error (GitHub eventual consistency). func retryOnNotFound(ctx context.Context, maxAttempts int, fn func() error) error { diff --git a/internal/binary/acquire.go b/internal/binary/acquire.go deleted file mode 100644 index 0f7e70d9ad..0000000000 --- a/internal/binary/acquire.go +++ /dev/null @@ -1,115 +0,0 @@ -package binary - -import ( - "fmt" - "os" - "path/filepath" -) - -// Source identifies how a Linux fullsend binary was obtained. -type Source int - -const ( - SourceExplicitPath Source = iota - SourceCheckoutBuild - SourceReleaseDownload -) - -// AcquireResult holds the path to an acquired binary and metadata for callers. -type AcquireResult struct { - TmpDir string // caller must RemoveAll when non-empty - Path string - Source Source -} - -// ResolveExplicit validates that path is a Linux ELF for arch. -func ResolveExplicit(path, arch string) error { - return ValidateLinuxBinary(path, arch) -} - -// ResolveForRun obtains a Linux binary using the run policy: -// release download (if released) → cross-compile → latest release. -func ResolveForRun(version, arch string) (AcquireResult, error) { - tmpDir, err := os.MkdirTemp("", "fullsend-linux-*") - if err != nil { - return AcquireResult{}, fmt.Errorf("creating temp dir: %w", err) - } - binaryPath := filepath.Join(tmpDir, "fullsend") - - // 1. Released version → download matching release asset. - if IsReleasedVersion(version) { - fmt.Fprintf(os.Stderr, "Downloading fullsend %s for linux/%s from GitHub Release...\n", version, arch) - if dlErr := DownloadRelease(version, arch, binaryPath); dlErr == nil { - fmt.Fprintf(os.Stderr, "Downloaded fullsend for linux/%s\n", arch) - return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceReleaseDownload}, nil - } else { - fmt.Fprintf(os.Stderr, "WARNING: release download failed: %v\n", dlErr) - } - } - - // 2. Try cross-compilation (requires Go toolchain + module checkout). - fmt.Fprintf(os.Stderr, "Cross-compiling fullsend for linux/%s...\n", arch) - if ccErr := CrossCompile(CrossCompileOpts{ - Version: version, - Arch: arch, - DestPath: binaryPath, - VersionStamp: "-crosscompiled", - }); ccErr == nil { - fmt.Fprintf(os.Stderr, "Cross-compiled fullsend for linux/%s\n", arch) - return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceCheckoutBuild}, nil - } else { - fmt.Fprintf(os.Stderr, "WARNING: cross-compilation failed: %v\n", ccErr) - } - - // 3. Last resort → download latest release. - fmt.Fprintf(os.Stderr, "Downloading latest fullsend release for linux/%s...\n", arch) - latestErr := DownloadLatestRelease(arch, binaryPath) - if latestErr == nil { - fmt.Fprintf(os.Stderr, "Downloaded latest fullsend for linux/%s\n", arch) - return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceReleaseDownload}, nil - } - fmt.Fprintf(os.Stderr, "WARNING: latest release download failed: %v\n", latestErr) - - os.RemoveAll(tmpDir) - return AcquireResult{}, fmt.Errorf("all strategies failed for linux/%s: provide --fullsend-binary or install Go toolchain", arch) -} - -// ResolveForVendor obtains a Linux binary using the vendoring policy: -// cross-compile from checkout → matching release (released CLI only) → fail. -// No latest-release fallback. -func ResolveForVendor(version, arch string) (AcquireResult, error) { - tmpDir, err := os.MkdirTemp("", "fullsend-linux-*") - if err != nil { - return AcquireResult{}, fmt.Errorf("creating temp dir: %w", err) - } - binaryPath := filepath.Join(tmpDir, "fullsend") - - // 1. Cross-compile from checkout. - fmt.Fprintf(os.Stderr, "Cross-compiling fullsend for linux/%s...\n", arch) - if ccErr := CrossCompile(CrossCompileOpts{ - Version: version, - Arch: arch, - DestPath: binaryPath, - VersionStamp: "-vendored", - }); ccErr == nil { - fmt.Fprintf(os.Stderr, "Cross-compiled fullsend for linux/%s\n", arch) - return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceCheckoutBuild}, nil - } else { - fmt.Fprintf(os.Stderr, "WARNING: cross-compilation failed: %v\n", ccErr) - } - - // 2. Release fetch only for released CLI versions. - if IsReleasedVersion(version) { - fmt.Fprintf(os.Stderr, "Downloading fullsend %s for linux/%s from GitHub Release...\n", version, arch) - if dlErr := DownloadRelease(version, arch, binaryPath); dlErr == nil { - fmt.Fprintf(os.Stderr, "Downloaded fullsend for linux/%s\n", arch) - return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceReleaseDownload}, nil - } else { - os.RemoveAll(tmpDir) - return AcquireResult{}, fmt.Errorf("cross-compilation unavailable and release download failed for v%s: %w", version, dlErr) - } - } - - os.RemoveAll(tmpDir) - return AcquireResult{}, fmt.Errorf("cannot vendor binary: not in fullsend source tree and CLI version %s is a dev build — use --fullsend-binary, run from a checkout, or use a released CLI", version) -} diff --git a/internal/binary/crosscompile.go b/internal/binary/crosscompile.go deleted file mode 100644 index d71b0407ae..0000000000 --- a/internal/binary/crosscompile.go +++ /dev/null @@ -1,64 +0,0 @@ -package binary - -import ( - "fmt" - "os" - "os/exec" - "path/filepath" - "strings" -) - -// CrossCompileOpts configures a cross-compilation build. -type CrossCompileOpts struct { - Version string // CLI version to embed (before stamp suffix) - Arch string - DestPath string - VersionStamp string // e.g. "-vendored", "-crosscompiled", or "" -} - -// ModuleRoot returns the fullsend module root directory, or an error if not -// inside a Go module checkout. -func ModuleRoot() (string, error) { - goPath, lookErr := exec.LookPath("go") - if lookErr != nil { - return "", fmt.Errorf("Go toolchain not found: %w", lookErr) - } - modRootCmd := exec.Command(goPath, "env", "GOMOD") - modOutput, err := modRootCmd.Output() - if err != nil { - return "", fmt.Errorf("finding module root: %w", err) - } - modPath := strings.TrimSpace(string(modOutput)) - if modPath == "" || modPath == os.DevNull { - return "", fmt.Errorf("not in a Go module") - } - return filepath.Dir(modPath), nil -} - -// CrossCompile builds a Linux fullsend binary and writes it to DestPath. -// Requires the Go toolchain and a fullsend module checkout (go env GOMOD). -func CrossCompile(opts CrossCompileOpts) error { - goPath, lookErr := exec.LookPath("go") - if lookErr != nil { - return fmt.Errorf("Go toolchain not found — install Go or use a released version of fullsend: %w", lookErr) - } - - modRoot, err := ModuleRoot() - if err != nil { - return fmt.Errorf("not in a Go module — run from the fullsend source tree or use a released version: %w", err) - } - - versionLD := opts.Version + opts.VersionStamp - buildCmd := exec.Command(goPath, "build", - "-ldflags", fmt.Sprintf("-X github.com/fullsend-ai/fullsend/internal/cli.version=%s", versionLD), - "-o", opts.DestPath, - "./cmd/fullsend/", - ) - buildCmd.Dir = modRoot - buildCmd.Env = append(os.Environ(), "GOTOOLCHAIN=auto", "GOOS=linux", "GOARCH="+opts.Arch, "CGO_ENABLED=0") - buildCmd.Stderr = os.Stderr - if err := buildCmd.Run(); err != nil { - return fmt.Errorf("cross-compiling for linux/%s: %w", opts.Arch, err) - } - return nil -} diff --git a/internal/binary/download.go b/internal/binary/download.go deleted file mode 100644 index 8714a34555..0000000000 --- a/internal/binary/download.go +++ /dev/null @@ -1,184 +0,0 @@ -package binary - -import ( - "archive/tar" - "bufio" - "bytes" - "compress/gzip" - "crypto/sha256" - "encoding/hex" - "encoding/json" - "fmt" - "io" - "net/http" - "os" - "path/filepath" - "strings" - "time" -) - -// ReleaseBaseURL is the GitHub releases download base URL. Tests may override. -// Not safe for concurrent test mutation. -var ReleaseBaseURL = "https://github.com/fullsend-ai/fullsend/releases/download" - -// HTTPClient is used for release downloads. Tests may override. -// Not safe for concurrent test mutation. -var HTTPClient = &http.Client{Timeout: 120 * time.Second} - -const defaultMaxDownloadSize = 200 * 1024 * 1024 // 200 MB compressed - -// maxDownloadSize caps release asset downloads. Tests may lower temporarily. -var maxDownloadSize = defaultMaxDownloadSize - -const maxBinarySize = 500 * 1024 * 1024 // 500 MB — reasonable upper bound for a Go binary - -// DownloadRelease downloads the fullsend binary for linux/{arch} from the -// GitHub Release matching the given version, verifies its SHA256 checksum -// against the release checksums.txt, and writes it to destPath. -func DownloadRelease(ver, arch, destPath string) error { - cleanVer := strings.TrimPrefix(ver, "v") - assetName := fmt.Sprintf("fullsend_%s_linux_%s.tar.gz", cleanVer, arch) - - expectedHash, err := downloadChecksumForAsset(ver, assetName) - if err != nil { - return fmt.Errorf("fetching checksum for %s: %w", assetName, err) - } - - url := fmt.Sprintf("%s/v%s/%s", ReleaseBaseURL, cleanVer, assetName) - resp, err := HTTPClient.Get(url) //nolint:gosec // URL is constructed from known constants - if err != nil { - return fmt.Errorf("fetching %s: %w", url, err) - } - defer resp.Body.Close() - - if resp.StatusCode != http.StatusOK { - return fmt.Errorf("GET %s returned %d", url, resp.StatusCode) - } - - maxSize := int64(maxDownloadSize) - var buf bytes.Buffer - if _, err := io.Copy(&buf, io.LimitReader(resp.Body, maxSize+1)); err != nil { - return fmt.Errorf("reading %s: %w", assetName, err) - } - if int64(buf.Len()) > maxSize { - return fmt.Errorf("download of %s exceeds maximum size (%d bytes)", assetName, maxSize) - } - - h := sha256.Sum256(buf.Bytes()) - actualHash := hex.EncodeToString(h[:]) - if actualHash != expectedHash { - return fmt.Errorf("checksum mismatch for %s: got %s, want %s", assetName, actualHash, expectedHash) - } - - return ExtractFullsendFromTarGz(bytes.NewReader(buf.Bytes()), destPath) -} - -func downloadChecksumForAsset(ver, assetName string) (string, error) { - cleanVer := strings.TrimPrefix(ver, "v") - url := fmt.Sprintf("%s/v%s/checksums.txt", ReleaseBaseURL, cleanVer) - - resp, err := HTTPClient.Get(url) //nolint:gosec // URL is constructed from known constants - if err != nil { - return "", fmt.Errorf("fetching checksums: %w", err) - } - defer resp.Body.Close() - - if resp.StatusCode != http.StatusOK { - return "", fmt.Errorf("GET %s returned %d", url, resp.StatusCode) - } - - scanner := bufio.NewScanner(io.LimitReader(resp.Body, 64*1024)) - for scanner.Scan() { - line := scanner.Text() - parts := strings.Fields(line) - if len(parts) == 2 && parts[1] == assetName { - hash := strings.ToLower(parts[0]) - if len(hash) != 64 { - return "", fmt.Errorf("invalid hash length for %s in checksums.txt", assetName) - } - if _, err := hex.DecodeString(hash); err != nil { - return "", fmt.Errorf("invalid hex hash for %s in checksums.txt: %w", assetName, err) - } - return hash, nil - } - } - if err := scanner.Err(); err != nil { - return "", fmt.Errorf("reading checksums: %w", err) - } - return "", fmt.Errorf("asset %s not found in checksums.txt", assetName) -} - -// DownloadLatestRelease resolves the latest release tag from the GitHub API -// and downloads the Linux binary for the given arch. -func DownloadLatestRelease(arch, destPath string) error { - tag, err := resolveLatestReleaseTag() - if err != nil { - return err - } - return DownloadRelease(tag, arch, destPath) -} - -func resolveLatestReleaseTag() (string, error) { - resp, err := HTTPClient.Get("https://api.github.com/repos/fullsend-ai/fullsend/releases/latest") //nolint:gosec - if err != nil { - return "", fmt.Errorf("fetching latest release: %w", err) - } - defer resp.Body.Close() - - if resp.StatusCode != http.StatusOK { - return "", fmt.Errorf("GitHub API returned %d", resp.StatusCode) - } - - var release struct { - TagName string `json:"tag_name"` - } - if err := json.NewDecoder(io.LimitReader(resp.Body, 1024*1024)).Decode(&release); err != nil { - return "", fmt.Errorf("parsing release JSON: %w", err) - } - if release.TagName == "" { - return "", fmt.Errorf("empty tag_name in latest release") - } - return release.TagName, nil -} - -// ExtractFullsendFromTarGz reads a tar.gz stream and extracts the "fullsend" -// binary to destPath. -func ExtractFullsendFromTarGz(r io.Reader, destPath string) error { - gz, err := gzip.NewReader(r) - if err != nil { - return fmt.Errorf("gzip reader: %w", err) - } - defer gz.Close() - - tr := tar.NewReader(gz) - for { - hdr, err := tr.Next() - if err == io.EOF { - return fmt.Errorf("fullsend binary not found in archive") - } - if err != nil { - return fmt.Errorf("reading tar: %w", err) - } - clean := filepath.Clean(hdr.Name) - if strings.Contains(clean, "..") || filepath.IsAbs(clean) { - continue - } - if filepath.Base(clean) == "fullsend" && hdr.Typeflag == tar.TypeReg { - f, err := os.OpenFile(destPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o755) - if err != nil { - return fmt.Errorf("creating %s: %w", destPath, err) - } - n, copyErr := io.Copy(f, io.LimitReader(tr, maxBinarySize+1)) - if copyErr != nil { - f.Close() - return fmt.Errorf("extracting fullsend: %w", copyErr) - } - if n > maxBinarySize { - f.Close() - os.Remove(destPath) - return fmt.Errorf("binary exceeds maximum size (%d bytes)", maxBinarySize) - } - return f.Close() - } - } -} diff --git a/internal/binary/download_test.go b/internal/binary/download_test.go deleted file mode 100644 index 23b20db993..0000000000 --- a/internal/binary/download_test.go +++ /dev/null @@ -1,580 +0,0 @@ -package binary - -import ( - "archive/tar" - "bytes" - "compress/gzip" - "crypto/sha256" - "encoding/hex" - "fmt" - "io" - "net/http" - "net/http/httptest" - "os" - "path/filepath" - "runtime" - "strings" - "sync/atomic" - "testing" - "time" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -type redirectTransport struct { - srvURL string - base http.RoundTripper -} - -func (t redirectTransport) RoundTrip(req *http.Request) (*http.Response, error) { - clone := req.Clone(req.Context()) - clone.URL.Scheme = "http" - clone.URL.Host = strings.TrimPrefix(strings.TrimPrefix(t.srvURL, "https://"), "http://") - if t.base == nil { - t.base = http.DefaultTransport - } - return t.base.RoundTrip(clone) -} - -func withTestReleaseServer(t *testing.T, srv *httptest.Server) { - t.Helper() - origClient := HTTPClient - origBaseURL := ReleaseBaseURL - HTTPClient = &http.Client{ - Transport: redirectTransport{srvURL: srv.URL}, - Timeout: 120 * time.Second, - } - ReleaseBaseURL = srv.URL - t.Cleanup(func() { - HTTPClient = origClient - ReleaseBaseURL = origBaseURL - }) -} - -func TestExtractFullsendFromTarGz_PathTraversal(t *testing.T) { - var buf bytes.Buffer - gw := gzip.NewWriter(&buf) - tw := tar.NewWriter(gw) - - content := []byte("malicious binary content") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "../../../tmp/fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - destPath := filepath.Join(t.TempDir(), "fullsend") - err = ExtractFullsendFromTarGz(&buf, destPath) - assert.Error(t, err) - assert.Contains(t, err.Error(), "not found in archive") -} - -func TestExtractFullsendFromTarGz_ValidEntry(t *testing.T) { - var buf bytes.Buffer - gw := gzip.NewWriter(&buf) - tw := tar.NewWriter(gw) - - content := []byte("valid binary content") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "fullsend_0.4.0_linux_amd64/fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - destPath := filepath.Join(t.TempDir(), "fullsend") - err = ExtractFullsendFromTarGz(&buf, destPath) - require.NoError(t, err) - - data, err := os.ReadFile(destPath) - require.NoError(t, err) - assert.Equal(t, "valid binary content", string(data)) -} - -func TestDownloadChecksumForAsset_ParsesLine(t *testing.T) { - body := "1b4f0e9851971998e732078544c96b36c3d01cedf7caa332359d6f1d83567014 fullsend_1.0.0_linux_arm64.tar.gz\n" + - "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752 fullsend_1.0.0_linux_amd64.tar.gz\n" - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - fmt.Fprint(w, body) - })) - defer srv.Close() - - origBaseURL := ReleaseBaseURL - ReleaseBaseURL = srv.URL - defer func() { ReleaseBaseURL = origBaseURL }() - - hash, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_amd64.tar.gz") - require.NoError(t, err) - assert.Equal(t, "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752", hash) -} - -func TestDownloadChecksumForAsset_AssetNotFound(t *testing.T) { - body := "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752 fullsend_1.0.0_linux_amd64.tar.gz\n" - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - fmt.Fprint(w, body) - })) - defer srv.Close() - - origBaseURL := ReleaseBaseURL - ReleaseBaseURL = srv.URL - defer func() { ReleaseBaseURL = origBaseURL }() - - _, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_arm64.tar.gz") - require.Error(t, err) - assert.Contains(t, err.Error(), "not found in checksums.txt") -} - -func TestDownloadChecksumForAsset_InvalidHex(t *testing.T) { - body := "ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ fullsend_1.0.0_linux_amd64.tar.gz\n" - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - fmt.Fprint(w, body) - })) - defer srv.Close() - - origBaseURL := ReleaseBaseURL - ReleaseBaseURL = srv.URL - defer func() { ReleaseBaseURL = origBaseURL }() - - _, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_amd64.tar.gz") - require.Error(t, err) - assert.Contains(t, err.Error(), "invalid hex hash") -} - -func TestDownloadReleaseBinary_ChecksumMismatch(t *testing.T) { - var tarBuf bytes.Buffer - gw := gzip.NewWriter(&tarBuf) - tw := tar.NewWriter(gw) - content := []byte("fake binary") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - wrongHash := "0000000000000000000000000000000000000000000000000000000000000000" - checksumBody := fmt.Sprintf("%s fullsend_1.0.0_linux_amd64.tar.gz\n", wrongHash) - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path == "/v1.0.0/checksums.txt" { - fmt.Fprint(w, checksumBody) - } else if r.URL.Path == "/v1.0.0/fullsend_1.0.0_linux_amd64.tar.gz" { - w.Write(tarBuf.Bytes()) - } else { - http.NotFound(w, r) - } - })) - defer srv.Close() - - origBaseURL := ReleaseBaseURL - ReleaseBaseURL = srv.URL - defer func() { ReleaseBaseURL = origBaseURL }() - - destPath := filepath.Join(t.TempDir(), "fullsend") - err = DownloadRelease("1.0.0", "amd64", destPath) - require.Error(t, err) - assert.Contains(t, err.Error(), "checksum mismatch") -} - -func TestDownloadReleaseBinary_ChecksumMatch(t *testing.T) { - var tarBuf bytes.Buffer - gw := gzip.NewWriter(&tarBuf) - tw := tar.NewWriter(gw) - content := []byte("good binary") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - tarBytes := tarBuf.Bytes() - h := sha256.Sum256(tarBytes) - correctHash := hex.EncodeToString(h[:]) - checksumBody := fmt.Sprintf("%s fullsend_2.0.0_linux_amd64.tar.gz\n", correctHash) - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path == "/v2.0.0/checksums.txt" { - fmt.Fprint(w, checksumBody) - } else if r.URL.Path == "/v2.0.0/fullsend_2.0.0_linux_amd64.tar.gz" { - w.Write(tarBytes) - } else { - http.NotFound(w, r) - } - })) - defer srv.Close() - - origBaseURL := ReleaseBaseURL - ReleaseBaseURL = srv.URL - defer func() { ReleaseBaseURL = origBaseURL }() - - destPath := filepath.Join(t.TempDir(), "fullsend") - err = DownloadRelease("2.0.0", "amd64", destPath) - require.NoError(t, err) - - data, err := os.ReadFile(destPath) - require.NoError(t, err) - assert.Equal(t, "good binary", string(data)) -} - -func TestDownloadRelease_Live(t *testing.T) { - if testing.Short() { - t.Skip("skipping download test in short mode") - } - - destPath := filepath.Join(t.TempDir(), "fullsend") - err := DownloadRelease("0.4.0", "amd64", destPath) - require.NoError(t, err) - - info, err := os.Stat(destPath) - require.NoError(t, err) - assert.True(t, info.Size() > 0) -} - -func TestCrossCompile_ProducesBinary(t *testing.T) { - if runtime.GOOS == "linux" { - t.Skip("cross-compilation test only meaningful on non-Linux hosts") - } - if testing.Short() { - t.Skip("skipping cross-compilation in short mode") - } - - tmpDir := t.TempDir() - binPath := filepath.Join(tmpDir, "fullsend") - err := CrossCompile(CrossCompileOpts{ - Version: "dev", - Arch: runtime.GOARCH, - DestPath: binPath, - VersionStamp: "-crosscompiled", - }) - require.NoError(t, err) - - info, err := os.Stat(binPath) - require.NoError(t, err) - assert.True(t, info.Size() > 0) -} - -func TestValidateLinuxBinary_RejectsNonELF(t *testing.T) { - tmp := filepath.Join(t.TempDir(), "not-elf") - require.NoError(t, os.WriteFile(tmp, []byte("#!/bin/sh\necho hello"), 0o755)) - err := ValidateLinuxBinary(tmp, "amd64") - require.Error(t, err) - assert.Contains(t, err.Error(), "not a valid ELF binary") -} - -func TestValidateLinuxBinary_RejectsMissing(t *testing.T) { - err := ValidateLinuxBinary("/tmp/nonexistent-fullsend-binary-12345", "amd64") - require.Error(t, err) -} - -func TestValidateLinuxBinary_AcceptsHostBinary(t *testing.T) { - if runtime.GOOS != "linux" { - t.Skip("host binary is only ELF on Linux") - } - exe, err := os.Executable() - require.NoError(t, err) - assert.NoError(t, ValidateLinuxBinary(exe, runtime.GOARCH)) -} - -func TestResolveForVendor_DevNoCheckoutFails(t *testing.T) { - // Force no module by running from a temp dir without go.mod. - origDir, err := os.Getwd() - require.NoError(t, err) - tmpDir := t.TempDir() - require.NoError(t, os.Chdir(tmpDir)) - t.Cleanup(func() { _ = os.Chdir(origDir) }) - - _, err = ResolveForVendor("dev", "amd64") - require.Error(t, err) - assert.Contains(t, err.Error(), "dev build") -} - -func TestResolveForVendor_NoLatestFallback(t *testing.T) { - var latestCalls atomic.Int32 - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if strings.Contains(r.URL.Path, "/releases/latest") { - latestCalls.Add(1) - } - http.NotFound(w, r) - })) - defer srv.Close() - - origClient := HTTPClient - origBaseURL := ReleaseBaseURL - HTTPClient = srv.Client() - ReleaseBaseURL = srv.URL - defer func() { - HTTPClient = origClient - ReleaseBaseURL = origBaseURL - }() - - origDir, err := os.Getwd() - require.NoError(t, err) - tmpDir := t.TempDir() - require.NoError(t, os.Chdir(tmpDir)) - t.Cleanup(func() { _ = os.Chdir(origDir) }) - - _, err = ResolveForVendor("0.4.0", "amd64") - require.Error(t, err) - assert.Equal(t, int32(0), latestCalls.Load(), "vendor path must not call latest release API") - assert.NotContains(t, err.Error(), "latest") -} - -func TestResolveForVendor_ReleaseFallback(t *testing.T) { - var tarBuf bytes.Buffer - gw := gzip.NewWriter(&tarBuf) - tw := tar.NewWriter(gw) - content := []byte("release binary") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - tarBytes := tarBuf.Bytes() - h := sha256.Sum256(tarBytes) - correctHash := hex.EncodeToString(h[:]) - checksumBody := fmt.Sprintf("%s fullsend_0.4.0_linux_amd64.tar.gz\n", correctHash) - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path == "/v0.4.0/checksums.txt" { - fmt.Fprint(w, checksumBody) - } else if r.URL.Path == "/v0.4.0/fullsend_0.4.0_linux_amd64.tar.gz" { - w.Write(tarBytes) - } else { - http.NotFound(w, r) - } - })) - defer srv.Close() - - origBaseURL := ReleaseBaseURL - ReleaseBaseURL = srv.URL - defer func() { ReleaseBaseURL = origBaseURL }() - - origDir, err := os.Getwd() - require.NoError(t, err) - tmpDir := t.TempDir() - require.NoError(t, os.Chdir(tmpDir)) - t.Cleanup(func() { _ = os.Chdir(origDir) }) - - result, err := ResolveForVendor("0.4.0", "amd64") - require.NoError(t, err) - t.Cleanup(func() { os.RemoveAll(result.TmpDir) }) - assert.Equal(t, SourceReleaseDownload, result.Source) - - data, err := os.ReadFile(result.Path) - require.NoError(t, err) - assert.Equal(t, "release binary", string(data)) -} - -func TestResolveForRun_PrefersReleaseBeforeCrossCompile(t *testing.T) { - // Build mock release assets. - var tarBuf bytes.Buffer - gw := gzip.NewWriter(&tarBuf) - tw := tar.NewWriter(gw) - content := []byte("release binary") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - tarBytes := tarBuf.Bytes() - h := sha256.Sum256(tarBytes) - correctHash := hex.EncodeToString(h[:]) - checksumBody := fmt.Sprintf("%s fullsend_0.4.0_linux_amd64.tar.gz\n", correctHash) - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path == "/v0.4.0/checksums.txt" { - fmt.Fprint(w, checksumBody) - } else if r.URL.Path == "/v0.4.0/fullsend_0.4.0_linux_amd64.tar.gz" { - w.Write(tarBytes) - } else { - http.NotFound(w, r) - } - })) - defer srv.Close() - - origBaseURL := ReleaseBaseURL - ReleaseBaseURL = srv.URL - defer func() { ReleaseBaseURL = origBaseURL }() - - // Run from non-module dir — cross-compile would fail if attempted after release. - origDir, err := os.Getwd() - require.NoError(t, err) - tmpDir := t.TempDir() - require.NoError(t, os.Chdir(tmpDir)) - t.Cleanup(func() { _ = os.Chdir(origDir) }) - - result, err := ResolveForRun("0.4.0", "amd64") - require.NoError(t, err) - t.Cleanup(func() { os.RemoveAll(result.TmpDir) }) - assert.Equal(t, SourceReleaseDownload, result.Source) -} - -func TestDownloadRelease_ExceedsMaxSize(t *testing.T) { - origLimit := maxDownloadSize - maxDownloadSize = 512 - t.Cleanup(func() { maxDownloadSize = origLimit }) - - content := bytes.Repeat([]byte("x"), 2000) - - var tarBuf bytes.Buffer - gw, err := gzip.NewWriterLevel(&tarBuf, gzip.NoCompression) - require.NoError(t, err) - tw := tar.NewWriter(gw) - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err = tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - tarBytes := tarBuf.Bytes() - h := sha256.Sum256(tarBytes) - checksumBody := fmt.Sprintf("%s fullsend_1.0.0_linux_amd64.tar.gz\n", hex.EncodeToString(h[:])) - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path == "/v1.0.0/checksums.txt" { - fmt.Fprint(w, checksumBody) - } else if r.URL.Path == "/v1.0.0/fullsend_1.0.0_linux_amd64.tar.gz" { - w.Write(tarBytes) - } else { - http.NotFound(w, r) - } - })) - defer srv.Close() - withTestReleaseServer(t, srv) - - destPath := filepath.Join(t.TempDir(), "fullsend") - err = DownloadRelease("1.0.0", "amd64", destPath) - require.Error(t, err) - assert.Contains(t, err.Error(), "exceeds maximum size") -} - -func TestResolveForRun_CrossCompileFallback(t *testing.T) { - if testing.Short() { - t.Skip("skipping cross-compilation in short mode") - } - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - http.NotFound(w, r) - })) - defer srv.Close() - withTestReleaseServer(t, srv) - - result, err := ResolveForRun("0.4.0", "amd64") - require.NoError(t, err) - t.Cleanup(func() { os.RemoveAll(result.TmpDir) }) - assert.Equal(t, SourceCheckoutBuild, result.Source) -} - -func TestResolveForRun_LatestReleaseFallback(t *testing.T) { - var tarBuf bytes.Buffer - gw := gzip.NewWriter(&tarBuf) - tw := tar.NewWriter(gw) - content := []byte("latest release binary") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - tarBytes := tarBuf.Bytes() - h := sha256.Sum256(tarBytes) - correctHash := hex.EncodeToString(h[:]) - checksumBody := fmt.Sprintf("%s fullsend_9.9.9_linux_amd64.tar.gz\n", correctHash) - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path == "/repos/fullsend-ai/fullsend/releases/latest" { - fmt.Fprint(w, `{"tag_name":"v9.9.9"}`) - } else if r.URL.Path == "/v9.9.9/checksums.txt" { - fmt.Fprint(w, checksumBody) - } else if r.URL.Path == "/v9.9.9/fullsend_9.9.9_linux_amd64.tar.gz" { - w.Write(tarBytes) - } else { - http.NotFound(w, r) - } - })) - defer srv.Close() - withTestReleaseServer(t, srv) - - origDir, err := os.Getwd() - require.NoError(t, err) - tmpDir := t.TempDir() - require.NoError(t, os.Chdir(tmpDir)) - t.Cleanup(func() { _ = os.Chdir(origDir) }) - - result, err := ResolveForRun("dev", "amd64") - require.NoError(t, err) - t.Cleanup(func() { os.RemoveAll(result.TmpDir) }) - assert.Equal(t, SourceReleaseDownload, result.Source) -} - -func TestResolveForRun_AllStrategiesFail(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - http.NotFound(w, r) - })) - defer srv.Close() - withTestReleaseServer(t, srv) - - origDir, err := os.Getwd() - require.NoError(t, err) - tmpDir := t.TempDir() - require.NoError(t, os.Chdir(tmpDir)) - t.Cleanup(func() { _ = os.Chdir(origDir) }) - - _, err = ResolveForRun("dev", "amd64") - require.Error(t, err) - assert.Contains(t, err.Error(), "all strategies failed") -} - -func TestResolveExplicit_ValidatesELF(t *testing.T) { - tmp := filepath.Join(t.TempDir(), "not-elf") - require.NoError(t, os.WriteFile(tmp, []byte("not binary"), 0o644)) - err := ResolveExplicit(tmp, "amd64") - require.Error(t, err) -} - -// Ensure io is used in download tests. -var _ = io.Discard diff --git a/internal/binary/validate.go b/internal/binary/validate.go deleted file mode 100644 index 64decfb0ae..0000000000 --- a/internal/binary/validate.go +++ /dev/null @@ -1,40 +0,0 @@ -package binary - -import ( - "debug/elf" - "fmt" -) - -// DefaultArch is the architecture used for vendored binaries (linux/amd64 GHA runners). -const DefaultArch = "amd64" - -var validArchs = map[string]bool{"amd64": true, "arm64": true} - -// ValidateLinuxBinary checks that the file at path is a Linux ELF executable -// for the expected architecture. Returns a descriptive error if the file is -// missing, not ELF, not Linux, or the wrong architecture. -func ValidateLinuxBinary(path, arch string) error { - f, err := elf.Open(path) - if err != nil { - return fmt.Errorf("not a valid ELF binary (is this a macOS Mach-O?): %w", err) - } - defer f.Close() - - if f.OSABI != elf.ELFOSABI_NONE && f.OSABI != elf.ELFOSABI_LINUX { - return fmt.Errorf("ELF OS/ABI is %s, expected Linux or NONE", f.OSABI) - } - - archToMachine := map[string]elf.Machine{ - "amd64": elf.EM_X86_64, - "arm64": elf.EM_AARCH64, - } - if expected, ok := archToMachine[arch]; ok && f.Machine != expected { - return fmt.Errorf("ELF machine is %s, expected %s for %s", f.Machine, expected, arch) - } - return nil -} - -// ValidArch reports whether arch is a supported linux target (amd64 or arm64). -func ValidArch(arch string) bool { - return validArchs[arch] -} diff --git a/internal/binary/version.go b/internal/binary/version.go deleted file mode 100644 index 82fcc42936..0000000000 --- a/internal/binary/version.go +++ /dev/null @@ -1,20 +0,0 @@ -package binary - -import "strings" - -// IsReleasedVersion returns true if version looks like a release tag -// (e.g. "0.4.0", "v0.4.0") rather than a dev build (e.g. "dev", -// "0.4.0-3-gabcdef", "0.4.0-vendored"). -func IsReleasedVersion(v string) bool { - v = strings.TrimPrefix(v, "v") - if v == "" || v == "dev" { - return false - } - // A released version is purely digits and dots (e.g. "0.4.0"). - for _, c := range v { - if c != '.' && (c < '0' || c > '9') { - return false - } - } - return true -} diff --git a/internal/binary/version_test.go b/internal/binary/version_test.go deleted file mode 100644 index 5c1f3213d7..0000000000 --- a/internal/binary/version_test.go +++ /dev/null @@ -1,28 +0,0 @@ -package binary - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -func TestIsReleasedVersion(t *testing.T) { - tests := []struct { - version string - expected bool - }{ - {"0.4.0", true}, - {"v0.4.0", true}, - {"1.0.0", true}, - {"dev", false}, - {"", false}, - {"0.4.0-3-gabcdef", false}, - {"0.4.0-vendored", false}, - {"0.4.0-crosscompiled", false}, - } - for _, tt := range tests { - t.Run(tt.version, func(t *testing.T) { - assert.Equal(t, tt.expected, IsReleasedVersion(tt.version), "version=%q", tt.version) - }) - } -} diff --git a/internal/cli/admin.go b/internal/cli/admin.go index 0e23ad809d..588658f262 100644 --- a/internal/cli/admin.go +++ b/internal/cli/admin.go @@ -9,6 +9,7 @@ import ( "net/url" "os" "os/exec" + "path/filepath" "regexp" "sort" "strconv" @@ -150,7 +151,6 @@ type perRepoInstallConfig struct { SkipMintCheck bool AppSet string VendorBinary bool - FullsendBinary string } // wifProviderPattern validates the full WIF provider resource name format @@ -227,7 +227,6 @@ func newInstallCmd() *cobra.Command { var dryRun bool var skipAppSetup bool var vendorBinary bool - var fullsendBinary string var enrollAllFlag bool var enrollNoneFlag bool var inferenceProject string @@ -272,9 +271,6 @@ Inference authentication: if err := appsetup.ValidateAppSet(appSet); err != nil { return fmt.Errorf("invalid --app-set: %w", err) } - if err := validateVendorBinaryFlags(vendorBinary, fullsendBinary); err != nil { - return err - } arg := args[0] if strings.Contains(arg, "/") { @@ -309,7 +305,6 @@ Inference authentication: SkipMintCheck: skipMintCheck, AppSet: appSet, VendorBinary: vendorBinary, - FullsendBinary: fullsendBinary, }) } @@ -496,7 +491,7 @@ Inference authentication: printer.Blank() if dryRun { - return runDryRun(ctx, client, printer, org, repos, roles, inferenceProvider, inferenceProviderName, skipMintCheck, mintURL, allRepos, vendorBinary, fullsendBinary) + return runDryRun(ctx, client, printer, org, repos, roles, inferenceProvider, inferenceProviderName, skipMintCheck, mintURL, vendorBinary, allRepos) } if err := checkInstallScopes(ctx, client, printer); err != nil { @@ -539,15 +534,14 @@ Inference authentication: agentCreds = creds } - return runInstall(ctx, client, printer, org, repos, roles, agentCreds, inferenceProvider, inferenceProviderName, vendorBinary, fullsendBinary, mintProvider, mintProject, mintRegion, mintSourceDir, mintSkipDeploy, mintURL, skipMintCheck, allRepos) + return runInstall(ctx, client, printer, org, repos, roles, agentCreds, inferenceProvider, inferenceProviderName, vendorBinary, mintProvider, mintProject, mintRegion, mintSourceDir, mintSkipDeploy, mintURL, skipMintCheck, allRepos) }, } cmd.Flags().StringVar(&agents, "agents", strings.Join(config.DefaultAgentRoles(), ","), "comma-separated agent roles") cmd.Flags().BoolVar(&dryRun, "dry-run", false, "preview changes without making them") cmd.Flags().BoolVar(&skipAppSetup, "skip-app-setup", false, "skip GitHub App creation/setup") - cmd.Flags().BoolVar(&vendorBinary, "vendor-fullsend-binary", false, "resolve and upload a linux/amd64 fullsend binary for CI") - cmd.Flags().StringVar(&fullsendBinary, "fullsend-binary", "", "path to a Linux fullsend binary to upload when vendoring (default: auto-resolve)") + cmd.Flags().BoolVar(&vendorBinary, "vendor-fullsend-binary", false, "cross-compile and vendor the fullsend binary for development iteration") cmd.Flags().BoolVar(&enrollAllFlag, "enroll-all", false, "enroll all repositories without prompting") cmd.Flags().BoolVar(&enrollNoneFlag, "enroll-none", false, "skip repository enrollment without prompting") cmd.Flags().StringVar(&inferenceProject, "inference-project", "", "GCP project ID for inference (Agent Platform)") @@ -584,7 +578,6 @@ func runPerRepoInstall(ctx context.Context, c perRepoInstallConfig) error { mintSkipDeploy := c.MintSkipDeploy skipMintCheck := c.SkipMintCheck vendorBinary := c.VendorBinary - fullsendBinary := c.FullsendBinary if strings.Contains(repoFullName, "://") || strings.HasPrefix(repoFullName, "www.") { return fmt.Errorf("expected owner/repo format, got a URL — use just the owner/repo portion (e.g. acme/widget)") @@ -837,7 +830,7 @@ func runPerRepoInstall(ctx context.Context, c perRepoInstallConfig) error { } if vendorBinary { printer.Blank() - printer.StepInfo(vendorDryRunMessage(fullsendBinary, layers.VendoredBinaryPathPerRepo)) + printer.StepInfo(fmt.Sprintf("Would cross-compile and upload vendored binary to %s", layers.VendoredBinaryPathPerRepo)) } else { printer.Blank() printer.StepInfo(fmt.Sprintf("Would remove stale vendored binary at %s (if present)", layers.VendoredBinaryPathPerRepo)) @@ -1026,12 +1019,22 @@ func runPerRepoInstall(ctx context.Context, c perRepoInstallConfig) error { printer.StepDone(fmt.Sprintf("Set %d repository secrets", len(repoSecrets))) if vendorBinary { - if err := acquireAndVendorFullsendBinary(ctx, client, printer, owner, repo, fullsendBinary); err != nil { + if err := vendorFullsendBinary(ctx, client, printer, owner, repo); err != nil { return fmt.Errorf("vendoring binary: %w", err) } } else { - if err := removeStaleVendoredBinary(ctx, client, printer, owner, repo, layers.VendoredBinaryPathPerRepo); err != nil { - return err + // Clean up any vendored binary left from a previous install. + // Mirrors VendorBinaryLayer.Install cleanup logic for per-org mode. + _, err := client.GetFileContent(ctx, owner, repo, layers.VendoredBinaryPathPerRepo) + if err == nil { + printer.StepStart("removing stale vendored binary") + if err := client.DeleteFile(ctx, owner, repo, layers.VendoredBinaryPathPerRepo, "chore: remove vendored binary"); err != nil { + printer.StepFail("failed to remove vendored binary") + return fmt.Errorf("deleting vendored binary: %w", err) + } + printer.StepDone("removed stale vendored binary") + } else if !forge.IsNotFound(err) { + return fmt.Errorf("checking for vendored binary: %w", err) } } @@ -1040,6 +1043,72 @@ func runPerRepoInstall(ctx context.Context, c perRepoInstallConfig) error { return nil } +// vendorFullsendBinary cross-compiles the fullsend binary for linux/amd64 +// and uploads it via layers.VendorBinary. Per-org mode uploads to bin/fullsend +// in the .fullsend config repo; per-repo mode uploads to .fullsend/bin/fullsend +// in the target repo. +func vendorFullsendBinary(ctx context.Context, client forge.Client, printer *ui.Printer, owner, repo string) error { + destPath := layers.VendoredBinaryPath + if repo != forge.ConfigRepoName { + destPath = layers.VendoredBinaryPathPerRepo + } + + printer.StepStart("Cross-compiling fullsend for linux/amd64") + + tmpBinary, err := os.CreateTemp("", "fullsend-linux-amd64-*") + if err != nil { + return fmt.Errorf("creating temp file: %w", err) + } + tmpBinary.Close() + defer os.Remove(tmpBinary.Name()) + + goPath, lookErr := exec.LookPath("go") + if lookErr != nil { + printer.StepFail("Go toolchain not found") + return fmt.Errorf("Go toolchain not found — install Go or use a released version of fullsend: %w", lookErr) + } + + modRootCmd := exec.Command(goPath, "env", "GOMOD") + modOutput, err := modRootCmd.Output() + if err != nil { + return fmt.Errorf("finding module root: %w", err) + } + modPath := strings.TrimSpace(string(modOutput)) + if modPath == "" || modPath == os.DevNull { + return fmt.Errorf("not in a Go module — run from the fullsend source tree or use a released version") + } + modRoot := filepath.Dir(modPath) + + buildCmd := exec.Command(goPath, "build", + "-ldflags", fmt.Sprintf("-X github.com/fullsend-ai/fullsend/internal/cli.version=%s-vendored", version), + "-o", tmpBinary.Name(), + "./cmd/fullsend/", + ) + buildCmd.Dir = modRoot + buildCmd.Env = append(os.Environ(), "GOTOOLCHAIN=auto", "GOOS=linux", "GOARCH=amd64", "CGO_ENABLED=0") + buildCmd.Stderr = os.Stderr + if err := buildCmd.Run(); err != nil { + printer.StepFail("Cross-compilation failed") + return fmt.Errorf("cross-compiling: %w", err) + } + printer.StepDone("Cross-compiled fullsend for linux/amd64") + + printer.StepStart(fmt.Sprintf("Uploading vendored binary to %s", destPath)) + if err := layers.VendorBinary(ctx, client, owner, repo, destPath, tmpBinary.Name()); err != nil { + printer.StepFail("Failed to upload vendored binary") + return err + } + + info, _ := os.Stat(tmpBinary.Name()) + if info != nil { + printer.StepDone(fmt.Sprintf("Uploaded vendored binary (%d MB)", info.Size()/(1024*1024))) + } else { + printer.StepDone("Uploaded vendored binary") + } + + return nil +} + func newUninstallCmd() *cobra.Command { var yolo bool var appSet string @@ -1133,7 +1202,7 @@ func newAnalyzeCmd() *cobra.Command { // runDryRun builds a layer stack with empty credentials and analyzes. // If discoveredRepos is non-nil, it will be used instead of calling ListOrgRepos. -func runDryRun(ctx context.Context, client forge.Client, printer *ui.Printer, org string, enabledRepos, roles []string, inferenceProvider inference.Provider, inferenceProviderName string, skipMintCheck bool, mintURL string, discoveredRepos []forge.Repository, vendorBinary bool, fullsendBinary string) error { +func runDryRun(ctx context.Context, client forge.Client, printer *ui.Printer, org string, enabledRepos, roles []string, inferenceProvider inference.Provider, inferenceProviderName string, skipMintCheck bool, mintURL string, vendorBinary bool, discoveredRepos []forge.Repository) error { printer.Header("Dry run - analyzing what install would do") printer.Blank() @@ -1194,7 +1263,11 @@ func runDryRun(ctx context.Context, client forge.Client, printer *ui.Printer, or } else { dispatcher = gcf.NewProvisioner(gcf.Config{}, nil) } - stack := buildLayerStack(org, client, cfg, printer, user, privateRepo, enabledRepos, agentCreds, enrolledRepoIDs, inferenceProvider, vendorBinary, makeVendorFunc(fullsendBinary), dispatcher) + var vendorFn layers.VendorFunc + if vendorBinary { + vendorFn = vendorFullsendBinary + } + stack := buildLayerStack(org, client, cfg, printer, user, privateRepo, enabledRepos, agentCreds, enrolledRepoIDs, inferenceProvider, vendorBinary, vendorFn, dispatcher) if err := runPreflight(ctx, stack, layers.OpInstall, client, printer); err != nil { return err @@ -1455,7 +1528,7 @@ func validateEnabledRepos(enabledRepos, discoveredNames []string) error { // runInstall performs the full installation. // If discoveredRepos is non-nil, it will be used instead of calling ListOrgRepos. -func runInstall(ctx context.Context, client forge.Client, printer *ui.Printer, org string, enabledRepos, roles []string, agentCreds []layers.AgentCredentials, inferenceProvider inference.Provider, inferenceProviderName string, vendorBinary bool, fullsendBinary, mintProvider, mintProject, mintRegion, mintSourceDir string, mintSkipDeploy bool, mintURL string, skipMintCheck bool, discoveredRepos []forge.Repository) error { +func runInstall(ctx context.Context, client forge.Client, printer *ui.Printer, org string, enabledRepos, roles []string, agentCreds []layers.AgentCredentials, inferenceProvider inference.Provider, inferenceProviderName string, vendorBinary bool, mintProvider, mintProject, mintRegion, mintSourceDir string, mintSkipDeploy bool, mintURL string, skipMintCheck bool, discoveredRepos []forge.Repository) error { var allRepos []forge.Repository var err error @@ -1547,7 +1620,7 @@ func runInstall(ctx context.Context, client forge.Client, printer *ui.Printer, o }, gcf.NewLiveGCFClient(mintProject)) } - stack := buildLayerStack(org, client, cfg, printer, user, privateRepo, enabledRepos, agentCreds, enrolledRepoIDs, inferenceProvider, vendorBinary, makeVendorFunc(fullsendBinary), disp) + stack := buildLayerStack(org, client, cfg, printer, user, privateRepo, enabledRepos, agentCreds, enrolledRepoIDs, inferenceProvider, vendorBinary, vendorFullsendBinary, disp) if err := runPreflight(ctx, stack, layers.OpInstall, client, printer); err != nil { return err diff --git a/internal/cli/github.go b/internal/cli/github.go index ed695b7213..93ddd9b36a 100644 --- a/internal/cli/github.go +++ b/internal/cli/github.go @@ -60,7 +60,6 @@ type githubSetupConfig struct { enrollAll bool enrollNone bool vendorBinary bool - fullsendBinary string dryRun bool } @@ -90,9 +89,6 @@ values (mint URL, WIF provider, project ID) are provided as flags.`, if err := appsetup.ValidateAppSet(cfg.appSet); err != nil { return fmt.Errorf("invalid --app-set: %w", err) } - if err := validateVendorBinaryFlags(cfg.vendorBinary, cfg.fullsendBinary); err != nil { - return err - } if err := validateMintURLHTTPS(cfg.mintURL); err != nil { return err @@ -100,7 +96,8 @@ values (mint URL, WIF provider, project ID) are provided as flags.`, _, _, isRepo := parseTarget(cfg.target) if isRepo { - for _, name := range perOrgOnlyFlags { + githubPerOrgOnly := append(perOrgOnlyFlags, "vendor-fullsend-binary") + for _, name := range githubPerOrgOnly { if cmd.Flags().Changed(name) { return fmt.Errorf("--%s is only valid for per-org setup (fullsend github setup )", name) } @@ -136,8 +133,7 @@ values (mint URL, WIF provider, project ID) are provided as flags.`, cmd.Flags().StringVar(&cfg.appSet, "app-set", appsetup.DefaultAppSet, "app set name prefix for GitHub Apps") cmd.Flags().BoolVar(&cfg.enrollAll, "enroll-all", false, "enroll all repositories without prompting") cmd.Flags().BoolVar(&cfg.enrollNone, "enroll-none", false, "skip repository enrollment without prompting") - cmd.Flags().BoolVar(&cfg.vendorBinary, "vendor-fullsend-binary", false, "resolve and upload a linux/amd64 fullsend binary for CI") - cmd.Flags().StringVar(&cfg.fullsendBinary, "fullsend-binary", "", "path to a Linux fullsend binary to upload when vendoring (default: auto-resolve)") + cmd.Flags().BoolVar(&cfg.vendorBinary, "vendor-fullsend-binary", false, "cross-compile and upload the fullsend binary") cmd.Flags().BoolVar(&cfg.dryRun, "dry-run", false, "preview changes without making them") return cmd @@ -271,13 +267,6 @@ func runGitHubSetupPerRepo(ctx context.Context, client forge.Client, printer *ui for _, name := range secretNames { printer.StepInfo(fmt.Sprintf(" %s", name)) } - if cfg.vendorBinary { - printer.Blank() - printer.StepInfo(vendorDryRunMessage(cfg.fullsendBinary, layers.VendoredBinaryPathPerRepo)) - } else { - printer.Blank() - printer.StepInfo(fmt.Sprintf("Would remove stale vendored binary at %s (if present)", layers.VendoredBinaryPathPerRepo)) - } return nil } @@ -317,16 +306,6 @@ func runGitHubSetupPerRepo(ctx context.Context, client forge.Client, printer *ui } printer.StepDone(fmt.Sprintf("Set %d repository secrets", len(repoSecrets))) - if cfg.vendorBinary { - if err := acquireAndVendorFullsendBinary(ctx, client, printer, owner, repo, cfg.fullsendBinary); err != nil { - return fmt.Errorf("vendoring binary: %w", err) - } - } else { - if err := removeStaleVendoredBinary(ctx, client, printer, owner, repo, layers.VendoredBinaryPathPerRepo); err != nil { - return err - } - } - printer.Blank() printer.StepDone(fmt.Sprintf("Per-repo setup complete for %s/%s", owner, repo)) return nil @@ -474,7 +453,7 @@ func runGitHubSetupPerOrg(ctx context.Context, client forge.Client, printer *ui. var vendorFn layers.VendorFunc if cfg.vendorBinary { - vendorFn = makeVendorFunc(cfg.fullsendBinary) + vendorFn = vendorFullsendBinary } stack := buildLayerStack(org, client, orgCfg, printer, user, privateRepo, enabledRepos, agentCreds, enrolledRepoIDs, inferenceProvider, cfg.vendorBinary, vendorFn, dispatcher) diff --git a/internal/cli/run.go b/internal/cli/run.go index 6cba7a97f2..91370aeb20 100644 --- a/internal/cli/run.go +++ b/internal/cli/run.go @@ -1,7 +1,14 @@ package cli import ( + "archive/tar" + "bufio" + "bytes" + "compress/gzip" "context" + "crypto/sha256" + "debug/elf" + "encoding/hex" "encoding/json" "fmt" "io" @@ -17,7 +24,6 @@ import ( "github.com/spf13/cobra" - "github.com/fullsend-ai/fullsend/internal/binary" "github.com/fullsend-ai/fullsend/internal/config" "github.com/fullsend-ai/fullsend/internal/envfile" "github.com/fullsend-ai/fullsend/internal/fetch" @@ -904,7 +910,7 @@ func bootstrapCommon(sandboxName, fullsendBinary string, h *harness.Harness) err if localBinary == "" { if needsCrossCompilation() { targetArch := sandboxArch() - result, err := binary.ResolveForRun(version, targetArch) + dir, binPath, err := resolveLinuxBinary(targetArch) if err != nil { if h.FailModeClosed() { return fmt.Errorf("could not obtain linux/%s binary for security scan (fail_mode: closed): %w\nUse --fullsend-binary to provide a pre-built Linux binary", targetArch, err) @@ -913,8 +919,8 @@ func bootstrapCommon(sandboxName, fullsendBinary string, h *harness.Harness) err fmt.Fprintf(os.Stderr, "WARNING: skipping sandbox-side security scan (fail_mode: open). Use --fullsend-binary to provide a pre-built Linux binary.\n") localBinary = "" } else { - tmpBinaryDir = result.TmpDir - localBinary = result.Path + tmpBinaryDir = dir + localBinary = binPath } } else { var err error @@ -928,8 +934,8 @@ func bootstrapCommon(sandboxName, fullsendBinary string, h *harness.Harness) err defer os.RemoveAll(tmpBinaryDir) } if localBinary != "" { - if err := binary.ValidateLinuxBinary(localBinary, sandboxArch()); err != nil { - return fmt.Errorf("fullsend binary %q is not valid for the sandbox: %w\nSet FULLSEND_SANDBOX_ARCH to override the target architecture", localBinary, err) + if err := validateLinuxBinary(localBinary); err != nil { + return fmt.Errorf("fullsend binary %q is not valid for the sandbox: %w", localBinary, err) } // Use UploadDir (tarball-based) instead of Upload for the binary. // Upload silently fails for large files (~16MB); the tarball @@ -1250,8 +1256,6 @@ func runOIDCRefresh(ctx context.Context, sandboxName, oidcURL, oidcAuth string, } } -var oidcHTTPClient = &http.Client{Timeout: 120 * time.Second} // matches pre-refactor shared httpClient timeout - func refreshOIDCToken(ctx context.Context, sandboxName, oidcURL, oidcAuth string) error { req, err := http.NewRequestWithContext(ctx, "GET", oidcURL, nil) if err != nil { @@ -1259,7 +1263,7 @@ func refreshOIDCToken(ctx context.Context, sandboxName, oidcURL, oidcAuth string } req.Header.Set("Authorization", oidcAuth) - resp, err := oidcHTTPClient.Do(req) + resp, err := httpClient.Do(req) if err != nil { return fmt.Errorf("fetching OIDC token: %w", err) } @@ -1618,6 +1622,31 @@ func needsCrossCompilation() bool { return runtime.GOOS != "linux" } +// validateLinuxBinary checks that the file at path is a Linux ELF executable +// for the expected sandbox architecture. Returns a descriptive error if the +// file is missing, not ELF, not Linux, or the wrong architecture. +func validateLinuxBinary(path string) error { + f, err := elf.Open(path) + if err != nil { + return fmt.Errorf("not a valid ELF binary (is this a macOS Mach-O?): %w", err) + } + defer f.Close() + + if f.OSABI != elf.ELFOSABI_NONE && f.OSABI != elf.ELFOSABI_LINUX { + return fmt.Errorf("ELF OS/ABI is %s, expected Linux or NONE", f.OSABI) + } + + arch := sandboxArch() + archToMachine := map[string]elf.Machine{ + "amd64": elf.EM_X86_64, + "arm64": elf.EM_AARCH64, + } + if expected, ok := archToMachine[arch]; ok && f.Machine != expected { + return fmt.Errorf("ELF machine is %s, expected %s for %s (set FULLSEND_SANDBOX_ARCH to override)", f.Machine, expected, arch) + } + return nil +} + // copyFile copies src to dst, preserving permissions. func copyFile(src, dst string) error { in, err := os.Open(src) @@ -1643,6 +1672,8 @@ func copyFile(src, dst string) error { return os.Chmod(dst, info.Mode()) } +var validArchs = map[string]bool{"amd64": true, "arm64": true} + // sandboxArch returns the target architecture for the sandbox binary. // Defaults to the host arch (correct when sandbox image matches host, e.g. // arm64 Mac → arm64 sandbox image). Override with FULLSEND_SANDBOX_ARCH @@ -1650,7 +1681,7 @@ func copyFile(src, dst string) error { // on an arm64 host via emulation). Only amd64 and arm64 are supported. func sandboxArch() string { if arch := os.Getenv("FULLSEND_SANDBOX_ARCH"); arch != "" { - if !binary.ValidArch(arch) { + if !validArchs[arch] { fmt.Fprintf(os.Stderr, "WARNING: FULLSEND_SANDBOX_ARCH=%q is not a supported architecture (amd64, arm64), using host arch %s\n", arch, runtime.GOARCH) return runtime.GOARCH } @@ -1659,6 +1690,263 @@ func sandboxArch() string { return runtime.GOARCH } +// resolveLinuxBinary obtains a Linux fullsend binary for the given arch. +// Strategy: download from GitHub Release first (fast, no toolchain needed), +// fall back to cross-compilation if the download fails or version is "dev". +// Returns the temp directory (caller must clean up), the binary path, and any error. +func resolveLinuxBinary(arch string) (tmpDir string, binaryPath string, err error) { + tmpDir, err = os.MkdirTemp("", "fullsend-linux-*") + if err != nil { + return "", "", fmt.Errorf("creating temp dir: %w", err) + } + binaryPath = filepath.Join(tmpDir, "fullsend") + + // 1. Released version → download matching release asset. + if isReleasedVersion(version) { + fmt.Fprintf(os.Stderr, "Downloading fullsend %s for linux/%s from GitHub Release...\n", version, arch) + if dlErr := downloadReleaseBinary(version, arch, binaryPath); dlErr == nil { + fmt.Fprintf(os.Stderr, "Downloaded fullsend for linux/%s\n", arch) + return tmpDir, binaryPath, nil + } else { + fmt.Fprintf(os.Stderr, "WARNING: release download failed: %v\n", dlErr) + } + } + + // 2. Dev build → try cross-compilation (requires Go toolchain + module in CWD). + fmt.Fprintf(os.Stderr, "Cross-compiling fullsend for linux/%s...\n", arch) + if ccErr := crossCompileFullsend(arch, binaryPath); ccErr == nil { + fmt.Fprintf(os.Stderr, "Cross-compiled fullsend for linux/%s\n", arch) + return tmpDir, binaryPath, nil + } else { + fmt.Fprintf(os.Stderr, "WARNING: cross-compilation failed: %v\n", ccErr) + } + + // 3. Last resort → download latest release (version won't match exactly, + // but the scan context command interface is stable across patch versions). + fmt.Fprintf(os.Stderr, "Downloading latest fullsend release for linux/%s...\n", arch) + if dlErr := downloadLatestReleaseBinary(arch, binaryPath); dlErr == nil { + fmt.Fprintf(os.Stderr, "Downloaded latest fullsend for linux/%s\n", arch) + return tmpDir, binaryPath, nil + } else { + fmt.Fprintf(os.Stderr, "WARNING: latest release download failed: %v\n", dlErr) + } + + os.RemoveAll(tmpDir) + return "", "", fmt.Errorf("all strategies failed for linux/%s: provide --fullsend-binary or install Go toolchain", arch) +} + +// isReleasedVersion returns true if version looks like a release tag +// (e.g. "0.4.0", "v0.4.0") rather than a dev build (e.g. "dev", +// "0.4.0-3-gabcdef", "0.4.0-vendored"). +func isReleasedVersion(v string) bool { + v = strings.TrimPrefix(v, "v") + if v == "" || v == "dev" { + return false + } + // A released version is purely digits and dots (e.g. "0.4.0"). + for _, c := range v { + if c != '.' && (c < '0' || c > '9') { + return false + } + } + return true +} + +var releaseBaseURL = "https://github.com/fullsend-ai/fullsend/releases/download" + +var httpClient = &http.Client{Timeout: 120 * time.Second} + +// downloadReleaseBinary downloads the fullsend binary for linux/{arch} from +// the GitHub Release matching the given version, verifies its SHA256 checksum +// against the release checksums.txt, and writes it to destPath. +func downloadReleaseBinary(ver, arch, destPath string) error { + cleanVer := strings.TrimPrefix(ver, "v") + assetName := fmt.Sprintf("fullsend_%s_linux_%s.tar.gz", cleanVer, arch) + + expectedHash, err := downloadChecksumForAsset(ver, assetName) + if err != nil { + return fmt.Errorf("fetching checksum for %s: %w", assetName, err) + } + + url := fmt.Sprintf("%s/v%s/%s", releaseBaseURL, cleanVer, assetName) + resp, err := httpClient.Get(url) //nolint:gosec // URL is constructed from known constants + if err != nil { + return fmt.Errorf("fetching %s: %w", url, err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + return fmt.Errorf("GET %s returned %d", url, resp.StatusCode) + } + + const maxDownloadSize = 200 * 1024 * 1024 // 200 MB compressed + var buf bytes.Buffer + if _, err := io.Copy(&buf, io.LimitReader(resp.Body, maxDownloadSize+1)); err != nil { + return fmt.Errorf("reading %s: %w", assetName, err) + } + if buf.Len() > maxDownloadSize { + return fmt.Errorf("%s exceeds maximum download size (%d bytes)", assetName, maxDownloadSize) + } + + h := sha256.Sum256(buf.Bytes()) + actualHash := hex.EncodeToString(h[:]) + if actualHash != expectedHash { + return fmt.Errorf("checksum mismatch for %s: got %s, want %s", assetName, actualHash, expectedHash) + } + + return extractFullsendFromTarGz(bytes.NewReader(buf.Bytes()), destPath) +} + +// downloadChecksumForAsset fetches the checksums.txt from the GitHub Release +// for the given version and returns the SHA256 hash for assetName. +// GoReleaser format: " \n" +func downloadChecksumForAsset(ver, assetName string) (string, error) { + cleanVer := strings.TrimPrefix(ver, "v") + url := fmt.Sprintf("%s/v%s/checksums.txt", releaseBaseURL, cleanVer) + + resp, err := httpClient.Get(url) //nolint:gosec // URL is constructed from known constants + if err != nil { + return "", fmt.Errorf("fetching checksums: %w", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + return "", fmt.Errorf("GET %s returned %d", url, resp.StatusCode) + } + + scanner := bufio.NewScanner(io.LimitReader(resp.Body, 64*1024)) + for scanner.Scan() { + line := scanner.Text() + parts := strings.Fields(line) + if len(parts) == 2 && parts[1] == assetName { + hash := strings.ToLower(parts[0]) + if len(hash) != 64 { + return "", fmt.Errorf("invalid hash length for %s in checksums.txt", assetName) + } + if _, err := hex.DecodeString(hash); err != nil { + return "", fmt.Errorf("invalid hex hash for %s in checksums.txt: %w", assetName, err) + } + return hash, nil + } + } + if err := scanner.Err(); err != nil { + return "", fmt.Errorf("reading checksums: %w", err) + } + return "", fmt.Errorf("asset %s not found in checksums.txt", assetName) +} + +// downloadLatestReleaseBinary resolves the latest release tag from the GitHub +// API and downloads the Linux binary for the given arch. +func downloadLatestReleaseBinary(arch, destPath string) error { + tag, err := resolveLatestReleaseTag() + if err != nil { + return err + } + return downloadReleaseBinary(tag, arch, destPath) +} + +func resolveLatestReleaseTag() (string, error) { + resp, err := httpClient.Get("https://api.github.com/repos/fullsend-ai/fullsend/releases/latest") //nolint:gosec + if err != nil { + return "", fmt.Errorf("fetching latest release: %w", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + return "", fmt.Errorf("GitHub API returned %d", resp.StatusCode) + } + + var release struct { + TagName string `json:"tag_name"` + } + if err := json.NewDecoder(io.LimitReader(resp.Body, 1024*1024)).Decode(&release); err != nil { + return "", fmt.Errorf("parsing release JSON: %w", err) + } + if release.TagName == "" { + return "", fmt.Errorf("empty tag_name in latest release") + } + return release.TagName, nil +} + +const maxBinarySize = 500 * 1024 * 1024 // 500 MB — reasonable upper bound for a Go binary + +// extractFullsendFromTarGz reads a tar.gz stream and extracts the "fullsend" +// binary to destPath. +func extractFullsendFromTarGz(r io.Reader, destPath string) error { + gz, err := gzip.NewReader(r) + if err != nil { + return fmt.Errorf("gzip reader: %w", err) + } + defer gz.Close() + + tr := tar.NewReader(gz) + for { + hdr, err := tr.Next() + if err == io.EOF { + return fmt.Errorf("fullsend binary not found in archive") + } + if err != nil { + return fmt.Errorf("reading tar: %w", err) + } + clean := filepath.Clean(hdr.Name) + if strings.Contains(clean, "..") || filepath.IsAbs(clean) { + continue + } + if filepath.Base(clean) == "fullsend" && hdr.Typeflag == tar.TypeReg { + f, err := os.OpenFile(destPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o755) + if err != nil { + return fmt.Errorf("creating %s: %w", destPath, err) + } + n, copyErr := io.Copy(f, io.LimitReader(tr, maxBinarySize+1)) + if copyErr != nil { + f.Close() + return fmt.Errorf("extracting fullsend: %w", copyErr) + } + if n > maxBinarySize { + f.Close() + os.Remove(destPath) + return fmt.Errorf("binary exceeds maximum size (%d bytes)", maxBinarySize) + } + return f.Close() + } + } +} + +// crossCompileFullsend builds a Linux fullsend binary for the given arch +// and writes it to destPath. Requires the Go toolchain. +func crossCompileFullsend(arch, destPath string) error { + goPath, lookErr := exec.LookPath("go") + if lookErr != nil { + return fmt.Errorf("Go toolchain not found — install Go or use a released version of fullsend: %w", lookErr) + } + + // Find the module root so `go build ./cmd/fullsend/` resolves correctly + // regardless of the caller's working directory. + modRootCmd := exec.Command(goPath, "env", "GOMOD") + modOutput, err := modRootCmd.Output() + if err != nil { + return fmt.Errorf("finding module root: %w", err) + } + modPath := strings.TrimSpace(string(modOutput)) + if modPath == "" || modPath == os.DevNull { + return fmt.Errorf("not in a Go module — run from the fullsend source tree or use a released version") + } + modRoot := filepath.Dir(modPath) + + buildCmd := exec.Command(goPath, "build", + "-ldflags", fmt.Sprintf("-X github.com/fullsend-ai/fullsend/internal/cli.version=%s-crosscompiled", version), + "-o", destPath, + "./cmd/fullsend/", + ) + buildCmd.Dir = modRoot + buildCmd.Env = append(os.Environ(), "GOTOOLCHAIN=auto", "GOOS=linux", "GOARCH="+arch, "CGO_ENABLED=0") + buildCmd.Stderr = os.Stderr + if err := buildCmd.Run(); err != nil { + return fmt.Errorf("cross-compiling for linux/%s: %w", arch, err) + } + return nil +} + func titleCase(s string) string { words := strings.Fields(s) for i, w := range words { diff --git a/internal/cli/run_test.go b/internal/cli/run_test.go index 8a91ad00f2..9c7f163ef7 100644 --- a/internal/cli/run_test.go +++ b/internal/cli/run_test.go @@ -1,8 +1,12 @@ package cli import ( + "archive/tar" "bytes" + "compress/gzip" "context" + "crypto/sha256" + "encoding/hex" "fmt" "io" "net/http" @@ -19,7 +23,6 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" - "github.com/fullsend-ai/fullsend/internal/binary" "github.com/fullsend-ai/fullsend/internal/ui" ) @@ -498,15 +501,16 @@ func TestSandboxArch_InvalidFallsBack(t *testing.T) { } func TestValidateLinuxBinary_RejectsNonELF(t *testing.T) { + // A plain text file should be rejected. tmp := filepath.Join(t.TempDir(), "not-elf") require.NoError(t, os.WriteFile(tmp, []byte("#!/bin/sh\necho hello"), 0o755)) - err := binary.ValidateLinuxBinary(tmp, "amd64") + err := validateLinuxBinary(tmp) require.Error(t, err) assert.Contains(t, err.Error(), "not a valid ELF binary") } func TestValidateLinuxBinary_RejectsMissing(t *testing.T) { - err := binary.ValidateLinuxBinary("/tmp/nonexistent-fullsend-binary-12345", "amd64") + err := validateLinuxBinary("/tmp/nonexistent-fullsend-binary-12345") require.Error(t, err) } @@ -516,7 +520,115 @@ func TestValidateLinuxBinary_AcceptsHostBinary(t *testing.T) { } exe, err := os.Executable() require.NoError(t, err) - assert.NoError(t, binary.ValidateLinuxBinary(exe, runtime.GOARCH)) + assert.NoError(t, validateLinuxBinary(exe)) +} + +func TestIsReleasedVersion(t *testing.T) { + tests := []struct { + version string + expected bool + }{ + {"0.4.0", true}, + {"v0.4.0", true}, + {"1.0.0", true}, + {"dev", false}, + {"", false}, + {"0.4.0-3-gabcdef", false}, + {"0.4.0-vendored", false}, + {"0.4.0-crosscompiled", false}, + } + for _, tt := range tests { + t.Run(tt.version, func(t *testing.T) { + assert.Equal(t, tt.expected, isReleasedVersion(tt.version), "version=%q", tt.version) + }) + } +} + +func TestExtractFullsendFromTarGz_PathTraversal(t *testing.T) { + // Create a tar.gz with a path-traversal entry named "../../../tmp/fullsend". + var buf bytes.Buffer + gw := gzip.NewWriter(&buf) + tw := tar.NewWriter(gw) + + content := []byte("malicious binary content") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "../../../tmp/fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + destPath := filepath.Join(t.TempDir(), "fullsend") + err = extractFullsendFromTarGz(&buf, destPath) + assert.Error(t, err, "should reject traversal entry and report binary not found") + assert.Contains(t, err.Error(), "not found in archive") +} + +func TestExtractFullsendFromTarGz_ValidEntry(t *testing.T) { + var buf bytes.Buffer + gw := gzip.NewWriter(&buf) + tw := tar.NewWriter(gw) + + content := []byte("valid binary content") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "fullsend_0.4.0_linux_amd64/fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + destPath := filepath.Join(t.TempDir(), "fullsend") + err = extractFullsendFromTarGz(&buf, destPath) + require.NoError(t, err) + + data, err := os.ReadFile(destPath) + require.NoError(t, err) + assert.Equal(t, "valid binary content", string(data)) +} + +func TestCrossCompileFullsend_ProducesBinary(t *testing.T) { + if runtime.GOOS == "linux" { + t.Skip("cross-compilation test only meaningful on non-Linux hosts") + } + if testing.Short() { + t.Skip("skipping cross-compilation in short mode") + } + + tmpDir := t.TempDir() + binPath := filepath.Join(tmpDir, "fullsend") + err := crossCompileFullsend(runtime.GOARCH, binPath) + require.NoError(t, err) + + info, err := os.Stat(binPath) + require.NoError(t, err) + assert.True(t, info.Size() > 0, "binary should be non-empty") +} + +func TestResolveLinuxBinary_Download(t *testing.T) { + if testing.Short() { + t.Skip("skipping download test in short mode") + } + + tmpDir := t.TempDir() + binPath := filepath.Join(tmpDir, "fullsend") + err := downloadReleaseBinary("0.4.0", "amd64", binPath) + require.NoError(t, err) + + info, err := os.Stat(binPath) + require.NoError(t, err) + assert.True(t, info.Size() > 0, "downloaded binary should be non-empty") + + // Verify the downloaded artifact is a valid Linux ELF for the requested arch. + t.Setenv("FULLSEND_SANDBOX_ARCH", "amd64") + assert.NoError(t, validateLinuxBinary(binPath), "downloaded binary should be a valid Linux/amd64 ELF") } func TestAgentWorkingDirExcludes_ContainsKnownPatterns(t *testing.T) { @@ -728,6 +840,148 @@ func TestRunHeartbeat_NoNoticeWhenNotCI(t *testing.T) { assert.Empty(t, buf.String(), "should not emit any ::notice:: when not in CI") } +func TestDownloadChecksumForAsset_ParsesLine(t *testing.T) { + body := "1b4f0e9851971998e732078544c96b36c3d01cedf7caa332359d6f1d83567014 fullsend_1.0.0_linux_arm64.tar.gz\n" + + "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752 fullsend_1.0.0_linux_amd64.tar.gz\n" + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprint(w, body) + })) + defer srv.Close() + + origBaseURL := releaseBaseURL + releaseBaseURL = srv.URL + defer func() { releaseBaseURL = origBaseURL }() + + hash, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_amd64.tar.gz") + require.NoError(t, err) + assert.Equal(t, "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752", hash) +} + +func TestDownloadChecksumForAsset_AssetNotFound(t *testing.T) { + body := "1b4f0e9851971998e732078544c96b36c3d01cedf7caa332359d6f1d83567014 fullsend_1.0.0_linux_amd64.tar.gz\n" + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprint(w, body) + })) + defer srv.Close() + + origBaseURL := releaseBaseURL + releaseBaseURL = srv.URL + defer func() { releaseBaseURL = origBaseURL }() + + _, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_arm64.tar.gz") + require.Error(t, err) + assert.Contains(t, err.Error(), "not found in checksums.txt") +} + +func TestDownloadChecksumForAsset_InvalidHex(t *testing.T) { + body := "ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ fullsend_1.0.0_linux_amd64.tar.gz\n" + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprint(w, body) + })) + defer srv.Close() + + origBaseURL := releaseBaseURL + releaseBaseURL = srv.URL + defer func() { releaseBaseURL = origBaseURL }() + + _, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_amd64.tar.gz") + require.Error(t, err) + assert.Contains(t, err.Error(), "invalid hex hash") +} + +func TestDownloadReleaseBinary_ChecksumMismatch(t *testing.T) { + // Build a valid tar.gz containing a "fullsend" binary. + var tarBuf bytes.Buffer + gw := gzip.NewWriter(&tarBuf) + tw := tar.NewWriter(gw) + content := []byte("fake binary") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + tarBytes := tarBuf.Bytes() + + // Serve a checksums.txt with a WRONG hash for the asset. + wrongHash := "0000000000000000000000000000000000000000000000000000000000000000" + checksumBody := fmt.Sprintf("%s fullsend_1.0.0_linux_amd64.tar.gz\n", wrongHash) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/v1.0.0/checksums.txt" { + fmt.Fprint(w, checksumBody) + } else if r.URL.Path == "/v1.0.0/fullsend_1.0.0_linux_amd64.tar.gz" { + w.Write(tarBytes) + } else { + http.NotFound(w, r) + } + })) + defer srv.Close() + + origBaseURL := releaseBaseURL + releaseBaseURL = srv.URL + defer func() { releaseBaseURL = origBaseURL }() + + destPath := filepath.Join(t.TempDir(), "fullsend") + err = downloadReleaseBinary("1.0.0", "amd64", destPath) + require.Error(t, err) + assert.Contains(t, err.Error(), "checksum mismatch") +} + +func TestDownloadReleaseBinary_ChecksumMatch(t *testing.T) { + var tarBuf bytes.Buffer + gw := gzip.NewWriter(&tarBuf) + tw := tar.NewWriter(gw) + content := []byte("good binary") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + tarBytes := tarBuf.Bytes() + h := sha256.Sum256(tarBytes) + correctHash := hex.EncodeToString(h[:]) + + checksumBody := fmt.Sprintf("%s fullsend_2.0.0_linux_amd64.tar.gz\n", correctHash) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/v2.0.0/checksums.txt" { + fmt.Fprint(w, checksumBody) + } else if r.URL.Path == "/v2.0.0/fullsend_2.0.0_linux_amd64.tar.gz" { + w.Write(tarBytes) + } else { + http.NotFound(w, r) + } + })) + defer srv.Close() + + origBaseURL := releaseBaseURL + releaseBaseURL = srv.URL + defer func() { releaseBaseURL = origBaseURL }() + + destPath := filepath.Join(t.TempDir(), "fullsend") + err = downloadReleaseBinary("2.0.0", "amd64", destPath) + require.NoError(t, err) + + data, err := os.ReadFile(destPath) + require.NoError(t, err) + assert.Equal(t, "good binary", string(data)) +} + func TestValidationFailMessage_UsesOutputWhenPresent(t *testing.T) { msg := validationFailMessage([]byte("check failed: lint errors"), fmt.Errorf("exit status 1")) assert.Equal(t, "check failed: lint errors", msg) diff --git a/internal/cli/vendor.go b/internal/cli/vendor.go deleted file mode 100644 index bf455a4f78..0000000000 --- a/internal/cli/vendor.go +++ /dev/null @@ -1,118 +0,0 @@ -package cli - -import ( - "context" - "fmt" - "os" - - "github.com/fullsend-ai/fullsend/internal/binary" - "github.com/fullsend-ai/fullsend/internal/forge" - "github.com/fullsend-ai/fullsend/internal/layers" - "github.com/fullsend-ai/fullsend/internal/ui" -) - -const vendorArch = binary.DefaultArch - -func validateVendorBinaryFlags(vendorBinary bool, fullsendBinary string) error { - if fullsendBinary != "" && !vendorBinary { - return fmt.Errorf("--fullsend-binary requires --vendor-fullsend-binary") - } - return nil -} - -// makeVendorFunc returns a VendorFunc closure that uploads a fullsend binary -// using the vendoring acquisition policy. -func makeVendorFunc(fullsendBinary string) layers.VendorFunc { - return func(ctx context.Context, client forge.Client, printer *ui.Printer, owner, repo string) error { - return acquireAndVendorFullsendBinary(ctx, client, printer, owner, repo, fullsendBinary) - } -} - -// acquireAndVendorFullsendBinary resolves a Linux binary and uploads it to the -// target repo using the vendoring policy. -func acquireAndVendorFullsendBinary(ctx context.Context, client forge.Client, printer *ui.Printer, owner, repo, fullsendBinary string) error { - destPath := layers.VendoredBinaryPath - if repo != forge.ConfigRepoName { - destPath = layers.VendoredBinaryPathPerRepo - } - - var ( - binPath string - source binary.Source - tmpDir string - ) - - if fullsendBinary != "" { - printer.StepStart(fmt.Sprintf("Using provided binary: %s", fullsendBinary)) - if err := binary.ResolveExplicit(fullsendBinary, vendorArch); err != nil { - printer.StepFail("Invalid --fullsend-binary") - return fmt.Errorf("validating --fullsend-binary: %w", err) - } - binPath = fullsendBinary - source = binary.SourceExplicitPath - printer.StepDone("Validated linux/amd64 ELF binary") - } else { - result, err := binary.ResolveForVendor(version, vendorArch) - if err != nil { - printer.StepFail("Failed to obtain binary for vendoring") - return err - } - tmpDir = result.TmpDir - binPath = result.Path - source = result.Source - } - - if tmpDir != "" { - defer os.RemoveAll(tmpDir) - } - - info, err := os.Stat(binPath) - if err != nil { - return fmt.Errorf("stat binary: %w", err) - } - - commitMsg := layers.VendorCommitMessage(source, version, destPath, info.Size()) - - printer.StepStart(fmt.Sprintf("Uploading vendored binary to %s", destPath)) - if err := layers.VendorBinary(ctx, client, owner, repo, destPath, binPath, commitMsg); err != nil { - printer.StepFail("Failed to upload vendored binary") - return err - } - - printer.StepDone(fmt.Sprintf("Uploaded vendored binary (%d MB)", info.Size()/(1024*1024))) - return nil -} - -// removeStaleVendoredBinary deletes a stale vendored binary when vendoring is disabled. -func removeStaleVendoredBinary(ctx context.Context, client forge.Client, printer *ui.Printer, owner, repo, destPath string) error { - _, err := client.GetFileContent(ctx, owner, repo, destPath) - if err != nil { - if forge.IsNotFound(err) { - return nil - } - return fmt.Errorf("checking for vendored binary: %w", err) - } - - printer.StepStart("removing stale vendored binary") - deleteMsg := layers.RemoveStaleBinaryCommitMessage(destPath) - if err := client.DeleteFile(ctx, owner, repo, destPath, deleteMsg); err != nil { - printer.StepFail("failed to remove vendored binary") - return fmt.Errorf("deleting vendored binary: %w", err) - } - printer.StepDone("removed stale vendored binary") - return nil -} - -// vendorDryRunMessage returns a dry-run line describing what vendoring would do. -func vendorDryRunMessage(fullsendBinary, destPath string) string { - if fullsendBinary != "" { - return fmt.Sprintf("Would upload provided binary from %s to %s", fullsendBinary, destPath) - } - if _, err := binary.ModuleRoot(); err == nil { - return fmt.Sprintf("Would cross-compile and upload vendored binary to %s", destPath) - } - if binary.IsReleasedVersion(version) { - return fmt.Sprintf("Would download release %s and upload vendored binary to %s", version, destPath) - } - return fmt.Sprintf("Would fail: dev CLI outside checkout cannot vendor to %s", destPath) -} diff --git a/internal/cli/vendor_test.go b/internal/cli/vendor_test.go deleted file mode 100644 index f8a4c60eae..0000000000 --- a/internal/cli/vendor_test.go +++ /dev/null @@ -1,84 +0,0 @@ -package cli - -import ( - "context" - "os" - "runtime" - "strings" - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - - "github.com/fullsend-ai/fullsend/internal/forge" - "github.com/fullsend-ai/fullsend/internal/layers" - "github.com/fullsend-ai/fullsend/internal/ui" -) - -func TestValidateVendorBinaryFlags(t *testing.T) { - require.NoError(t, validateVendorBinaryFlags(false, "")) - require.NoError(t, validateVendorBinaryFlags(true, "")) - require.NoError(t, validateVendorBinaryFlags(true, "/tmp/fullsend")) - - err := validateVendorBinaryFlags(false, "/tmp/fullsend") - require.Error(t, err) - assert.Contains(t, err.Error(), "--fullsend-binary requires --vendor-fullsend-binary") -} - -func TestInstallCmd_HasFullsendBinaryFlag(t *testing.T) { - cmd := newInstallCmd() - flag := cmd.Flags().Lookup("fullsend-binary") - require.NotNil(t, flag, "expected --fullsend-binary flag") - assert.Equal(t, "", flag.DefValue) -} - -func TestGitHubSetupCmd_HasFullsendBinaryFlag(t *testing.T) { - cmd := newGitHubSetupCmd() - flag := cmd.Flags().Lookup("fullsend-binary") - require.NotNil(t, flag, "expected --fullsend-binary flag") -} - -func TestVendorDryRunMessage(t *testing.T) { - msg := vendorDryRunMessage("/tmp/fullsend", layers.VendoredBinaryPathPerRepo) - assert.Contains(t, msg, "/tmp/fullsend") - assert.Contains(t, msg, layers.VendoredBinaryPathPerRepo) -} - -func TestAcquireAndVendorFullsendBinary_ExplicitPath(t *testing.T) { - if runtime.GOOS != "linux" { - t.Skip("needs Linux ELF binary") - } - exe, err := os.Executable() - require.NoError(t, err) - - client := &forge.FakeClient{} - var buf strings.Builder - printer := ui.New(&buf) - - err = acquireAndVendorFullsendBinary(context.Background(), client, printer, "org", "my-repo", exe) - require.NoError(t, err) - - key := "org/my-repo/" + layers.VendoredBinaryPathPerRepo - require.Contains(t, client.FileContents, key) - require.NotEmpty(t, client.CreatedFiles) - assert.Contains(t, client.CreatedFiles[0].Message, "\n\n") - assert.Contains(t, client.CreatedFiles[0].Message, "Source: --fullsend-binary") -} - -func TestAcquireAndVendorFullsendBinary_CheckoutBuild(t *testing.T) { - if testing.Short() { - t.Skip("skipping cross-compile in short mode") - } - - client := &forge.FakeClient{} - var buf strings.Builder - printer := ui.New(&buf) - - err := acquireAndVendorFullsendBinary(context.Background(), client, printer, "org", forge.ConfigRepoName, "") - require.NoError(t, err) - - key := "org/" + forge.ConfigRepoName + "/" + layers.VendoredBinaryPath - require.Contains(t, client.FileContents, key) - require.NotEmpty(t, client.CreatedFiles) - assert.Contains(t, client.CreatedFiles[0].Message, "cross-compiled from checkout") -} diff --git a/internal/harness/harness.go b/internal/harness/harness.go index bf5686a171..85ba2ff2ff 100644 --- a/internal/harness/harness.go +++ b/internal/harness/harness.go @@ -217,6 +217,8 @@ type Harness struct { SandboxTimeoutSeconds int `yaml:"sandbox_timeout_seconds,omitempty"` Security *SecurityConfig `yaml:"security,omitempty"` AllowedRemoteResources []string `yaml:"allowed_remote_resources,omitempty"` + AllowRuntimeFetch bool `yaml:"allow_runtime_fetch,omitempty"` + MaxRuntimeFetches int `yaml:"max_runtime_fetches,omitempty"` Forge map[string]*ForgeConfig `yaml:"forge,omitempty"` } @@ -290,6 +292,12 @@ func (h *Harness) Validate() error { if h.ValidationLoop != nil && h.ValidationLoop.Script == "" { return fmt.Errorf("validation_loop.script is required when validation_loop is set") } + if h.MaxRuntimeFetches < 0 { + return fmt.Errorf("max_runtime_fetches must be non-negative, got %d", h.MaxRuntimeFetches) + } + if !h.AllowRuntimeFetch && h.MaxRuntimeFetches != 0 { + return fmt.Errorf("max_runtime_fetches requires allow_runtime_fetch: true") + } if err := h.validateSecurity(); err != nil { return err } diff --git a/internal/harness/harness_test.go b/internal/harness/harness_test.go index 9c9d3d34ad..9c94114470 100644 --- a/internal/harness/harness_test.go +++ b/internal/harness/harness_test.go @@ -1074,6 +1074,67 @@ func TestMatchingAllowedPrefix(t *testing.T) { }) } +func TestValidate_AllowRuntimeFetch(t *testing.T) { + t.Run("allow_runtime_fetch true with max is valid", func(t *testing.T) { + h := &Harness{ + Agent: "code", + AllowRuntimeFetch: true, + MaxRuntimeFetches: 5, + } + assert.NoError(t, h.Validate()) + }) + + t.Run("allow_runtime_fetch true with zero max is valid", func(t *testing.T) { + h := &Harness{ + Agent: "code", + AllowRuntimeFetch: true, + } + assert.NoError(t, h.Validate()) + }) + + t.Run("max without allow is invalid", func(t *testing.T) { + h := &Harness{ + Agent: "code", + MaxRuntimeFetches: 5, + } + err := h.Validate() + require.Error(t, err) + assert.Contains(t, err.Error(), "max_runtime_fetches requires allow_runtime_fetch") + }) + + t.Run("negative max is invalid", func(t *testing.T) { + h := &Harness{ + Agent: "code", + AllowRuntimeFetch: true, + MaxRuntimeFetches: -1, + } + err := h.Validate() + require.Error(t, err) + assert.Contains(t, err.Error(), "max_runtime_fetches must be non-negative") + }) + + t.Run("backward compatible without new fields", func(t *testing.T) { + h := &Harness{Agent: "code"} + assert.NoError(t, h.Validate()) + assert.False(t, h.AllowRuntimeFetch) + assert.Equal(t, 0, h.MaxRuntimeFetches) + }) +} + +func TestLoad_RuntimeFetchFields(t *testing.T) { + content := `agent: code +allow_runtime_fetch: true +max_runtime_fetches: 10 +` + path := filepath.Join(t.TempDir(), "harness.yaml") + require.NoError(t, os.WriteFile(path, []byte(content), 0o644)) + + h, err := Load(path) + require.NoError(t, err) + assert.True(t, h.AllowRuntimeFetch) + assert.Equal(t, 10, h.MaxRuntimeFetches) +} + // --- Role and slug field tests --- func TestLoad_RoleAndSlug(t *testing.T) { diff --git a/internal/layers/vendor.go b/internal/layers/vendor.go index 6ddd0639e5..0bbddb2c16 100644 --- a/internal/layers/vendor.go +++ b/internal/layers/vendor.go @@ -4,9 +4,7 @@ import ( "context" "fmt" "os" - "strings" - "github.com/fullsend-ai/fullsend/internal/binary" "github.com/fullsend-ai/fullsend/internal/forge" ) @@ -20,7 +18,7 @@ const ( // VendorBinary uploads a pre-built fullsend binary to the given destPath. // CI workflows detect this file and use it instead of downloading from // GitHub releases, enabling development iteration without cutting a release. -func VendorBinary(ctx context.Context, client forge.Client, owner, repo, destPath, binaryPath, commitMsg string) error { +func VendorBinary(ctx context.Context, client forge.Client, owner, repo, destPath, binaryPath string) error { const maxBinarySize = 100 * 1024 * 1024 // 100 MB (GitHub Contents API limit) info, err := os.Stat(binaryPath) if err != nil { @@ -36,62 +34,9 @@ func VendorBinary(ctx context.Context, client forge.Client, owner, repo, destPat if err != nil { return fmt.Errorf("reading binary %s: %w", binaryPath, err) } - if err := client.CreateOrUpdateFile(ctx, owner, repo, destPath, commitMsg, data); err != nil { + if err := client.CreateOrUpdateFile(ctx, owner, repo, + destPath, "chore: vendor fullsend binary for development", data); err != nil { return fmt.Errorf("uploading vendored binary: %w", err) } return nil } - -// VendorCommitMessage returns a GitHub commit message (title + body) for upload. -func VendorCommitMessage(source binary.Source, version, destPath string, sizeBytes int64) string { - const arch = "linux/amd64" - var title string - var bodyLines []string - - switch source { - case binary.SourceExplicitPath: - title = "chore: vendor fullsend binary for development" - bodyLines = []string{ - "Source: --fullsend-binary", - fmt.Sprintf("Path: %s", destPath), - fmt.Sprintf("Size: %d bytes", sizeBytes), - fmt.Sprintf("Arch: %s", arch), - } - case binary.SourceCheckoutBuild: - title = "chore: vendor fullsend binary for development" - bodyLines = []string{ - "Source: cross-compiled from checkout", - fmt.Sprintf("CLI version: %s", version), - fmt.Sprintf("Binary stamp: %s-vendored", version), - fmt.Sprintf("Path: %s", destPath), - fmt.Sprintf("Size: %d bytes", sizeBytes), - fmt.Sprintf("Arch: %s", arch), - } - case binary.SourceReleaseDownload: - cleanVer := strings.TrimPrefix(version, "v") - title = fmt.Sprintf("chore: vendor fullsend v%s binary from release", cleanVer) - bodyLines = []string{ - fmt.Sprintf("Source: GitHub Release v%s", cleanVer), - fmt.Sprintf("Path: %s", destPath), - fmt.Sprintf("Size: %d bytes", sizeBytes), - fmt.Sprintf("Arch: %s", arch), - "Note: binary retains release version stamp (no -vendored suffix)", - } - default: - title = "chore: vendor fullsend binary for development" - bodyLines = []string{fmt.Sprintf("Path: %s", destPath)} - } - - return title + "\n\n" + strings.Join(bodyLines, "\n") -} - -// RemoveStaleBinaryCommitMessage returns title + body for stale binary deletion. -func RemoveStaleBinaryCommitMessage(destPath string) string { - title := "chore: remove vendored fullsend binary" - body := strings.Join([]string{ - "Reason: --vendor-fullsend-binary not set; removing stale binary so CI uses released versions", - fmt.Sprintf("Path: %s", destPath), - "Note: re-run install with --vendor-fullsend-binary to upload again", - }, "\n") - return title + "\n\n" + body -} diff --git a/internal/layers/vendor_test.go b/internal/layers/vendor_test.go deleted file mode 100644 index 4c19c5936b..0000000000 --- a/internal/layers/vendor_test.go +++ /dev/null @@ -1,69 +0,0 @@ -package layers - -import ( - "strings" - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - - "github.com/fullsend-ai/fullsend/internal/binary" -) - -func TestVendorCommitMessage_HasTitleAndBody(t *testing.T) { - tests := []struct { - name string - source binary.Source - ver string - path string - size int64 - want []string - }{ - { - name: "explicit path", - source: binary.SourceExplicitPath, - ver: "dev", - path: ".fullsend/bin/fullsend", - size: 1024, - want: []string{"Source: --fullsend-binary", "Path: .fullsend/bin/fullsend", "Size: 1024 bytes"}, - }, - { - name: "checkout build", - source: binary.SourceCheckoutBuild, - ver: "dev", - path: "bin/fullsend", - size: 2048, - want: []string{"Source: cross-compiled from checkout", "Binary stamp: dev-vendored", "Path: bin/fullsend"}, - }, - { - name: "release download", - source: binary.SourceReleaseDownload, - ver: "0.4.0", - path: "bin/fullsend", - size: 4096, - want: []string{"Source: GitHub Release v0.4.0", "no -vendored suffix"}, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - msg := VendorCommitMessage(tt.source, tt.ver, tt.path, tt.size) - require.Contains(t, msg, "\n\n", "commit message must have title and body separated by blank line") - for _, line := range tt.want { - assert.Contains(t, msg, line) - } - }) - } -} - -func TestRemoveStaleBinaryCommitMessage_HasTitleAndBody(t *testing.T) { - msg := RemoveStaleBinaryCommitMessage(".fullsend/bin/fullsend") - require.Contains(t, msg, "\n\n") - assert.Contains(t, msg, "chore: remove vendored fullsend binary") - assert.Contains(t, msg, "Path: .fullsend/bin/fullsend") - assert.Contains(t, msg, "--vendor-fullsend-binary not set") -} - -func TestVendorCommitMessage_ReleaseTitle(t *testing.T) { - msg := VendorCommitMessage(binary.SourceReleaseDownload, "v0.4.0", "bin/fullsend", 100) - assert.True(t, strings.HasPrefix(msg, "chore: vendor fullsend v0.4.0 binary from release")) -} diff --git a/internal/layers/vendorbinary.go b/internal/layers/vendorbinary.go index 901920a0fc..15d326540b 100644 --- a/internal/layers/vendorbinary.go +++ b/internal/layers/vendorbinary.go @@ -83,8 +83,7 @@ func (l *VendorBinaryLayer) Install(ctx context.Context) error { } l.ui.StepStart("removing stale vendored binary") - deleteMsg := RemoveStaleBinaryCommitMessage(path) - if err := l.client.DeleteFile(ctx, l.org, l.repo, path, deleteMsg); err != nil { + if err := l.client.DeleteFile(ctx, l.org, l.repo, path, "chore: remove vendored binary"); err != nil { l.ui.StepFail("failed to remove vendored binary") return fmt.Errorf("deleting vendored binary: %w", err) } @@ -118,10 +117,10 @@ func (l *VendorBinaryLayer) Analyze(ctx context.Context) (*LayerReport, error) { if l.enabled { report.Status = StatusInstalled - report.Details = append(report.Details, fmt.Sprintf("vendored binary present at %s", l.binaryPath())) + report.Details = append(report.Details, "vendored binary present") } else { report.Status = StatusDegraded - report.Details = append(report.Details, fmt.Sprintf("stale vendored binary present at %s", l.binaryPath())) + report.Details = append(report.Details, "stale vendored binary present") report.WouldFix = append(report.WouldFix, "delete vendored binary") } return report, nil diff --git a/internal/layers/vendorbinary_test.go b/internal/layers/vendorbinary_test.go index 72ee7d1e05..d0c1304cb7 100644 --- a/internal/layers/vendorbinary_test.go +++ b/internal/layers/vendorbinary_test.go @@ -4,7 +4,6 @@ import ( "bytes" "context" "errors" - "strings" "testing" "github.com/stretchr/testify/assert" @@ -90,8 +89,6 @@ func TestVendorBinaryLayer_DisabledDeletesBinary(t *testing.T) { assert.Equal(t, "test-org", client.DeletedFiles[0].Owner) assert.Equal(t, ".fullsend", client.DeletedFiles[0].Repo) assert.Equal(t, "bin/fullsend", client.DeletedFiles[0].Path) - assert.Contains(t, client.DeletedFiles[0].Message, "\n\n") - assert.Contains(t, client.DeletedFiles[0].Message, "Path: bin/fullsend") // File should no longer be in FileContents _, ok := client.FileContents["test-org/.fullsend/bin/fullsend"] @@ -146,7 +143,7 @@ func TestVendorBinaryLayer_Analyze_EnabledPresent(t *testing.T) { require.NoError(t, err) assert.Equal(t, "vendor-binary", report.Name) assert.Equal(t, StatusInstalled, report.Status) - assert.True(t, strings.Contains(strings.Join(report.Details, " "), "vendored binary present at")) + assert.Contains(t, report.Details, "vendored binary present") } func TestVendorBinaryLayer_Analyze_EnabledAbsent(t *testing.T) { @@ -172,7 +169,7 @@ func TestVendorBinaryLayer_Analyze_DisabledPresent(t *testing.T) { report, err := layer.Analyze(context.Background()) require.NoError(t, err) assert.Equal(t, StatusDegraded, report.Status) - assert.True(t, strings.Contains(strings.Join(report.Details, " "), "stale vendored binary present at")) + assert.Contains(t, report.Details, "stale vendored binary present") assert.Contains(t, report.WouldFix, "delete vendored binary") } @@ -248,7 +245,7 @@ func TestVendorBinaryLayer_PerRepo_Analyze_EnabledPresent(t *testing.T) { report, err := layer.Analyze(context.Background()) require.NoError(t, err) assert.Equal(t, StatusInstalled, report.Status) - assert.True(t, strings.Contains(strings.Join(report.Details, " "), "vendored binary present at")) + assert.Contains(t, report.Details, "vendored binary present") } func TestVendorBinaryLayer_PerRepo_Analyze_DisabledPresent(t *testing.T) { @@ -264,7 +261,7 @@ func TestVendorBinaryLayer_PerRepo_Analyze_DisabledPresent(t *testing.T) { report, err := layer.Analyze(context.Background()) require.NoError(t, err) assert.Equal(t, StatusDegraded, report.Status) - assert.True(t, strings.Contains(strings.Join(report.Details, " "), "stale vendored binary present at")) + assert.Contains(t, report.Details, "stale vendored binary present") } func TestVendorBinaryLayer_PerRepo_EnabledCallsVendorFn(t *testing.T) { diff --git a/internal/resolve/resolve.go b/internal/resolve/resolve.go index 9b4bfaca8d..60a28b1cbf 100644 --- a/internal/resolve/resolve.go +++ b/internal/resolve/resolve.go @@ -15,8 +15,9 @@ import ( ) const ( - DefaultMaxDepth = 10 - DefaultMaxResources = 50 + DefaultMaxDepth = 10 + DefaultMaxResources = 50 + DefaultMaxRuntimeFetches = 10 ) // Dependency records a single URL that was resolved to a local cache path. @@ -466,3 +467,117 @@ func resolveSkillTransitiveDeps(ctx context.Context, parentURL, skillDirPath str return nil } + +// ResolveSkillURL resolves a single URL-referenced skill directory for runtime +// fetch. Like static skill resolution, it uses the forge API to list and fetch +// the skill directory contents, verifies the tree hash, and caches the result. +// The audit entry uses FetchType "runtime" to distinguish from static resolution. +// No transitive resolution — runtime-fetched skills are leaf nodes. +// +// Callers must gate on h.AllowRuntimeFetch and enforce MaxRuntimeFetches +// (or DefaultMaxRuntimeFetches when zero) before calling this function. +func ResolveSkillURL(ctx context.Context, rawURL string, h *harness.Harness, opts ResolveOpts) (Dependency, string, error) { + const field = "runtime_skill" + + cleanURL, expectedHash, hasHash := harness.ParseIntegrityHash(rawURL) + if !hasHash { + return Dependency{}, "", fmt.Errorf("%s: URL must include #sha256=... integrity hash", field) + } + if !strings.HasPrefix(cleanURL, "https://") { + return Dependency{}, "", fmt.Errorf("%s: URL scheme must be https: %s", field, cleanURL) + } + + allowedBy := h.MatchingAllowedPrefix(cleanURL) + if allowedBy == "" { + return Dependency{}, "", fmt.Errorf("%s: URL %q is not in allowed_remote_resources", field, cleanURL) + } + + forgeInfo, err := forge.ParseForgeURL(cleanURL) + if err != nil { + return Dependency{}, "", fmt.Errorf("%s: skill URLs must be hosted on a supported forge: %w", field, err) + } + + treePath, dirEntry, err := fetch.CacheGetDir(opts.WorkspaceRoot, expectedHash) + if err != nil { + return Dependency{}, "", fmt.Errorf("%s: cache lookup: %w", field, err) + } + + cacheHit := treePath != "" + fetchedAt := time.Now().UTC() + + if !cacheHit { + if opts.ForgeClient == nil { + return Dependency{}, "", fmt.Errorf("%s: ForgeClient is required to resolve skill URL %s (not cached)", field, cleanURL) + } + if opts.FetchPolicy.Offline { + return Dependency{}, "", fmt.Errorf("%s: offline mode, no cache entry for %s", field, cleanURL) + } + + dirPath := forgeInfo.Path + entries, err := opts.ForgeClient.ListDirectoryContents(ctx, forgeInfo.Owner, forgeInfo.Repo, dirPath, forgeInfo.Ref, true) + if err != nil { + return Dependency{}, "", fmt.Errorf("%s: listing directory at %s: %w", field, cleanURL, err) + } + + files := make(map[string][]byte) + for _, e := range entries { + if e.Type != "file" { + continue + } + var fullPath string + if dirPath == "" { + fullPath = e.Path + } else { + fullPath = dirPath + "/" + e.Path + } + content, err := opts.ForgeClient.GetFileContentAtRef(ctx, forgeInfo.Owner, forgeInfo.Repo, fullPath, forgeInfo.Ref) + if err != nil { + return Dependency{}, "", fmt.Errorf("%s: fetching file %s from %s: %w", field, e.Path, cleanURL, err) + } + files[e.Path] = content + } + + actualHash := fetch.ComputeTreeHash(files) + if actualHash != expectedHash { + return Dependency{}, "", fmt.Errorf("%s: integrity check failed for %s: expected %s, got %s", field, cleanURL, expectedHash, actualHash) + } + + if _, err := fetch.CachePutDir(opts.WorkspaceRoot, cleanURL, files); err != nil { + return Dependency{}, "", fmt.Errorf("%s: cache write: %w", field, err) + } + + cachePath, err := fetch.CachePath(opts.WorkspaceRoot, expectedHash) + if err != nil { + return Dependency{}, "", fmt.Errorf("%s: computing cache path: %w", field, err) + } + treePath = filepath.Join(cachePath, "tree") + } else { + fetchedAt = dirEntry.FetchTime + } + + if opts.AuditLogPath != "" { + if err := fetch.AppendFetchAudit(opts.AuditLogPath, fetch.FetchAuditEntry{ + TraceID: opts.TraceID, + FetchTime: fetchedAt, + URL: cleanURL, + SHA256: expectedHash, + FetchType: "runtime", + AllowedBy: allowedBy, + CacheHit: cacheHit, + }); err != nil { + return Dependency{}, "", fmt.Errorf("%s: writing fetch audit: %w", field, err) + } + } + + dep := Dependency{ + Field: field, + URL: cleanURL, + LocalPath: treePath, + SHA256: expectedHash, + FetchedAt: fetchedAt, + CacheHit: cacheHit, + Type: "directory", + } + + return dep, treePath, nil +} diff --git a/internal/resolve/resolve_test.go b/internal/resolve/resolve_test.go index e9ed2f1058..ef198387e9 100644 --- a/internal/resolve/resolve_test.go +++ b/internal/resolve/resolve_test.go @@ -1192,3 +1192,241 @@ func TestResolveHarness_NilForgeClientWithSkillURL(t *testing.T) { require.Error(t, err) assert.Contains(t, err.Error(), "ForgeClient is required") } + +// --- ResolveSkillURL tests (runtime fetch, directory model) --- + +func TestResolveSkillURL_ValidFetch(t *testing.T) { + fc := &forge.FakeClient{} + files := map[string][]byte{ + "SKILL.md": []byte("# Runtime Skill\nFetched at runtime."), + } + treeHash := registerSkillDir(fc, "skills/runtime", files) + + root := t.TempDir() + rawURL := forgeSkillURL("skills/runtime", treeHash) + h := &harness.Harness{ + AllowedRemoteResources: []string{testForgeBase}, + } + + dep, localPath, err := ResolveSkillURL(context.Background(), rawURL, h, ResolveOpts{ + WorkspaceRoot: root, + ForgeClient: fc, + }) + require.NoError(t, err) + + assert.Equal(t, "runtime_skill", dep.Field) + assert.Equal(t, forgeSkillCleanURL("skills/runtime"), dep.URL) + assert.Equal(t, treeHash, dep.SHA256) + assert.Equal(t, "directory", dep.Type) + assert.False(t, dep.CacheHit) + assert.True(t, strings.HasSuffix(localPath, "/tree")) + + got, err := os.ReadFile(filepath.Join(localPath, "SKILL.md")) + require.NoError(t, err) + assert.Equal(t, files["SKILL.md"], got) +} + +func TestResolveSkillURL_CacheHit(t *testing.T) { + fc := &forge.FakeClient{} + files := map[string][]byte{ + "SKILL.md": []byte("cached runtime skill"), + } + treeHash := registerSkillDir(fc, "skills/cached", files) + + root := t.TempDir() + cleanURL := forgeSkillCleanURL("skills/cached") + _, err := fetch.CachePutDir(root, cleanURL, files) + require.NoError(t, err) + + rawURL := forgeSkillURL("skills/cached", treeHash) + h := &harness.Harness{ + AllowedRemoteResources: []string{testForgeBase}, + } + + dep, localPath, err := ResolveSkillURL(context.Background(), rawURL, h, ResolveOpts{ + WorkspaceRoot: root, + ForgeClient: fc, + }) + require.NoError(t, err) + assert.True(t, dep.CacheHit) + + got, err := os.ReadFile(filepath.Join(localPath, "SKILL.md")) + require.NoError(t, err) + assert.Equal(t, files["SKILL.md"], got) +} + +func TestResolveSkillURL_HashMismatch(t *testing.T) { + fc := &forge.FakeClient{} + files := map[string][]byte{ + "SKILL.md": []byte("tampered content"), + } + registerSkillDir(fc, "skills/bad", files) + + wrongHash := fetch.ComputeTreeHash(map[string][]byte{ + "SKILL.md": []byte("expected content"), + }) + rawURL := forgeSkillURL("skills/bad", wrongHash) + h := &harness.Harness{ + AllowedRemoteResources: []string{testForgeBase}, + } + + _, _, err := ResolveSkillURL(context.Background(), rawURL, h, ResolveOpts{ + WorkspaceRoot: t.TempDir(), + ForgeClient: fc, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "integrity check failed") +} + +func TestResolveSkillURL_URLNotInAllowlist(t *testing.T) { + fc := &forge.FakeClient{} + files := map[string][]byte{ + "SKILL.md": []byte("skill content"), + } + treeHash := registerSkillDir(fc, "skills/blocked", files) + + rawURL := forgeSkillURL("skills/blocked", treeHash) + h := &harness.Harness{ + AllowedRemoteResources: []string{"https://other-domain.com/"}, + } + + _, _, err := ResolveSkillURL(context.Background(), rawURL, h, ResolveOpts{ + WorkspaceRoot: t.TempDir(), + ForgeClient: fc, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "not in allowed_remote_resources") +} + +func TestResolveSkillURL_MissingHash(t *testing.T) { + h := &harness.Harness{ + AllowedRemoteResources: []string{testForgeBase}, + } + + rawURL := forgeSkillCleanURL("skills/nohash") + _, _, err := ResolveSkillURL(context.Background(), rawURL, h, ResolveOpts{ + WorkspaceRoot: t.TempDir(), + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "integrity hash") +} + +func TestResolveSkillURL_NonHTTPSRejected(t *testing.T) { + fakeHash := strings.Repeat("a", 64) + h := &harness.Harness{ + AllowedRemoteResources: []string{"http://github.com/"}, + } + + _, _, err := ResolveSkillURL(context.Background(), fmt.Sprintf("http://github.com/test-org/test-repo/tree/main/skills/test#sha256=%s", fakeHash), h, ResolveOpts{ + WorkspaceRoot: t.TempDir(), + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "scheme must be https") +} + +func TestResolveSkillURL_NonForgeURLRejected(t *testing.T) { + fakeHash := strings.Repeat("a", 64) + h := &harness.Harness{ + AllowedRemoteResources: []string{"https://example.com/"}, + } + + _, _, err := ResolveSkillURL(context.Background(), fmt.Sprintf("https://example.com/skills/test#sha256=%s", fakeHash), h, ResolveOpts{ + WorkspaceRoot: t.TempDir(), + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "supported forge") +} + +func TestResolveSkillURL_AuditEntry(t *testing.T) { + fc := &forge.FakeClient{} + files := map[string][]byte{ + "SKILL.md": []byte("audited runtime skill"), + } + treeHash := registerSkillDir(fc, "skills/audited", files) + + root := t.TempDir() + auditPath := filepath.Join(root, "audit", "fetch-audit.jsonl") + rawURL := forgeSkillURL("skills/audited", treeHash) + h := &harness.Harness{ + AllowedRemoteResources: []string{testForgeBase}, + } + + _, _, err := ResolveSkillURL(context.Background(), rawURL, h, ResolveOpts{ + WorkspaceRoot: root, + ForgeClient: fc, + TraceID: "runtime-trace", + AuditLogPath: auditPath, + }) + require.NoError(t, err) + + f, err := os.Open(auditPath) + require.NoError(t, err) + defer f.Close() + + var entry fetch.FetchAuditEntry + scanner := bufio.NewScanner(f) + require.True(t, scanner.Scan()) + require.NoError(t, json.Unmarshal(scanner.Bytes(), &entry)) + + assert.Equal(t, "runtime-trace", entry.TraceID) + assert.Equal(t, "runtime", entry.FetchType) + assert.Equal(t, treeHash, entry.SHA256) + assert.False(t, entry.CacheHit) +} + +func TestResolveSkillURL_OfflineMiss(t *testing.T) { + fakeHash := strings.Repeat("a", 64) + rawURL := forgeSkillURL("skills/offline", fakeHash) + h := &harness.Harness{ + AllowedRemoteResources: []string{testForgeBase}, + } + + _, _, err := ResolveSkillURL(context.Background(), rawURL, h, ResolveOpts{ + WorkspaceRoot: t.TempDir(), + FetchPolicy: fetch.FetchPolicy{Offline: true}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "offline") +} + +func TestResolveSkillURL_OfflineHit(t *testing.T) { + files := map[string][]byte{ + "SKILL.md": []byte("cached skill for offline runtime"), + } + treeHash := fetch.ComputeTreeHash(files) + root := t.TempDir() + + cleanURL := forgeSkillCleanURL("skills/offline") + _, err := fetch.CachePutDir(root, cleanURL, files) + require.NoError(t, err) + + rawURL := forgeSkillURL("skills/offline", treeHash) + h := &harness.Harness{ + AllowedRemoteResources: []string{testForgeBase}, + } + + dep, localPath, err := ResolveSkillURL(context.Background(), rawURL, h, ResolveOpts{ + WorkspaceRoot: root, + FetchPolicy: fetch.FetchPolicy{Offline: true}, + }) + require.NoError(t, err) + assert.True(t, dep.CacheHit) + + got, err := os.ReadFile(filepath.Join(localPath, "SKILL.md")) + require.NoError(t, err) + assert.Equal(t, files["SKILL.md"], got) +} + +func TestResolveSkillURL_NilForgeClient(t *testing.T) { + fakeHash := strings.Repeat("a", 64) + rawURL := forgeSkillURL("skills/test", fakeHash) + h := &harness.Harness{ + AllowedRemoteResources: []string{testForgeBase}, + } + + _, _, err := ResolveSkillURL(context.Background(), rawURL, h, ResolveOpts{ + WorkspaceRoot: t.TempDir(), + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "ForgeClient is required") +} diff --git a/internal/scaffold/fullsend-repo/.github/actions/mint-token/action.yml b/internal/scaffold/fullsend-repo/.github/actions/mint-token/action.yml new file mode 100644 index 0000000000..baa0b74ada --- /dev/null +++ b/internal/scaffold/fullsend-repo/.github/actions/mint-token/action.yml @@ -0,0 +1,72 @@ +--- +name: Mint Token +description: >- + Exchange a GitHub OIDC token for a role-scoped installation token via the + fullsend token mint. Requires id-token: write permission on the calling job. + +inputs: + role: + description: Agent role name (e.g. triage, coder, review, fullsend) + required: true + repos: + description: Comma-separated repo names to scope the token to + required: true + mint_url: + description: URL of the token mint (typically from vars.FULLSEND_MINT_URL) + required: true + +outputs: + token: + description: Scoped GitHub installation token + value: ${{ steps.mint.outputs.token }} + +runs: + using: composite + steps: + - name: Mint token via OIDC + id: mint + shell: bash + env: + MINT_URL: ${{ inputs.mint_url }} + ROLE: ${{ inputs.role }} + REPOS: ${{ inputs.repos }} + run: | + set -euo pipefail + if [[ -z "$MINT_URL" ]]; then + echo "::error::FULLSEND_MINT_URL is not set" + exit 1 + fi + echo "::add-mask::$MINT_URL" + if [[ -z "$REPOS" ]]; then + echo "::error::repos input is required" + exit 1 + fi + OIDC_TOKEN=$(curl -sSf --retry 3 --retry-delay 2 --retry-all-errors \ + -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \ + "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=fullsend-mint" | jq -r '.value') + if [[ -z "$OIDC_TOKEN" || "$OIDC_TOKEN" == "null" ]]; then + echo "::error::Failed to obtain OIDC token" + exit 1 + fi + echo "::add-mask::$OIDC_TOKEN" + REPOS_JSON=$(echo "$REPOS" | jq -Rc 'split(",") | map(select(length > 0))') + BODY=$(jq -nc --arg role "$ROLE" --argjson repos "$REPOS_JSON" \ + '{role: $role, repos: $repos}') + echo "Requesting token: role=$ROLE repos=$REPOS" + MINT_RESPONSE=$(curl -sSf --retry 5 --retry-delay 5 --retry-all-errors \ + -H "Authorization: Bearer $OIDC_TOKEN" \ + -H "Content-Type: application/json" \ + -d "$BODY" \ + "${MINT_URL}/v1/token") + echo "::add-mask::$MINT_RESPONSE" + TOKEN=$(echo "$MINT_RESPONSE" | jq -r '.token') + if [[ -z "$TOKEN" || "$TOKEN" == "null" ]]; then + echo "::error::Token mint returned no token for role=$ROLE" + exit 1 + fi + echo "::add-mask::$TOKEN" + GRANTED_REPOS=$(echo "$MINT_RESPONSE" | jq -r '.granted_repos | if . then map(strings) | join(",") else empty end') + GRANTED_PERMS=$(echo "$MINT_RESPONSE" | jq -r '.granted_permissions | if . then to_entries | map("\(.key)=\(.value)") | join(",") else empty end') + REPO_SELECTION=$(echo "$MINT_RESPONSE" | jq -r '.repository_selection // empty') + echo "Granted scope: repos=${GRANTED_REPOS:-} permissions=${GRANTED_PERMS:-} repo_selection=${REPO_SELECTION:-}" + echo "token=$TOKEN" >> "$GITHUB_OUTPUT" diff --git a/internal/scaffold/fullsend-repo/.github/actions/setup-gcp/action.yml b/internal/scaffold/fullsend-repo/.github/actions/setup-gcp/action.yml new file mode 100644 index 0000000000..2f664f67f9 --- /dev/null +++ b/internal/scaffold/fullsend-repo/.github/actions/setup-gcp/action.yml @@ -0,0 +1,38 @@ +--- +name: Setup GCP +description: Authenticate to Google Cloud via Workload Identity Federation, mask credentials, and prepare sandbox credentials + +inputs: + gcp_wif_provider: + description: 'Workload Identity Federation provider resource name' + required: true + gcp_project_id: + description: 'GCP project ID — passed to google-github-actions/auth so the runner env has GOOGLE_CLOUD_PROJECT' + required: false + +runs: + using: composite + steps: + - name: Pre-mask GCP credential file path + shell: bash + run: echo "::add-mask::${GITHUB_WORKSPACE}/gha-creds-" + + - name: Authenticate to Google Cloud (WIF) + uses: google-github-actions/auth@v3 + with: + workload_identity_provider: ${{ inputs.gcp_wif_provider }} + project_id: ${{ inputs.gcp_project_id }} + + - name: Mask GCP credential file paths + shell: bash + run: | + for var in GOOGLE_GHA_CREDS_PATH GOOGLE_APPLICATION_CREDENTIALS CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE; do + val="${!var:-}" + if [[ -n "${val}" ]]; then + echo "::add-mask::${val}" + fi + done + + - name: Prepare sandbox credentials + shell: bash + run: bash scripts/prepare-sandbox-credentials.sh diff --git a/internal/scaffold/fullsend-repo/.github/actions/validate-enrollment/action.yml b/internal/scaffold/fullsend-repo/.github/actions/validate-enrollment/action.yml new file mode 100644 index 0000000000..40e705bdd4 --- /dev/null +++ b/internal/scaffold/fullsend-repo/.github/actions/validate-enrollment/action.yml @@ -0,0 +1,57 @@ +--- +name: Validate Enrollment +description: Validate that source repository is enrolled and extract repository metadata + +inputs: + source_repo: + description: 'Source repository in owner/repo format' + required: true + +outputs: + name: + description: 'Repository name (without owner)' + value: ${{ steps.extract.outputs.name }} + +runs: + using: composite + steps: + - name: Validate source repo is enrolled + shell: bash + env: + SOURCE_REPO: ${{ inputs.source_repo }} + run: | + set -euo pipefail + : "${SOURCE_REPO:?SOURCE_REPO is required}" + : "${GITHUB_REPOSITORY_OWNER:?GITHUB_REPOSITORY_OWNER is required}" + if [[ ! "$SOURCE_REPO" =~ ^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$ ]]; then + echo "::error::Invalid source_repo format: must be owner/repo" + exit 1 + fi + REPO_OWNER="${SOURCE_REPO%%/*}" + if [[ "$REPO_OWNER" != "$GITHUB_REPOSITORY_OWNER" ]]; then + echo "::error::source_repo owner does not match org" + exit 1 + fi + REPO_NAME="${SOURCE_REPO#*/}" + if [[ ! -f config.yaml ]]; then + echo "::error::config.yaml not found" + exit 1 + fi + if ! command -v yq &> /dev/null; then + echo "::error::yq command not found" + exit 1 + fi + ENABLED=$(yq ".repos.\"$REPO_NAME\".enabled" config.yaml) + if [[ "$ENABLED" != "true" ]]; then + echo "::error::repo is not enabled in config.yaml" + exit 1 + fi + echo "Validation passed for ${SOURCE_REPO}" + + - name: Extract repo parts + id: extract + shell: bash + env: + SOURCE_REPO: ${{ inputs.source_repo }} + run: | + echo "name=${SOURCE_REPO##*/}" >> "${GITHUB_OUTPUT}" diff --git a/internal/scaffold/fullsend-repo/skills/code-review/SKILL.md b/internal/scaffold/fullsend-repo/skills/code-review/SKILL.md index f67c35a17a..3d4031b855 100644 --- a/internal/scaffold/fullsend-repo/skills/code-review/SKILL.md +++ b/internal/scaffold/fullsend-repo/skills/code-review/SKILL.md @@ -184,12 +184,6 @@ readability or correctness. - Do documentation files reference behavior, APIs, or configurations changed by this PR? - Are any docs now stale as a result of the change? -- **Rename/deprecation completeness:** When a PR renames or removes an - identifier, grep for stale references using a bare-word pattern - (`\bOLD_NAME\b`) in addition to any syntax-specific pattern (e.g., - `OLD_NAME:` for YAML). Documentation files (`.md`, `.adoc`, `.rst`) - often reference field names in prose without syntax suffixes and will - be missed by syntax-specific patterns alone. #### Cross-repo contracts diff --git a/internal/scaffold/fullsend-repo/skills/docs-review/SKILL.md b/internal/scaffold/fullsend-repo/skills/docs-review/SKILL.md index 35a9d67aa6..e3dcab9402 100644 --- a/internal/scaffold/fullsend-repo/skills/docs-review/SKILL.md +++ b/internal/scaffold/fullsend-repo/skills/docs-review/SKILL.md @@ -94,15 +94,6 @@ and exit — there is nothing to check. Write a shell script that takes the identifiers from step 2 and greps for each one across the documentation files from step 3. - -**Rename/deprecation PRs:** When a PR renames or removes an identifier, -use a bare-word pattern (`\bOLD_NAME\b`) in addition to any -syntax-specific pattern (e.g., `OLD_NAME:` for YAML). Documentation -files often reference field names in prose without syntax suffixes and -will be missed by syntax-specific patterns alone. See the -`docs-currency` sub-agent's "Rename/deprecation pattern strategy" -section for the full approach. - Run the script in a single Bash call: ```bash diff --git a/internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/docs-currency.md b/internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/docs-currency.md index 71f47995ef..33c8a826fa 100644 --- a/internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/docs-currency.md +++ b/internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/docs-currency.md @@ -18,21 +18,3 @@ references to renamed/removed identifiers. Extract identifiers from the diff, then search documentation files for references. Flag docs that reference identifiers modified or removed in this PR. - -## Rename/deprecation pattern strategy - -When a PR renames or removes an identifier (config key, CLI flag, API -field, function name, etc.), search for stale references using **both** -broad and syntax-specific grep patterns: - -1. **Bare-word pattern** (`\bOLD_NAME\b`) — catches all mentions - including prose, comments, backtick-wrapped references, and code. - Run this first and evaluate hits in context. -2. **Syntax-specific pattern** (e.g., `OLD_NAME:` for YAML keys, - `--OLD_NAME` for CLI flags) — catches structured usage in config - and code files. - -Documentation files (`.md`, `.adoc`, `.rst`) frequently reference field -names in prose without syntax-specific suffixes (e.g., "set the -`repository` field"). Always include the bare-word pattern when scanning -these file types — a syntax-specific pattern alone will miss them. diff --git a/internal/scaffold/scaffold_test.go b/internal/scaffold/scaffold_test.go index 10429942f5..a863968957 100644 --- a/internal/scaffold/scaffold_test.go +++ b/internal/scaffold/scaffold_test.go @@ -57,6 +57,9 @@ func TestFullsendRepoFilesExist(t *testing.T) { ".github/workflows/review.yml", ".github/workflows/fix.yml", ".github/workflows/repo-maintenance.yml", + ".github/actions/mint-token/action.yml", + ".github/actions/setup-gcp/action.yml", + ".github/actions/validate-enrollment/action.yml", ".github/scripts/setup-agent-env.sh", "agents/triage.md", "agents/code.md", @@ -477,6 +480,59 @@ func TestRetroWorkflowContent(t *testing.T) { assert.Contains(t, s, "issues: write") } +func TestSetupGcpActionContent(t *testing.T) { + content, err := FullsendRepoFile(".github/actions/setup-gcp/action.yml") + require.NoError(t, err) + s := string(content) + // Verify inputs (composite actions cannot access vars/secrets directly) + assert.Contains(t, s, "inputs:") + assert.Contains(t, s, "gcp_wif_provider:") + assert.Contains(t, s, "gcp_project_id:") + assert.NotContains(t, s, "gcp_wif_sa_email:") + assert.NotContains(t, s, "gcp_auth_mode:") + assert.NotContains(t, s, "gcp_sa_key_json:") + assert.NotContains(t, s, "credentials_json:") + // Verify pre-mask step + assert.Contains(t, s, "Pre-mask GCP credential file path") + assert.Contains(t, s, "GITHUB_WORKSPACE}/gha-creds-") + // Verify WIF authentication + assert.Contains(t, s, "google-github-actions/auth@v3") + assert.Contains(t, s, "workload_identity_provider:") + assert.Contains(t, s, "project_id:") + assert.NotContains(t, s, "service_account:") + // Verify credential masking + assert.Contains(t, s, "Mask GCP credential file paths") + assert.Contains(t, s, "::add-mask::") + assert.Contains(t, s, "GOOGLE_GHA_CREDS_PATH") + assert.Contains(t, s, "GOOGLE_APPLICATION_CREDENTIALS") + assert.Contains(t, s, "CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE") + // Verify sandbox preparation + assert.Contains(t, s, "prepare-sandbox-credentials.sh") +} + +func TestValidateEnrollmentActionContent(t *testing.T) { + content, err := FullsendRepoFile(".github/actions/validate-enrollment/action.yml") + require.NoError(t, err) + s := string(content) + // Verify inputs declarations + assert.Contains(t, s, "inputs:") + assert.Contains(t, s, "source_repo:") + assert.Contains(t, s, "required: true") + // Verify outputs contract + assert.Contains(t, s, "outputs:") + assert.Contains(t, s, "name:") + assert.Contains(t, s, "steps.extract.outputs.name") + // Verify step ID matches output reference + assert.Contains(t, s, "id: extract") + // Verify SOURCE_REPO env var wiring + assert.Contains(t, s, "SOURCE_REPO: ${{ inputs.source_repo }}") + // Verify enrollment validation is inlined (not a script reference that + // could be overwritten by customized/scripts/). + assert.NotContains(t, s, "validate-source-repo.sh") + assert.Contains(t, s, "config.yaml not found") + assert.Contains(t, s, "repo is not enabled in config.yaml") +} + func TestValidateSourceRepoContent(t *testing.T) { content, err := FullsendRepoFile("scripts/validate-source-repo.sh") require.NoError(t, err) @@ -630,6 +686,22 @@ func TestRepoMaintenanceTokenCoversAllRepos(t *testing.T) { "repo-list step must extract both enabled and disabled repos so the minted token covers them for unenrollment") } +func TestMintTokenActionContent(t *testing.T) { + content, err := FullsendRepoFile(".github/actions/mint-token/action.yml") + require.NoError(t, err) + s := string(content) + assert.Contains(t, s, "Mint Token") + assert.Contains(t, s, "OIDC") + assert.Contains(t, s, "audience=fullsend-mint") + assert.Contains(t, s, "/v1/token") + assert.Contains(t, s, "::add-mask::$OIDC_TOKEN") + assert.Contains(t, s, "::add-mask::$TOKEN") + assert.Contains(t, s, "ACTIONS_ID_TOKEN_REQUEST_TOKEN") + assert.Contains(t, s, "ACTIONS_ID_TOKEN_REQUEST_URL") + assert.Contains(t, s, "jq -nc --arg role") + assert.NotContains(t, s, "create-github-app-token") +} + func TestReconcileReposContent(t *testing.T) { content, err := FullsendRepoFile("scripts/reconcile-repos.sh") require.NoError(t, err) diff --git a/skills/merge-queue/SKILL.md b/skills/merge-queue/SKILL.md index 7932d97788..b539d3a62c 100644 --- a/skills/merge-queue/SKILL.md +++ b/skills/merge-queue/SKILL.md @@ -1,21 +1,18 @@ --- name: merge-queue description: >- - Use when you need to add a PR to a GitHub merge queue, check what's currently - queued, or find out why a PR was removed from the queue. The gh CLI has no - built-in merge-queue commands, so this skill provides scripts that use the + Use when you need to add a PR to a GitHub merge queue. The gh CLI has no + built-in merge-queue command, so this skill provides a script that uses the GraphQL API. -allowed-tools: Bash(bash skills/merge-queue/scripts/*:*) +allowed-tools: Bash(bash skills/merge-queue/scripts/enqueue-pr.sh:*) --- # Merge Queue -## Enqueue a PR - Run `bash skills/merge-queue/scripts/enqueue-pr.sh [PR_NUMBER_OR_URL]` to enqueue a PR. Omit the argument to enqueue the current branch's PR. -### Accepted input formats +## Accepted input formats - **PR number:** `652` (uses the current repo context from `gh`) - **PR URL:** `https://github.com/owner/repo/pull/652` @@ -23,20 +20,6 @@ Omit the argument to enqueue the current branch's PR. The `owner/repo#number` format is **not supported** — use a URL or number instead. -## Check queue status - -Run `bash skills/merge-queue/scripts/queue-status.sh [OWNER/REPO] [BRANCH]` to list PRs currently in the merge queue. - -Both arguments are optional — defaults to the current repo and `main` branch. - -Shows each entry's position, state, PR title/URL, author, enqueuer, and estimated time to merge. - -## Investigate dequeue reasons - -Run `bash skills/merge-queue/scripts/dequeue-reason.sh ` to find out why a PR was removed from the merge queue. - -Shows each removal event's timestamp, reason (e.g. `failed_checks`, `merge_conflict`), and the commit SHA at the time of removal. - ## Prerequisites - `gh` CLI authenticated with write access to the target repository diff --git a/skills/merge-queue/scripts/dequeue-reason.sh b/skills/merge-queue/scripts/dequeue-reason.sh deleted file mode 100755 index 9a3896d99b..0000000000 --- a/skills/merge-queue/scripts/dequeue-reason.sh +++ /dev/null @@ -1,64 +0,0 @@ -#!/usr/bin/env bash -# Shows why a PR was removed from the merge queue. -# Usage: dequeue-reason.sh -# -# Queries the PR timeline for RemovedFromMergeQueueEvent entries and -# prints the reason, timestamp, and commit SHA for each removal. -# Requires: gh CLI authenticated, jq. - -set -euo pipefail - -pr="${1:?Usage: dequeue-reason.sh }" - -# Resolve to owner/repo and PR number -if [[ "$pr" =~ ^https://github.com/([^/]+/[^/]+)/pull/([0-9]+) ]]; then - repo="${BASH_REMATCH[1]}" - number="${BASH_REMATCH[2]}" -elif [[ "$pr" =~ ^[0-9]+$ ]]; then - repo="$(gh repo view --json nameWithOwner -q .nameWithOwner)" - number="$pr" -else - echo "Error: provide a PR number or URL" >&2 - exit 1 -fi - -owner="${repo%%/*}" -name="${repo##*/}" - -result="$(gh api graphql -f query=' - query($owner: String!, $name: String!, $number: Int!) { - repository(owner: $owner, name: $name) { - pullRequest(number: $number) { - title - url - timelineItems(last: 20, itemTypes: [REMOVED_FROM_MERGE_QUEUE_EVENT]) { - nodes { - ... on RemovedFromMergeQueueEvent { - createdAt - reason - beforeCommit { abbreviatedOid } - } - } - } - } - } - } -' -f owner="$owner" -f name="$name" -F number="$number")" - -title="$(echo "$result" | jq -r '.data.repository.pullRequest.title')" -url="$(echo "$result" | jq -r '.data.repository.pullRequest.url')" -count="$(echo "$result" | jq '.data.repository.pullRequest.timelineItems.nodes | length')" - -if [[ "$count" -eq 0 ]]; then - echo "${url} ${title}" - echo " No merge queue removals found." - exit 0 -fi - -echo "${url} ${title}" -echo "${count} removal(s):" -echo "" -echo "$result" | jq -r ' - .data.repository.pullRequest.timelineItems.nodes[] | - " \(.createdAt) reason: \(.reason) commit: \(.beforeCommit.abbreviatedOid // "unknown")" -' diff --git a/skills/merge-queue/scripts/queue-status.sh b/skills/merge-queue/scripts/queue-status.sh deleted file mode 100755 index 79653c3593..0000000000 --- a/skills/merge-queue/scripts/queue-status.sh +++ /dev/null @@ -1,51 +0,0 @@ -#!/usr/bin/env bash -# Lists PRs currently in the merge queue for a branch. -# Usage: queue-status.sh [OWNER/REPO] [BRANCH] -# -# Defaults: OWNER/REPO from current gh repo context, BRANCH=main -# Requires: gh CLI authenticated, jq. - -set -euo pipefail - -repo="${1:-$(gh repo view --json nameWithOwner -q .nameWithOwner)}" -branch="${2:-main}" -owner="${repo%%/*}" -name="${repo##*/}" - -result="$(gh api graphql -f query=' - query($owner: String!, $name: String!, $branch: String!) { - repository(owner: $owner, name: $name) { - mergeQueue(branch: $branch) { - entries(first: 50) { - nodes { - position - state - estimatedTimeToMerge - enqueuedAt - enqueuer { login } - pullRequest { - number - title - url - author { login } - } - } - } - } - } - } -' -f owner="$owner" -f name="$name" -f branch="$branch")" - -count="$(echo "$result" | jq '.data.repository.mergeQueue.entries.nodes | length')" - -if [[ "$count" -eq 0 ]]; then - echo "Merge queue for ${repo}:${branch} is empty." - exit 0 -fi - -echo "Merge queue for ${repo}:${branch} — ${count} enqueued:" -echo "" -echo "$result" | jq -r ' - .data.repository.mergeQueue.entries.nodes[] | - " #\(.position) [\(.state)] \(.pullRequest.url) \(.pullRequest.title)\n by \(.pullRequest.author.login), enqueued \(.enqueuedAt) by \(.enqueuer.login) ETA: \(.estimatedTimeToMerge // "unknown")s" -'