diff --git a/docs/guides/dev/behaviour-drivers.md b/docs/guides/dev/behaviour-drivers.md index 7739506446..c9c354f237 100644 --- a/docs/guides/dev/behaviour-drivers.md +++ b/docs/guides/dev/behaviour-drivers.md @@ -8,9 +8,12 @@ Behaviour tests isolate forge-specific code behind drivers so Gherkin scenarios |-----------|---------|----------------| | `scm.Driver` | `pkg/behaviourtest/drivers/scm` | Issues, comments, labels (via GetIssue), file commits | | `ci.Driver` | `pkg/behaviourtest/drivers/ci` | Workflow polling, logs, artifact download | -| `install.Driver` | `pkg/behaviourtest/drivers/install` | Provision and tear down fullsend in the acquired pool org | +| `install.Driver` | `pkg/behaviourtest/drivers/install` | Unified repo allocation: leases pool slots, lazily creates/installs repos, and manages mint lifecycle via `AllocateRepo`/`DeallocateRepo`/`Finalize`/`Capacity` | +| `install.Factory` | `pkg/behaviourtest/drivers/install` | Constructs a unified `Driver` from suite-level parameters; suite code selects a Factory and does not compose `MintDriver` + ensurer + pool itself (#6135) | +| `install.MintDriver` | `pkg/behaviourtest/drivers/install` | Provision and tear down fullsend mint in the acquired pool org (internal to composed `Driver`) | | `install.State` | `pkg/behaviourtest/drivers/install` | Post-install config paths (script commits, workflow polling) | -| `install.RepoEnsurer` | `pkg/behaviourtest/drivers/install` | Lazily create and install numbered pool repos on demand; caches by org/repo key; concurrent-safe via singleflight | +| `install.StateProvider` | `pkg/behaviourtest/drivers/install` | Optional interface for `Driver` implementations that carry suite-level install state (`InstallState() State`) | +| `install.RepoEnsurer` | `pkg/behaviourtest/drivers/install` | Lazily create and install numbered pool repos on demand; caches by org/repo key; concurrent-safe via singleflight (internal to composed `Driver`) | v1 reference implementations: @@ -28,11 +31,11 @@ BEHAVIOUR_CI=githubactions # future: tekton, gitlabci BEHAVIOUR_INSTALL_MODE=per-repo # v1 default and only supported value ``` -The suite in `e2e/behaviour/suite_test.go` (or an external runner) acquires a pool org via `pkg/e2etest`, runs pre-install cleanup, calls `install.Driver.Install` (which deploys the mint), constructs SCM and CI drivers, creates a `world.RepoPool` (a buffered-channel lease pool of logical repo names), then runs godog with `pkg/behaviourtest/suite.InitScenario`. `InitScenario` clones a template `*world.World` per scenario and leases a unique repo name from the pool for the scenario's duration. Unsupported `BEHAVIOUR_INSTALL_MODE` values fail at suite startup. +The suite in `e2e/behaviour/suite_test.go` (or an external runner) acquires a pool org via `pkg/e2etest`, runs pre-install cleanup, selects an `install.Factory` (e.g. `cfmint.NewFactory`), and calls it to obtain a unified `install.Driver`. The Factory internally creates a `MintDriver`, performs suite setup (mint deploy via `NewComposedDriver`), and wires the internal repo-name pool and `RepoEnsurer` behind the Driver — the suite does not compose these itself. The suite then constructs SCM and CI drivers and runs godog with `pkg/behaviourtest/suite.InitScenario`. `InitScenario` clones a template `*world.World` per scenario; repo allocation (slot lease + lazy create/install) is handled by the unified driver's `AllocateRepo` when the `Given the enrolled test repository` step runs. Unsupported `BEHAVIOUR_INSTALL_MODE` values fail at suite startup. ### Install driver (v1 per-repo) -The install driver only manages the **mint lifecycle**: the cfmint driver deploys a Cloudflare Worker preview mint and tears it down; the legacy driver holds a pre-configured mint URL. Neither driver runs `github setup`, post-install validation, or `TeardownPerRepoInstall` on any target repository — that responsibility belongs to the `RepoEnsurer`, which lazily creates and installs numbered pool repos (`test-repo-01` … `test-repo-12`) on demand via `EnsureRepo`. +The unified `install.Driver` (created by `install.NewComposedDriver`) wraps a `MintDriver` with an internal pool and `RepoEnsurer`. The `MintDriver` only manages the **mint lifecycle**: the cfmint driver deploys a Cloudflare Worker preview mint and tears it down; the legacy driver holds a pre-configured mint URL. Neither mint driver runs `github setup`, post-install validation, or `TeardownPerRepoInstall` on any target repository — that responsibility belongs to the internal `RepoEnsurer`, which lazily creates and installs numbered pool repos (`test-repo-01` … `test-repo-12`) on demand via `AllocateRepo`. Pool orgs must already have shared GitHub Apps, org-level mint enrollment, and per-repo mint enrollment for each numbered repo (one-time GCP admin step on the hosted mint project). The driver does not run `fullsend admin install` or `fullsend mint enroll`. See [e2e-testing.md](e2e-testing.md#behaviour-tests-and-per-repo-mint-enrollment). diff --git a/docs/guides/dev/behaviour-testing.md b/docs/guides/dev/behaviour-testing.md index cb509412c9..67778ee82d 100644 --- a/docs/guides/dev/behaviour-testing.md +++ b/docs/guides/dev/behaviour-testing.md @@ -208,9 +208,9 @@ scenario or when `-v` output from multiple scenarios would interleave. In CI, the test runner mints cross-org `e2e` installation tokens via OIDC (same as admin e2e) for GitHub API operations. Triage workflows on the pool org's `test-repo` mint same-org `triage` tokens from vendored reusable workflows; those require per-repo mint enrollment (`PER_REPO_WIF_REPOS`) on the hosted mint project. Pool `test-repo` repos are enrolled once by a GCP admin — not during CI install. The install driver provisions repo-scoped inference WIF via `fullsend inference provision` before `github setup`. See [e2e-testing.md](e2e-testing.md#behaviour-tests-and-per-repo-mint-enrollment). -### Lazy create+install (`RepoEnsurer`) +### Lazy create+install (`AllocateRepo`) -The `Given the enrolled test repository` step lazily creates and installs numbered pool repos (`test-repo-NN`) on demand via `RepoEnsurer`. When a scenario leases a repo name from the pool and an ensurer is configured, the step calls `EnsureRepo(ctx, org, repoName)` which: +The `Given the enrolled test repository` step lazily creates and installs numbered pool repos (`test-repo-NN`) on demand via the unified `install.Driver.AllocateRepo`. When a scenario calls `AllocateRepo`, the composed driver leases a slot from its internal pool and delegates to the internal `RepoEnsurer`, which calls `EnsureRepo(ctx, org, repoName)` to: 1. Creates the repo if it does not exist (the forge's `auto_init` provides the initial commit). 2. Validates post-install files; if validation fails, runs `fullsend github setup` (and inference provision when configured). @@ -395,24 +395,47 @@ require github.com/fullsend-ai/fullsend v0.x.y // released tag, not @main ### API changes -**`suite.InitScenario` signature change:** The function signature changed from `InitScenario(sc, w)` to `InitScenario(sc, template, pool)` starting in the release that includes this change. Instead of passing a single `*world.World`, callers now pass a template `*world.World` (cloned per scenario) and a `*world.RepoPool` (used to lease unique repo names). Update your `suite_test.go` accordingly: +**`suite.InitScenario` signature change:** The function signature changed from `InitScenario(sc, template, pool)` to `InitScenario(sc, template)` starting in the release that includes this change. The `*world.RepoPool` parameter was removed because repo allocation is now handled by the unified `install.Driver` on the World template — `AllocateRepo` is called by the `Given the enrolled test repository` step, and `DeallocateRepo` is called in the After hook. Select an `install.Factory` and call it to obtain a Driver, then set `World.RepoDriver` on the template: ```go -pool, err := world.NewRepoPool(12) +// Select an install.Factory — suite code does not compose +// MintDriver + ensurer + pool itself. +factory := cfmint.NewFactory(ctx, e2eCfg, 12, cfmintCfg) + +driver, err := factory(org, client, token, binary, gcpProjectID, t.Logf) if err != nil { - t.Fatalf("creating repo pool: %v", err) + t.Fatalf("creating install driver: %v", err) +} +t.Cleanup(func() { + if err := driver.Finalize(context.Background()); err != nil { + t.Logf("driver finalize: %v", err) + } +}) + +// Extract install state from the driver. +var mintState install.State +if sp, ok := driver.(install.StateProvider); ok { + mintState = sp.InstallState() } -template := &world.World{ /* ... driver fields ... */ } +template := &world.World{ + Install: mintState, + RepoDriver: driver, + // ... other driver fields ... +} suiteRunner := godog.TestSuite{ ScenarioInitializer: func(sc *godog.ScenarioContext) { - suite.InitScenario(sc, template, pool) + suite.InitScenario(sc, template) }, // ... } ``` +**`World.Ensurer` → `World.RepoDriver` field change:** The `Ensurer install.RepoEnsurer` field on `world.World` has been replaced by `RepoDriver install.Driver`. External runners that set `World.Ensurer` must switch to `World.RepoDriver` and pass a unified `install.Driver` (created via an `install.Factory`). + +**`install.Driver` → `install.MintDriver` rename:** The former `install.Driver` interface (with `Install`/`Teardown` methods) has been renamed to `install.MintDriver`. The `install.Driver` name now refers to the unified repo-allocation interface (`AllocateRepo`/`DeallocateRepo`/`Finalize`/`Capacity`). Update any code that references the old `install.Driver` type. The `cfmint.NewDriver` and `legacy.NewDriver` functions now return `install.MintDriver`. + **`steps.Register` signature change:** The function signature changed from `Register(ctx, w)` (where `ctx` was a `*godog.ScenarioContext` and `w` was a `*world.World`) to `Register(sc)` starting in the same release. Step definitions no longer receive `*world.World` as a parameter. Instead, they accept `context.Context` and extract the per-scenario World via `world.FromContext(ctx)`. **`scm.Driver.DeleteRepo` addition:** The `scm.Driver` interface now includes a `DeleteRepo(ctx context.Context, owner, repo string) error` method. `CleanupScenario` calls it to delete ephemeral fork repos after each scenario. External `scm.Driver` implementations must add this method — return `forge.ErrNotFound` when the repository does not exist. diff --git a/docs/guides/dev/e2e-testing.md b/docs/guides/dev/e2e-testing.md index 17456d15cd..5ade6ac86f 100644 --- a/docs/guides/dev/e2e-testing.md +++ b/docs/guides/dev/e2e-testing.md @@ -78,7 +78,7 @@ Prefer **`wrangler versions upload --name=mint-test --preview-alias=…`** so ru Behaviour tests install fullsend in **per-repo** mode (`fullsend github setup`). Triage workflows mint same-org `triage` tokens from vendored reusable workflows; that requires per-repo mint enrollment (`PER_REPO_WIF_REPOS`). The install driver does **not** run `mint enroll` — pool org behaviour repos must be enrolled once by a GCP admin on the hosted mint project. -Admin e2e uses the singular `halfsend-NN/test-repo` name. Behaviour tests lease numbered `halfsend-NN/test-repo-01` … `test-repo-12` names from a `RepoPool`; these repos are **lazily created and installed** on demand by `RepoEnsurer` (see [behaviour-testing.md](behaviour-testing.md#lazy-createinstall-repoensurer)). Pre-provisioning numbered repos in the pool org is no longer required — mint enrollment for those names is still pre-provisioned so it is not on the critical path. Enroll base names only — do **not** enroll `*-fork` names (forks are ephemeral PR sources and mint against the enrolled base repo). GitHub repositories need not exist yet — enroll is a mint allowlist / WIF-provider update only. +Admin e2e uses the singular `halfsend-NN/test-repo` name. Behaviour tests lease numbered `halfsend-NN/test-repo-01` … `test-repo-12` slots via the unified `install.Driver.AllocateRepo`; these repos are **lazily created and installed** on demand by the composed driver (see [behaviour-testing.md](behaviour-testing.md#lazy-createinstall-allocaterepo)). Pre-provisioning numbered repos in the pool org is no longer required — mint enrollment for those names is still pre-provisioned so it is not on the critical path. Enroll base names only — do **not** enroll `*-fork` names (forks are ephemeral PR sources and mint against the enrolled base repo). GitHub repositories need not exist yet — enroll is a mint allowlist / WIF-provider update only. Inference (`E2E_GCP_PROJECT_ID`) and mint (`it-gcp-konflux-dev-fullsend` for the hosted mint) may be different GCP projects. The behaviour install driver runs `fullsend inference provision /test-repo` using CI credentials on the inference project (same access model as admin e2e), then passes the repo-scoped WIF provider to `github setup`. `E2E_GCP_WIF_PROVIDER` authenticates the CI job itself; it is not written to pool org repos. diff --git a/docs/guides/infrastructure/mint-administration.md b/docs/guides/infrastructure/mint-administration.md index d883637d66..8f39ff833d 100644 --- a/docs/guides/infrastructure/mint-administration.md +++ b/docs/guides/infrastructure/mint-administration.md @@ -76,7 +76,7 @@ Pass this URL as `--mint-url` when running `fullsend github setup`, or set the ` `roles/owner` covers all of the above for users with broad access. - **Behaviour / e2e pool orgs:** Enroll `halfsend-NN/test-repo` (admin e2e) and `halfsend-NN/test-repo-01` … `test-repo-12` (lazily created and installed on demand by `RepoEnsurer` — see [behaviour-testing.md](../dev/behaviour-testing.md#lazy-createinstall-repoensurer)) on the hosted mint (`PER_REPO_WIF_REPOS`). Run `fullsend mint enroll owner/repo` once per name — not from CI; do not enroll `*-fork` names. Repos need not exist at enrollment time — enroll is a mint allowlist / WIF-provider update only; `RepoEnsurer` creates the repos when a behaviour scenario first leases them. See [e2e-testing.md](../dev/e2e-testing.md#behaviour-tests-and-per-repo-mint-enrollment). + **Behaviour / e2e pool orgs:** Enroll `halfsend-NN/test-repo` (admin e2e) and `halfsend-NN/test-repo-01` … `test-repo-12` (lazily created and installed on demand by the composed `install.Driver` — see [behaviour-testing.md](../dev/behaviour-testing.md#lazy-createinstall-allocaterepo)) on the hosted mint (`PER_REPO_WIF_REPOS`). Run `fullsend mint enroll owner/repo` once per name — not from CI; do not enroll `*-fork` names. Repos need not exist at enrollment time — enroll is a mint allowlist / WIF-provider update only; `AllocateRepo` creates the repos when a behaviour scenario first leases them. See [e2e-testing.md](../dev/e2e-testing.md#behaviour-tests-and-per-repo-mint-enrollment). An administrator can grant all required roles with a single script: diff --git a/e2e/behaviour/suite_test.go b/e2e/behaviour/suite_test.go index 82bb2a9cfd..3f2e5c25ab 100644 --- a/e2e/behaviour/suite_test.go +++ b/e2e/behaviour/suite_test.go @@ -24,9 +24,9 @@ import ( ) // poolSize is the number of enrolled test-repo-NN repos in the pool org. -// GODOG_CONCURRENCY must not exceed this — extra workers would block in -// pool.Acquire with no warning because the pool org only has test-repo-01 -// through test-repo-12 with per-repo mint enrollment. +// This determines the driver's Capacity — the maximum number of +// concurrently allocated repos. Scenarios that exceed this block in +// AllocateRepo until a slot frees. const poolSize = 12 // suiteName identifies this test suite. It is used to derive the CF Worker @@ -46,9 +46,6 @@ func TestBehaviourSuite(t *testing.T) { } concurrency = n } - if concurrency > poolSize { - t.Fatalf("GODOG_CONCURRENCY=%d exceeds repo pool size %d", concurrency, poolSize) - } cfg := env.LoadRunnerConfig() if err := cfg.Validate(); err != nil { @@ -70,10 +67,10 @@ func TestBehaviourSuite(t *testing.T) { binary := e2etest.BuildCLIBinary(t) - // Construct the CF mint driver with caller-provided parameters. - // The driver does not hardcode pool size or test-repo-NN assumptions; - // the calling code passes allowed orgs, per-repo WIF repos, and - // workflow host repos. + // Build a Factory that captures the cfmint configuration. + // The Factory is the #6135 consolidation seam — suite code + // selects a Factory and does not compose MintDriver + ensurer + + // pool itself. // // AllowedOrgs is explicitly empty ("") — this is per-repo mode, so // we must not dual-enroll the pool org as an allowed org. The CLI's @@ -82,7 +79,7 @@ func TestBehaviourSuite(t *testing.T) { // WorkflowHostRepos registers the pool repos whose vendored workflows // need to mint tokens. Without this, the mint rejects // job_workflow_ref values from pool repos → 401. - installDriver, err := cfmint.NewDriver(client, token, binary, e2eCfg.GCPProjectID, t.Logf, cfmint.Config{ + factory := cfmint.NewFactory(ctx, e2eCfg, poolSize, cfmint.Config{ PEMDir: e2eCfg.CFMintPEMDir, SuiteName: suiteName, AllowedOrgs: "", @@ -90,55 +87,49 @@ func TestBehaviourSuite(t *testing.T) { WorkflowHostRepos: buildWorkflowHostRepos(org), AppSet: "fullsend-test", }) + + e2etest.CleanupStaleResources(ctx, client, token, org, t) + + // Call the Factory to construct the unified Driver. The Factory + // creates the MintDriver, calls mint.Install, and wires the + // internal pool + RepoEnsurer behind the Driver interface. + driver, err := factory(org, client, token, binary, e2eCfg.GCPProjectID, t.Logf) if err != nil { t.Fatalf("creating install driver: %v", err) } - e2etest.CleanupStaleResources(ctx, client, token, org, t) - - // Register teardown before Install so that a partially deployed - // preview mint is cleaned up even if Install fails partway through. - var installState install.State + // Register Finalize for teardown — it reclaims any outstanding + // leases, logs them, and tears down the mint (e.g. CF preview). t.Cleanup(func() { - if installState == nil { - return - } - teardownCtx := context.Background() - if teardownErr := installDriver.Teardown(teardownCtx, org, installState); teardownErr != nil { - t.Logf("install teardown: %v", teardownErr) + if finalizeErr := driver.Finalize(context.Background()); finalizeErr != nil { + t.Logf("driver finalize: %v", finalizeErr) } }) - installState, err = installDriver.Install(ctx, org) - if err != nil { - t.Fatalf("installing fullsend on %s: %v", org, err) + // Advisory: excess workers block in AllocateRepo (#6135). + if concurrency > driver.Capacity() { + t.Logf("WARNING: GODOG_CONCURRENCY=%d exceeds driver capacity %d; excess workers may block in AllocateRepo", concurrency, driver.Capacity()) } - // The install state carries the mint URL from the selected driver. - // Thread it to the ensurer so additional pool repos use the same - // mint endpoint. - if m, ok := installState.(install.MintURLProvider); ok && m.MintURL() != "" { - t.Logf("using mint URL for ensurer: %s", m.MintURL()) - e2eCfg.MintURL = m.MintURL() - } - - pool, err := world.NewRepoPool(poolSize) - if err != nil { - t.Fatalf("creating repo pool: %v", err) - } - - ensurer := install.NewRepoEnsurer(e2eCfg, client, token, binary, t.Logf) - // The install driver only manages the mint lifecycle — it does not // install on any specific repo. RepoName and RepoFull are set per- - // scenario by the ensurer when "Given the enrolled test repository" + // scenario by AllocateRepo when "Given the enrolled test repository" // acquires a leased pool repo. + // + // The Driver carries the mint install state internally; extract it + // via the StateProvider interface so the suite does not thread State + // separately. + var mintState install.State + if sp, ok := driver.(install.StateProvider); ok { + mintState = sp.InstallState() + } + template := &world.World{ Config: cfg, SCM: scmgh.New(client), CI: gaci.New(client, token), - Install: installState, - Ensurer: ensurer, + Install: mintState, + RepoDriver: driver, Org: org, Token: token, Logf: t.Logf, @@ -148,7 +139,7 @@ func TestBehaviourSuite(t *testing.T) { suiteRunner := godog.TestSuite{ Name: "behaviour", - ScenarioInitializer: func(sc *godog.ScenarioContext) { suite.InitScenario(sc, template, pool) }, + ScenarioInitializer: func(sc *godog.ScenarioContext) { suite.InitScenario(sc, template) }, Options: &godog.Options{ Format: "pretty", Paths: []string{"features"}, diff --git a/pkg/behaviourtest/drivers/install/cfmint/driver.go b/pkg/behaviourtest/drivers/install/cfmint/driver.go index 0622119208..6caf80492b 100644 --- a/pkg/behaviourtest/drivers/install/cfmint/driver.go +++ b/pkg/behaviourtest/drivers/install/cfmint/driver.go @@ -1,12 +1,13 @@ -// Package cfmint implements an install.Driver that deploys a temporary +// Package cfmint implements an install.MintDriver that deploys a temporary // Cloudflare Worker preview mint for behaviour tests. The preview mint is // self-contained: all configuration (PEMs, allowlists, provenance) is // passed at deploy time. Teardown abandons the preview alias. // // The driver only manages the mint lifecycle (deploy + teardown). It does // not run github setup, post-install validation, or per-repo teardown on -// any target repository — that responsibility belongs to the RepoEnsurer -// which handles leased pool repos on demand. +// any target repository — that responsibility belongs to the composed +// install.Driver which handles leased pool repos on demand via +// AllocateRepo. package cfmint import ( @@ -78,7 +79,7 @@ func NewDriver( token, binary, gcpProjectID string, logf func(string, ...any), cfg Config, -) (install.Driver, error) { +) (install.MintDriver, error) { if cfg.PEMDir == "" { return nil, fmt.Errorf("cfmint: PEMDir is required (no PEMs materialized)") } @@ -112,6 +113,33 @@ func NewDriver( }, nil } +// NewFactory returns an install.Factory that builds a composed Driver +// backed by a Cloudflare mint preview. The returned closure captures +// ctx, env config, pool size, and cfmint Config; the Factory's +// org/client/token/binary/gcpProjectID are supplied by the suite at +// call time. +// +// The Factory calls NewDriver and install.NewComposedDriver internally +// so the suite does not compose MintDriver + pool + ensurer itself. +func NewFactory(ctx context.Context, e2eCfg e2etest.EnvConfig, poolSize int, cfg Config) install.Factory { + return func( + org string, + client forge.Client, + token, binary, gcpProjectID string, + logf func(string, ...any), + ) (install.Driver, error) { + md, err := NewDriver(client, token, binary, gcpProjectID, logf, cfg) + if err != nil { + return nil, fmt.Errorf("creating cfmint driver: %w", err) + } + d, _, err := install.NewComposedDriver(ctx, md, org, e2eCfg, client, token, binary, poolSize, logf) + if err != nil { + return nil, err + } + return d, nil + } +} + // WorkerName derives the CF Worker script name from a suite name. // The name clearly indicates it is the e2e/BT worker for that suite. func WorkerName(suiteName string) string { @@ -166,8 +194,8 @@ func (d *driver) Install(_ context.Context, org string) (install.State, error) { } // The driver only manages the mint lifecycle. Per-repo github setup - // and post-install validation are handled by the RepoEnsurer for each - // leased pool repo. + // and post-install validation are handled by the composed + // install.Driver for each leased pool repo. return install.NewPerRepoState(org, "", mintURL), nil } diff --git a/pkg/behaviourtest/drivers/install/cfmint/driver_test.go b/pkg/behaviourtest/drivers/install/cfmint/driver_test.go index a0a817b1d9..5846caa627 100644 --- a/pkg/behaviourtest/drivers/install/cfmint/driver_test.go +++ b/pkg/behaviourtest/drivers/install/cfmint/driver_test.go @@ -217,8 +217,8 @@ func TestDriver_Implements_Install_Driver(t *testing.T) { }) require.NoError(t, err) - // Verify it implements install.Driver. - var _ install.Driver = d + // Verify it implements install.MintDriver. + var _ install.MintDriver = d } func TestPerRepoState_MintURL(t *testing.T) { diff --git a/pkg/behaviourtest/drivers/install/composed.go b/pkg/behaviourtest/drivers/install/composed.go new file mode 100644 index 0000000000..04885b13ba --- /dev/null +++ b/pkg/behaviourtest/drivers/install/composed.go @@ -0,0 +1,197 @@ +package install + +import ( + "context" + "errors" + "fmt" + "sort" + "sync" + + "github.com/fullsend-ai/fullsend/internal/forge" + "github.com/fullsend-ai/fullsend/pkg/e2etest" +) + +// composedDriver wraps a MintDriver, an internal slot pool, and a +// RepoEnsurer behind the unified Driver interface. Suite and scenario +// lifecycle code speaks only to this surface. +// +// Thread safety: all exported methods are safe for concurrent callers. +type composedDriver struct { + mint MintDriver + mintState State + ensurer RepoEnsurer + org string + logf func(string, ...any) + + // Internal slot pool. + poolSize int + names chan string // available slot names + + mu sync.Mutex + outstanding map[string]struct{} // currently allocated repo names +} + +// NewComposedDriver creates a unified Driver that wraps a MintDriver +// with internal pool and RepoEnsurer management. It calls +// mint.Install to perform suite-level setup before returning so that +// setup failures fail the suite before scenarios run. +// +// If Install fails after a partial deploy, the constructor calls +// mint.Teardown to clean up before returning the error. +// +// The returned State is the mint-level install state (org-scoped, empty +// repo). Callers use it to seed the template World's Install field. +// Per-scenario repo-scoped state is constructed after AllocateRepo. +func NewComposedDriver( + ctx context.Context, + mint MintDriver, + org string, + e2eCfg e2etest.EnvConfig, + client forge.Client, + token, binary string, + poolSize int, + logf func(string, ...any), +) (Driver, State, error) { + // Step 1: install the mint (deploy preview, etc.). + mintState, err := mint.Install(ctx, org) + if err != nil { + // Attempt teardown in case the deploy partially succeeded + // (e.g. cfmint set previewAlias before deployCFMint failed). + _ = mint.Teardown(ctx, org, nil) + return nil, nil, fmt.Errorf("installing mint: %w", err) + } + + // Step 2: thread mint URL to the ensurer config so additional + // pool repos use the same mint endpoint. + if m, ok := mintState.(MintURLProvider); ok && m.MintURL() != "" { + logf("using mint URL for ensurer: %s", m.MintURL()) + e2eCfg.MintURL = m.MintURL() + } + + // Step 3: create the internal slot pool. + ch := make(chan string, poolSize) + for i := 1; i <= poolSize; i++ { + ch <- fmt.Sprintf("test-repo-%02d", i) + } + + // Step 4: create the internal ensurer. + ensurer := NewRepoEnsurer(e2eCfg, client, token, binary, logf) + + d := &composedDriver{ + mint: mint, + mintState: mintState, + ensurer: ensurer, + org: org, + logf: logf, + poolSize: poolSize, + names: ch, + outstanding: make(map[string]struct{}), + } + return d, mintState, nil +} + +// AllocateRepo leases a slot from the internal pool and ensures the +// repo is created and has fullsend installed. Blocks until a slot is +// free or ctx is cancelled. +func (d *composedDriver) AllocateRepo(ctx context.Context) (string, error) { + // Acquire a slot name from the pool. + var name string + select { + case name = <-d.names: + case <-ctx.Done(): + return "", fmt.Errorf("acquiring repo slot: %w", ctx.Err()) + } + + // Ensure the repo is created and installed. + if _, err := d.ensurer.EnsureRepo(ctx, d.org, name); err != nil { + // Release the slot back on failure. + d.names <- name + return "", fmt.Errorf("ensuring repo %s/%s: %w", d.org, name, err) + } + + d.mu.Lock() + d.outstanding[name] = struct{}{} + d.mu.Unlock() + + return name, nil +} + +// DeallocateRepo returns a previously allocated repo slot. Errors on +// unknown name or double-release. +func (d *composedDriver) DeallocateRepo(_ context.Context, repoName string) error { + d.mu.Lock() + if _, ok := d.outstanding[repoName]; !ok { + d.mu.Unlock() + return fmt.Errorf("DeallocateRepo: %q is not an outstanding allocation (possible double-release)", repoName) + } + delete(d.outstanding, repoName) + d.mu.Unlock() + + // Return the name to the pool. The channel buffer equals poolSize + // and this name was removed during AllocateRepo, so the send is + // guaranteed non-blocking. + d.names <- repoName + return nil +} + +// Finalize tears down suite-scoped resources. If leases are still +// outstanding it reclaims them, logs the names, completes teardown, +// and returns an error so leaked After-hooks fail CI without stranding +// resources (e.g. CF preview mints). +func (d *composedDriver) Finalize(ctx context.Context) error { + // Collect outstanding leases. + d.mu.Lock() + outstanding := make([]string, 0, len(d.outstanding)) + for name := range d.outstanding { + outstanding = append(outstanding, name) + } + d.mu.Unlock() + + // Deterministic order for logging. + sort.Strings(outstanding) + + var leakErr error + if len(outstanding) > 0 { + d.logf("Finalize: reclaiming %d outstanding allocations: %v", len(outstanding), outstanding) + for _, name := range outstanding { + d.names <- name + } + d.mu.Lock() + for _, name := range outstanding { + delete(d.outstanding, name) + } + d.mu.Unlock() + leakErr = fmt.Errorf("Finalize: %d outstanding allocations were not deallocated: %v", len(outstanding), outstanding) + } + + // Tear down mint infrastructure (e.g. CF preview worker). + var teardownErr error + if d.mint != nil && d.mintState != nil { + if err := d.mint.Teardown(ctx, d.org, d.mintState); err != nil { + d.logf("Finalize: mint teardown: %v", err) + teardownErr = fmt.Errorf("mint teardown: %w", err) + } + } + + return errors.Join(leakErr, teardownErr) +} + +// Capacity returns the max concurrent outstanding allocations (the +// driver's real parallelism ceiling). +func (d *composedDriver) Capacity() int { + return d.poolSize +} + +// InstallState returns the suite-level install state obtained from the +// MintDriver during construction. The suite uses this via the +// StateProvider interface to seed World.Install without threading State +// through the Factory return value. +func (d *composedDriver) InstallState() State { + return d.mintState +} + +// Compile-time checks. +var ( + _ Driver = (*composedDriver)(nil) + _ StateProvider = (*composedDriver)(nil) +) diff --git a/pkg/behaviourtest/drivers/install/composed_test.go b/pkg/behaviourtest/drivers/install/composed_test.go new file mode 100644 index 0000000000..c1744a6efc --- /dev/null +++ b/pkg/behaviourtest/drivers/install/composed_test.go @@ -0,0 +1,385 @@ +package install + +import ( + "context" + "fmt" + "sync" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/fullsend-ai/fullsend/pkg/e2etest" +) + +// --- fakeMintDriver for composed driver tests --- + +type fakeMintDriver struct { + installState State + installErr error + teardownErr error + teardownCalls int + mu sync.Mutex +} + +func (f *fakeMintDriver) Install(_ context.Context, _ string) (State, error) { + if f.installErr != nil { + return nil, f.installErr + } + return f.installState, nil +} + +func (f *fakeMintDriver) Teardown(_ context.Context, _ string, _ State) error { + f.mu.Lock() + defer f.mu.Unlock() + f.teardownCalls++ + return f.teardownErr +} + +func (f *fakeMintDriver) TeardownCallCount() int { + f.mu.Lock() + defer f.mu.Unlock() + return f.teardownCalls +} + +var _ MintDriver = (*fakeMintDriver)(nil) + +// --- NewComposedDriver tests --- + +func TestNewComposedDriver_Success(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", "https://mint.test"), + } + e2eCfg := e2etest.EnvConfig{} + + drv, mintState, err := NewComposedDriver( + context.Background(), mint, "org", e2eCfg, + nil, "tok", "/bin/true", 3, t.Logf, + ) + require.NoError(t, err) + require.NotNil(t, drv) + require.NotNil(t, mintState) + + assert.Equal(t, 3, drv.Capacity()) + assert.Equal(t, "org", mintState.ConfigOwner()) +} + +func TestNewComposedDriver_InstallFailure_CallsTeardown(t *testing.T) { + mint := &fakeMintDriver{ + installErr: fmt.Errorf("deploy exploded"), + } + e2eCfg := e2etest.EnvConfig{} + + drv, mintState, err := NewComposedDriver( + context.Background(), mint, "org", e2eCfg, + nil, "tok", "/bin/true", 3, t.Logf, + ) + require.Error(t, err) + assert.Nil(t, drv) + assert.Nil(t, mintState) + assert.Contains(t, err.Error(), "installing mint") + + // Teardown should have been called for cleanup. + assert.Equal(t, 1, mint.TeardownCallCount(), + "Teardown should be called to clean up a partial deploy") +} + +func TestNewComposedDriver_ThreadsMintURL(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", "https://preview.test"), + } + e2eCfg := e2etest.EnvConfig{} + + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2eCfg, + nil, "tok", "/bin/true", 3, t.Logf, + ) + require.NoError(t, err) + + // Type-assert to the concrete composedDriver and inspect the + // internal RepoEnsurer to verify the mint URL was threaded through. + cd, ok := drv.(*composedDriver) + require.True(t, ok, "expected *composedDriver, got %T", drv) + + re, ok := cd.ensurer.(*repoEnsurer) + require.True(t, ok, "expected *repoEnsurer, got %T", cd.ensurer) + + assert.Equal(t, "https://preview.test", re.e2eCfg.MintURL, + "mint URL should be threaded from MintDriver install state to the internal RepoEnsurer config") +} + +// --- AllocateRepo / DeallocateRepo tests --- + +func TestAllocateRepo_ReturnsDistinctNames(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + } + // Use a fake ensurer that always succeeds. + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", 3, t.Logf, + ) + require.NoError(t, err) + + // Replace the ensurer with a fake that always succeeds. + cd := drv.(*composedDriver) + cd.ensurer = newFakeEnsurer() + + names := make(map[string]bool) + for range 3 { + name, err := drv.AllocateRepo(context.Background()) + require.NoError(t, err) + assert.False(t, names[name], "duplicate name %q", name) + names[name] = true + } + assert.Len(t, names, 3) +} + +func TestAllocateRepo_BlocksWhenExhausted(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + } + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", 1, t.Logf, + ) + require.NoError(t, err) + cd := drv.(*composedDriver) + cd.ensurer = newFakeEnsurer() + + // Exhaust the single slot. + name, err := drv.AllocateRepo(context.Background()) + require.NoError(t, err) + + // Next allocate should block and fail with cancelled context. + cancelledCtx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond) + defer cancel() + + _, err = drv.AllocateRepo(cancelledCtx) + require.Error(t, err) + assert.Contains(t, err.Error(), "acquiring repo slot") + + // Release and try again — should succeed. + require.NoError(t, drv.DeallocateRepo(context.Background(), name)) + name2, err := drv.AllocateRepo(context.Background()) + require.NoError(t, err) + assert.Equal(t, name, name2) +} + +func TestAllocateRepo_EnsureFailure_ReleasesSlot(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + } + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", 1, t.Logf, + ) + require.NoError(t, err) + + // Use a failing ensurer. + cd := drv.(*composedDriver) + cd.ensurer = &failingEnsurer{err: fmt.Errorf("install boom")} + + _, err = drv.AllocateRepo(context.Background()) + require.Error(t, err) + assert.Contains(t, err.Error(), "ensuring repo") + + // The slot should have been released back — replace ensurer and + // verify the slot is available. + cd.ensurer = newFakeEnsurer() + name, err := drv.AllocateRepo(context.Background()) + require.NoError(t, err) + assert.NotEmpty(t, name) +} + +func TestDeallocateRepo_UnknownName(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + } + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", 3, t.Logf, + ) + require.NoError(t, err) + + err = drv.DeallocateRepo(context.Background(), "not-allocated") + require.Error(t, err) + assert.Contains(t, err.Error(), "not an outstanding allocation") +} + +func TestDeallocateRepo_DoubleRelease(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + } + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", 3, t.Logf, + ) + require.NoError(t, err) + cd := drv.(*composedDriver) + cd.ensurer = newFakeEnsurer() + + name, err := drv.AllocateRepo(context.Background()) + require.NoError(t, err) + + require.NoError(t, drv.DeallocateRepo(context.Background(), name)) + + // Second release should error. + err = drv.DeallocateRepo(context.Background(), name) + require.Error(t, err) + assert.Contains(t, err.Error(), "double-release") +} + +// --- Finalize tests --- + +func TestFinalize_NoOutstanding_TearsDownMint(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + } + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", 3, t.Logf, + ) + require.NoError(t, err) + + err = drv.Finalize(context.Background()) + require.NoError(t, err) + + assert.Equal(t, 1, mint.TeardownCallCount()) +} + +func TestFinalize_WithOutstanding_ReclaimsAndReturnsError(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + } + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", 3, t.Logf, + ) + require.NoError(t, err) + cd := drv.(*composedDriver) + cd.ensurer = newFakeEnsurer() + + // Allocate but don't deallocate. + name, err := drv.AllocateRepo(context.Background()) + require.NoError(t, err) + + err = drv.Finalize(context.Background()) + require.Error(t, err) + assert.Contains(t, err.Error(), "outstanding allocations were not deallocated") + assert.Contains(t, err.Error(), name) + + // Mint teardown should still have been called. + assert.Equal(t, 1, mint.TeardownCallCount()) +} + +func TestFinalize_CombinesLeakAndTeardownErrors(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + teardownErr: fmt.Errorf("teardown boom"), + } + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", 3, t.Logf, + ) + require.NoError(t, err) + cd := drv.(*composedDriver) + cd.ensurer = newFakeEnsurer() + + // Allocate but don't deallocate — this creates a leak error. + _, err = drv.AllocateRepo(context.Background()) + require.NoError(t, err) + + err = drv.Finalize(context.Background()) + require.Error(t, err) + assert.Contains(t, err.Error(), "outstanding allocations were not deallocated") + assert.Contains(t, err.Error(), "mint teardown") +} + +func TestFinalize_TeardownErrorOnly(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + teardownErr: fmt.Errorf("teardown boom"), + } + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", 3, t.Logf, + ) + require.NoError(t, err) + + // No outstanding allocations, but teardown fails. + err = drv.Finalize(context.Background()) + require.Error(t, err) + assert.Contains(t, err.Error(), "mint teardown") +} + +func TestCapacity(t *testing.T) { + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + } + for _, size := range []int{1, 5, 12} { + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", size, t.Logf, + ) + require.NoError(t, err) + assert.Equal(t, size, drv.Capacity()) + } +} + +// --- Concurrent access test --- + +func TestComposedDriver_ConcurrentAllocateDeallocate(t *testing.T) { + t.Parallel() + + mint := &fakeMintDriver{ + installState: NewPerRepoState("org", "", ""), + } + const poolSize = 4 + drv, _, err := NewComposedDriver( + context.Background(), mint, "org", e2etest.EnvConfig{}, + nil, "tok", "/bin/true", poolSize, t.Logf, + ) + require.NoError(t, err) + cd := drv.(*composedDriver) + cd.ensurer = newFakeEnsurer() + + const goroutines = 20 + var wg sync.WaitGroup + + for range goroutines { + wg.Add(1) + go func() { + defer wg.Done() + name, allocErr := drv.AllocateRepo(context.Background()) + if allocErr != nil { + t.Errorf("AllocateRepo: %v", allocErr) + return + } + // Simulate scenario work. + time.Sleep(time.Millisecond) + if deallocErr := drv.DeallocateRepo(context.Background(), name); deallocErr != nil { + t.Errorf("DeallocateRepo: %v", deallocErr) + } + }() + } + wg.Wait() + + // All slots should be returned. + err = drv.Finalize(context.Background()) + require.NoError(t, err, "no outstanding allocations expected after all goroutines finished") +} + +// --- Test helpers --- + +// failingEnsurer always returns an error. +type failingEnsurer struct { + err error +} + +func (f *failingEnsurer) EnsureRepo(_ context.Context, _, _ string) (State, error) { + return nil, f.err +} + +var _ RepoEnsurer = (*failingEnsurer)(nil) diff --git a/pkg/behaviourtest/drivers/install/driver.go b/pkg/behaviourtest/drivers/install/driver.go index 1ac9f1a8ab..77d210a0da 100644 --- a/pkg/behaviourtest/drivers/install/driver.go +++ b/pkg/behaviourtest/drivers/install/driver.go @@ -1,15 +1,68 @@ package install -import "context" +import ( + "context" -// Driver provisions and tears down fullsend in an acquired pool org. -// Driver is used only during suite setup (single-threaded) and is not + "github.com/fullsend-ai/fullsend/internal/forge" +) + +// Factory constructs a unified Driver from suite-level parameters. +// Implementations capture env config, pool size, and backend-specific +// settings (PEMs, platform flags) in a closure so the suite does not +// thread those details. +// +// See #6135 for the consolidation contract. +type Factory func( + org string, + client forge.Client, + token, binary, gcpProjectID string, + logf func(string, ...any), +) (Driver, error) + +// MintDriver provisions and tears down fullsend in an acquired pool org. +// MintDriver is used only during suite setup (single-threaded) and is not // shared across concurrent scenarios. -type Driver interface { +// +// Renamed from Driver so that the Driver name can refer to the unified +// repo-allocation interface (#6135). +type MintDriver interface { Install(ctx context.Context, org string) (State, error) Teardown(ctx context.Context, org string, state State) error } +// Driver is the unified repo-allocation interface that suite and +// scenario lifecycle code speaks to. It combines mint/environment +// lifecycle, repo-name leasing, and lazy repo creation behind a +// single surface. Implementations must be safe for concurrent use. +// +// See #6135 for the full contract. +type Driver interface { + // AllocateRepo leases a slot and makes that repo ready. It blocks + // until a slot is free or ctx is cancelled. Returns repo name only + // (org is fixed for the driver). + AllocateRepo(ctx context.Context) (repoName string, err error) + + // DeallocateRepo returns a previously allocated repo. Errors on + // unknown name or double-release. + DeallocateRepo(ctx context.Context, repoName string) error + + // Finalize tears down suite-scoped resources. If leases are still + // outstanding, it reclaims them (logging the names), completes + // teardown, and returns an error so leaked After-hooks fail CI + // without stranding resources. + Finalize(ctx context.Context) error + + // Capacity returns the max concurrent outstanding allocations. + Capacity() int +} + +// StateProvider is optionally implemented by Driver values that carry +// suite-level install state. The suite uses this to seed World.Install +// without threading State separately through the Factory return value. +type StateProvider interface { + InstallState() State +} + // State describes where behaviour tests find fullsend configuration after install. // // Concurrency: the PerRepoState implementation is a read-only snapshot @@ -39,8 +92,9 @@ type State interface { } // MintURLProvider is optionally implemented by State values that carry -// the effective mint URL. The suite uses this to thread the mint URL -// from the install driver to the RepoEnsurer. +// the effective mint URL. The composed driver uses this to thread the +// mint URL from the MintDriver install output to the internal +// RepoEnsurer. type MintURLProvider interface { MintURL() string } diff --git a/pkg/behaviourtest/drivers/install/legacy/driver.go b/pkg/behaviourtest/drivers/install/legacy/driver.go index 102b9ad96d..3735007a69 100644 --- a/pkg/behaviourtest/drivers/install/legacy/driver.go +++ b/pkg/behaviourtest/drivers/install/legacy/driver.go @@ -1,11 +1,11 @@ -// Package legacy implements an install.Driver that uses a pre-configured +// Package legacy implements an install.MintDriver that uses a pre-configured // mint URL. This is the original install path retained as a separate driver // so other test configurations can select it. // // The driver only manages the mint URL lifecycle. It does not run github // setup, post-install validation, or per-repo teardown on any target -// repository — that responsibility belongs to the RepoEnsurer which -// handles leased pool repos on demand. +// repository — that responsibility belongs to the composed install.Driver +// which handles leased pool repos on demand via AllocateRepo. package legacy import ( @@ -26,14 +26,14 @@ type driver struct { logf func(string, ...any) } -// NewDriver creates a legacy install driver that uses the provided +// NewDriver creates a legacy mint driver that uses the provided // mintURL. The driver only holds the mint URL; per-repo install is -// handled by the RepoEnsurer for each leased pool repo. +// handled by the composed install.Driver for each leased pool repo. func NewDriver( client forge.Client, token, binary, mintURL, gcpProjectID string, logf func(string, ...any), -) (install.Driver, error) { +) (install.MintDriver, error) { if mintURL == "" { return nil, fmt.Errorf("legacy: mintURL is required") } @@ -49,13 +49,12 @@ func NewDriver( func (d *driver) Install(_ context.Context, org string) (install.State, error) { // The driver only provides the mint URL. Per-repo github setup and - // post-install validation are handled by the RepoEnsurer for each - // leased pool repo. + // post-install validation are handled by the composed install.Driver + // for each leased pool repo. return install.NewPerRepoState(org, "", d.mintURL), nil } func (d *driver) Teardown(_ context.Context, _ string, _ install.State) error { // The legacy driver has no mint infrastructure to tear down. - // Per-repo teardown is handled by the RepoEnsurer. return nil } diff --git a/pkg/behaviourtest/drivers/install/legacy/driver_test.go b/pkg/behaviourtest/drivers/install/legacy/driver_test.go index d9704aed4a..099c73b1e2 100644 --- a/pkg/behaviourtest/drivers/install/legacy/driver_test.go +++ b/pkg/behaviourtest/drivers/install/legacy/driver_test.go @@ -21,8 +21,8 @@ func TestNewDriver_OK(t *testing.T) { require.NoError(t, err) require.NotNil(t, d) - // Verify it implements install.Driver. - var _ install.Driver = d + // Verify it implements install.MintDriver. + var _ install.MintDriver = d } func TestInstall_ReturnsStateWithMintURL(t *testing.T) { diff --git a/pkg/behaviourtest/steps/triage.go b/pkg/behaviourtest/steps/triage.go index 838cec473b..7c2cc45841 100644 --- a/pkg/behaviourtest/steps/triage.go +++ b/pkg/behaviourtest/steps/triage.go @@ -7,6 +7,7 @@ import ( "github.com/cucumber/godog" + "github.com/fullsend-ai/fullsend/pkg/behaviourtest/drivers/install" "github.com/fullsend-ai/fullsend/pkg/behaviourtest/drivers/scm" "github.com/fullsend-ai/fullsend/pkg/behaviourtest/world" ) @@ -36,25 +37,38 @@ func registerTriageSteps(sc *godog.ScenarioContext) { } func givenEnrolledTestRepository(ctx context.Context, w *world.World) error { - // When a leased repo name is available (from the pool) and an ensurer - // is configured, lazily create and install the leased repo. This - // removes the requirement for pre-existing repos in the pool org. - if w.LeasedRepoName != "" && w.Ensurer != nil { - st, err := w.Ensurer.EnsureRepo(ctx, w.Org, w.LeasedRepoName) - if err != nil { - return fmt.Errorf("ensuring leased repo %s/%s: %w", w.Org, w.LeasedRepoName, err) - } - w.Install = st - w.RepoOwner = w.Org - w.RepoName = w.LeasedRepoName - w.RepoFull = w.Org + "/" + w.LeasedRepoName + if w.RepoDriver == nil { + return fmt.Errorf("no repo driver configured; configure a unified install.Driver on the World template") + } + + // Guard against double-call: if a repo is already allocated for this + // scenario, treat the second call as a no-op. Without this guard each + // invocation would lease a new slot while overwriting LeasedRepoName, + // silently leaking the first slot until Finalize reclaims it. + if w.LeasedRepoName != "" { return nil } - // No leased repo and no ensurer — the scenario cannot proceed without - // a repo. The install driver only manages the mint lifecycle and does - // not target a specific repo. - return fmt.Errorf("no leased repo name or ensurer configured; use a pool to acquire a repo for each scenario") + // AllocateRepo leases a pool slot and ensures the repo is created + // and has fullsend installed. This replaces the former two-step + // flow of pool.Acquire (in Before hook) + Ensurer.EnsureRepo (here). + repoName, err := w.RepoDriver.AllocateRepo(ctx) + if err != nil { + return fmt.Errorf("allocating repo: %w", err) + } + + w.LeasedRepoName = repoName + w.RepoOwner = w.Org + w.RepoName = repoName + w.RepoFull = w.Org + "/" + repoName + + // Construct per-repo install state for this allocated repo. + // The mint URL is internal to the driver and not needed by + // step-level code; TriageWorkflowRepo, ConfigPathPrefix, etc. + // are derived from org and repo name. + w.Install = install.NewPerRepoState(w.Org, repoName, "") + + return nil } func givenEnrolledRepository(w *world.World, fullName string) error { diff --git a/pkg/behaviourtest/steps/triage_test.go b/pkg/behaviourtest/steps/triage_test.go new file mode 100644 index 0000000000..dbb47b2c58 --- /dev/null +++ b/pkg/behaviourtest/steps/triage_test.go @@ -0,0 +1,57 @@ +package steps + +import ( + "context" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/fullsend-ai/fullsend/pkg/behaviourtest/drivers/install" + "github.com/fullsend-ai/fullsend/pkg/behaviourtest/world" +) + +// fakeRepoDriver is a minimal install.Driver for unit testing step-level +// repo allocation. It tracks how many times AllocateRepo is called. +type fakeRepoDriver struct { + allocateCalls int + nextName string +} + +func (f *fakeRepoDriver) AllocateRepo(_ context.Context) (string, error) { + f.allocateCalls++ + return f.nextName, nil +} + +func (f *fakeRepoDriver) DeallocateRepo(_ context.Context, _ string) error { return nil } +func (f *fakeRepoDriver) Finalize(_ context.Context) error { return nil } +func (f *fakeRepoDriver) Capacity() int { return 3 } + +var _ install.Driver = (*fakeRepoDriver)(nil) + +func TestGivenEnrolledTestRepository_DoubleCallGuard(t *testing.T) { + drv := &fakeRepoDriver{nextName: "test-repo-01"} + w := &world.World{ + Org: "test-org", + RepoDriver: drv, + } + + // First call should allocate. + err := givenEnrolledTestRepository(context.Background(), w) + require.NoError(t, err) + assert.Equal(t, 1, drv.allocateCalls) + assert.Equal(t, "test-repo-01", w.LeasedRepoName) + + // Second call is a no-op — should not allocate again. + err = givenEnrolledTestRepository(context.Background(), w) + require.NoError(t, err) + assert.Equal(t, 1, drv.allocateCalls, "second call should not allocate another slot") +} + +func TestGivenEnrolledTestRepository_NilDriver(t *testing.T) { + w := &world.World{Org: "test-org"} + + err := givenEnrolledTestRepository(context.Background(), w) + require.Error(t, err) + assert.Contains(t, err.Error(), "no repo driver configured") +} diff --git a/pkg/behaviourtest/suite/init.go b/pkg/behaviourtest/suite/init.go index 65e178816b..fc4cb19590 100644 --- a/pkg/behaviourtest/suite/init.go +++ b/pkg/behaviourtest/suite/init.go @@ -14,62 +14,55 @@ import ( ) // InitScenario registers tag-based skips, Before/After hooks, and shared steps. -// Each scenario receives its own World cloned from template. If pool is non-nil, -// a repo name is leased from it for the scenario's duration. -func InitScenario(sc *godog.ScenarioContext, template *world.World, pool *world.RepoPool) { +// Each scenario receives its own World cloned from template. Repo allocation +// and deallocation are handled by the unified install.Driver on the World — +// AllocateRepo is called by the "Given the enrolled test repository" step, +// and DeallocateRepo is called in the After hook for scenarios that allocated. +func InitScenario(sc *godog.ScenarioContext, template *world.World) { sc.Before(func(ctx context.Context, scenario *godog.Scenario) (context.Context, error) { - return beforeScenario(ctx, tagNames(scenario.Tags), template, pool) + return beforeScenario(ctx, tagNames(scenario.Tags), template) }) sc.After(func(ctx context.Context, scenario *godog.Scenario, err error) (context.Context, error) { - return afterScenario(ctx, pool, err) + return afterScenario(ctx, err) }) steps.Register(sc) } -// beforeScenario clones the template World, resets scenario fields, and -// optionally acquires a pool lease. Extracted for unit testing without -// live godog infrastructure. -func beforeScenario(ctx context.Context, tags []string, template *world.World, pool *world.RepoPool) (context.Context, error) { +// beforeScenario clones the template World and resets scenario fields. +// Repo allocation is deferred to the step that needs it (via +// World.RepoDriver.AllocateRepo) rather than eagerly acquiring a pool +// slot here, so scenarios that never allocate a repo do not consume a +// slot. +func beforeScenario(ctx context.Context, tags []string, template *world.World) (context.Context, error) { if err := SkipErrorForTagNames(tags, template); err != nil { return ctx, err } w := template.Clone() resetScenarioWorld(w) - if pool != nil { - name, err := pool.Acquire(ctx) - if err != nil { - return ctx, fmt.Errorf("acquiring pool repo name: %w", err) - } - w.LeasedRepoName = name - } - ctx = world.WithWorld(ctx, w) return ctx, nil } -// afterScenario runs scenario cleanup and releases the pool lease. -// Extracted for unit testing. Release errors are surfaced as test -// failures rather than panicking the godog runner. -// -// pool.Release is deferred so the lease is returned even if -// CleanupScenario panics. Named return values allow the deferred -// closure to surface a release error when no scenario error exists. -func afterScenario(ctx context.Context, pool *world.RepoPool, scenarioErr error) (_ context.Context, retErr error) { +// afterScenario runs scenario cleanup and deallocates the repo if one +// was allocated. Deallocation is deferred so the slot is returned even +// if CleanupScenario panics. Named return values allow the deferred +// closure to surface a deallocation error when no scenario error exists. +func afterScenario(ctx context.Context, scenarioErr error) (_ context.Context, retErr error) { retErr = scenarioErr w := world.FromContext(ctx) if w == nil { return ctx, retErr } - if pool != nil && w.LeasedRepoName != "" { + if w.RepoDriver != nil && w.LeasedRepoName != "" { name := w.LeasedRepoName defer func() { - if releaseErr := pool.Release(name); releaseErr != nil { + if deallocErr := w.RepoDriver.DeallocateRepo(ctx, name); deallocErr != nil { if w.Logf != nil { - w.Logf("releasing pool repo name: %v", releaseErr) + w.Logf("deallocating repo %s: %v", name, deallocErr) } if retErr == nil { - retErr = fmt.Errorf("releasing pool repo name: %w", releaseErr) + retErr = fmt.Errorf("deallocating repo %s: %w", name, deallocErr) } } }() diff --git a/pkg/behaviourtest/suite/init_test.go b/pkg/behaviourtest/suite/init_test.go index 781e5aaf48..e20e649220 100644 --- a/pkg/behaviourtest/suite/init_test.go +++ b/pkg/behaviourtest/suite/init_test.go @@ -2,8 +2,8 @@ package suite import ( "context" + "fmt" "testing" - "time" "github.com/cucumber/godog" messages "github.com/cucumber/messages/go/v21" @@ -12,11 +12,12 @@ import ( "github.com/fullsend-ai/fullsend/internal/forge" "github.com/fullsend-ai/fullsend/pkg/behaviourtest/drivers/env" + "github.com/fullsend-ai/fullsend/pkg/behaviourtest/drivers/install" "github.com/fullsend-ai/fullsend/pkg/behaviourtest/world" ) // panickingSCM is a fake scm.Driver whose CloseIssue panics. -// Used to verify that afterScenario's deferred pool.Release runs +// Used to verify that afterScenario's deferred DeallocateRepo runs // even when steps.CleanupScenario panics during issue cleanup. type panickingSCM struct{} @@ -77,6 +78,36 @@ func (p *panickingSCM) CreateForkChangeProposal(context.Context, string, string, return nil, nil } +// fakeDriver is a minimal install.Driver for unit testing the suite +// lifecycle hooks. It tracks allocations and deallocations. +type fakeDriver struct { + capacity int + allocated []string + deallocated []string + allocateErr error + deallocateRv error + nextName string +} + +func (f *fakeDriver) AllocateRepo(_ context.Context) (string, error) { + if f.allocateErr != nil { + return "", f.allocateErr + } + name := f.nextName + f.allocated = append(f.allocated, name) + return name, nil +} + +func (f *fakeDriver) DeallocateRepo(_ context.Context, name string) error { + f.deallocated = append(f.deallocated, name) + return f.deallocateRv +} + +func (f *fakeDriver) Finalize(_ context.Context) error { return nil } +func (f *fakeDriver) Capacity() int { return f.capacity } + +var _ install.Driver = (*fakeDriver)(nil) + func TestTagNames(t *testing.T) { names := tagNames([]*messages.PickleTag{{Name: "@foo"}, {Name: "@bar"}}) assert.Equal(t, []string{"@foo", "@bar"}, names) @@ -164,7 +195,7 @@ func TestBeforeScenario_ClonesAndResetsWorld(t *testing.T) { IssueNumber: 42, // scenario field — should be zeroed by reset } - ctx, err := beforeScenario(context.Background(), nil, template, nil) + ctx, err := beforeScenario(context.Background(), nil, template) require.NoError(t, err) w := world.FromContext(ctx) @@ -176,47 +207,17 @@ func TestBeforeScenario_ClonesAndResetsWorld(t *testing.T) { assert.False(t, w.ScenarioStart.IsZero(), "ScenarioStart should be set") } -func TestBeforeScenario_AcquiresPoolLease(t *testing.T) { - template := &world.World{Org: "test-org"} - pool, err := world.NewRepoPool(3) - require.NoError(t, err) - - ctx, err := beforeScenario(context.Background(), nil, template, pool) - require.NoError(t, err) - - w := world.FromContext(ctx) - require.NotNil(t, w) - assert.NotEmpty(t, w.LeasedRepoName) - assert.Contains(t, w.LeasedRepoName, "test-repo-") -} - -func TestBeforeScenario_PoolAcquireFailure(t *testing.T) { - template := &world.World{Org: "test-org"} - pool, err := world.NewRepoPool(1) - require.NoError(t, err) - - // Exhaust the pool. - _, err = pool.Acquire(context.Background()) - require.NoError(t, err) - - // Acquire with a cancelled context should fail. - cancelledCtx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond) - defer cancel() - - _, err = beforeScenario(cancelledCtx, nil, template, pool) - require.Error(t, err) - assert.Contains(t, err.Error(), "acquiring pool repo name") -} - -func TestBeforeScenario_NilPool(t *testing.T) { +func TestBeforeScenario_NoPoolAcquire(t *testing.T) { + // With the unified driver, beforeScenario no longer acquires a + // pool lease. Allocation is deferred to the step. template := &world.World{Org: "test-org"} - ctx, err := beforeScenario(context.Background(), nil, template, nil) + ctx, err := beforeScenario(context.Background(), nil, template) require.NoError(t, err) w := world.FromContext(ctx) require.NotNil(t, w) - assert.Empty(t, w.LeasedRepoName, "no pool → no leased name") + assert.Empty(t, w.LeasedRepoName, "Before hook should not allocate a repo") } func TestAfterScenario_NilWorld(t *testing.T) { @@ -225,71 +226,74 @@ func TestAfterScenario_NilWorld(t *testing.T) { origErr := godog.ErrSkip ctx := context.Background() // no World stored - _, err := afterScenario(ctx, nil, origErr) + _, err := afterScenario(ctx, origErr) assert.Equal(t, origErr, err, "original error should be preserved") } -func TestAfterScenario_ReleasesPoolLease(t *testing.T) { - pool, err := world.NewRepoPool(1) - require.NoError(t, err) - - name, err := pool.Acquire(context.Background()) - require.NoError(t, err) - - w := &world.World{LeasedRepoName: name} +func TestAfterScenario_DeallocatesRepo(t *testing.T) { + drv := &fakeDriver{capacity: 3, nextName: "test-repo-01"} + w := &world.World{ + LeasedRepoName: "test-repo-01", + RepoDriver: drv, + } ctx := world.WithWorld(context.Background(), w) - _, err = afterScenario(ctx, pool, nil) + _, err := afterScenario(ctx, nil) require.NoError(t, err) - // The released name should be available for re-acquisition. - got, err := pool.Acquire(context.Background()) - require.NoError(t, err) - assert.Equal(t, name, got) + assert.Equal(t, []string{"test-repo-01"}, drv.deallocated) } -func TestAfterScenario_DoubleReleaseSurfacesError(t *testing.T) { - pool, err := world.NewRepoPool(2) - require.NoError(t, err) - - name, err := pool.Acquire(context.Background()) - require.NoError(t, err) - - // Release the name before After — simulates a double-release. - require.NoError(t, pool.Release(name)) - - w := &world.World{LeasedRepoName: name} +func TestAfterScenario_DeallocateError_SurfacedWhenNoScenarioError(t *testing.T) { + drv := &fakeDriver{ + capacity: 3, + deallocateRv: fmt.Errorf("double-release"), + } + w := &world.World{ + LeasedRepoName: "test-repo-01", + RepoDriver: drv, + Logf: func(string, ...any) {}, + } ctx := world.WithWorld(context.Background(), w) - // After should surface the release error, not panic. - _, err = afterScenario(ctx, pool, nil) + _, err := afterScenario(ctx, nil) require.Error(t, err) - assert.Contains(t, err.Error(), "releasing pool repo name") + assert.Contains(t, err.Error(), "deallocating repo") } func TestAfterScenario_PreservesOriginalError(t *testing.T) { - pool, err := world.NewRepoPool(2) - require.NoError(t, err) - - name, err := pool.Acquire(context.Background()) - require.NoError(t, err) - - w := &world.World{LeasedRepoName: name} + drv := &fakeDriver{capacity: 3} + w := &world.World{ + LeasedRepoName: "test-repo-01", + RepoDriver: drv, + } ctx := world.WithWorld(context.Background(), w) origErr := assert.AnError - _, err = afterScenario(ctx, pool, origErr) - // Original error is preserved; release error is logged but not + _, err := afterScenario(ctx, origErr) + // Original error is preserved; dealloc error is logged but not // returned when there is already an error from the scenario. assert.Equal(t, origErr, err) } -func TestAfterScenario_ReleasesLeaseOnCleanupPanic(t *testing.T) { - pool, err := world.NewRepoPool(1) +func TestAfterScenario_NoDriverOrNoLease(t *testing.T) { + // No driver → no deallocation attempt. + w := &world.World{LeasedRepoName: "test-repo-01"} + ctx := world.WithWorld(context.Background(), w) + _, err := afterScenario(ctx, nil) require.NoError(t, err) - name, err := pool.Acquire(context.Background()) + // No leased name → no deallocation attempt. + drv := &fakeDriver{capacity: 3} + w2 := &world.World{RepoDriver: drv} + ctx2 := world.WithWorld(context.Background(), w2) + _, err = afterScenario(ctx2, nil) require.NoError(t, err) + assert.Empty(t, drv.deallocated, "no deallocation when LeasedRepoName is empty") +} + +func TestAfterScenario_DeallocatesOnCleanupPanic(t *testing.T) { + drv := &fakeDriver{capacity: 1, nextName: "test-repo-01"} // Build a World whose cleanup will panic: panickingSCM.CloseIssue // panics, and IssueNumber > 0 triggers that code path in @@ -297,19 +301,18 @@ func TestAfterScenario_ReleasesLeaseOnCleanupPanic(t *testing.T) { w := &world.World{ SCM: &panickingSCM{}, IssueNumber: 1, - LeasedRepoName: name, + LeasedRepoName: "test-repo-01", + RepoDriver: drv, } ctx := world.WithWorld(context.Background(), w) // afterScenario should panic (from CleanupScenario), but the - // deferred pool.Release must still run during stack unwinding. + // deferred DeallocateRepo must still run during stack unwinding. assert.Panics(t, func() { - afterScenario(ctx, pool, nil) //nolint:errcheck // panic prevents return + afterScenario(ctx, nil) //nolint:errcheck // panic prevents return }) - // The deferred Release ran: the leased name is back in the pool - // and can be re-acquired. - got, err := pool.Acquire(context.Background()) - require.NoError(t, err) - assert.Equal(t, name, got, "deferred Release should have returned the name to the pool") + // The deferred DeallocateRepo ran. + assert.Equal(t, []string{"test-repo-01"}, drv.deallocated, + "deferred DeallocateRepo should have run despite cleanup panic") } diff --git a/pkg/behaviourtest/world/world.go b/pkg/behaviourtest/world/world.go index 9974481f07..69735cb15e 100644 --- a/pkg/behaviourtest/world/world.go +++ b/pkg/behaviourtest/world/world.go @@ -72,10 +72,11 @@ type World struct { // for this scenario's duration. Empty when no pool is configured. LeasedRepoName string - // Ensurer lazily creates and installs repos on demand. Shared across - // scenarios (like other driver fields) and safe for concurrent use. - // Nil when lazy ensure is not configured. - Ensurer install.RepoEnsurer + // RepoDriver is the unified repo-allocation driver that owns pool + // leasing and lazy repo creation/install. Shared across scenarios + // (like other driver fields) and safe for concurrent use. Nil when + // repo allocation is not configured. + RepoDriver install.Driver // KillSwitchActivated records whether this scenario activated the // repo-level kill switch. CleanupScenario uses this to deactivate