From 3fcdc422d03687db12ec195454784f57b9712de0 Mon Sep 17 00:00:00 2001 From: fullsend-code <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Thu, 23 Jul 2026 04:51:36 +0000 Subject: [PATCH 1/2] test(#5493): fold PR-to-issue negative into PR-label positive scenario MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove the standalone "PR does not trigger issue-only harness" scenario and fold its negative assertion into the "PR label dispatches PR-only harness" scenario. The positive scenario already opens a PR, labels it, and waits for pr-ping to complete — the negative check piggy-backs on the same dispatch round-trip without a separate 90s settle wait. The merged scenario now: - Registers both pr-ping (change_proposal) and issue-only-ping (work_item) with the same label, verifying entity kind filtering - Asserts pr-ping workflow completes and dummy proves execution - Asserts pr-ping was dispatched exactly 1 time - Asserts issue-only-ping agent did not run This follows the enabled/disabled pattern already established by the "Disabled harness is not dispatched while enabled one triggers" scenario. Closes #5493 Signed-off-by: Barak Korren --- .../features/dispatch/dispatch.feature | 22 +++++++++---------- 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/e2e/behaviour/features/dispatch/dispatch.feature b/e2e/behaviour/features/dispatch/dispatch.feature index 85dfe38f2d..16e7c93aee 100644 --- a/e2e/behaviour/features/dispatch/dispatch.feature +++ b/e2e/behaviour/features/dispatch/dispatch.feature @@ -32,32 +32,28 @@ Feature: Harness CEL dispatch And the agent will succeed to Prove execution And the harness "issue-ping" was dispatched exactly 1 time - Scenario: PR does not trigger issue-only harness - Given a custom harness "issue-ping" with: + Scenario: PR label dispatches PR-only harness but not issue-only harness + Given a custom harness "pr-ping" with: """ agent: agents/triage.md role: triage - slug: fullsend-ai-issue-ping + slug: fullsend-ai-pr-ping model: opus image: ghcr.io/fullsend-ai/fullsend-sandbox:latest trigger: | - event.entity.kind == "work_item" + event.entity.kind == "change_proposal" && event.transition.kind == "label_changed" - && event.transition.label.name == "ready-for-ping" + && event.transition.label.name == "ready-for-pr-ping" """ - When a pull request is opened - Then the harness "issue-ping" agent did not run - - Scenario: PR label dispatches PR-only harness - Given a custom harness "pr-ping" with: + And a custom harness "issue-only-ping" with: """ agent: agents/triage.md role: triage - slug: fullsend-ai-pr-ping + slug: fullsend-ai-issue-only-ping model: opus image: ghcr.io/fullsend-ai/fullsend-sandbox:latest trigger: | - event.entity.kind == "change_proposal" + event.entity.kind == "work_item" && event.transition.kind == "label_changed" && event.transition.label.name == "ready-for-pr-ping" """ @@ -69,6 +65,8 @@ Feature: Harness CEL dispatch And the pull request is labeled "ready-for-pr-ping" Then the harness "pr-ping" workflow completes successfully And the agent will succeed to Prove PR execution + And the harness "pr-ping" was dispatched exactly 1 time + And the harness "issue-only-ping" agent did not run Scenario: Disabled harness is not dispatched while enabled one triggers Given a custom harness "enabled-ping" with: From 034f8468177a41a9180a2b1f18f950a67e97cb2a Mon Sep 17 00:00:00 2001 From: Barak Korren Date: Thu, 23 Jul 2026 11:22:07 +0300 Subject: [PATCH 2/2] ci(e2e): raise behaviour job timeout to 45m MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Healthy behaviour suites run ~24–29m and were flaking against the 30m job ceiling after the folded dispatch scenario already passed. Align the Makefile go test -timeout with the workflow budget. Signed-off-by: Barak Korren Co-authored-by: Cursor --- .github/workflows/e2e.yml | 2 +- Makefile | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.yml index d1d093b3b2..64518432ba 100644 --- a/.github/workflows/e2e.yml +++ b/.github/workflows/e2e.yml @@ -189,7 +189,7 @@ jobs: !cancelled() && (github.event_name != 'pull_request_target' || needs.gate.outputs.authorized == 'true') runs-on: ubuntu-24.04 - timeout-minutes: 30 + timeout-minutes: 45 permissions: contents: read id-token: write diff --git a/Makefile b/Makefile index ddc1a858ae..2d48a58ae0 100644 --- a/Makefile +++ b/Makefile @@ -171,7 +171,7 @@ e2e-test: go test -tags e2e -v -count=1 -timeout 30m ./e2e/admin/ behaviour-test: - go test -tags behaviour -v -count=1 -timeout 30m ./e2e/behaviour/ + go test -tags behaviour -v -count=1 -timeout 45m ./e2e/behaviour/ # Functional agent evals — run agents against ephemeral GitHub repos and judge results. # Required env: EVAL_ORG (GitHub org for ephemeral repos), plus GCP creds for Vertex AI.