Skip to content

ci: skip secret-dependent jobs on fork PRs - #765

Closed
slin1237 wants to merge 2 commits into
mainfrom
slin/ci-secrets
Closed

slin1237 wants to merge 2 commits into
mainfrom
slin/ci-secrets

Conversation

@slin1237

@slin1237 slin1237 commented Mar 15, 2026 •

Copy link
Copy Markdown
Member

Summary

Fork PRs can now run the full integration test suite (including secret-dependent jobs) after a maintainer adds the safe-to-test label.

What changed

1. pr-test-rust.yml — skip secret jobs on fork PRs

Added fork detection guard to 8 secret-dependent jobs so they show as "skipped" (not "failed") on fork PRs:

  • benchmarks, e2e-1gpu-chat/embeddings/gateway, e2e-2gpu-responses/pd, e2e-vendor, go-bindings-e2e

Non-secret jobs (lint, build, unit tests) always run for everyone.

2. pr-test-fork-integration.yml — NEW label-gated workflow

Runs secret-dependent jobs for fork PRs via pull_request_target:

  • Triggered when maintainer adds safe-to-test label
  • Checks out the PR's HEAD SHA (not the base branch)
  • Has access to repo secrets (API keys, HF_TOKEN)
  • Guard job ensures it only runs for fork PRs with the correct label

3. e2e-gpu-job.yml — added checkout_ref input

The reusable workflow now accepts an optional checkout_ref input so callers can specify which git ref to checkout (needed for pull_request_target which defaults to the base branch).

Security model

  1. Contributor opens fork PR
  2. pr-test-rust.yml runs non-secret CI automatically (lint, build, unit tests)
  3. Maintainer reviews the fork's code
  4. Maintainer adds safe-to-test label
  5. pr-test-fork-integration.yml triggers, runs integration tests with secrets
  6. If contributor pushes new commits, maintainer must remove and re-add the label after reviewing new code (workflow only triggers on labeled, not synchronize)

Why

GitHub never passes secrets to pull_request events from forks. The only way to get secrets for fork PRs is pull_request_target, which runs in the base repo's context. A separate workflow with a label gate is the safest approach — it isolates the pull_request_target trigger and requires explicit maintainer approval.

Test plan

  • Fork guard conditions verified in pr-test-rust.yml
  • Guard job in fork workflow checks both label name and fork status
  • All checkout steps in fork workflow use PR_HEAD_SHA
  • Create safe-to-test label in the repo
  • Verify on a fork PR: non-secret jobs run, secret jobs skipped, label triggers integration workflow

Summary by CodeRabbit

  • Chores
    • Refined CI to run tests only when pull request source meets repository gating, reducing unnecessary jobs for forked PRs.
  • New Features
    • Added a fork PR integration pipeline with a safe-to-test label gate and broad end-to-end, benchmark, and artifact workflows across GPU/CPU matrices.
    • Introduced an option to checkout a specific git ref for targeted test runs.

Fork PRs don't have access to repo secrets (API keys, HF_TOKEN),
causing e2e tests, vendor tests, benchmarks, and go-bindings-e2e
to fail with missing credentials after maintainer approval.

Add fork detection guard to all secret-dependent jobs:
  github.event.pull_request.head.repo.full_name == github.repository

This skips these jobs on fork PRs while still running them on:
- push to main
- internal PRs (same repo)

Affected jobs: benchmarks, e2e-1gpu-chat, e2e-1gpu-embeddings,
e2e-1gpu-gateway, e2e-2gpu-responses, e2e-2gpu-pd, e2e-vendor
(anthropic-messages, openai-responses, openai-realtime,
xai-responses), go-bindings-e2e.

Cascading jobs (e2e-2gpu-chat, e2e-4gpu-chat) auto-skip when
their dependencies skip. The finish job already handles skipped
results correctly (checks for "failure", not "skipped").

Fork PRs will still run: pre-commit, python-lint, unit-tests,
build-wheel, python-unit-tests, go-unit-tests, grpc-proto-build-check.

Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Note

Gemini is unable to generate a summary for this pull request due to the file types involved not being currently supported.

@coderabbitai

coderabbitai Bot commented Mar 15, 2026 •

Copy link
Copy Markdown

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: fd28f64a-bc35-42ea-9366-89acb5293a07

📥 Commits

Reviewing files that changed from the base of the PR and between 2e1ec27 and 2faadc5.

📒 Files selected for processing (2)
  • .github/workflows/e2e-gpu-job.yml
  • .github/workflows/pr-test-fork-integration.yml

📝 Walkthrough

Walkthrough

Adds CI workflow gating and a new fork-safe integration workflow: tightens pull_request job conditionals to require the PR head repo match the workflow repo; adds a workflow_call input to e2e GPU jobs to allow checking out a specific ref; introduces a fork-focused pull_request_target workflow that runs gated builds, artifact production, and multi-engine E2E/benchmark jobs.

Changes

Cohort / File(s) Summary
PR gating updates
​.github/workflows/pr-test-rust.yml
Added head.repo.full_name == repository checks to multiple pull_request job conditionals to prevent jobs from running for forks.
E2E GPU job inputs
​.github/workflows/e2e-gpu-job.yml
Added optional checkout_ref workflow_call input and updated Checkout step to use it when provided (falls back to github.sha).
Fork integration workflow (new, large)
​.github/workflows/pr-test-fork-integration.yml
Introduces a pull_request_target workflow gated by a safe-to-test label for fork PRs; builds wheel/FFI/wasm, uploads artifacts, and runs matrixed E2E jobs across GPUs, engines, and vendors with controlled secret exposure.

Sequence Diagram(s)

sequenceDiagram
    participant Contributor as Contributor (fork PR)
    participant GitHub as GitHub Actions
    participant BaseRepo as Base repo (pull_request_target)
    participant Runner as Runner (build/test)
    participant Storage as Artifact Storage

    Note over Contributor,GitHub: Fork PR labeled "safe-to-test"
    Contributor->>GitHub: Open PR (fork)
    GitHub->>BaseRepo: trigger pull_request_target workflow (reads label)
    BaseRepo->>Runner: Checkout PR head SHA via base repo context
    Runner->>Runner: Build wheel / Go FFI / WASM / client types
    Runner->>Storage: Upload artifacts
    GitHub->>Runner: Start matrixed E2E jobs (download artifacts)
    Runner->>Storage: Download artifacts
    Runner->>Runner: Run E2E tests (GPU/CPU matrices)
    Runner->>Storage: Upload test results / sccache stats
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested labels

tests

Suggested reviewers

  • CatherineSue
  • key4ng
  • XinyueZhang369

Poem

🐰 I hopped from fork to repo bright,

"Label me safe," I begged one night.
Builders hummed and wheels did bake,
Artifacts hopped back in my wake,
Tests danced, and CI cheered—what a sight! 🥕

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'ci: skip secret-dependent jobs on fork PRs' clearly and concisely describes the main change: adding fork detection to skip CI jobs that require secrets on fork pull requests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch slin/ci-secrets
📝 Coding Plan
  • Generate coding plan for human review comments

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added the ci CI/CD configuration changes label Mar 15, 2026
Create pr-test-fork-integration.yml — a separate workflow that runs
secret-dependent jobs for fork PRs via `pull_request_target`.

Flow:
1. Fork PR opened → pr-test-rust.yml runs non-secret jobs (lint,
   build, unit tests). Secret-dependent jobs are skipped.
2. Maintainer reviews the fork's code
3. Maintainer adds `safe-to-test` label
4. This workflow triggers via pull_request_target (labeled event),
   checks out the PR's HEAD SHA, runs all integration tests with
   access to repo secrets

Security:
- Only triggers on `labeled` event (not synchronize) — if the
  contributor pushes new commits, maintainer must remove and re-add
  the label after reviewing the new code
- Guard job checks both the label name AND that it's a fork PR
  (internal PRs are handled by pr-test-rust.yml)
- All checkout steps use the PR's HEAD SHA, not a branch ref

Also adds `checkout_ref` input to e2e-gpu-job.yml reusable workflow
so callers can specify which ref to checkout (needed for
pull_request_target which defaults to the base branch).

Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
@slin1237 slin1237 closed this Mar 15, 2026
@slin1237
slin1237 deleted the slin/ci-secrets branch March 15, 2026 02:50

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2faadc5c1d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

on:
pull_request_target:
branches: [main]
types: [labeled]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Trigger fork integration tests on new commits

pull_request_target is limited to types: [labeled], so after safe-to-test is added the secret-dependent workflow runs only once for that label event. If the fork author pushes another commit afterward, pr-test-rust.yml now skips the secret jobs on forks and this workflow does not retrigger, leaving the latest PR head SHA without integration coverage. Add synchronize (and gate on the label being present) so approved fork PRs are revalidated on subsequent pushes.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci CI/CD configuration changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant