Skip to content

[CI] CI: add extended test workflow - #4458

Merged
gyohuangxin merged 8 commits into
mainfrom
ci/extended-test-dispatch
Sep 2, 2026
Merged

gyohuangxin merged 8 commits into
mainfrom
ci/extended-test-dispatch

Conversation

@gyohuangxin

Copy link
Copy Markdown
Member

No description provided.

@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4458 --add-label <label>

@gyohuangxin
gyohuangxin marked this pull request as ready for review September 1, 2026 14:43
@gyohuangxin
gyohuangxin requested review from a team and a lite review from Copilot September 1, 2026 14:43
@github-actions github-actions Bot changed the title CI: add extended test workflow [CI] CI: add extended test workflow Sep 1, 2026
@github-actions github-actions Bot added the CI label Sep 1, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new GitHub Actions workflow to optionally trigger and gate PRs on an external “extended test” run, based on PR title or an explicit label, and then wait for a specific commit status context to report success/failure.

Changes:

  • Introduces .github/workflows/extended-test.yaml using pull_request_target with same-repo-only gating.
  • Builds and sends a repository_dispatch payload (PR metadata + source run info) to a configured target repository.
  • Polls the commit status context extended-tests/aiter until success/failure or a timeout.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +65 to +67
"source_workflow_run_id": os.environ["GITHUB_RUN_ID"],
"source_workflow_run_attempt": os.environ["GITHUB_RUN_ATTEMPT"],
"triggering_actor": os.environ["GITHUB_TRIGGERING_ACTOR"],
Comment thread .github/workflows/extended-test.yaml Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 1, 2026 15:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

.github/workflows/extended-test.yaml:7

  • To match the repository’s existing pull_request_target workflows and avoid dispatching extended tests for PRs targeting non-main branches, add a branches: [main] filter under pull_request_target (see .github/workflows/pr-title-tags.yaml and pr-welcome-comment.yaml).
on:
  pull_request_target:
    types: [opened, synchronize, reopened, ready_for_review, edited, labeled]

.github/workflows/extended-test.yaml:67

  • GITHUB_TRIGGERING_ACTOR is not guaranteed to be present in the runner environment for this event; using it with os.environ[...] can raise KeyError and fail the workflow. Prefer a safe fallback to GITHUB_ACTOR (always set) to keep dispatch payload generation reliable.
                  "triggering_actor": os.environ["GITHUB_TRIGGERING_ACTOR"],

@leo-automation leo-automation left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks good overall. I noticed two small things we may want to adjust:

  1. The workflow currently includes the architecture name in the public title-matching condition. Could we make the test label-only and remove those title checks?
  2. Since edited is included in the event types, changing the PR title or description while the label is present can restart the test. Could we remove edited, since the label and new commits already provide the triggers we need?

@zufayu
zufayu requested a review from valarLip September 2, 2026 01:22
Copilot AI review requested due to automatic review settings September 2, 2026 02:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The new workflow has branch/draft scoping and a paginated-status polling issue that can cause unintended runs or timeouts.

Review details

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

.github/workflows/extended-test.yaml:8

  • pull_request_target trigger is not scoped to the default branch. Other pull_request_target workflows in this repo restrict to branches: [main], which avoids running this secret-using workflow on PRs targeting release/feature branches unintentionally.
    .github/workflows/extended-test.yaml:31
  • This workflow can dispatch extended tests for draft PRs (e.g., on opened/synchronize) if the title matches. Most other PR-triggered CI workflows in this repo gate on pull_request.draft == false, and ready_for_review is already included in the event types.
    .github/workflows/extended-test.yaml:121
  • The commit-statuses endpoint is paginated; fetching only the first 100 statuses can miss the desired context if there are many statuses on the commit (e.g., repeated CI runs). That can cause this workflow to wait until timeout even though the status exists.
  • Files reviewed: 1/1 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@gyohuangxin
gyohuangxin merged commit ded4e3e into main Sep 2, 2026
76 checks passed
@gyohuangxin
gyohuangxin deleted the ci/extended-test-dispatch branch September 2, 2026 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants