Skip to content

QVAC-16473 infra: add suite filtering and PR-triggered e2e test workflows for SDK#1653

Merged
lauripiisang merged 9 commits into
mainfrom
test/sdk-run-smoke-suite
Apr 22, 2026
Merged

QVAC-16473 infra: add suite filtering and PR-triggered e2e test workflows for SDK#1653
lauripiisang merged 9 commits into
mainfrom
test/sdk-run-smoke-suite

Conversation

@lauripiisang

@lauripiisang lauripiisang commented Apr 17, 2026

Copy link
Copy Markdown
Contributor

🎯 What problem does this PR solve?

  • SDK e2e tests (test-sdk.yml) could only be triggered manually with no suite filtering, no automated PR triggers, and no label-based control
  • No way to run just the smoke suite (~84 tests, ~88s desktop) vs full suite (~388 tests) from the workflow dispatch UI
  • Release branch PRs with SDK changes had no automatic e2e validation
  • Android Device Farm test runs did not upload device logs as artifacts (iOS did)

📝 How does it solve it?

Suite filtering (test-sdk.yml + reusable workflows)

  • Adds suite choice input to manual dispatch: full (default, no --suite flag), smoke (--suite smoke), custom (uses companion suite-custom string input)
  • Adds exclude-suite string input for excluding tagged suites (e.g. slow,flaky)
  • A resolve job maps the choice+custom pair to a plain string, then passes it through to all three reusable workflows (test-desktop-sdk.yml, test-android-sdk.yml, test-ios-sdk.yml)
  • Reusable workflows append --suite / --exclude-suite flags to the qvac-test run:producer command

workflow_call support (test-sdk.yml)

  • Adds workflow_call trigger with matching inputs (string types, same defaults) so other workflows can call it programmatically
  • The resolve job handles both workflow_dispatch (choice → string) and workflow_call (string passthrough)

PR trigger workflow (on-pr-test-sdk.yml)

  • Label-based: applying test-e2e-smoke runs smoke suite on all platforms; test-e2e-full runs full suite
  • Release branch auto-trigger: PRs targeting release-* with packages/sdk/ changes automatically run full suite
  • Authorization: uses the shared authorize-pr composite action with safe-to-test label gate for external fork PRs
  • Success label: applies e2e-tested label to the PR when all tests pass
  • Uses pull_request_target for secrets access (Device Farm, MQTT)
  • Path filter packages/sdk/** gates opened/synchronize events; labeled events bypass path filter per GitHub behavior

Android Device Farm log uploads

  • Added Device Farm artifact download and upload steps to test-android-sdk.yml cleanup job, mirroring the existing iOS implementation
  • Waits for runs to complete, downloads Customer artifacts, uploads as device-farm-logs-android-* with 30-day retention
  • Bumped cleanup job timeout from 5 to 10 minutes to accommodate log downloads

Other changes

  • Default targets changed from desktop to all
  • Updated desktop-platforms default to use mac-mini-m4-gpu (aligned with upstream rename)

🧪 How was it tested?

  • test-sdk.yml manual dispatch can be tested on this branch with all suite/exclude-suite/filter combinations
  • Limitation: on-pr-test-sdk.yml uses pull_request_target, which loads the workflow from the base branch — label triggers and release-branch auto-triggers can only be tested after merge to main. Post-merge verification:
    1. Open a test PR, apply test-e2e-smoke label, verify smoke suite runs on all platforms
    2. Apply test-e2e-full label, verify full suite runs
    3. Verify e2e-tested label is applied on success
    4. Open a PR targeting a release-* branch with SDK changes, verify auto-trigger
  • Android Device Farm log upload can be verified on the next Android test run by checking for device-farm-logs-android-* artifact

- Add workflow_call trigger + suite/exclude-suite inputs to test-sdk.yml
- Thread suite/exclude-suite through desktop, android, and iOS reusable workflows
- Create on-pr-test-sdk.yml for label-based and release-branch triggers
- Add suite choice dropdown (full/smoke/custom) for manual dispatch
@lauripiisang lauripiisang requested review from a team as code owners April 17, 2026 14:06
Comment thread .github/workflows/on-pr-test-sdk.yml Fixed
@github-actions

github-actions Bot commented Apr 17, 2026

Copy link
Copy Markdown
Contributor

Tier-based Approval Status

**PR Tier:** TIER1

**Current Status:** ✅ APPROVED

**Requirements:**
- 1 Team Member approval ✅ (3/1)
- 1 Team Lead OR Management approval ✅ (1/1)



---
*This comment is automatically updated when reviews change.*

Remove git fetch/diff of PR head refs that triggered "checkout of
untrusted code in trusted context" alert. Use sparse checkout (only
authorize-pr action) and rely on the trigger-level paths filter for
SDK change detection on release branch PRs.
… capture

Replace inline logcat consumption with background capture to a persistent
log file, matching the iOS pymobiledevice3 pattern. Adds full unfiltered
logcat dump and React Native log extraction in post_test.
The STOPPING state can take 10+ minutes on Device Farm. Previous 5-min
wait (30x10s) was insufficient, resulting in empty artifact downloads.
Now waits up to 30 min (120x15s) and merges stop+wait+download into a
single step. Applied to both iOS and Android cleanup jobs.
@lauripiisang

Copy link
Copy Markdown
Contributor Author

/review

@lauripiisang

Copy link
Copy Markdown
Contributor Author

/review

@lauripiisang lauripiisang merged commit a1b74ad into main Apr 22, 2026
15 of 16 checks passed
@lauripiisang lauripiisang deleted the test/sdk-run-smoke-suite branch April 22, 2026 10:15
Proletter pushed a commit that referenced this pull request May 24, 2026
…lows for SDK (#1653)

* infra: add suite filtering and PR-triggered e2e test workflows for SDK

- Add workflow_call trigger + suite/exclude-suite inputs to test-sdk.yml
- Thread suite/exclude-suite through desktop, android, and iOS reusable workflows
- Create on-pr-test-sdk.yml for label-based and release-branch triggers
- Add suite choice dropdown (full/smoke/custom) for manual dispatch

* infra: add Device Farm artifact download and upload to Android SDK test workflow

* infra: fix CodeQL alert in on-pr-test-sdk.yml

Remove git fetch/diff of PR head refs that triggered "checkout of
untrusted code in trusted context" alert. Use sparse checkout (only
authorize-pr action) and rely on the trigger-level paths filter for
SDK change detection on release branch PRs.

* infra: improve Android Device Farm logging with continuous background capture

Replace inline logcat consumption with background capture to a persistent
log file, matching the iOS pymobiledevice3 pattern. Adds full unfiltered
logcat dump and React Native log extraction in post_test.

* infra: increase Device Farm cleanup wait to 30 min for artifact download

The STOPPING state can take 10+ minutes on Device Farm. Previous 5-min
wait (30x10s) was insufficient, resulting in empty artifact downloads.
Now waits up to 30 min (120x15s) and merges stop+wait+download into a
single step. Applied to both iOS and Android cleanup jobs.

* fix: fix security issue caused by allowing out-of-date labeled prs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants