Skip to content

test: verify GitHub-hosted runner fallback on a fork - #1

Closed
Project516 wants to merge 1 commit into
ci-fallback-test-basefrom
ci/fork-github-hosted-runners
Closed

test: verify GitHub-hosted runner fallback on a fork#1
Project516 wants to merge 1 commit into
ci-fallback-test-basefrom
ci/fork-github-hosted-runners

Conversation

@Project516

Copy link
Copy Markdown
Owner

Throwaway PR to prove the runner fallback actually schedules and passes on a fork. Not for merge.

Blacksmith runner labels only resolve for the pingdotgg org, so every job
in CI, the mobile fingerprint check, and the Windows test lane queues
forever on a fork. Pick the runner with case() on the repository owner so
forks get the equivalent GitHub-hosted runner from the same workflow file.
@github-actions github-actions Bot added size:S vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. labels Sep 12, 2026

@project516-review-bot project516-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR replaces hardcoded Blacksmith runner labels with case() expressions to fall back to GitHub-hosted runners on forks. Two critical bugs: (1) case() is not a valid GitHub Actions expression function — the workflow will fail to parse/evaluate on forks; the correct pattern is github.repository_owner == 'pingdotgg' && '<blacksmith>' || '<github-hosted>'. (2) macos-26 is not a valid GitHub-hosted macOS runner label (should be macos-14). Also verify ubuntu-24.04 and windows-2025 are available as GitHub-hosted labels before relying on them.


review-bot, model inclusionai/ling-3.0-flash-sante:free, verdict request_changes

Comment thread .github/workflows/ci.yml
group: ci-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

# Blacksmith runners only exist under the pingdotgg org. Every job picks its

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

case() is not a GitHub Actions expression function. Use github.repository_owner == 'pingdotgg' && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-24.04' instead, or the workflow will fail on forks.

Comment thread .github/workflows/ci.yml
# output empty, and that must run the lint rather than silently skip it.
if: ${{ !cancelled() && needs.mobile_native_changes.outputs.changed != 'false' }}
runs-on: blacksmith-6vcpu-macos-26
runs-on: ${{ case(github.repository_owner == 'pingdotgg', 'blacksmith-6vcpu-macos-26', 'macos-26') }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

case() is invalid here too — same fix as above. Also macos-26 is not a valid GitHub-hosted runner label; use macos-14 (or the current GA macOS version).

Comment thread .github/workflows/ci.yml
release_smoke:
name: Release Smoke
runs-on: blacksmith-8vcpu-ubuntu-2404
runs-on: ${{ case(github.repository_owner == 'pingdotgg', 'blacksmith-8vcpu-ubuntu-2404', 'ubuntu-24.04') }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

case() is not a valid GitHub Actions expression function — same fix as line 16.

fingerprint:
name: Native fingerprint diff
runs-on: blacksmith-8vcpu-ubuntu-2404
runs-on: ${{ case(github.repository_owner == 'pingdotgg', 'blacksmith-8vcpu-ubuntu-2404', 'ubuntu-24.04') }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

case() is not a valid GitHub Actions expression function — same fix as line 16.

test:
name: Test (${{ inputs.package || 'all non-server' }})
runs-on: blacksmith-8vcpu-windows-2025
runs-on: ${{ case(github.repository_owner == 'pingdotgg', 'blacksmith-8vcpu-windows-2025', 'windows-2025') }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

case() is not a valid GitHub Actions expression function — same fix as line 16. Also confirm windows-2025 is a generally-available GitHub-hosted label before relying on it.

@github-actions

Copy link
Copy Markdown

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 13.6 KiB 15.1 KiB
Codex Thread snapshot wire 7.0 KiB 7.3 KiB
Codex Live turn WebSocket wire 6.6 KiB 7.8 KiB
Codex Live turn WebSocket decoded 57.1 KiB 66.4 KiB
Codex Live turn messages 10 21
Claude Total thread wire 13.6 KiB 15.1 KiB
Claude Thread snapshot wire 7.1 KiB 7.3 KiB
Claude Live turn WebSocket wire 6.5 KiB 7.8 KiB
Claude Live turn WebSocket decoded 57.8 KiB 66.4 KiB
Claude Live turn messages 9 21

Baseline: unavailable · PR result: 36affdd · Source CI: failure

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 113.9 KiB
  • Claude decoded thread snapshot: 114.6 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@Project516

Copy link
Copy Markdown
Owner Author

Verification complete, results captured in pingdotgg#11438.

@Project516 Project516 closed this Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant