Use native Depot registry auth for canary

[ndizazzo]

Summary

• run registry timing samples on fresh Depot runners
• use Depot's native short-lived Actions job registry credential
• verify the injected Mesh-LLM organization identity and remove OIDC/pull-token authentication
• update registry canary guidance and focused tests

Validation

• python3 -m unittest scripts.tests.test_depot_registry_canary_workflow scripts.tests.test_summarize_depot_registry_pulls
• actionlint .github/workflows/depot-registry-canary.yml
• git diff --check

Summary by CodeRabbit

• CI Improvements

  • Registry canary jobs now use pre-authenticated Depot-managed runners with short-lived job credentials.
  • Added organization identity verification before pulling images.
  • Removed manual registry login, token generation, and stored secret requirements.

• Documentation

  • Updated registry migration, metrics, and CI guidance to reflect the new authentication workflow.

• Tests

  • Updated validation to confirm native runner authentication and prevent token-based login handling.

[coderabbitai]

📝 Walkthrough

Walkthrough

The registry canary now uses pre-authenticated depot-ubuntu-24.04 runners and short-lived Depot Registry credentials. It validates DEPOT_ORG_ID before pulls. Documentation and tests now reflect the native authentication flow.

Changes

Depot Registry authentication

Layer / File(s)	Summary
Native runner authentication and organization guard .github/workflows/depot-registry-canary.yml	The workflow removes OIDC permissions, Depot CLI setup, explicit login, and DEPOT_PROJECT_ID. Pull jobs use pre-authenticated Depot runners and validate DEPOT_ORG_ID.
Authentication documentation and workflow validation ci/DEPOT_MIGRATION.md, ci/METRICS.md, ci/ci.md, scripts/tests/test_depot_registry_canary_workflow.py	Documentation and tests now describe and verify short-lived Depot job credentials, runner selection, organization validation, and the removal of token handling.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• Mesh-LLM/mesh-llm#1117: Uses and validates depot-ubuntu-24.04 runners in another workflow.
• Mesh-LLM/mesh-llm#1156: Updates the same canary workflow, documentation, and tests.
• Mesh-LLM/mesh-llm#1157: Changes the same canary back from Depot runners to GitHub-hosted runners and OIDC authentication.

Suggested reviewers: i386

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely summarizes the main change: using native Depot registry authentication for the canary.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch agent/use-native-depot-registry-auth

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

scripts/tests/test_depot_registry_canary_workflow.py (1)

19-37: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Scope the authentication assertions to jobs.pull.

The assertions search the entire workflow text. They can pass if the Depot runner or DEPOT_ORG_ID guard moves to policy while pull uses a hosted runner. The count() check has the same gap and does not prove that the guard runs before docker pull. Parse the workflow or isolate jobs.pull before checking its runner and guard. Keep token-removal assertions global.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/tests/test_depot_registry_canary_workflow.py` around lines 19 - 37,
The tests test_registry_auth_is_native_and_job_scoped and
test_canary_uses_fresh_runner_samples_and_exact_digest must isolate or parse
jobs.pull before asserting its Depot runner, DEPOT_ORG_ID guard, authentication
message, and ordering before docker pull. Keep token-removal assertions global,
while ensuring the runner count and pull-job checks cannot pass based on the
policy job.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@scripts/tests/test_depot_registry_canary_workflow.py`:
- Around line 19-37: The tests test_registry_auth_is_native_and_job_scoped and
test_canary_uses_fresh_runner_samples_and_exact_digest must isolate or parse
jobs.pull before asserting its Depot runner, DEPOT_ORG_ID guard, authentication
message, and ordering before docker pull. Keep token-removal assertions global,
while ensuring the runner count and pull-job checks cannot pass based on the
policy job.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4e7a34b0-d85c-4881-bcae-5a44603abf48

📥 Commits

Reviewing files that changed from the base of the PR and between bfd020c and e41d382.

📒 Files selected for processing (5)

• .github/workflows/depot-registry-canary.yml
• ci/DEPOT_MIGRATION.md
• ci/METRICS.md
• ci/ci.md
• scripts/tests/test_depot_registry_canary_workflow.py