[Mihaiii]

The Delegate agent moved from the coder_eval_uipath plugin (delegate-sdk) into coder-eval as the built-in delegate agent, which drives the @uipath/delegate-stdio host. Port this workflow to it, as coder_eval_uipath's daily.sh and daily-windows.ps1 do:

• Rename the agent input option delegate-sdk to delegate.
• Do not install the coder_eval_uipath plugin into coder-eval's environment. Linux still checks the repo out, for the docker overlay and the evalboard uploader. Windows no longer checks it out.
• Use the agent's env names: DELEGATE_ENV, DELEGATE_AUTH_TOKEN, DELEGATE_TENANT_ID, DELEGATE_ORG_ID, DELEGATE_ORG_SLUG and DELEGATE_TENANT_SLUG. Drop DELEGATE_STALL_TIMEOUT_S, which only the studio-web agent reads.
• Install @uipath/delegate-stdio from the preview dist-tag, which the nightly tracks, not alpha.

Needs a coder-eval release that contains the built-in delegate-stdio agent (tests/.coder-eval-version), and coder_eval_uipath main with the overlay that no longer installs the plugin for delegate.

Before merging this one ensure that:

1. A new version of coder_eval (which has the migration code) is relased and referenced
2. The coder_eval_uipath change is merged.

[github-actions]

Claude finished @Mihaiii's task in 2m 20s —— View job

---

PR Review: ci(run-coder-eval)!: run coder-eval's built-in delegate agent

• Read review criteria and project conventions
• Read full diff
• Read changed files in full context
• Check for secrets/tokens/hardcoded paths
• Post review findings

---

Summary

Migrates the run-coder-eval.yml workflow from the delegate-sdk agent (registered by the coder_eval_uipath plugin) to coder-eval's built-in delegate agent. The plugin is no longer installed into coder-eval's environment; the coder_eval_uipath checkout remains for the Docker overlay (Linux) and the evalboard uploader only. Env vars are renamed to the agent's canonical names, DELEGATE_STALL_TIMEOUT_S is dropped, and @uipath/delegate-stdio now installs from the preview dist-tag instead of alpha.

Change-by-Change Review

1. .github/scripts/refresh-auth.sh

Severity: OK

Single comment update: delegate-sdk → delegate in the comment at line 28. No logic changes. Correct.

2. .github/workflows/run-coder-eval.yml — input renames & comment updates

Severity: OK

All workflow input descriptions, comments, and the agent choice list are consistently updated: delegate-sdk → delegate throughout. The coder_eval_uipath_ref input description now accurately reflects its narrowed role (overlay + uploader, not plugin). The removed comments about the plugin's coder_eval.plugins entry point, monkeypatching, and --with resolution are appropriate — that code path is gone.

3. .github/workflows/run-coder-eval.yml — Linux job: checkout gate & plugin removal

Severity: OK

• Checkout condition updated: inputs.agent == 'delegate' (line 247). Correct.
• The checkout comment no longer describes plugin installation — just the overlay and uploader. Accurate.
• The extra_packages output variable and the composite action's extra-packages input are both removed. Consistent: no plugin, no extra package to inject.
• The pre-flight assertion for the checkout moved from the prep step (where it gated plugin availability) into the overlay step (line 349–352, where it gates the Docker build context). Correct relocation — the checkout is now mandatory only for the overlay build, not for the agent registration.

4. .github/workflows/run-coder-eval.yml — Linux job: env var renames

Severity: OK

Auth extraction loop updated consistently in both Linux (line 522–526) and Windows (line 1215–1219):

• AUTH_TOKEN → DELEGATE_AUTH_TOKEN
• TENANT_ID → DELEGATE_TENANT_ID
• ORG_ID → DELEGATE_ORG_ID
• ORG_LOGICAL_NAME → DELEGATE_ORG_SLUG
• TENANT_NAME → DELEGATE_TENANT_SLUG

The mask condition correctly gates on DELEGATE_AUTH_TOKEN (line 535). The DELEGATE_SDK_ENV → DELEGATE_ENV rename is consistent (lines 511, 1206). DELEGATE_STALL_TIMEOUT_S is dropped from both the env lines and the docker passthrough list. The docker env passthrough list (line 566–569) uses the new names and omits the dropped var. All correct.

5. .github/workflows/run-coder-eval.yml — Linux job: overlay step

Severity: OK

• Condition updated to inputs.agent == 'delegate' (line 344).
• Dist-tag changed: alpha → preview (line 355). Matches the PR description and nightly tracking.
• The checkout assertion is now an explicit if [ ! -f ... ] guard with a clear error message (lines 349–352), moved here from the old prep-step plugin assertion. Good — fails fast with a readable message before the docker build.

6. .github/workflows/run-coder-eval.yml — Windows job: checkout & install changes

Severity: OK

• The entire coder_eval_uipath checkout step is removed from the Windows job (was lines 898–906 on main). Correct: Windows doesn't need the overlay or evalboard uploader, and the plugin is no longer installed.
• The delegate-sdk case in the Install coder-eval step (old line 396) is removed. delegate now falls through to the * case (coder-eval[litellm]==$CE_VERSION), which is correct — no plugin to co-install.
• Auth file renamed: .delegate-sdk-auth → .delegate-auth (lines 1066, 1163, 1334). Consistent everywhere.
• @uipath/delegate-stdio install uses preview dist-tag (line 1108). Matches Linux.
• Stop shared delegate interop condition updated (line 1306). Correct.

7. .github/workflows/run-coder-eval.yml — Windows Install @uipath/delegate-stdio comment

Severity: Low

Line 1085: comment says "verify it and retry explicitly at the version @uipath/delegate-sdk pins." This refers to the npm package @uipath/delegate-sdk (line 1111 reads its optionalDependencies to get the runtime version), which is distinct from the coder-eval agent type that was renamed. The npm package name hasn't changed, so this is technically correct. However, it could momentarily confuse a reader who just saw every other delegate-sdk reference renamed. A minor clarification (e.g., "the version the @uipath/delegate-sdk npm package pins") would help, but not blocking.

What's Missing

• Pre-merge dependency verification: The PR description notes two prerequisites: (1) a coder-eval release containing the built-in delegate agent, referenced by tests/.coder-eval-version, and (2) the corresponding coder_eval_uipath change merged. These cannot be verified from the diff alone — the person merging should confirm both are in place, or the delegate overlay build / --type delegate will fail at runtime.
• Nothing else identified.

Area Ratings

Area	Status	Notes
Frontmatter	N/A	No skill files changed
E2E Tests	N/A	CI workflow only
Skill Body	N/A	No skill files changed
References & Assets	N/A	No skill files changed
Repo Hygiene	OK	No secrets, tokens, or personal paths. Changes scoped to CI.

Issues for Manual Review

1. Pre-merge gates: Confirm the coder-eval release with the built-in delegate agent is published and tests/.coder-eval-version pins that version. Confirm the coder_eval_uipath overlay change (no plugin install) is merged to main before this lands.
2. preview dist-tag availability: Verify that @uipath/delegate-stdio@preview resolves on the GitHub Packages npm registry. The PR switches from alpha — if preview isn't published yet, the npm install will fail.
3. Saved dispatch configurations: This is a breaking change (the ! in the title). Anyone with bookmarked or scripted dispatches using agent: delegate-sdk will need to update to agent: delegate.

Conclusion

Clean, well-executed migration. The rename is thorough and consistent across both jobs (Linux/Windows), all env var mappings, the docker passthrough list, and the auth file paths. The plugin removal correctly narrows the coder_eval_uipath checkout to its remaining roles (overlay + uploader). No code issues found. The one low-severity finding (npm package name in a comment) is cosmetic. Approve once the two pre-merge dependencies in the PR description are confirmed satisfied.

---