test: update CI setup for running GPU unit tests

[jgongd]

Ticket

Description

test-unit-harness-gpu-tf, test-unit-harness-pytorch2-gpu, test-unit-harness-gpu-parallel, and test-unit-harness-gpu-deepspeed were ran on:

• determined-ai/container-runner-gpu
• determined-ai/container-runner-multi-gpu

These runners were hosted on https://console.cloud.google.com/kubernetes/clusters/details/us-central1/ci-runners-cluster. We no longer have ci-runners-cluster on GCP. Use GPU instance on CircleCI instead.

Test Plan

Checklist

• Changes have been manually QA'd
• New features have been approved by the corresponding PM
• User-facing API changes have the "User-facing API Change" label
• Release notes have been added as a separate file under docs/release-notes/
  See Release Note for details.
• Licenses have been included for new code which was copied and/or modified from any external code

[netlify]

✅ Deploy Preview for determined-ui canceled.

Name	Link
🔨 Latest commit	4bbc843
🔍 Latest deploy log	https://app.netlify.com/sites/determined-ui/deploys/673ff2a928435a000872ccca

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 54.35%. Comparing base (81b2fce) to head (4bbc843).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10230      +/-   ##
==========================================
+ Coverage   53.67%   54.35%   +0.68%     
==========================================
  Files        1255     1256       +1     
  Lines      155754   155779      +25     
  Branches     3617     3618       +1     
==========================================
+ Hits        83607    84681    +1074     
+ Misses      72014    70965    -1049     
  Partials      133      133              

Flag	Coverage Δ
backend	45.83% <ø> (+<0.01%)	⬆️
harness	71.26% <ø> (+3.89%)	⬆️
web	54.45% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 23 files with indirect coverage changes

---- 🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests
• JS Bundle Analysis - Avoid shipping oversized bundles