Skip to content

ci(rccl): schedule the coco TLS weekly job - #13373

Open
i-kosarev wants to merge 1 commit into
developfrom
users/ilkosare/rccl-tls-ci
Open

i-kosarev wants to merge 1 commit into
developfrom
users/ilkosare/rccl-tls-ci

Conversation

@i-kosarev

@i-kosarev i-kosarev commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

No scheduled coco run builds RCCL with TLS_BACKEND=OPENSSL3, so socket encryption is never tested.

Technical Details

  • .github/scripts/rccl_coco_matrix.py: schedules the weekly coco job ruby64-rccl-tls-weekly (ROCm/tRCCL#612). The job builds RCCL with -DTLS_BACKEND=OPENSSL3 and runs the socket encryption gtests and encrypted rccl-tests.

Merge after #13368, #13369 and ROCm/tRCCL#612.

Issue Tracking

JIRA ID: AICOMRCCL-2488

Test Plan

coco matrix script tests.

Test Result

Matrix script tests pass.

Submission Checklist

Copilot AI balanced review requested due to automatic review settings October 13, 2026 16:20
@github-actions github-actions Bot added the github actions Pull requests that update GitHub Actions code label Oct 13, 2026
@therock-pr-bot

Copy link
Copy Markdown

✅ All Policy Checks Passed

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The workflow omits its source-level CMake configuration from the PR path filters.

1 open finding
What changed in this PR

Adds CI coverage for RCCL’s OpenSSL 3 TLS backend and schedules weekly TLS validation.

Changes:

  • Adds a CPU-only TLS compilation workflow.
  • Registers the weekly Ruby TLS Coco job.
  • Updates weekly matrix expectations.
File Description
.github/​workflows/​rccl-tls-build.yml Adds targeted TLS compilation CI.
.github/​scripts/​rccl_coco_matrix.py Adds the weekly TLS job.
.github/​scripts/​tests/​rccl_coco_matrix_test.py Updates weekly job assertions.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/rccl-tls-build.yml Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Prerequisite merges and the external coco job configuration require human confirmation.

0 open findings

1 resolved since last review

🧠 Review effort: Balanced

@i-kosarev
i-kosarev marked this pull request as ready for review October 15, 2026 15:27
@i-kosarev
i-kosarev requested a review from cmcknigh October 29, 2026 11:56
@mch

mch commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Doesn't CoCo build RCCL? I think this could be fully in CoCo, but I'm not sure.

Adds ruby64-rccl-tls-weekly (ROCm/tRCCL#612) to the scheduled catalogue.
The job builds RCCL with -DTLS_BACKEND=OPENSSL3 and runs the socket
encryption gtests and encrypted rccl-tests.
@i-kosarev
i-kosarev force-pushed the users/ilkosare/rccl-tls-ci branch from f836625 to 97c7ac0 Compare November 6, 2026 19:07
@i-kosarev
i-kosarev requested a review from a team as a code owner November 6, 2026 19:07
@i-kosarev i-kosarev changed the title ci(rccl): build the TLS backend weekly and schedule the coco TLS job ci(rccl): schedule the coco TLS weekly job Nov 6, 2026
@i-kosarev
i-kosarev requested a review from mch November 6, 2026 19:13
@i-kosarev

Copy link
Copy Markdown
Contributor Author

Doesn't CoCo build RCCL? I think this could be fully in CoCo, but I'm not sure.

Agreed, dropped the TLS build guard. This PR now only schedules the coco TLS weekly job.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

ROCm/tRCCL#612 was inaccessible, leaving the external TLS job’s availability on the scheduled 1.4.x harness ref and required prerequisite merges unverified.

No open issues 🎉

Review effort: Balanced

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

pr-rocm-systems-13373-oci-rccl-pr-smoke

View this job on CoCo

Builds

Stack Component Version Commit Status Build time
pr-13373_rccl-perf_rocm-10.1-ga gcc-13.2 13.2.0 13.2.0 cached
pr-13373_rccl-perf_rocm-10.1-ga rocm-10.1-ga 10.1.0 10.1.0 cached
pr-13373_rccl-perf_rocm-10.1-ga openmpi-5.0.8 5.0.8 5.0.8 cached
pr-13373_rccl-perf_rocm-10.1-ga rccl-pr-13373 97c7ac0 97c7ac0 cached
pr-13373_rccl-perf_rocm-10.1-ga rccl-tests-develop-perf develop 8b73a22 cached
pr-13373_rccl-perf_rocm-10.2-latest gcc-13.2 13.2.0 13.2.0 cached
pr-13373_rccl-perf_rocm-10.2-latest rocm-10.2-latest 10.2.0rc2 10.2.0rc2 cached
pr-13373_rccl-perf_rocm-10.2-latest openmpi-5.0.8 5.0.8 5.0.8 cached
pr-13373_rccl-perf_rocm-10.2-latest rccl-pr-13373 97c7ac0 97c7ac0 cached
pr-13373_rccl-perf_rocm-10.2-latest rccl-tests-develop-perf develop 8b73a22 cached
pr-13373_rccl-smoke_rocm-10.1-ga gcc-13.2 13.2.0 13.2.0 cached
pr-13373_rccl-smoke_rocm-10.1-ga rocm-10.1-ga 10.1.0 10.1.0 cached
pr-13373_rccl-smoke_rocm-10.1-ga openmpi-5.0.8 5.0.8 5.0.8 cached
pr-13373_rccl-smoke_rocm-10.1-ga rccl-pr-13373-smoke 97c7ac0 97c7ac0 cached
pr-13373_rccl-smoke_rocm-10.1-ga rccl_gtest-pr-13373 97c7ac0 97c7ac0 cached
rocm-10.1-ga_rccl-develop-perf gcc-13.2 13.2.0 13.2.0 cached
rocm-10.1-ga_rccl-develop-perf rocm-10.1-ga 10.1.0 10.1.0 cached
rocm-10.1-ga_rccl-develop-perf openmpi-5.0.8 5.0.8 5.0.8 cached
rocm-10.1-ga_rccl-develop-perf rccl-develop-200406f9cb56 200406f 200406f cached
rocm-10.1-ga_rccl-develop-perf rccl-tests-develop-perf develop 8b73a22 cached

Failures

No failing tests in this run.

15 tests skipped.

Performance regressions

pr-13373_rccl-perf_rocm-10.1-ga vs rocm-10.1-ga_rccl-develop-perf

Readings: flat 2070.

No confirmed regressions or improvements.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

pr-rocm-systems-13373-ruby64-rccl-pr-smoke

View this job on CoCo

Builds

Stack Component Version Commit Status Build time
pr-13373_rccl-perf_rocm-10.1-ga gcc-11.5 11.5.0 11.5.0 cached
pr-13373_rccl-perf_rocm-10.1-ga rocm-10.1-ga 10.1.0 10.1.0 cached
pr-13373_rccl-perf_rocm-10.1-ga openmpi-4.0.2 4.0.2rc3 4.0.2rc3 cached
pr-13373_rccl-perf_rocm-10.1-ga rccl-pr-13373 97c7ac0 97c7ac0 cached
pr-13373_rccl-perf_rocm-10.1-ga rcclt-develop-perf develop c565458 cached
pr-13373_rccl-perf_rocm-10.2-latest gcc-11.5 11.5.0 11.5.0 cached
pr-13373_rccl-perf_rocm-10.2-latest rocm-10.2-latest 10.2.0rc2 10.2.0rc2 cached
pr-13373_rccl-perf_rocm-10.2-latest openmpi-4.0.2 4.0.2rc3 4.0.2rc3 cached
pr-13373_rccl-perf_rocm-10.2-latest rccl-pr-13373 97c7ac0 97c7ac0 cached
pr-13373_rccl-perf_rocm-10.2-latest rcclt-develop-perf develop c565458 cached
pr-13373_rccl-smoke_rocm-10.1-ga gcc-11.5 11.5.0 11.5.0 cached
pr-13373_rccl-smoke_rocm-10.1-ga rocm-10.1-ga 10.1.0 10.1.0 cached
pr-13373_rccl-smoke_rocm-10.1-ga openmpi-4.0.2 4.0.2rc3 4.0.2rc3 cached
pr-13373_rccl-smoke_rocm-10.1-ga rccl-pr-13373-smoke 97c7ac0 97c7ac0 cached
pr-13373_rccl-smoke_rocm-10.1-ga rccl_gtest-pr-13373 97c7ac0 97c7ac0 cached
rocm-10.1-ga_rccl-develop-perf gcc-11.5 11.5.0 11.5.0 cached
rocm-10.1-ga_rccl-develop-perf rocm-10.1-ga 10.1.0 10.1.0 cached
rocm-10.1-ga_rccl-develop-perf openmpi-4.0.2 4.0.2rc3 4.0.2rc3 cached
rocm-10.1-ga_rccl-develop-perf rccl-develop-200406f9cb56 200406f 200406f cached
rocm-10.1-ga_rccl-develop-perf rcclt-develop-perf develop c565458 cached

Failures

No failing tests in this run.

15 tests skipped.

Performance regressions

pr-13373_rccl-perf_rocm-10.1-ga vs rocm-10.1-ga_rccl-develop-perf

Readings: flat 2070.

Fails the run: 0 regressed point(s) past 8%, each confirmed by an adjacent size.

No confirmed regressions or improvements.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github actions Pull requests that update GitHub Actions code organization: ROCm

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants