[yhuiYH]

Motivation

Install openmpi package onto RHEL CI docker images

Technical Details

This will allow MPI tests to be run on CI. Seems like find_package(MPI) does work on the Ubuntu and SLES CI images.

JIRA ID

Test Plan

Test Result

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[venkat1361]

Code Coverage Report

Code Coverage Report

Tests Only

Samples Only

Tests + Samples

[yhuiYH]

@venkat1361 I split this out into it's own PR so the docker image can be updated. Then I want to test against it later.

[Copilot]

Pull request overview

Adds OpenMPI development support to the RHEL CI Docker image so rocprofiler-sdk can enable/execute MPI-related tests on RHEL runners.

Changes:

• Install openmpi-devel on RHEL CI images.
• Attempt to make OpenMPI binaries available by extending PATH during the RHEL install step.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[randomizedcoder]

G'day @yhuiYH and @dayatsin-amd — thanks for adding the openmpi package to the RHEL CI images, getting MPI test coverage on CI is a great step forward.

Just had a look at the CI failure here and wanted to share what I found in case it saves you some time.

The one failing check (Core • mi325 • ubuntu-22.04) does not appear to be related to your Dockerfile changes. Here's the evidence:

1. The PR only touches RHEL, but the failure is on ubuntu-22.04

The two changes in this PR are both RHEL-specific — adding openmpi-devel to the dnf install line and adding /usr/lib64/openmpi/bin to PATH. The ubuntu-22.04 docker image uses a completely separate code path in Dockerfile.ci (the apt-get section), so these changes don't affect it at all.

2. The failing test passed twice then failed on its 3rd run — classic flaky/race behavior

Test #488 (rocprofv3-test-summary-inp-yaml.test_otf2_data) ran three times within this single CI run:

Time	Result
01:29:29	Passed (0.65 sec)
01:41:21	Passed (0.65 sec)
01:50:46	Failed (0.49 sec)

Same test, same image, same code — non-deterministic failure.

3. The error is a duplicate Location definition in the OTF2 trace

otf2.error.TraceReaderError: Semantic error in the input trace: Duplicate definition:
Location [7]: 'name': 'Thread 103685, Memory Operation at CPU Agent-0',
'type': LocationType.GPU, 'number_of_events': 2,
'group': LocationGroup [3001] 'AMD EPYC 9534 64-Core Processor'

The test fails in its pytest fixture setup because rocprofv3 generated a trace with a duplicate Location [7] definition, and the otf2 Python reader rejects it. This is a race condition in the trace output — the test itself is correct. The test has no relationship to openmpi.

4. The develop branch also had a Core • mi325 • ubuntu-22.04 failure recently

Run 24346198222 on develop (Apr 13) also failed on Core • mi325 • ubuntu-22.04 (that time it was a thread-trace-sample timeout rather than the OTF2 test, but it shows this job is prone to non-deterministic failures on develop too).

I totally sympathise with the challenge here — we've been trying to apply and test patches across multiple systems and distros ourselves, and it's genuinely tricky when unrelated integration tests are flaky on CI. A re-run of the workflow should get you a clean pass.

If it would be helpful, I'm happy to dig into the rocprofiler-sdk OTF2 writer to try and track down the source of the duplicate Location definition and put up a fix PR. Just let me know.

Hope this helps!