Skip to content

AIRUNTIME-2896 - Fix streams perf tests - #13682

Merged
TomSang merged 1 commit into
developfrom
users/taosang/AIRUNTIME-2896-Fix-streams-perf-tests-1
Oct 21, 2026
Merged

TomSang merged 1 commit into
developfrom
users/taosang/AIRUNTIME-2896-Fix-streams-perf-tests-1

Conversation

@TomSang

@TomSang TomSang commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Motivation

Fix the following tests' hang in some devices,

StreamScenarioPerformance.exe Performance_KernelLaunchLatency_IncreasingNumberOfStreams StreamScenarioPerformance.exe Performance_hipPerfMultiStreamKernelLaunch

Technical Details

When memory is not enough, some test cases will be skipped.

Issue Tracking

AIRUNTIME-2896

Test Plan

All tests should pass

Test Result

Pass

Submission Checklist

@TomSang
TomSang requested a review from a team as a code owner October 17, 2026 01:28
Copilot AI balanced review requested due to automatic review settings October 17, 2026 01:28
@therock-pr-bot

Copy link
Copy Markdown

❌ PR Check — Action Required

Check Status Details
📝 PR Description ❌ Fail Error: PR description must reference a JIRA ID, ISSUE ID, or a GitHub closing keyword.
Expected: include a JIRA ID / ISSUE ID line (separator : or -, or omitted; value may be a JIRA key, a number with/without #, or a link), OR a closing keyword + issue reference. Accepted examples:
• JIRA ID : TESTAUTO-6039
• JIRA ID - #330
• JIRA ID #330
• JIRA ID (on separate line)
ROCM-25757
• ISSUE ID : TESTUTO-3334
• ISSUE ID #3334
• ISSUE ID - TESTAUTO-3433
• ISSUE ID (on separate line)
AIRUNTIME-2352
• ISSUE ID : https://github.com/<org_name>/<repo_name>/issues/1234
• Closes #10
• Fixes octo-org/octo-repo#100
• Resolves: #123
• #123
• https://github.com/<org_name>/<repo_name>/issues/123
Current: no valid JIRA/ISSUE/closing-keyword reference found
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ⚠️ Warning Error: Source/code files changed without an accompanying unit test.
Expected: add at least one test file named like test_<name>.py / test_<name>.cpp (or <name>_test.*).
Current: code file(s) changed: projects/hip-tests/catch/performance/scenarios/stream/hipPerfIncreasingNumberOfStreams.cc, projects/hip-tests/catch/performance/scenarios/stream/hipPerfMultiStreamKernelLaunch.cc; no test file found
🔎 pre-commit ⏳ Pending ⏳ Still running…
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —

⚠️ 1 policy check(s) failed. Please address the issues above before this PR can be Reviewed.

🚫 Please fix the failed policies

  • ❌ PR Description

The Not ready to Review label was added to this PR. Once all policies pass, the label is removed automatically.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

  • ❌ PR Description

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Calibration can exhaust memory before budgeting applies, and checked event cleanup can prevent the intended skip.

2 open issues
What changed in this PR

This PR aims to prevent resource-exhaustion hangs in HIP multi-stream performance tests by skipping unsupported stream counts.

Changes:

  • Adds shared stream-cost calibration, memory budgeting, and exhaustion reporting.
  • Integrates skip handling into both performance scenarios.
  • Removes the 5,000-stream case.
File Description
projects/​hip-tests/​catch/​performance/​scenarios/​stream/​stream_create_helper.hh Adds memory-aware stream creation.
projects/​hip-tests/​catch/​performance/​scenarios/​stream/​hipPerfMultiStreamKernelLaunch.cc Uses the helper to skip unsupported experiments.
projects/​hip-tests/​catch/​performance/​scenarios/​stream/​hipPerfIncreasingNumberOfStreams.cc Adds skip handling and removes the largest stream count.

Review effort: Balanced

@TomSang
TomSang force-pushed the users/taosang/AIRUNTIME-2896-Fix-streams-perf-tests-1 branch from 5995cdc to aa2f218 Compare October 18, 2026 02:22
@TomSang
TomSang force-pushed the users/taosang/AIRUNTIME-2896-Fix-streams-perf-tests-1 branch from aa2f218 to fbbaccf Compare October 19, 2026 01:22

@mangupta mangupta left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI-assisted pre-review (for @mangupta)

@TomSang
TomSang force-pushed the users/taosang/AIRUNTIME-2896-Fix-streams-perf-tests-1 branch 3 times, most recently from dd57f92 to 44e0f5b Compare October 19, 2026 21:54
@mangupta

Copy link
Copy Markdown
Contributor

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (44e0f5bdea67) — no new findings at or above the posting threshold.

Fix the following tests' hang in some devices,

StreamScenarioPerformance.exe Performance_KernelLaunchLatency_IncreasingNumberOfStreams
StreamScenarioPerformance.exe Performance_hipPerfMultiStreamKernelLaunch
@TomSang
TomSang force-pushed the users/taosang/AIRUNTIME-2896-Fix-streams-perf-tests-1 branch from 44e0f5b to 6dcc339 Compare October 20, 2026 02:30
@mangupta

Copy link
Copy Markdown
Contributor

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (6dcc339bf52a) — no new findings at or above the posting threshold.

@TomSang
TomSang merged commit e57ed1a into develop Oct 21, 2026
401 of 413 checks passed
@TomSang
TomSang deleted the users/taosang/AIRUNTIME-2896-Fix-streams-perf-tests-1 branch October 21, 2026 15:45
@systems-assistant

Copy link
Copy Markdown
Contributor

TheRock Submodule Bump Activity

Newest first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants