Repository navigation
test(rccl): suppress core dumps in expected-crash tests - #11744
Merged
dhanavikram merged 2 commits intoSep 17, 2026
Merged
Conversation
✅ All Policy Checks Passed
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
![amd-argus[bot]](https://avatars.githubusercontent.com/in/4740020?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
ARGUS
Bottom line: Looks approval ready. The only open question is whether the CI runners use a piped core_pattern, which would blunt the speed win, and that does not need another pass from me.
Comment @argus review to re-run, or @argus re-review after pushing.
Signed-off-by: Dhanavikram Sekar <dsekar@amd.com>
dhanavikram
force-pushed
the
users/dhanavikram/aicomrccl-2364-death-test-speed
branch
from
September 17, 2026 17:42
c1035d5 to
50d6b27
Compare
pvallem
approved these changes
Sep 17, 2026
Signed-off-by: Dhanavikram Sekar <dsekar@amd.com>
Contributor
Author
|
@argus re-review |
mch
approved these changes
Sep 17, 2026
mch
left a comment
mch
left a comment
Contributor
There was a problem hiding this comment.
Excellent, it's taking about 9 seconds to run all of the tests on my laptop now.
aduforest
approved these changes
Sep 17, 2026
dhanavikram
deleted the
users/dhanavikram/aicomrccl-2364-death-test-speed
branch
September 17, 2026 20:26
Contributor
TheRock Submodule Bump ActivityNewest first
|
corey-derochie-amd
pushed a commit
that referenced
this pull request
Oct 14, 2026
## Motivation The RCCL host-only init microtests contain expected-crash tests that deliberately raise SIGFPE or SIGSEGV. When core dumps are enabled, a shuffled death test can inherit a large address space from earlier tests. Writing the resulting core dump can turn a millisecond-scale assertion into a 20+ second test. ## Technical Details Disable core dumps inside the three expected-crash child processes before executing the code under test. The limit is changed only inside the GoogleTest death-test child. The parent test process retains its original core-dump policy, so unexpected suite crashes remain debuggable. ## Issue Tracking JIRA ID: AICOMRCCL-2364 ## Test Plan Built the following targets: - `rccl-UnitTestsMicroInit` - `rccl-UnitTestsMicroInit-uncached` - `rccl-UnitTestsMicroInit-faultinj` Ran the complete `InitMicrotest.*` and `InitMicrotestIsolated.*` suites through `run_host_tests.sh` with coverage and shuffling. ``` GTEST_FILTER='InitMicrotest.*:InitMicrotestIsolated.*' HOST_TEST_SHUFFLE='--gtest_shuffle --gtest_random_seed=79347' ./run_host_tests.sh run --gtest_death_test_style=threadsafe --gtest_brief=1 ``` ``` ./test/host/run_host_tests.sh all ``` ## Test Result - Normal init variant: 690 passed - Uncached init variant: 689 passed - Fault-injection init variant: 690 passed - Expected-crash subset completed in 4–8 ms per binary - Slowest death/isolated test was approximately 0.051 seconds - No core files were generated. --------- Signed-off-by: Dhanavikram Sekar <dsekar@amd.com>
corey-derochie-amd
pushed a commit
that referenced
this pull request
Oct 15, 2026
## Motivation The RCCL host-only init microtests contain expected-crash tests that deliberately raise SIGFPE or SIGSEGV. When core dumps are enabled, a shuffled death test can inherit a large address space from earlier tests. Writing the resulting core dump can turn a millisecond-scale assertion into a 20+ second test. ## Technical Details Disable core dumps inside the three expected-crash child processes before executing the code under test. The limit is changed only inside the GoogleTest death-test child. The parent test process retains its original core-dump policy, so unexpected suite crashes remain debuggable. ## Issue Tracking JIRA ID: AICOMRCCL-2364 ## Test Plan Built the following targets: - `rccl-UnitTestsMicroInit` - `rccl-UnitTestsMicroInit-uncached` - `rccl-UnitTestsMicroInit-faultinj` Ran the complete `InitMicrotest.*` and `InitMicrotestIsolated.*` suites through `run_host_tests.sh` with coverage and shuffling. ``` GTEST_FILTER='InitMicrotest.*:InitMicrotestIsolated.*' HOST_TEST_SHUFFLE='--gtest_shuffle --gtest_random_seed=79347' ./run_host_tests.sh run --gtest_death_test_style=threadsafe --gtest_brief=1 ``` ``` ./test/host/run_host_tests.sh all ``` ## Test Result - Normal init variant: 690 passed - Uncached init variant: 689 passed - Fault-injection init variant: 690 passed - Expected-crash subset completed in 4–8 ms per binary - Slowest death/isolated test was approximately 0.051 seconds - No core files were generated. --------- Signed-off-by: Dhanavikram Sekar <dsekar@amd.com>
corey-derochie-amd
pushed a commit
that referenced
this pull request
Oct 15, 2026
## Motivation The RCCL host-only init microtests contain expected-crash tests that deliberately raise SIGFPE or SIGSEGV. When core dumps are enabled, a shuffled death test can inherit a large address space from earlier tests. Writing the resulting core dump can turn a millisecond-scale assertion into a 20+ second test. ## Technical Details Disable core dumps inside the three expected-crash child processes before executing the code under test. The limit is changed only inside the GoogleTest death-test child. The parent test process retains its original core-dump policy, so unexpected suite crashes remain debuggable. ## Issue Tracking JIRA ID: AICOMRCCL-2364 ## Test Plan Built the following targets: - `rccl-UnitTestsMicroInit` - `rccl-UnitTestsMicroInit-uncached` - `rccl-UnitTestsMicroInit-faultinj` Ran the complete `InitMicrotest.*` and `InitMicrotestIsolated.*` suites through `run_host_tests.sh` with coverage and shuffling. ``` GTEST_FILTER='InitMicrotest.*:InitMicrotestIsolated.*' HOST_TEST_SHUFFLE='--gtest_shuffle --gtest_random_seed=79347' ./run_host_tests.sh run --gtest_death_test_style=threadsafe --gtest_brief=1 ``` ``` ./test/host/run_host_tests.sh all ``` ## Test Result - Normal init variant: 690 passed - Uncached init variant: 689 passed - Fault-injection init variant: 690 passed - Expected-crash subset completed in 4–8 ms per binary - Slowest death/isolated test was approximately 0.051 seconds - No core files were generated. --------- Signed-off-by: Dhanavikram Sekar <dsekar@amd.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The RCCL host-only init microtests contain expected-crash tests that deliberately raise SIGFPE or SIGSEGV.
When core dumps are enabled, a shuffled death test can inherit a large address space from earlier tests. Writing the resulting core dump can turn a millisecond-scale assertion into a 20+ second test.
Technical Details
Disable core dumps inside the three expected-crash child processes before executing the code under test.
The limit is changed only inside the GoogleTest death-test child. The parent test process retains its original core-dump policy, so unexpected suite crashes remain debuggable.
Issue Tracking
JIRA ID: AICOMRCCL-2364
Test Plan
Built the following targets:
rccl-UnitTestsMicroInitrccl-UnitTestsMicroInit-uncachedrccl-UnitTestsMicroInit-faultinjRan the complete
InitMicrotest.*andInitMicrotestIsolated.*suites throughrun_host_tests.shwith coverage and shuffling.Test Result