Skip to content

test: GPU-free regression test for env/SSH memory-limit isolation (#2125) - #272

Closed
mohitt31 wants to merge 1 commit into
Project-HAMi:mainfrom
mohitt31:test/env-isolation-2125
Closed

test: GPU-free regression test for env/SSH memory-limit isolation (#2125)#272
mohitt31 wants to merge 1 commit into
Project-HAMi:mainfrom
mohitt31:test/env-isolation-2125

Conversation

@mohitt31

@mohitt31 mohitt31 commented Aug 12, 2026

Copy link
Copy Markdown

Description

Adds a hardware-free CI characterization test for the memory-limit isolation gap in Project-HAMi/HAMi#2125.

test/test_env_isolation.c links the production shared-region implementation directly and calls no CUDA/NVML entry point, so it runs in CI without a GPU — the same GPU-free linking pattern as the existing test/test_postinit_owner_death.c (--gc-sections drops the GPU-facing paths).

It pins get_current_device_memory_limit(0) — the value oom_check() gates every cudaMalloc on — across the process-startup shapes from the issue:

Case Startup shape Enforced limit
[A] configured process, fresh region 1 GiB (enforced)
[C] env-stripped process, first writer of a fresh region 0 (leak)
[D] configured workload joining a region an env-less process seeded with 0 first 0 (leak)

Notes

  • Assertions encode current behavior, so the test is green on today's code and documents the boundary.
  • The two leak assertions are tagged [FIX FLIPS THIS]; whichever fix maintainers choose flips them from 0 to the configured limit, turning this into the issue's acceptance test.
  • Intentionally fix-agnostic — a CI test for whoever implements the fix, not a fix itself.

Repro harness (also GPU-free): https://gist.github.com/mohitt31/a6a825f429ccce3edbaafe50425fa37c

Testing

Built and run locally against the production shared-region source; all four assertions pass (env-isolation characterization tests passed, exit 0).

Summary by CodeRabbit

  • New Features

    • Added coverage for shared-memory limit behavior across isolated processes.
    • Added scenarios for configured limits, environment-stripped processes, and later processes affected by earlier configuration.
    • Added bounded execution, resource cleanup, and failure reporting for these checks.
  • Tests

    • Added a GPU-free automated test with a 30-second timeout.
    • Updated test build settings to use a focused, non-CUDA configuration.

@hami-robot

hami-robot Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mohitt31
Once this PR has been reviewed and has the lgtm label, please assign archlitchi for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hami-robot

hami-robot Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Welcome @mohitt31! It looks like this is your first PR to Project-HAMi/HAMi-core 🎉

@hami-robot hami-robot Bot added the size/L label Aug 12, 2026
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9600b83a-0ea2-4d7a-9d78-e245007b94a8

📥 Commits

Reviewing files that changed from the base of the PR and between 427a6db and ae52f8a.

📒 Files selected for processing (2)
  • test/CMakeLists.txt
  • test/test_env_isolation.c
🚧 Files skipped from review as they are similar to previous changes (2)
  • test/CMakeLists.txt
  • test/test_env_isolation.c

📝 Walkthrough

Walkthrough

This change adds a GPU-free environment-isolation regression test. It checks shared-region memory-limit behavior across configured, environment-stripped, and poisoned-region processes. The test uses a focused non-CUDA build and runs under CTest with a 30-second timeout.

Changes

Environment isolation test

Layer / File(s) Summary
Isolation characterization test
test/test_env_isolation.c
Adds forked probes, environment and cache control, shared-region result reporting, bounded child execution, cleanup, and checks for three first-writer scenarios.
Focused build and CTest wiring
test/CMakeLists.txt
Applies the focused non-CUDA build and reduced link configuration to test_env_isolation. Registers the test with a 30-second timeout.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested labels: enhancement

Suggested reviewers: chaunceyjiang, archlitchi

Poem

I hop through caches, quiet and bright,
Fork little probes to check limits right.
No GPU clouds disturb the run,
Thirty seconds, then the test is done.
Shared regions report what writers knew. 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the new GPU-free regression test and its memory-limit isolation focus.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot added the enhancement New feature or request label Aug 12, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
test/CMakeLists.txt (1)

57-59: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Raise the CTest timeout above the test's own worst-case budget.

test_env_isolation runs four probes, and each probe allows TEST_TIMEOUT_MS of 5000 ms. The worst case reaches 20 s plus process overhead, which matches or exceeds this 20 s CTest timeout. CTest then kills the test instead of printing the mismatch report. Use a value above the internal budget.

♻️ Proposed timeout adjustment
-set_tests_properties(env_isolation PROPERTIES TIMEOUT 20)
+set_tests_properties(env_isolation PROPERTIES TIMEOUT 30)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/CMakeLists.txt` around lines 57 - 59, Increase the CTest TIMEOUT for
env_isolation above the test_env_isolation internal 20-second worst-case budget,
allowing additional process overhead so the test can finish and report failures
instead of being killed.
test/test_env_isolation.c (1)

65-82: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Terminate and reap the child when the bounded wait expires.

On timeout, wait_child_bounded returns -1 and leaves the child running and unreaped. The child holds the shared region and the cache file, so a later probe in the same run can observe state from the stuck child. Send SIGKILL and reap before you return the error.

♻️ Proposed cleanup on timeout
         if (waited < 0 || now_ms() >= deadline) {
+            kill(child, SIGKILL);
+            while (waitpid(child, &status, 0) < 0 && errno == EINTR) {
+            }
             return -1;
         }

Add the required header:

 `#include` <errno.h>
+#include <signal.h>
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/test_env_isolation.c` around lines 65 - 82, Update wait_child_bounded so
timeout or waitpid failure terminates the still-running child with SIGKILL, then
reaps it with waitpid before returning -1; add the required signal header and
preserve the existing successful-child behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/CMakeLists.txt`:
- Around line 57-59: Increase the CTest TIMEOUT for env_isolation above the
test_env_isolation internal 20-second worst-case budget, allowing additional
process overhead so the test can finish and report failures instead of being
killed.

In `@test/test_env_isolation.c`:
- Around line 65-82: Update wait_child_bounded so timeout or waitpid failure
terminates the still-running child with SIGKILL, then reaps it with waitpid
before returning -1; add the required signal header and preserve the existing
successful-child behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2779adb7-06d4-457b-ae38-f7fccfde1cd2

📥 Commits

Reviewing files that changed from the base of the PR and between 5496322 and 427a6db.

📒 Files selected for processing (2)
  • test/CMakeLists.txt
  • test/test_env_isolation.c

… (#2125)

Adds test/test_env_isolation.c, a hardware-free characterization test for the
memory-limit shared-region source of truth described in Project-HAMi/HAMi#2125.
It links the production shared-region implementation directly and calls no
CUDA/NVML entry point, mirroring test_postinit_owner_death.c, so it runs in CI
without a GPU.

The test pins get_current_device_memory_limit(0) -- the value oom_check() gates
every cudaMalloc on -- across the process-startup shapes from the issue:
  [A] configured process, fresh region        -> enforced
  [C] env-stripped process, first writer       -> unlimited (leak)
  [D] configured workload joining a region an  -> unlimited (leak)
      env-less process seeded with 0 first

Assertions encode CURRENT behavior, so the test is green on today's code and
documents the boundary. The two leak assertions are tagged [FIX FLIPS THIS];
whichever fix the maintainers choose flips them from 0 to the configured limit,
turning this into the issue's acceptance test. This is intentionally a
fix-agnostic CI test, not a fix.

Signed-off-by: mohitt31 <mohitprajapati3112@gmail.com>
@mohitt31
mohitt31 force-pushed the test/env-isolation-2125 branch from 427a6db to ae52f8a Compare August 12, 2026 06:29
@mohitt31

Copy link
Copy Markdown
Author

Thanks for the review. Addressed both nitpicks in the latest push:

  • test/test_env_isolation.c: wait_child_bounded now SIGKILLs and reaps the child on timeout before returning, so a stuck probe can't leak shared-region/cache state into a later probe (added <signal.h>).
  • test/CMakeLists.txt: raised the env_isolation CTest timeout from 20s to 30s, above the test's own 4×5s worst-case budget, so a slow run reports the mismatch instead of being killed.

Rebuilt and re-ran locally — all four assertions still pass.

@mesutoezdil

Copy link
Copy Markdown
Contributor

This is being closed because it does not comply with the contribution guidelines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants