Skip to content

fix: prevent pkill etcd from killing other containers' processes - #1227

Merged
dpressle merged 1 commit into
ai-dynamo:mainfrom
dpressle:etcd_kill_fixup
Jan 19, 2026
Merged

dpressle merged 1 commit into
ai-dynamo:mainfrom
dpressle:etcd_kill_fixup

Conversation

@dpressle

@dpressle dpressle commented Jan 18, 2026 •

Copy link
Copy Markdown
Contributor

What?

Ensures each container only terminates its own etcd instance, allowing parallel test execution without interference.

Why?

When running multiple test containers in parallel (e.g., SLURM enroot), pkill etcd would kill etcd processes across all containers, causing the second container to fail with exit code 1.

How?

Replace pkill etcd with targeted process termination using captured PID.

Changes:

  • Capture etcd PID immediately after backgrounding: ETCD_PID=$!
  • Replace pkill etcd with kill -9 $ETCD_PID 2>/dev/null || true
  • Use SIGKILL (-9) for immediate, reliable termination in test environment
  • Add error suppression to prevent script failure if process already exited

Replace `pkill etcd` with targeted process termination using captured PID.
When running multiple test containers in parallel (e.g., SLURM enroot),
`pkill etcd` would kill etcd processes across all containers, causing the
second container to fail with exit code 1.

Changes:
- Capture etcd PID immediately after backgrounding: ETCD_PID=$!
- Replace `pkill etcd` with `kill -9 $ETCD_PID 2>/dev/null || true`
- Use SIGKILL (-9) for immediate, reliable termination in test environment
- Add error suppression to prevent script failure if process already exited

This ensures each container only terminates its own etcd instance, allowing
parallel test execution without interference.

Signed-off-by: Daniel Pressler <danielpr@nvidia.com>
@github-actions

Copy link
Copy Markdown

👋 Hi dpressle! Thank you for contributing to ai-dynamo/nixl.

Your PR reviewers will review your contribution then trigger the CI to test your changes.

🚀

@dpressle

Copy link
Copy Markdown
Contributor Author

/build

@dpressle
dpressle marked this pull request as ready for review January 18, 2026 15:51
@dpressle
dpressle requested a review from a team as a code owner January 18, 2026 15:51
@dpressle
dpressle requested a review from ovidiusm January 18, 2026 15:51
@dpressle
dpressle merged commit 1985604 into ai-dynamo:main Jan 19, 2026
21 of 22 checks passed
@dpressle
dpressle deleted the etcd_kill_fixup branch January 19, 2026 09:51

This branch was previously deployed

2 inactive deployments
SWX_AWS — e7ef8574 Deployed Jan 18, 2026 by copy-pr-bot[bot] via Run AWS Tests - Python Tests #5070
GITLAB — e7ef8574 Deployed Jan 18, 2026 by copy-pr-bot[bot] via Trigger CI Pipeline #6385
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants