Skip to content

CI: Increase test sleep - #1081

Merged
brminich merged 10 commits into
ai-dynamo:mainfrom
Alexey-Rivkin:increase_test_sleep
Jan 12, 2026
Merged

brminich merged 10 commits into
ai-dynamo:mainfrom
Alexey-Rivkin:increase_test_sleep

Conversation

@Alexey-Rivkin

@Alexey-Rivkin Alexey-Rivkin commented Dec 4, 2025

Copy link
Copy Markdown
Contributor

What?

Increase hardcoded sleep durations across CI scripts, unit tests, and examples.

Why?

Prevent random test failures during high-concurrency runs where system performance is slower, causing fixed waits to expire prematurely.

How?

Scaled up sleep intervals (typically by 3x–5x, with larger increases for tight polling loops) in:

  • GitLab CI shell scripts (test_python.sh, test_cpp.sh, etc.)
  • C++ GTests (metadata_exchange, telemetry_test, device_api/utils.cu, etc.)
  • Plugin unit tests (nixl_hf3fs_test.cpp, nixl_hf3fs_mt_test.cpp)
  • Benchmark and example code (etcd_rt.cpp, nixl_p2p_storage_example.py)

@github-actions

github-actions Bot commented Dec 4, 2025

Copy link
Copy Markdown

👋 Hi Alexey-Rivkin! Thank you for contributing to ai-dynamo/nixl.

Your PR reviewers will review your contribution then trigger the CI to test your changes.

🚀

@Alexey-Rivkin

Copy link
Copy Markdown
Contributor Author

The safest and most reliable way to start an etcd member and wait until it is ready to serve is by repeatedly polling its health check endpoint or using the etcdctl command until it returns a healthy status.

This approach replaces an unreliable sleep command with a precise check for the service's operational readiness. use AI to generate proper check

10edcd9

@Alexey-Rivkin

Copy link
Copy Markdown
Contributor Author

/build

brminich
brminich previously approved these changes Dec 9, 2025
Comment thread test/gtest/telemetry_test.cpp
ovidiusm
ovidiusm previously approved these changes Dec 9, 2025
@Alexey-Rivkin

Copy link
Copy Markdown
Contributor Author

/build

Comment thread .ci/scripts/common.sh
dpressle
dpressle previously approved these changes Dec 10, 2025
@dpressle

Copy link
Copy Markdown
Contributor

Tested successfully with 8 executers see builds 180-183

@Alexey-Rivkin

Copy link
Copy Markdown
Contributor Author

/build

3 similar comments
@Alexey-Rivkin

Copy link
Copy Markdown
Contributor Author

/build

@Alexey-Rivkin

Copy link
Copy Markdown
Contributor Author

/build

@dpressle

Copy link
Copy Markdown
Contributor

/build

brminich
brminich previously approved these changes Jan 5, 2026
@dpressle

dpressle commented Jan 6, 2026

Copy link
Copy Markdown
Contributor

/build

@brminich

brminich commented Jan 8, 2026

Copy link
Copy Markdown
Contributor

/ok to test 3f15bdf

@brminich

brminich commented Jan 8, 2026

Copy link
Copy Markdown
Contributor

/build

@Alexey-Rivkin

Copy link
Copy Markdown
Contributor Author

/build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants