CI/BUILD: Change TMPDIR name. - #1910
Conversation
|
👋 Hi ColinNV! Thank you for contributing to ai-dynamo/nixl. Your PR reviewers will review your contribution then trigger the CI to test your changes. 🚀 |
|
/build |
|
👀 Investigating |
|
👀 Investigating |
|
👀 Investigating |
|
👀 Investigating |
|
👀 Investigating |
|
👀 Investigating |
|
🤖 CI Triage Agent — I have a clear diagnosis. This is a pre-flight CI policy check, not a test failure or hang. The Note the log timestamps show continuous activity ending immediately at the policy error — this is a deterministic exit-1 failure, not a hang or timeout. Summary: The Root cause: PR #1910 (commit 689c405) modified Implicated commit: [REDACTED:Hex High Entropy String] (PR #1910 merge commit). The offending change is the modification to File: Suggested fix: In PR #1910, increment
Alternatively, if the Related: PR #1910 (this change); prior tag-bumping precedent in commit
|
|
🤖 CI Triage Agent — I have a confident diagnosis. This is a CI policy/gate check, not a test failure or hang. The Summary: The Root cause: PR #1910 modified a monitored CI file ( Implicated commit: [REDACTED:Hex High Entropy String] (merge commit for PR #1910). The offending change is the modification to File: Suggested fix: In PR #1910, since
Keep the tag value identical across all five files. If the Related: none found (this is a repo-local policy gate; the guard was most recently exercised by CI-image changes in commits #1868/#1869 touching
|
|
🤖 CI Triage Agent — The diagnosis is clear and confirmed by the logs. This is a deterministic guardrail failure — not a hang, not a flaky test. The Summary: The Root cause: PR #1910 (commit 689c405) modified
Implicated commit: [REDACTED:Hex High Entropy String] (PR #1910 merge). The change to File: Suggested fix: In PR #1910, bump
All five must be updated — the script fails if any one is not bumped (see the OR condition on line 112). After bumping, the CI image will be rebuilt with the updated Related: none found specific to this failure.
|
|
🤖 CI Triage Agent — I have a confident, evidence-based diagnosis. Summary: The Root cause: PR #1910 modified Implicated commit: [REDACTED:Hex High Entropy String] (merge commit for PR #1910). The File: Suggested fix: In PR #1910, bump Related: none (no matching issues/PRs found; the search hits above are unrelated).
|
|
🤖 CI Triage Agent — Confirmed. The current tag is Summary: The Root cause: PR #1910 modified Implicated commit: File: Suggested fix: Increment Related: Recent commits that touched
|
|
🤖 CI Triage Agent — The diagnosis is clear and confirmed by the logs. This is a deterministic gate failure, not a hang or timeout. The Summary: Checkout stage failed — the Root cause: PR #1910 (merge commit 689c405) modifies Implicated commit: [REDACTED:Hex High Entropy String] (PR #1910 merge). The change to File: Suggested fix: In PR #1910, bump
If the change to Related: none found specific to this failure. (Recent
|
|
/build |
|
👀 Investigating |
|
👀 Investigating |
|
👀 Investigating |
|
👀 Investigating |
|
👀 Investigating |
|
👀 Investigating |
|
🤖 CI Triage Agent — I have a clear diagnosis. This is not a hang or a test failure — it's a policy/guard check in the CI init hook. The I've confirmed the current tag is Summary: The "Checkout source code" stage failed — the cidemo init hook aborted because Root cause: PR #1910 changed a CI file ( Implicated commit: [REDACTED:Hex High Entropy String] (PR #1910 merge commit; the head commit [REDACTED:Hex High Entropy String] introduced the File: Suggested fix: Increment
Keep the tag value consistent across all five files, then re-run the pipeline. (Alternatively, if the Related: PR #1910. none (no matching issues found for this guard message).
|
|
🤖 CI Triage Agent — I have a confident diagnosis. The failure is a policy gate in the checkout hook, not a build/test failure. Summary: The "Checkout source code" stage failed because the Root cause: PR #1910 (commit Implicated commit: File: Suggested fix: In PR #1910, bump
All five must be updated — the guard fails if any one is left unchanged. Alternatively, if the Related: none found directly; recent CI-file changes that correctly bumped the tag include PR #1868 and PR #1869 (both by editing
|
|
🤖 CI Triage Agent — I have a confident diagnosis. This is a CI-policy gate failure, not a test/build failure. Summary: The "Checkout source code" stage failed in the Root cause: The This is a legitimate gate failure, not a flake, hang, or infra problem. Without a tag bump, the sanitizer images would be rebuilt/reused inconsistently against the modified dependency-build script. Implicated commit: [REDACTED:Hex High Entropy String] (the PR #1910 merge commit) — the change to File: Suggested fix: In PR #1910, bump
Keep the tag identical across all matrices. If the Related: none found (this is a policy check in
|
|
🤖 CI Triage Agent — I have a confident diagnosis. The failure is a CI policy guardrail, not a build/test error. Summary: The "Checkout source code" stage failed because the Root cause: PR #1910 (merge commit Implicated commit: File: Suggested fix: In PR #1910, bump the
Keep the tag identical across all five files (the check requires every one to differ from Related: none found specific to this failure. Note that
|
|
🤖 CI Triage Agent — The diagnosis is complete and unambiguous — this is a CI policy guardrail, not a test/infrastructure failure. Summary: The "Checkout source code" stage failed because PR #1910 modified a CI file ( Root cause: The Implicated commit: The failure is caused by the PR's own change to File: Suggested fix: In PR #1910, increment
The tag must be changed in all files, since the hook fails if any single one is unchanged (line 112). If the Related: none
|
|
🤖 CI Triage Agent — The diagnosis is confirmed. This is a clean, deterministic CI-policy failure — not a hang or timeout. Summary: The "Checkout source code" stage failed because PR #1910 modified a CI file ( Root cause: The Implicated commit: [REDACTED:Hex High Entropy String] (merge commit for PR #1910). The offending diff is the change to File: Suggested fix: In PR #1910, increment
Bump the tag consistently, e.g. Related: none found specific to this failure; the CI-tag guard was recently exercised by CI-file changes in commits 3ab92dc (UCX bump) and 9ac7ada (CUDA 13.3 base image switch).
|
|
/build |
|
👀 Investigating |
|
🤖 CI Triage Agent — I now have a clear picture. Let me analyze both failures. Stage 177 ( Stage 235 ( Let me look at the timestamps closely:
The This is not a hang and not a code bug — the one-hour gap is the Log requirement satisfied: I successfully fetched both Jenkins stage logs. Summary: The Root cause: Cluster resource starvation, not a code defect. In the Implicated commit: unknown — no code change is implicated; this is an infrastructure/scheduling failure. Commit File: unknown (failure is in the Jenkins pipeline's SLURM allocation step, not in a source file) — the relevant config is the Suggested fix: Re-run the build — the GB200 partition was contended. If this recurs frequently, mitigate at the infrastructure level rather than in code: (1) add automatic retry/backoff around the Related: none found.
|
📝 WalkthroughWalkthroughThis PR bumps the Jenkins CI_IMAGE_TAG from 20260707-1 to 20260709-1 across five build/test matrix YAML files, and updates .gitlab/build.sh to create and use a dedicated BUILD_TMP temporary directory (via mktemp -d) in place of TMPDIR for all download, build, and cleanup operations. ChangesJenkins CI Image Tag Bump
Estimated code review effort: 1 (Trivial) | ~3 minutes build.sh BUILD_TMP Migration
Estimated code review effort: 2 (Simple) | ~12 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.gitlab/build.sh:
- Line 43: The BUILD_TMP temporary directory created in the build script is only
cleaned up at the end, so failures in the long download/build chain can leave
large artifacts behind. Add a cleanup trap near the BUILD_TMP initialization in
the build script so it always removes the directory on exit or error, and make
sure the trap covers the existing final cleanup path without breaking normal
completion.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 514fc58b-bad1-4634-a2b2-305331056df4
📒 Files selected for processing (6)
.ci/jenkins/lib/build-matrix.yaml.ci/jenkins/lib/test-dl-ep-matrix.yaml.ci/jenkins/lib/test-dl-matrix.yaml.ci/jenkins/lib/test-matrix.yaml.ci/jenkins/lib/test-sanitizer-matrix.yaml.gitlab/build.sh
6 participants
What?
Rename
TMPDIRinbuild.sh.Why?
Prevent interference with externally set
TMPDIR.Symptom is
nvccbeing unable to create temporary files.Summary by CodeRabbit
Bug Fixes
Chores