Make cubin symlink setup idempotent under concurrent startup

[kaixih]

Summary

Make ensure_symlink() tolerate a concurrent startup race where another process creates the intended cubin symlink after the pre-check but before symlink_to().

Root Cause

During multi-rank SGLang startup, multiple ranks can set up the same FlashInfer cubin include symlink concurrently. If one rank creates the symlink after another rank checks link.exists() / link.is_symlink() but before it calls link.symlink_to(target), the second rank can fail with:

FileExistsError: [Errno 17] File exists

This is harmless when the symlink now points to the same intended target, so treat that case as success. Stale links, files, directories, or links to a different target still follow the existing behavior and raise/remove as before.

Validation

• PYTHONPYCACHEPREFIX=/private/tmp/pycache-flashinfer python3 -m py_compile flashinfer/jit/cubin_loader.py tests/utils/test_cubin_loader.py
• Manual focused race simulation for:
  • same-target concurrent symlink creation returns successfully
  • different-target concurrent symlink creation still raises FileExistsError

Note: local pytest could not collect the repo test directly on this machine because the lightweight local environment lacks FlashInfer test dependencies such as torch / tvm_ffi.

Summary by CodeRabbit

• Bug Fixes

  • Enhanced symlink creation error handling to gracefully handle cases where symlinks already exist with the correct target, improving system robustness.

• Tests

  • Added test coverage for concurrent symlink creation scenarios, verifying correct behavior when conflicts occur and ensuring proper error handling when symlinks point to different targets.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ce99e95a-a98d-4ecc-bea1-457df0c7bb0e

📥 Commits

Reviewing files that changed from the base of the PR and between 9035311 and bcfa052.

📒 Files selected for processing (2)

• flashinfer/jit/cubin_loader.py
• tests/utils/test_cubin_loader.py

---

📝 Walkthrough

Walkthrough

The ensure_symlink() function now handles concurrent symlink creation races by catching FileExistsError. If a symlink already exists pointing to the intended target, the operation succeeds silently; otherwise the error is re-raised. New tests verify both benign and problematic race scenarios.

Changes

Symlink Race Condition Handling

Layer / File(s)	Summary
Symlink conflict resolution logic flashinfer/jit/cubin_loader.py	ensure_symlink() wraps symlink creation in try/except to handle FileExistsError, tolerating it when the existing link already points to the target.
Concurrent race test coverage tests/utils/test_cubin_loader.py	Tests verify that ensure_symlink() tolerates benign FileExistsError when an existing symlink points to the expected target and re-raises when it points elsewhere.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related issues

• Race condition in ensure_symlink causes FileExistsError with tensor parallelism #2980: Directly related—both changes address the same concurrent symlink creation race in ensure_symlink() with the same conflict-resolution approach.

Possibly related PRs

• flashinfer-ai/flashinfer#2979: Addresses the same concurrent symlink creation race but uses a filelock-based approach instead of exception handling.

Suggested labels

run-ci

Suggested reviewers

• yzh119
• sricketts
• yongwww
• cyx-6
• samuellees

Poem

🐰 A symlink in haste, two processes race,
Creating at once in the same time and place,
One wins, one fails—no matter, we say,
If they both point the same way! 🔗✨

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	❓ Inconclusive	The description provides a clear Summary, Root Cause, and Validation section explaining the problem and how it was tested; however, it doesn't follow the provided template structure with required sections like Description, Related Issues, or the Checklist items.	Consider reformatting the description to match the template structure: add a formal Description section, link Related Issues, and include the Pre-commit Checks and Tests checklist items to ensure compliance with repository conventions.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: making cubin symlink setup idempotent under concurrent startup, which matches the PR's primary objective of handling FileExistsError races.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request enhances the robustness of the ensure_symlink function in flashinfer/jit/cubin_loader.py by implementing a try-except block to handle FileExistsError, which prevents failures during concurrent symlink creation if the existing link points to the correct target. A new test file, tests/utils/test_cubin_loader.py, was also added to verify this behavior and ensure that errors are still raised if the existing symlink points to a different target. I have no feedback to provide.

[yongwww]

/bot run

[flashinfer-bot]

GitLab MR !691 has been created, and the CI pipeline #51947596 is currently running. I'll report back once the pipeline job completes.