Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion flashinfer/jit/cubin_loader.py
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,9 @@ def ensure_symlink(
This is used to map C++ include paths (e.g.
``CUBIN_DIR/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export``) to the
canonical artifact directory where ``get_artifact()`` stores downloaded files.

This function is safe to call concurrently from multiple processes (e.g.
tensor-parallel workers) that may race to create the same symlink.
Comment on lines +242 to +243

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The claim that this function is safe to call concurrently is not entirely accurate because the cleanup logic (lines 247-254) still contains TOCTOU (Time-of-Check to Time-of-Use) race conditions. If multiple processes attempt to remove a stale file or directory at the same time, one will likely fail with a FileNotFoundError when calling link.unlink() or shutil.rmtree(link). To make this truly safe for concurrent workers, these cleanup operations should be wrapped in a try-except FileNotFoundError block or use missing_ok=True for unlink (available in Python 3.8+).

Comment on lines +242 to +243

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The docstring states that this function is safe to call concurrently, but the cleanup logic for stale links (lines 251-254) still contains a race condition. If multiple processes attempt to unlink() or rmtree() the same stale path simultaneously, one will succeed while others may fail with a FileNotFoundError. To make this fully safe for concurrent workers, the cleanup phase should handle the case where the file has already been removed by another process (e.g., by catching FileNotFoundError or using missing_ok=True for unlink).

"""
link = pathlib.Path(link)
target = pathlib.Path(target)
Expand All @@ -250,7 +253,13 @@ def ensure_symlink(
else:
shutil.rmtree(link)
link.parent.mkdir(parents=True, exist_ok=True)
link.symlink_to(target)
try:
link.symlink_to(target)
except FileExistsError:
# Another process created the symlink between our check and create.
if link.is_symlink() and link.resolve() == target.resolve():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Comparing link.resolve() with target.resolve() can be unreliable if target is a relative path, as target.resolve() resolves against the current working directory instead of the symlink's parent directory. Resolving target relative to link.parent provides a more robust comparison that works correctly for both absolute and relative paths.

Suggested change
if link.is_symlink() and link.resolve() == target.resolve():
if link.is_symlink() and link.resolve() == (link.parent / target).resolve():

return
raise


def verify_symlinked_headers(
Expand Down
Loading