Repository navigation
Conversation
Set the pinned request state and exit code before publishing their pointers or starting a proxy progress thread. cudaMallocHost does not initialize the allocation contents, so stale values can otherwise be interpreted as a device global-exit request. Related integration: sgl-project/sglang#39210 Signed-off-by: Ziang Li <ziangli@umich.edu>
Member
|
Thank you for your contribution. We are somewhat busy finishing up the 3.8 release but will take a look. From first glance I don't see any problems but will go through init in more detail before committing ot that stance. |
benhg
marked this pull request as ready for review
September 23, 2026 01:57
This was referenced Sep 24, 2026
Member
|
This has been merged internally and should come back out to GitHub in the next push from our internal mirror. |
5 tasks done
This was referenced Oct 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@HumansAnd
Initialize the proxy's global-exit request state and exit code immediately after allocating their pinned host storage. This prevents allocation contents from being interpreted as an application request to exit.
calloc()clearsproxy_state_t, including its pointer fields, but does not initialize the two subsequentcudaMallocHost()allocations. Both the minimal and full proxy progress paths readglobal_exit_request_state; a value equal toPROXY_GLOBAL_EXIT_REQUESTED(2) causesnvshmem_global_exit(*global_exit_code).PROXY_GLOBAL_EXIT_NOT_REQUESTEDand0beforenvshmemi_proxy_prep_minimal_state(), device-state publication, and either progress-thread creation. This follows the existing initialization of the proxy channel counters. No API, transport selection, device request protocol, or runtime option changes.Evidence and validation
The missing initialization is still present on
develatb0d9d3dc08fc3ee0840fdb6f3a2c11932d85a2e2. This PR changes one file by two lines.git diff --checkpasses. No behavioral regression test is added.The following observations were collected separately on NVSHMEM 3.4.5, before porting the same two stores to this branch:
590.48.01;lmsysorg/sglang:nightly-dev-cu13-20260918-20518d85(sha256:b518f4f8cd15664cf0f733e9bf4fd2105c9c994882a369b247364394db99f596), PyTorch2.13.0+cu130, CUDA13.0, NVSHMEM4Py0.3.1, SGLang6d8a58f177a1488e10e8567a3fa4b4fccd0a26e2, FlashInferad0a5e5e78e57070ec7c582efe733cb55cd8839f.4, max running requests8, static memory fraction0.80, and EAGLE draft settings3/1/4. This is allocation-history sensitive; a standalone deterministic reproducer is not yet available.2and code0. The same proxy pointers later reachednvshmem_global_exit@pltfrom the full progress path with exit argument0. No inferior code, data, or computational register values were changed; debugger stops did perturb timing. The allocator's prior writer was not observed.A separate matched source-build comparison used the public
v3.4.5-0source (131da55f643ac87c810ba0bc51d359258bf433a1) with the two stores as the only original-source difference. Both builds used CUDA compiler13.0.88, GCC13.3,sm_103, and identical build settings. UCX was disabled because its headers were unavailable; tests, examples, Python, packaging, and bitcode builds were disabled. These are not claimed to be binary-equivalent to the distributed wheel.nvshmem API called before nvshmem_initmessagesLimits: The current
develpatch has not been built or run on a GPU. The matched 3.4.5 builds completed with existing CMake/PMIx warnings. The startup comparison sent no performance requests and establishes no numerical or performance result. Legitimate deviceglobal_exitpropagation, other transports, and multi-node behavior were not tested. The pristine run's initially incomplete cleanup check was retained; a separate later idle check passed. The patched server was stopped deliberately after readiness with no recorded remaining owners or cleanup errors.