[https://nvbugs/5351244][fix] CHERRY-PICK test_mpi_session (#7501) (#7900)

Signed-off-by: Yan Chunwei <[email protected]>
Signed-off-by: Ludwig Schneider <[email protected]>

pre-commit changes

Signed-off-by: Ludwig Schneider <[email protected]>

clang formatting

Signed-off-by: Ludwig Schneider <[email protected]>

safe guarding NCCL 2.27 build

Signed-off-by: Ludwig Schneider <[email protected]>

fixing precommit formatting

Signed-off-by: Ludwig Schneider <[email protected]>

most of code rabbit comments

Signed-off-by: Ludwig Schneider <[email protected]>

adding missing semi-colon

Signed-off-by: Ludwig Schneider <[email protected]>

removing unused comment lines

Signed-off-by: Ludwig Schneider <[email protected]>

Clarifying the test on how to compre residual chunked and unchunked.

Signed-off-by: Ludwig Schneider <[email protected]>

fixing pre-commit

Signed-off-by: Ludwig Schneider <[email protected]>

fixing pre-commit

Signed-off-by: Ludwig Schneider <[email protected]>

fixing missing variable, rebase complete and tested

Signed-off-by: Ludwig Schneider <[email protected]>

using a grid stride loop with less blocks launched for large message sizes

Signed-off-by: Ludwig Schneider <[email protected]>

using functioning grid stride loop for NCCL_DEVICE. It helps with better performance at larger message sizes

Signed-off-by: Ludwig Schneider <[email protected]>

initial oneshot implementation

Signed-off-by: Ludwig Schneider <[email protected]>

minor tweaks to include one shot

fixes

Signed-off-by: Ludwig Schneider <[email protected]>

enabling grid stride loop, but no perf benefit.

Signed-off-by: Ludwig Schneider <[email protected]>

addressing review feedback

Signed-off-by: Ludwig Schneider <[email protected]>

fix formatting

Signed-off-by: Ludwig Schneider <[email protected]>

Author: Superjomn

.codespellignore

@@ -0,0 +1 @@
+commIter

.pre-commit-config.yaml

@@ -1442,7 +1442,7 @@ repos:
         additional_dependencies:
         - tomli
         # add ignore words list
-        args: ["-L", "Mor,ans,thirdparty", "--skip", "ATTRIBUTIONS-*.md,*.svg", "--skip", "security_scanning/*"]
+        args: ["-L", "Mor,ans,thirdparty", "--skip", "ATTRIBUTIONS-*.md,*.svg", "--skip", "security_scanning/*", "-I", ".codespellignore"]
 -   repo: https://github.com/astral-sh/ruff-pre-commit
     rev: v0.9.4
     hooks:

cpp/tensorrt_llm/kernels/nccl_device/CMakeLists.txt

@@ -1,34 +1,28 @@
-# CMakeLists.txt for nccl_device
-# This directory contains CUDA kernels and host launcher code
+# CMakeLists.txt for nccl_device This directory contains CUDA kernels and host
+# launcher code
 
 # Enable CUDA
 enable_language(CUDA)
 
 # Create CUDA library
-add_library(tensorrt_llm_nccl_device
-    config.cu
-)
+add_library(tensorrt_llm_nccl_device config.cu)
 
 # Set properties for the CUDA library
-set_target_properties(tensorrt_llm_nccl_device PROPERTIES
-    CUDA_STANDARD 17
-    CUDA_SEPARABLE_COMPILATION ON
-    POSITION_INDEPENDENT_CODE ON
-)
+set_target_properties(
+  tensorrt_llm_nccl_device
+  PROPERTIES CUDA_STANDARD 17 CUDA_SEPARABLE_COMPILATION ON
+             POSITION_INDEPENDENT_CODE ON)
 
 # Include directories
-target_include_directories(tensorrt_llm_nccl_device PUBLIC
-    ${CMAKE_CURRENT_SOURCE_DIR}
-    ${CMAKE_CURRENT_SOURCE_DIR}/../..
-)
+target_include_directories(
+  tensorrt_llm_nccl_device PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}
+                                  ${CMAKE_CURRENT_SOURCE_DIR}/../..)
 
 # Link libraries
-target_link_libraries(tensorrt_llm_nccl_device
-    tensorrt_llm_common
-)
+target_link_libraries(tensorrt_llm_nccl_device tensorrt_llm_common)
 
 # Install target
-install(TARGETS tensorrt_llm_nccl_device
-    LIBRARY DESTINATION lib
-    ARCHIVE DESTINATION lib
-)
+install(
+  TARGETS tensorrt_llm_nccl_device
+  LIBRARY DESTINATION lib
+  ARCHIVE DESTINATION lib)