[Misc] Delay EPLB Nixl import until needed #41805
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors vllm/distributed/eplb/eplb_communicator.py by moving the imports of NixlWrapper and nixl_agent_config from the module level into the has_nixl function and the communicator's init method. This change facilitates lazy loading of these dependencies. As there were no review comments provided, I have no additional feedback to offer.
|
I think there is already PR: #41392 that does similar things |
|
@yewentao256 #41392 looks maybe a bit overly complicated, are you sure we don't just need lazy imports for eplb? |
yewentao256
left a comment
There was a problem hiding this comment.
LGTM, thanks for the work!
I think both of the PRs can be landed as they are doing different things.
This PR fixes the issue for now and previous PR will avoid similiar issues in the future.
@NickLucche Would appreciate your approval for previous PR as well.
|
LGTM! This is helpful. I think later on if needed we could also wrap the imports in |
Signed-off-by: NickLucche <nlucches@redhat.com> Signed-off-by: alexagriffith <agriffith96@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com> Signed-off-by: Libin Tang <libin.tang@intel.com>
Starting a simple colocated vllm instance with a dense model (ep obv disabled) still brings up this unrelated logs that show up when importing nixl
This PR simply delays importing nixl until needed, that is when eplb with nixl backend has been requested by the user
cc @ilmarkov