Skip to content

[Hardware][TPU] Optionally import for TPU backend#18269

Merged
DarkLight1337 merged 16 commits intovllm-project:mainfrom
lsy323:lsiyuan/tpu-commons-fallback
May 17, 2025
Merged

[Hardware][TPU] Optionally import for TPU backend#18269
DarkLight1337 merged 16 commits intovllm-project:mainfrom
lsy323:lsiyuan/tpu-commons-fallback

Conversation

@lsy323
Copy link
Collaborator

@lsy323 lsy323 commented May 16, 2025

No description provided.

@github-actions
Copy link

github-actions bot commented May 16, 2025

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

CAROLZXYZXY and others added 15 commits May 16, 2025 17:22
@CAROLZXYZXY @lsy323
[CI/Build] Fix TPU V1 Test mixed use of & and && across tests (vllm-p…
566398e 
…roject#17968)

Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@jianzs @lsy323
[Core] Use platform-agnostic device control for DP engine core (vllm-…
9bff53d 
…project#17245)

Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
delete runner, import from tpu commons
df3a9d3 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
Small clean up for the sys.path manipulation.
d81d960 
The same system path workaround can be achieved by `PYTHONPATH=$HOME python ...` without the hardcode in the source.

Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
Use the TPUWorker implementation from tpu_commons.
082813f 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
import workers from tpu commons
01c26b2 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
import tpu platform from tpu commons
c8af4d7 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
import tpu communicators from tpu commons
54d6f35 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
remove sampler
237bca7 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
fallback to tpu commons worker import
c82130e 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
Revert "remove sampler"
00710d9 
This reverts commit 3b4fc66e7daa3a79eb6669e2e97c359bc8c19539.

Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
Revert "import tpu platform from tpu commons"
847cb34 
This reverts commit 996c8b3d3ea835fedc9af4eebe433f276ff9a286.

Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
Revert "import tpu communicators from tpu commons"
1c7103c 
This reverts commit b767f6f90959e07d175f3b3070ff793688180c45.

Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
fall back for communicator and platform
20fe20d 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323
recover test script
206dad4 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@lsy323 lsy323 force-pushed the lsiyuan/tpu-commons-fallback branch from 911cf45 to 206dad4 Compare May 16, 2025 17:22
@mergify
Copy link

mergify bot commented May 16, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @lsy323.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added tpu Related to Google TPUs needs-rebase labels May 16, 2025
@lsy323
Merge branch 'main' into lsiyuan/tpu-commons-fallback
d8161c2 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
@mergify mergify bot removed the needs-rebase label May 16, 2025
@lsy323
Copy link
Collaborator Author

lsy323 commented May 16, 2025

cc @yarongmu-google @hfan @WoosukKwon

@WoosukKwon
Copy link
Collaborator

WoosukKwon commented May 16, 2025

QQ: Why not adding tpu_commons as a dependency? Why is it optional?

@yarongmu-google
Copy link
Contributor

yarongmu-google commented May 16, 2025

QQ: Why not adding tpu_commons as a dependency? Why is it optional?

I think we are trying to be extra carful, by being able to fall back to the existing implementation until the installation is super stable. More specifically, we want to create a wheel for tpu_commons with nightly, alpha, beta, prod etc. Until that system is in, it may be safer to have both so concumers/developers are not slowed down by anything.

So you are absolutely right, in the long run, we will add tpu_commons as a dependency.

WoosukKwon
WoosukKwon approved these changes May 16, 2025
View reviewed changes
Copy link
Collaborator

@WoosukKwon WoosukKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
@yarongmu-google Thanks for the explanation!

@lsy323 lsy323 changed the title [hardware][tpu] Optionally import for TPU backend [Hardware][TPU] Optionally import for TPU backend May 16, 2025
@lsy323 lsy323 added the ready ONLY add when PR is ready to merge/full CI is needed label May 17, 2025
@DarkLight1337 DarkLight1337 merged commit 48ac2be into vllm-project:main May 17, 2025
77 checks passed
@lsy323 lsy323 deleted the lsiyuan/tpu-commons-fallback branch May 20, 2025 16:47
zzzyq pushed a commit to zzzyq/vllm that referenced this pull request May 24, 2025
@lsy323 @CAROLZXYZXY @jianzs
[Hardware][TPU] Optionally import for TPU backend (vllm-project#18269)
a00fe6c 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Carol Zheng <cazheng@google.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Hongmin Fan <fanhongmin@google.com>
Signed-off-by: Yuqi Zhang <yuqizhang@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@WoosukKwon WoosukKwon WoosukKwon approved these changes

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat

@njhill njhill Awaiting requested review from njhill

@ywang96 ywang96 Awaiting requested review from ywang96

@comaniac comaniac Awaiting requested review from comaniac

@alexm-redhat alexm-redhat Awaiting requested review from alexm-redhat

Assignees

No one assigned

Labels

ci/build ready ONLY add when PR is ready to merge/full CI is needed tpu Related to Google TPUs v1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@lsy323 @WoosukKwon @yarongmu-google @DarkLight1337 @CAROLZXYZXY @jianzs