Skip to content

🎨 Remove new_token_ids from warmup#292

Closed
prashantgupta24 wants to merge 8 commits intomainfrom
fix-092-compiler-bug
Closed

🎨 Remove new_token_ids from warmup#292
prashantgupta24 wants to merge 8 commits intomainfrom
fix-092-compiler-bug

Conversation

@prashantgupta24
Copy link
Copy Markdown
Collaborator

@prashantgupta24 prashantgupta24 commented Jul 9, 2025 •
edited
Loading

Description

🎨 Remove new_token_ids from warmup since new_token_ids are not used anymore.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Jul 9, 2025

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

@prashantgupta24 prashantgupta24 changed the title 🔥 remove new_token_ids from warmup decode [WIP] Fix the compiler issue with the new changes Jul 9, 2025
prashantgupta24
prashantgupta24 commented Jul 9, 2025
View reviewed changes
Comment thread vllm_spyre/v1/worker/spyre_model_runner.py Outdated
@prashantgupta24
Copy link
Copy Markdown
Collaborator Author

prashantgupta24 commented Jul 9, 2025

bot:test
MARKERS="cb and spyre"

@prashantgupta24 prashantgupta24 mentioned this pull request Jul 9, 2025
Merged
yannicks1
yannicks1 approved these changes Jul 9, 2025
View reviewed changes
Comment thread vllm_spyre/v1/worker/spyre_model_runner.py Outdated
@prashantgupta24 prashantgupta24 changed the title [WIP] Fix the compiler issue with the new changes 🎨 Remove new_token_ids from warmup Jul 9, 2025
@prashantgupta24
🔥 remove new_token_ids from warmup decode
83059cb 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
joerunde
joerunde reviewed Jul 9, 2025
View reviewed changes
Comment thread vllm_spyre/v1/worker/spyre_worker.py Outdated
@prashantgupta24
🔥 remove new_token_ids from sb warmup too
a9f34c8 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
@prashantgupta24
🙈 ignore the random type error
99aad58 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
@prashantgupta24
🎨 make it consistent
dcb8f0c 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
@joerunde
Copy link
Copy Markdown
Collaborator

joerunde commented Jul 18, 2025

@prashantgupta24 do we wanna get this merged?

If so we should definitely test to triple-check that this works with the upcoming compiler changes for continuous batching

@prashantgupta24
Copy link
Copy Markdown
Collaborator Author

prashantgupta24 commented Jul 18, 2025

No hurry as such, don't want to add anything before the release lol

@prashantgupta24
🚧
a4a188f 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
@prashantgupta24
Merge remote-tracking branch 'upstream/main' into fix-092-compiler-bug
b00f92e 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
@prashantgupta24
Merge remote-tracking branch 'upstream/main' into fix-092-compiler-bug
fa34d66
@prashantgupta24
🐛 whoops
fd699ef 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
@prashantgupta24 prashantgupta24 marked this pull request as ready for review July 30, 2025 17:44
@prashantgupta24
Copy link
Copy Markdown
Collaborator Author

prashantgupta24 commented Jul 30, 2025

bot:test
MARKERS="spyre and not quantized"

1 similar comment
@prashantgupta24
Copy link
Copy Markdown
Collaborator Author

prashantgupta24 commented Jul 30, 2025

bot:test
MARKERS="spyre and not quantized"

maxdebayser
maxdebayser approved these changes Jul 30, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@maxdebayser maxdebayser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@prashantgupta24
Copy link
Copy Markdown
Collaborator Author

prashantgupta24 commented Jul 30, 2025 •
edited
Loading

Failed one of the tests on spyre

tests/e2e/test_spyre_cb_scheduler_steps.py::test_new_sequence_joins_during_decode[sendnn-ibm-ai-platform/micro-g3.3-8b-instruct-1b] FAILED

will investigate, seems flaky...

@prashantgupta24
Copy link
Copy Markdown
Collaborator Author

prashantgupta24 commented Aug 1, 2025 •
edited
Loading

bot:test
MARKERS="spyre and cb" - all cb tests passed ! (170)

@prashantgupta24
Copy link
Copy Markdown
Collaborator Author

prashantgupta24 commented Aug 1, 2025

bot:test
MARKERS="spyre and not quantized"

@prashantgupta24
Copy link
Copy Markdown
Collaborator Author

prashantgupta24 commented Aug 1, 2025

Failed 2 different tests this time -

tests/e2e/test_spyre_cb_scheduler_steps.py::test_prompt_too_long_for_current_tkv[sendnn-ibm-ai-platform/micro-g3.3-8b-instruct-1b] FAILED
tests/e2e/test_spyre_cb_scheduler_steps.py::test_requested_tokens_not_fitting_remaining_space[sendnn-ibm-ai-platform/micro-g3.3-8b-instruct-1b] FAILED

Seems flaky since running only CB tests made them all pass 🤔

@prashantgupta24 prashantgupta24 marked this pull request as draft August 1, 2025 17:59
@prashantgupta24
Copy link
Copy Markdown
Collaborator Author

prashantgupta24 commented Aug 1, 2025

Too much flakiness going on, going to convert it to a draft for now.

@prashantgupta24
Copy link
Copy Markdown
Collaborator Author

prashantgupta24 commented Aug 25, 2025

bot:test

@joerunde
Copy link
Copy Markdown
Collaborator

joerunde commented Oct 6, 2025

@prashantgupta24 do we need to revive this or okay to close as stale?

@prashantgupta24 prashantgupta24 deleted the fix-092-compiler-bug branch March 5, 2026 17:01
rafvasq pushed a commit to rafvasq/sendnn-inference that referenced this pull request Mar 11, 2026
@gkumbhat
Update pre compile job (torch-spyre#292)
78176a7 
* ♻️ Remove workspace

Signed-off-by: Gaurav-Kumbhat <Gaurav.Kumbhat@ibm.com>

* ✨ Add support for compiler env variables

Signed-off-by: Gaurav-Kumbhat <Gaurav.Kumbhat@ibm.com>

---------

Signed-off-by: Gaurav-Kumbhat <Gaurav.Kumbhat@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@joerunde joerunde joerunde left review comments

@maxdebayser maxdebayser maxdebayser approved these changes

@yannicks1 yannicks1 yannicks1 approved these changes

@tdoublep tdoublep Awaiting requested review from tdoublep tdoublep is a code owner

@nikolaospapandreou nikolaospapandreou Awaiting requested review from nikolaospapandreou nikolaospapandreou is a code owner

@sducouedic sducouedic Awaiting requested review from sducouedic sducouedic is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@prashantgupta24 @joerunde @maxdebayser @yannicks1