Skip to content

Conversation

@lk-chen
Copy link
Contributor

@lk-chen lk-chen commented May 14, 2025

Why are these changes needed?

vLLM changed (non-external-facing) api in chat_utils.py, we need to adapt the argument change. see vllm-project/vllm#18098

Related issue number

FIX #52975

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@lk-chen lk-chen requested a review from a team as a code owner May 14, 2025 07:33
@lk-chen lk-chen requested a review from kouroshHakha May 14, 2025 07:33
@lk-chen lk-chen added the go add ONLY when ready to merge, run all tests label May 14, 2025
@hainesmichaelc hainesmichaelc added the community-contribution Contributed by the community label May 15, 2025
@lk-chen lk-chen changed the title retry with model_config arg [Fix] retry with model_config arg May 15, 2025
@lk-chen lk-chen requested a review from GeneDer May 15, 2025 19:37
Copy link
Member

@GeneDer GeneDer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits and suggestions, but thanks for fixing it!

@masoudcharkhabi masoudcharkhabi added serve Ray Serve Related Issue llm labels May 15, 2025
Signed-off-by: Linkun <[email protected]>
@lk-chen lk-chen changed the title [Fix] retry with model_config arg [serve.llm][Fix] retry with model_config arg May 15, 2025
@kouroshHakha kouroshHakha enabled auto-merge (squash) May 15, 2025 23:20
Copy link
Member

@GeneDer GeneDer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments!

@kouroshHakha kouroshHakha merged commit dff4182 into ray-project:master May 15, 2025
6 checks passed
@lk-chen lk-chen deleted the vllm_chat_utils_api_change branch May 16, 2025 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-backlog community-contribution Contributed by the community go add ONLY when ready to merge, run all tests llm serve Ray Serve Related Issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Serve][LLM] unexpected keyword argument 'trust_remote_code' with vllm>0.8.5post1

5 participants