[bugfix][mm] change get_num_encoder_tokens to get_num_encoder_embeds in recompute_schedule.py

[HF-001]

What this PR does / why we need it?

adapt to: vllm-project/vllm#30475.

just change get_num_encoder_tokens() to get_num_encoder_embeds() in recompute_schedule.py, which seems that it is currently not in use. The get_num_encoder_tokens() function in VLLM no longer exists.

• vLLM version: v0.12.0
• vLLM main: vllm-project/vllm@ad32e3e

[gemini-code-assist]

Code Review

This pull request aims to optimize the encoder cache manager by operating on embeddings instead of tokens, which should reduce memory consumption. The change in vllm_ascend/core/recompute_scheduler.py reflects this by updating how the encoder compute budget is restored upon preemption. While this change aligns with the PR's goal, it introduces a potential critical issue due to a unit mismatch. The encoder compute budget is restored using a count of embeddings, but it appears to be initialized and used elsewhere within the same file as a count of tokens. This inconsistency could lead to incorrect resource allocation and scheduling behavior.

[gemini-code-assist]

There appears to be a potential unit mismatch in the accounting of encoder_compute_budget. This change restores the budget using the number of embeddings (get_num_encoder_embeds), which implies the budget is tracked in units of embeddings.

However, encoder_compute_budget is initialized with self.max_num_encoder_input_tokens on line 116. This same configuration value is also used as num_encoder_tokens when calling self.kv_cache_manager.allocate_slots on lines 451-458. This suggests the budget is still measured in tokens.

If one embedding does not correspond to exactly one token, this will lead to incorrect budget calculations, potentially causing scheduling failures or overallocation.

For example:

• If encoder_compute_budget is in tokens, then restoring it with the number of embeddings is incorrect. It should be restored with the number of tokens corresponding to the preempted embeddings.
• If encoder_compute_budget is meant to be in embeddings, then the configuration max_num_encoder_input_tokens is misnamed and its usage for num_encoder_tokens in allocate_slots is likely incorrect, as that method probably expects a token count for allocating cross-attention KV cache.

This inconsistency could lead to critical issues. Please verify the units used for encoder_compute_budget throughout the scheduler and ensure they are consistent.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[wangxiyuan]

please make sure both vllm main and vllm v0.13.0 works. Thanks

[HF-001]

please make sure both vllm main and vllm v0.13.0 works. Thanks

@wangxiyuan Thank you for the reminder. I will resolve it soon.

[HF-001]

please make sure both vllm main and vllm v0.13.0 works. Thanks

@wangxiyuan hi, I have confirmed that the changes made to this PR are fully compatible with the latest versions such as vllm main, vllm v0.13.0.rc2, vllm v0.13.0.rc3, vllm v0.13.0.rc4, and vllm v0.13.0, vllm v0.14.0rc0, etc.