Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] 希望加入生成参数allowed_token_ids #2905

Open
Marquis03 opened this issue Dec 17, 2024 · 0 comments
Open

[Feature] 希望加入生成参数allowed_token_ids #2905

Marquis03 opened this issue Dec 17, 2024 · 0 comments

Comments

@Marquis03
Copy link

Motivation

vLLM中存在生成参数allowed_token_ids,可以控制LLM在推理时仅从给定的token中生成。
类似于LMDeploy中bad_token_ids的反选?

我在LMDeploy的bad_token_ids加入了除我希望生成的token以外所有token的ids,但这会导致程序崩溃。

希望能在LMDeploy的生成参数中加入类似allowed_token_ids功能的参数,或者能否告诉我现有的版本能够如何实现这样的功能。

Related resources

vLLM: https://github.com/vllm-project/vllm/blob/main/vllm/sampling_params.py#L87

Additional context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@Marquis03 and others