Skip to content

Revert "[Misc] toy_proxy_server handle min_tokens" (#39706) #40138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
vllm-agent wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Revert "[Misc] toy_proxy_server handle min_tokens" (#39706) #40138

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions tests/v1/kv_connector/nixl_integration/toy_proxy_server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,6 @@ async def send_request_to_service(
req_data["max_completion_tokens"] = 1
if "stream_options" in req_data:
del req_data["stream_options"]
# These args are not supported for P
min_tokens = req_data.pop("min_tokens", None)
min_completion_tokens = req_data.pop("min_completion_tokens", None)
headers = {
"Authorization": f"Bearer {os.environ.get('OPENAI_API_KEY')}",
"X-Request-Id": request_id,
Expand All @@ -190,10 +187,6 @@ async def send_request_to_service(
# otherwise, it would http.ReadError
await response.aread()

# Add back the min_tokens and min_completion_tokens so D can use them
req_data["min_tokens"] = min_tokens
req_data["min_completion_tokens"] = min_completion_tokens

return response


Expand Down