Skip to content

vllm runner: implement tool calling (tool dispatch currently raises) #638

Description

@ttupper92618

Follow-up from the vLLM v1 engine scope (#580 arc) and PR #635: the vllm runner raises whenever task_params.tools is set, so the Qwen2.5 vLLM starter cards ship with tooling declared OFF (a platform-truth override; the models themselves support OpenAI tool calling). vLLM's OpenAI server supports tool calling natively (--enable-auto-tool-choice + per-family parsers), so the runner work is: pass tools through the proxied request, map the streamed tool-call deltas into Skulk ToolCallChunks, and configure the tool parser per family. When this lands, flip the [tooling] overrides in resources/inference_model_cards/Qwen--Qwen2.5-*-Instruct.toml back to the family default (each card carries a note pointing here).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions