fix(browsecomp): accept both vLLM /tokenize response shapes

[mjmikulski]

save_model_call_using_vllm_tokenize_endpoint reads len(response["tokens"]), which only exists on older vLLM builds; mainstream >=0.19.1 returns {"count": N, "max_model_len": ...} and omits the token list unless token ids are requested, so the pre-call context-reset estimation dies with KeyError: 'tokens' on every sample. Prefer the explicit count, fall back to the token list, and fail loudly when neither is present.

What does this PR do?

Checklist

• I have read the contributing guidelines.
• The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
• Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
• Pre-commit checks pass locally (pre-commit run --all-files) (so CI lint/format/copyright pass).
• All commits have DCO sign-off (git commit -s) (so the DCO check passes).

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[mjmikulski]

/claude review

[claude]

SHIP — LGTM, no reliability concerns.

Focused robustness fix for _count_prompt_tokens: vLLM's /tokenize response omits the tokens list unless token ids are requested (mainstream >=0.19.1 returns {"count": N, ...}), so the old len(tokenize_response["tokens"]) would KeyError against current vLLM and silently break context-reset decisions. New helper prefers the explicit count, falls back to len(tokens) for older builds, and raises loudly (not a silent 0) when neither is present — the right failure mode for a value feeding reset logic. Test asserts all four cases including count-wins-over-tokens and the loud-failure path. No async, API-surface, or scoring concerns.

[mjmikulski]

/ok to test 6c3a546