Skip to content

Add Cohere ChatV2 render endpoint - #53219

Merged
DarkLight1337 merged 3 commits into
vllm-project:mainfrom
andrewbcohere:andrewbcohere/chatv2-render
Aug 24, 2026
Merged

Add Cohere ChatV2 render endpoint#53219
DarkLight1337 merged 3 commits into
vllm-project:mainfrom
andrewbcohere:andrewbcohere/chatv2-render

Conversation

@andrewbcohere

@andrewbcohere andrewbcohere commented Aug 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Signed-off-by: Andrew Berneshawi andrewberneshawi@cohere.com

Purpose

Add /cohere/v2/chat/render endpoint to match other endpoint prompt render capability

Test Plan

Add unit and integration tests
pytest tests/entrypoints/cohere/test_api_router.py -q
python -m pytest tests/entrypoints/cohere/test_chat_v2.py -q

Test Result

pytest tests/entrypoints/cohere/test_api_router.py -q
............................ [100%]
28 passed, 3 warnings in 6.27s
python -m pytest tests/entrypoints/cohere/test_chat_v2.py -q
...... [100%]
6 passed, 15 warnings in 34.86s

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

@andrewbcohere
Initial render impl
02969a4 
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
@mergify mergify Bot added cohere Related to Cohere models frontend labels Aug 21, 2026
@mergify

mergify Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @andrewbcohere.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Aug 21, 2026
@andrewbcohere andrewbcohere changed the title Initial render impl Add Cohere chat v2 render endpoint Aug 21, 2026
@mergify mergify Bot removed the needs-rebase label Aug 21, 2026
@andrewbcohere andrewbcohere changed the title Add Cohere chat v2 render endpoint Add Cohere ChatV2 render endpoint Aug 21, 2026
@andrewbcohere
andrewbcohere marked this pull request as ready for review August 21, 2026 15:22
claude[bot]
claude Bot reviewed Aug 21, 2026
View reviewed changes

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify

mergify Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @andrewbcohere.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Aug 21, 2026
@DarkLight1337 DarkLight1337 added the verified Run pre-commit for new contributors without triggering other tests label Aug 24, 2026
@mergify mergify Bot removed the needs-rebase label Aug 24, 2026
@DarkLight1337
DarkLight1337 enabled auto-merge (squash) August 24, 2026 11:18
@DarkLight1337

DarkLight1337 commented Aug 24, 2026

Copy link
Copy Markdown
Member

/ci run

@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 24, 2026
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #85305 for commit bab66c5a551e.

@DarkLight1337
DarkLight1337 merged commit 41ae917 into vllm-project:main Aug 24, 2026
92 of 93 checks passed
am-cohere pushed a commit to am-cohere/vllm that referenced this pull request Sep 1, 2026
@andrewbcohere
Add Cohere ChatV2 render endpoint (vllm-project#53219)
1a35a1b
mikeshawcode pushed a commit to mikeshawcode/vllm that referenced this pull request Sep 1, 2026
@andrewbcohere @mikeshawcode
Add Cohere ChatV2 render endpoint (vllm-project#53219)
f5dea26 
Signed-off-by: mikeshawcode <michaelwshaw2@gmail.com>
vrdn-23 added a commit to vrdn-23/vllm that referenced this pull request Oct 8, 2026
@vrdn-23 @claude
Merge origin/main into pydantic-settings refactor: resolve envs.py co…
e959e8a 
…nflicts

Took the branch's vllm/envs.py wholesale: main's legacy TYPE_CHECKING block
and environment_variables dict are already superseded by the pydantic
*Settings models here. Then ported main's semantic delta from the 57
main-side commits that touched vllm/envs.py since 03a8d0b:

  vllm-project#52431 VLLM_USE_MODELSCOPE: description (bool already accepts "1")
  vllm-project#52216 prefix_cache_retention_interval promoted to an arg (see vllm-project#55353)
  vllm-project#48484 + VLLM_REPLICATE_EMBED
  vllm-project#52633 + VLLM_CPU_CI_ENV
  vllm-project#52182 - VLLM_TEST_FORCE_FP8_MARLIN
  vllm-project#52523 VLLM_API_KEY not a compile factor (branch already marked it)
  vllm-project#52998 VLLM_ALLREDUCE_USE_FLASHINFER default -> True
  vllm-project#53040 + VLLM_DISABLE_DSV4_MEGAMOE_SHARED_EXPERT_FUSION
  vllm-project#53053 + VLLM_KIMI_K3_GEMM_AR
  vllm-project#50501 + VLLM_XPU_INC_WNA16_BACKEND (case-sensitive Literal)
  vllm-project#52018 + VLLM_B12X_MOE_FP4_FORCE_A16
  vllm-project#43018 + VLLM_KV_OFFLOAD_MAX_BATCH_DESCRIPTORS
  vllm-project#51718 VLLM_KV_CACHE_LAYOUT Literal widened to 8 layouts + description
  vllm-project#51896 + VLLM_MAX_MEDIA_DOWNLOAD_SIZE_MB
  vllm-project#53219 VLLM_ENABLE_COHERE_API: description
  vllm-project#53559 - VLLM_TRITON_ATTN_USE_TD shim (already absent on the branch)
  vllm-project#53561 VLLM_MAX_AUDIO_CLIP_FILESIZE_MB: description
  vllm-project#53378 VLLM_ELASTIC_EP_SCALE_UP_LAUNCH not a compile factor
  vllm-project#53141 VLLM_ROCM_USE_AITER_FP4_ASM_GEMM removed; restored by vllm-project#57055
  vllm-project#54160 + VLLM_ENABLE_HPC_OPS
  vllm-project#53536 + VLLM_XPU_FORCE_N_CONTIG_WEIGHT
  vllm-project#53576 + VLLM_ALLREDUCE_USE_FLASHINFER_PCIE_IPC
  vllm-project#54632 + VLLM_MAX_EMBED_DECODE_BYTES (not a compile factor)
  vllm-project#54579 + VLLM_ENABLE_SCALE_OUT_ENDPOINTS; removed by vllm-project#55176 (net zero)
  vllm-project#53491 VLLM_GPU_SYNC_CHECK: description
  vllm-project#54371 + VLLM_PLE_CPU_OFFLOAD; reworked vllm-project#56512, dropped vllm-project#57937 (net zero)
  vllm-project#48247 VLLM_ROCM_USE_AITER_CUSTOM_AR: description
  vllm-project#55353 - VLLM_MM_HASHER_ALGORITHM (+ _lower_mm_hasher),
         - VLLM_PREFIX_CACHE_RETENTION_INTERVAL
  vllm-project#54985 - VLLM_NIXL_EP_MAX_NUM_RANKS
  vllm-project#56512 VLLM_PLE_CPU_OFFLOAD rework (net zero, see vllm-project#54371)
  vllm-project#55176 - VLLM_ENABLE_SCALE_OUT_ENDPOINTS (net zero, see vllm-project#54579)
  vllm-project#53940 + VLLM_ROCM_USE_AITER_MOE_SITUV2 (case-insensitive Literal)
  vllm-project#41074 + VLLM_SKIP_VERSION_SUFFIX (not a compile factor)
  vllm-project#48599 - envs.validate_environ (moved to Platform.validate_environ,
         which reads the branch's environment_variables shim)
  vllm-project#52136 pydocstyle docstring restyle of legacy helpers (no-op here)
  vllm-project#57212 docstring rejoin of legacy helpers (no-op here)
  vllm-project#56913 + FORCE_COLOR (alias, non-"0" is True, not a compile factor)
  vllm-project#57055 VLLM_ROCM_USE_AITER_FP4_ASM_GEMM restored (net zero, see vllm-project#53141)
  vllm-project#57142 shm buffer name autogen -> new _generate_shm_name() (<=30 chars)
  vllm-project#56685 VLLM_HUMMING_INPUT_QUANT_CONFIG: description
  vllm-project#57937 - VLLM_PLE_CPU_OFFLOAD (net zero, see vllm-project#54371)
  vllm-project#50814 + VLLM_MXFP4_EMULATION_DEQUANT_AT_LOAD
  vllm-project#57428 + VLLM_ENABLE_GEMM_RS, - VLLM_KIMI_K3_GEMM_RS
  vllm-project#51600 - VLLM_XPU_ENABLE_XPU_GRAPH
  vllm-project#58582 + VLLM_TRITON_JIT_WARMUP_NUM_THREADS (not a compile factor)
  vllm-project#55145 + VLLM_XPU_PP_MICROBATCH
  vllm-project#58201 VLLM_ROCM_USE_AITER_MOE_SITUV2 6-way choice,
         - VLLM_ROCM_USE_AITER_MOE_SITUV2_A8W4
  vllm-project#58919 + VLLM_MOONCAKE_CONNECTOR_TIMEOUT
  vllm-project#56861 + VLLM_ROCM_AITER_MLA_DCP_VERIFY (case-insensitive Literal)
  vllm-project#53492 + VLLM_ROCM_USE_AITER_TRITON_SPARSE_MLA
  vllm-project#57991 VLLM_DEEPEP_V2_ALLOW_HYBRID_MODE default True (branch already
         True) + description
  vllm-project#58819 + VLLM_ROCM_USE_AITER_MOE_A4W4_DSV4 (tri-state, bool(int(v)))
  vllm-project#50300 + VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT
  vllm-project#54024 VLLM_CPU_INT4_W4A8: description
  vllm-project#59781 - VLLM_ENGINE_ITERATION_TIMEOUT_S
  vllm-project#57995 + VLLM_FLASHINFER_MOE_A2A_LOW_PRECISION_COMBINE
  vllm-project#59965 VLLM_ROCM_AITER_MLA_DCP_VERIFY default route (final "auto")

Also VLLM_SSM_CONV_STATE_LAYOUT's description follows vllm-project#51718's layout rename.

Parity: 311 branch fields vs 311 main runtime entries, no difference in
either direction. Every var's value matches main's lambdas in an empty
environment and under 15 override scenarios, except case normalization of
the two lowercased ROCm choices (both callers already call .lower()).

Intentionally not ported: main's new consumer tests that match
env_with_choices' error text (test_invalid_backend_value_raises,
test_dcp_verify_env_rejects_unknown_route, and the ROCm-gated
test_aiter_moe_situv2_rejects_unknown_activation) fail on this branch:
pydantic's ValidationError names the field, not the env var. Invalid values
are still rejected. Tracked as a branch-wide follow-up.

Other conflicts and merge fallout:
- requirements/docs.txt, requirements/test/{cpu,cuda,rocm,xpu}.txt: took
  main's lockfiles and regenerated them with the pip-compile pre-commit
  hooks; they differ from main only by the branch's pydantic-settings lines.
- tests/test_envs_pydantic.py: dropped the mm-hasher test, pointed the
  validate_environ tests at Platform.validate_environ, updated the shm-name
  prefix assertion.
- check-test-tethering (new on main): wired test_envs_pydantic.py into the
  CPU params/env job; allowlisted tests/docs/test_env_vars_gen.py, which
  cannot run from a CI job's tests/ cwd (tests/docs shadows docs/).

Tests: pytest tests/test_envs.py tests/test_envs_pydantic.py
tests/docs/test_env_vars_gen.py -> 55 passed; pre-commit run --files on all
changed files -> clean.

AI assistance (Claude) was used for this merge resolution.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
@DarkLight1337 DarkLight1337 DarkLight1337 approved these changes
@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat
@aarnphm aarnphm Awaiting requested review from aarnphm
@NickLucche NickLucche Awaiting requested review from NickLucche
@AndreasKaratzas AndreasKaratzas Awaiting requested review from AndreasKaratzas

Assignees

No one assigned

Labels

cohere Related to Cohere models frontend ready ONLY add when PR is ready to merge/full CI is needed verified Run pre-commit for new contributors without triggering other tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@andrewbcohere @DarkLight1337