Support for Cohere Command-R

[tombolano]

Cohere released the model "Command-R", a multilingual model optimized for long context tasks such as retrieval augmented generation (RAG) and using external APIs and tools.

Release note: https://txt.cohere.com/command-r/
Weights: https://huggingface.co/CohereForAI/c4ai-command-r-v01

The evaluation results shown by Cohere are really good, it beats Mixtral, Llama2 70B, and ChatGPT 3.5 for RAG and tool usage tasks.

In the llama.cpp repository there is a discussion (ggml-org/llama.cpp#6033) that provides some useful comments about its implementation.

[EwoutH]

They now also released a larger, 104B parameter model: C4AI Command R+

[zhang001122]

Yeah, c4ai-command R+ is really nedd trt llm to support, and no more work for llama2, as the development of llm is super fast,
Gays please hurry up

[user-0a]

Would also like support for this! Thank you for all of the hard work @ncomly-nvidia

[imnoahcook]

I would also like to request support for CommandR and CommandR+ plus, they are currently the best open source models.

[aikitoria]

Yes please. Command-R+ support is needed!

[here4dadata]

+1 for even more visibility

[syuoni]

Hi all,

The Command-R and Aya models have been supported on main branch. See: https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/commandr

I'm closing this issue. Thanks!