Code

Create INT8 KV Cache on Qserve#2446

Code

Open

dleunji wants to merge 2 commits intoNVIDIA:mainNVIDIA/TensorRT-LLM:mainfrom dleunji:feat/qserve-int8kvdleunji/TensorRT-LLM:feat/qserve-int8kvCopy head branch name to clipboard

+76-5

Commits

Commits on Nov 14, 2024

• Create INT8 KV Cache on Qserve

  dleunji

  committed
  dbec956

  Copy full SHA for dbec956

• fix unused module

  lkm2835

  committed
  6a6ec86

  Copy full SHA for 6a6ec86