Code

[ascend]feat: support kv int8#2736

Code

Merged

lvhan028 merged 9 commits intoInternLM:mainInternLM/lmdeploy:mainfrom DeepLink-org:ascend_kv_int8DeepLink-org/lmdeploy:ascend_kv_int8Copy head branch name to clipboardDec 6, 2024

+199-11

Commits

Commits on Nov 13, 2024

• [ascend]feat: support kv int8 quant

  yao-fengchen

  committed
  15fda35

  Copy full SHA for 15fda35

• update code

  yao-fengchen

  committed
  ba5b58d

  Copy full SHA for ba5b58d

• fix error of argument missing

  yao-fengchen

  committed
  f52e8d3

  Copy full SHA for f52e8d3

• update params

  yao-fengchen

  committed
  f377fb3

  Copy full SHA for f377fb3

Commits on Dec 3, 2024

• Merge branch 'InternLM:main' into ascend_kv_int8

  jinminxi104

  authored
  80a0bbf

  Copy full SHA for 80a0bbf

Commits on Dec 4, 2024

• fix not iterable error when quant_meta is None.

  Reinerzhou

  committed
  3a6518e

  Copy full SHA for 3a6518e

• Update ascend_en_get_started.md for kvcache quant

  jinminxi104

  authored
  12b2305

  Copy full SHA for 12b2305

• Update ascend_cn_get_started.md for kvcache quant

  jinminxi104

  authored
  7db154a

  Copy full SHA for 7db154a

Commits on Dec 5, 2024

• Merge branch 'InternLM:main' into ascend_kv_int8

  jinminxi104

  authored
  24da10d

  Copy full SHA for 24da10d