Code

Infer ext rebase#11

Code

Open

jinminxi104 wants to merge 8 commits intoinfer_extDeepLink-org/lmdeploy:infer_extfrom infer_ext_rebaseDeepLink-org/lmdeploy:infer_ext_rebaseCopy head branch name to clipboard

+684-5

Commits

Commits on Aug 20, 2024

• support ascend using infer_ext

  

  CyCle1024

  authored and

  jinminxi104

  committed
  af4e89c

  Copy full SHA for af4e89c

• fix(ascend): make infer_ext using TND format q,k,v in paged_token_attention

  

  CyCle1024

  authored and

  jinminxi104

  committed
  d0e901a

  Copy full SHA for d0e901a

• support ascend using infer_ext

  

  yao-fengchen

  authored and

  jinminxi104

  committed
  90ed8fb

  Copy full SHA for 90ed8fb

• support ascend mixtral

  

  yao-fengchen

  authored and

  jinminxi104

  committed
  f170fb8

  Copy full SHA for f170fb8

• feat: change infer_ext ops function param order (#2)

  

  CyCle1024

  authored and

  jinminxi104

  committed
  41d9985

  Copy full SHA for 41d9985

• feat: support ascend qwen2 and qwen2_moe (#6)

  Show description for 33f7048
  

  yao-fengchen

  authored and

  jinminxi104

  committed
  33f7048

  Copy full SHA for 33f7048

• ascend: align attention mask to 32bytes (#7)

  

  JackWeiw

  authored and

  jinminxi104

  committed
  55413f2

  Copy full SHA for 55413f2

• fix attn args (#9)

  jinminxi104

  committed
  a459fdc

  Copy full SHA for a459fdc