Code

Add 4-bit Adam#478

Code

Merged

msaroufim merged 22 commits intopytorch:mainpytorch/ao:mainfrom gau-nernst:4bit_adamgau-nernst/ao:4bit_adamCopy head branch name to clipboardJul 6, 2024

+492-198

Commits

Commits on Jul 3, 2024

• add 4bit

  gau-nernst

  committed
  af8cfe2

• rename

  gau-nernst

  committed
  fc53c07

Commits on Jul 4, 2024

• Merge branch 'pytorch:main' into 4bit_adam

  gau-nernst

  authored
  f2262f0

• simplify 4bit

  gau-nernst

  committed
  8f56895

• add rank1 scaling

  gau-nernst

  committed
  5b97b14

• add lpmm to benchmark

  gau-nernst

  committed
  37bfb12

• remove rank-1 scaling

  gau-nernst

  committed
  58af742

• update

  gau-nernst

  committed
  d402e01

Commits on Jul 5, 2024

• clean

  gau-nernst

  committed
  b694bdc

• rename

  gau-nernst

  committed
  189d500

• update test

  gau-nernst

  committed
  b870663

• fix

  gau-nernst

  committed
  27af0e3

• fix

  gau-nernst

  committed
  20ff160

• update adam

  gau-nernst

  committed
  3f42c3d

• add AdamW 4bit

  gau-nernst

  committed
  9bfce3a

• update

  gau-nernst

  committed
  82a578c

• remove lpmm from dev cuz CI can't compile

  gau-nernst

  committed
  bd6c5fb

• fix test

  gau-nernst

  committed
  eea1967

• update README

  gau-nernst

  committed
  b3c8c25

• Update README.md

  gau-nernst

  authored
  cbc3f05

• update readme. small fixes

  gau-nernst

  committed
  c4bb9e7

Commits on Jul 6, 2024

• remove zero padding

  gau-nernst

  committed
  cb6176e