Code

Improve our mup implementation#837

Code

Draft

Quentin-Anthony wants to merge 5 commits intomainEleutherAI/gpt-neox:mainfrom improved-mupEleutherAI/gpt-neox:improved-mupCopy head branch name to clipboard

+84-16

Commits

Commits on Mar 10, 2023

• Add remaining mup params from paper's list. Add prototype support for deferred init

  Quentin-Anthony

  committed
  bf59c12

Commits on Mar 15, 2023

• Merge branch 'main' into improved-mup

  Quentin-Anthony

  authored
  5f09348

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  585efea

Commits on Apr 12, 2023

• Fixed syntax errors in training.py and added requirements-mup file

  curt-tigges

  committed
  4ab0c08

Commits on Apr 24, 2023

• Fixed misc bugs in mup implementation

  curt-tigges

  committed
  54f82ca