Code

DeepSpeed Integration#5954

Code

Merged

SeanNaren merged 68 commits intomasterLightning-AI/pytorch-lightning:masterfrom feature/deepspeed2Lightning-AI/pytorch-lightning:feature/deepspeed2Copy head branch name to clipboardFeb 17, 2021

Commits

Commits on Feb 13, 2021

• Add initial deepspeed changes

  

  SeanNaren

  committed
  2163721

  Copy full SHA for 2163721

Commits on Feb 14, 2021

• Address code review

  

  SeanNaren

  committed
  14c7b61

  Copy full SHA for 14c7b61

• Merge branch 'master' into feature/deepspeed2

  SeanNaren

  authored
  11a8ab9

  Copy full SHA for 11a8ab9

Commits on Feb 15, 2021

• Move static method outside of function

  

  SeanNaren

  committed
  e8ab7fd

  Copy full SHA for e8ab7fd

• Fixes

  

  SeanNaren

  committed
  5b1e091

  Copy full SHA for 5b1e091

• Add missing annotation

  

  SeanNaren

  committed
  d6d90be

  Copy full SHA for d6d90be

• Remove seed setting

  

  SeanNaren

  committed
  ab4efdf

  Copy full SHA for ab4efdf

• Merge branch 'master' into feature/deepspeed2

  SeanNaren

  authored
  91abbc4

  Copy full SHA for 91abbc4

• Doc changes

  

  SeanNaren

  committed
  bffb916

  Copy full SHA for bffb916

• Doc changes, add address reviews

  

  SeanNaren

  committed
  5c4444d

  Copy full SHA for 5c4444d

• Fix docs

  

  SeanNaren

  committed
  978470c

  Copy full SHA for 978470c

• Try fixing issue by moving to torch adam

  

  SeanNaren

  committed
  41389b9

  Copy full SHA for 41389b9

Commits on Feb 16, 2021

• Clean up check

  

  SeanNaren

  committed
  b1cf9c0

  Copy full SHA for b1cf9c0

• Changes, better APIs!

  

  SeanNaren

  committed
  beea306

  Copy full SHA for beea306

• Merge branch 'master' into feature/deepspeed2

  

  SeanNaren

  committed
  64da158

  Copy full SHA for 64da158

• Add wrapper, swap to git install revision

  

  SeanNaren

  committed
  2c659fe

  Copy full SHA for 2c659fe

• Add special test

  

  SeanNaren

  committed
  4b295cd

  Copy full SHA for 4b295cd

• Merge branch 'master' into feature/deepspeed2

  SeanNaren

  authored
  bb39215

  Copy full SHA for bb39215

• Add warning

  

  SeanNaren

  committed
  caeac52

  Copy full SHA for caeac52

• Address review

  

  SeanNaren

  committed
  fcc2f99

  Copy full SHA for fcc2f99

• Add better disclaimer

  

  SeanNaren

  committed
  91cc1e0

  Copy full SHA for 91cc1e0

• Merge branch 'master' into feature/deepspeed2

  SeanNaren

  authored
  05c4c51

  Copy full SHA for 05c4c51

• Turn off ZeRO for testing due to compilation

  

  SeanNaren

  committed
  37542d6

  Copy full SHA for 37542d6

• Add description on modifying parameters via the plugin

  

  SeanNaren

  committed
  a11695d

  Copy full SHA for a11695d

• Doc strings clear

  

  SeanNaren

  committed
  f4585f0

  Copy full SHA for f4585f0

• Merge branch 'master' into feature/deepspeed2

  Show description for 887112d
  

  SeanNaren

  committed
  887112d

  Copy full SHA for 887112d

• Small doc fixes

  

  SeanNaren

  committed
  52b654d

  Copy full SHA for 52b654d

• Merge branch 'master' into feature/deepspeed2

  Show description for 17e2252
  

  SeanNaren

  committed
  17e2252

  Copy full SHA for 17e2252

• Fix hash, reduce test

  

  SeanNaren

  committed
  b06bd2c

  Copy full SHA for b06bd2c

• Added CI change

  

  SeanNaren

  committed
  83535fb

  Copy full SHA for 83535fb

• Move to azure pipeline

  

  SeanNaren

  committed
  a1e487d

  Copy full SHA for a1e487d

• Fix test name

  

  SeanNaren

  committed
  535800c

  Copy full SHA for 535800c

• Merge branch 'master' into feature/deepspeed2

  Show description for f326068
  

  SeanNaren

  committed
  f326068

  Copy full SHA for f326068

• Add missing flag

  

  SeanNaren

  committed
  471ccdf

  Copy full SHA for 471ccdf

• Remove sudo...

  

  SeanNaren

  committed
  e458c19

  Copy full SHA for e458c19

• Try conda instead

  

  SeanNaren

  committed
  9826ca8

  Copy full SHA for 9826ca8

• Swap to conda base

  

  SeanNaren

  committed
  45ea290

  Copy full SHA for 45ea290

• Try suggested install

  

  SeanNaren

  committed
  9272a95

  Copy full SHA for 9272a95

• Apply suggestions from code review

  Borda

  authored
  e06ec29

  Copy full SHA for e06ec29

• Apply suggestions from code review

  Borda

  authored
  41cca05

  Copy full SHA for 41cca05

Commits on Feb 17, 2021

• Revert "Apply suggestions from code review"

  Show description for 37f2d9d
  

  SeanNaren

  committed
  37f2d9d

  Copy full SHA for 37f2d9d

• Revert "Apply suggestions from code review"

  Show description for c015530
  

  SeanNaren

  committed
  c015530

  Copy full SHA for c015530

• Remove setter

  

  SeanNaren

  committed
  054b320

  Copy full SHA for 054b320

• Merge branch 'master' into feature/deepspeed2

  tchaton

  committed
  87e0a92

  Copy full SHA for 87e0a92

• Address most review

  

  SeanNaren

  committed
  301c32d

  Copy full SHA for 301c32d

• Move out function, remove DeepSpeed from requirements

  

  SeanNaren

  committed
  3fda074

  Copy full SHA for 3fda074

• Install deepspeed/mpi4py within container

  

  SeanNaren

  committed
  d969d28

  Copy full SHA for d969d28

• Use special tests, move to master commit for deepspeed

  

  SeanNaren

  committed
  5d993ec

  Copy full SHA for 5d993ec

• Export path

  

  SeanNaren

  committed
  62f3048

  Copy full SHA for 62f3048

• Force compile to happen first

  

  SeanNaren

  committed
  ec79096

  Copy full SHA for ec79096

• Remove!

  

  SeanNaren

  committed
  894a6dd

  Copy full SHA for 894a6dd

• Debugging ninja

  

  SeanNaren

  committed
  e735358

  Copy full SHA for e735358

• Fix error in optimizer step logic

  

  SeanNaren

  committed
  60063f2

  Copy full SHA for 60063f2

• Attempt to fix symbolic link

  

  SeanNaren

  committed
  5aa9acc

  Copy full SHA for 5aa9acc

• Reverse to aid debugging

  

  SeanNaren

  committed
  b68a539

  Copy full SHA for b68a539

• Export path again

  

  SeanNaren

  committed
  0878927

  Copy full SHA for 0878927

• Clean up mess

  

  SeanNaren

  committed
  7dd17d3

  Copy full SHA for 7dd17d3

• var

  Borda

  committed
  3450eac

  Copy full SHA for 3450eac

• Revert "var"

  Show description for 0b9e7d5
  

  SeanNaren

  committed
  0b9e7d5

  Copy full SHA for 0b9e7d5

• Address review, add todo

  

  SeanNaren

  committed
  e0a2d6b

  Copy full SHA for e0a2d6b

• Add note about unsupported functionality

  

  SeanNaren

  committed
  c565bb1

  Copy full SHA for c565bb1

• Merge branch 'master' into feature/deepspeed2

  

  SeanNaren

  committed
  a9ba173

  Copy full SHA for a9ba173

• Update docs/source/advanced/multi_gpu.rst

  Show description for fbaf86f
  

  SeanNaren

  and

  awaelchli

  authored
  fbaf86f

  Copy full SHA for fbaf86f

• Address review

  

  SeanNaren

  committed
  1e7dcd6

  Copy full SHA for 1e7dcd6

• Remove import

  

  SeanNaren

  committed
  ea1d78c

  Copy full SHA for ea1d78c

• Add tmpdir

  

  SeanNaren

  committed
  10da87e

  Copy full SHA for 10da87e

• Add note

  

  SeanNaren

  committed
  f789b77

  Copy full SHA for f789b77

• Add note

  

  SeanNaren

  committed
  31d6267

  Copy full SHA for 31d6267