Skip to content
/ CO2 Public

CO2: Efficient Distributed Training with Full Communication-Computation Overlap

License

Notifications You must be signed in to change notification settings

OpenNLPLab/CO2

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 

Repository files navigation

CO2

CO2: Efficient Distributed Training with Full Communication-Computation Overlap (Paper: https://arxiv.org/abs/2401.16265).

Please refer to the following repos for trying CO2:

  • fairseq-CO2: provides an example for utilizing CO2 within Fairseq framework.
  • fairscale-CO2: integrates CO2 within Fairscale framework.

Citation

If you find our work useful, please cite the following paper:

@article{sun2024co2,
  title={CO2: Efficient Distributed Training with Full Communication-Computation Overlap},
  author={Sun, Weigao and Qin, Zhen and Sun, Weixuan and Li, Shidi and Li, Dong and Shen, Xuyang and Qiao, Yu and Zhong, Yiran},
  journal={arXiv preprint arXiv:2401.16265},
  year={2024}
}

About

CO2: Efficient Distributed Training with Full Communication-Computation Overlap

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published