Skip to content
/ DDG Public

A PyTorch implementation of the paper "Decoupled Parallel Backpropagation with Convergence Guarantee"

Notifications You must be signed in to change notification settings

slowbull/DDG

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Introduction

This repository is the PyTorch code for the paper Decoupled Parallel Backpropagation with Convergence Guarantee" by Zhouyuan Huo, Bin Gu, Qian Yang and Huang, Heng, ICML 2018.

For any question or suggestions, feel free to contact [email protected]

Dependencies

  • Python 3.6.2 (Anaconda)
  • PyTorch 0.3.1
  • CUDA 8.0

Run

Clone and cd into the repo directory, run:

python main.py ./data/cifar.python --dataset cifar10 --arch resnet_ddg --epochs 300 --learning_rate 0.01 --schedule 150 225 --gammas 0.1 0.1 --batch_size 128 --manualSeed 2 --depth 110 --splits 2

Result

Experimental results for optimizing ResNet110 on CIFAR10 using DDG.

Note that to obtain the speedup in the paper, the code should be run in a machine with 4 GPUs.

Citation

@article{huo2018decoupled,
  title={Decoupled Parallel Backpropagation with Convergence Guarantee},
  author={Huo, Zhouyuan and Gu, Bin and Yang, Qian and Huang, Heng},
  journal={arXiv preprint arXiv:1804.10574},
  year={2018}
}

About

A PyTorch implementation of the paper "Decoupled Parallel Backpropagation with Convergence Guarantee"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages