Using pytorch to implement DQN (Deep Q Network) / DDQN (Double DQN) / Atari DDQN.

Dependency

• python 3.6
• pytorch 0.4+
• tensorboard
• gym

Train

DQN:

dqn.py --train --env CartPole-v0

DDQN:

ddqn.py --train --env CartPole-v0

Atari DDQN:

atari_ddqn.py --train --env PongNoFrameskip-v4

Parameters need to be manually adjusted within the file.

You can use the tensorboard to see the training.

tensorboard --logdir=out/CartPole-v0-run0

Test

For dqn.py, ddqn.py and atari_ddqn.py, you use --test like this:

ddqn.py --test --env CartPole-v0 --model_path out/CartPole-v0-run23/model_best.pkl

It will render graphical interface.

Result

CartPole-v0

I trained CartPole-v0 environment with dqn and ddqn. (Blue is dqn, and orange is ddqn)

PongNoFrameskip-v4

Training Atari game PongNoFrameskip-v4 two million step.

Reference

• paper Deep Reinforcement Learning with Double Q-learning
• paper Playing Atari with Deep Reinforcement Learning
• repository RL-Adventure