documents/leaderboard/IPAM-AMD-Car_Racing.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:fcc8a719c3dd112747c2ba0559a2b06d57e73c7836a421776ce18ef641a38d8b"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## DDQN with Dropout: CarRacing-v0\n",
      "\n",
      " * [Patrik Gerber*](https://www.linkedin.com/in/patrik-robert-gerber/), [University of Oxford](http://www.stats.ox.ac.uk/)\n",
      " * [Jiajing Guan*](https://www.linkedin.com/in/jiajing-guan-a5606379/), [George Mason University](https://www2.gmu.edu/)\n",
      " * [Elvis Nunez*](https://www.linkedin.com/in/elvis-nunez-ba7666102/), [Johns Hopkins University](https://engineering.jhu.edu/ams/)\n",
      " * [Kaman Phamdo*](https://kaman.phamdo.com), [University of Maryland](https://www.umd.edu/)\n",
      " * [Nicholas Malaya](http://nicholasmalaya.github.io/), [AMD Research](https://www.amd.com/en-us/who-we-are/corporate-information/research)\n",
      " \n",
      "\\*: denotes equal contribution\n",
      " \n",
      "## Summary\n",
      "\n",
      "A Double Deep Q-Network (DDQN) \\[2\\] was trained using a modified version of the DeepMind Atari2600 architecture\\[1\\], a form of curriculum learning\\[3\\], and dropout for regularization\\[4\\]. This model was trained for 900 iterations and tested on OpenAI's [CarRacing-v0](https://gym.openai.com/envs/CarRacing-v0/), where it attained a score of 906.67 \u00b1 23.6, sufficient to solve the game. The action space was discretized into 5 possible actions (turn left, turn right, accelerate, brake and no action). A video of the trained agent can be found [here](https://drive.google.com/file/d/1DQU4yCsq6nbVJB6WKoXlED9YFGDselIu/view).\n",
      "\n",
      "The code is available in the [GitHub repository](https://github.com/AMD-RIPS/RL-2018). The exact code used to generate the submission is in the **`src/DQN_Agent`** directory, in the **reproducibility** branch. From this directory, the trained model can be run using the command, \n",
      "\n",
      "<pre>\n",
      "python main.py validate\n",
      "</pre>\n",
      "\n",
      "The README.txt file in the **`src/DQN_Agent`** directory provides additional usage instructions. Please contact: <nicholas.malaya@amd.com> for further questions and correspondence. An ArXiv paper is in preparation and will detail this work (and the implications of it) in greater detail. The key summary points are:\n",
      "\n",
      "\n",
      "* 6 layer Double Deep Q Learning (DDQN)\\[2\\]\n",
      "* 3 hidden-layer convolutions:\n",
      "   * CONV, 32 8x8 kernels with stride 4, ReLU activations\n",
      "   * CONV, 64 4x4 kernels with stride 2, ReLU activations with dropout applied at rate 50%. \n",
      "   * CONV, 64 3x3 kernels with stride 1, ReLU activations\n",
      "* Two FC layers: \n",
      "   * 512 neurons, ReLU activations\n",
      "   * 5 output neurons, linear activations\n",
      "* Implemented using TensorFlow and OpenAI Gym\n",
      "\n",
      "## Attribution\n",
      "\n",
      "This project was jointly supported by [Advanced Micro Devices](https://www.amd.com) and NSF Grant DMS-0931852. The work was perfomed at the [Institute for Pure & Applied Mathematics](http://www.ipam.ucla.edu/) at UCLA during the 2018 [Research in Industrial Projects for Students (RIPS)](https://www.ipam.ucla.edu/programs/student-research-programs/research-in-industrial-projects-for-students-rips-2018/?tab=overview). \n",
      "\n",
      "\n",
      "## References\n",
      "\n",
      "1. [Playing Atari with Deep Reinforcement Learning](https://arxiv.org/pdf/1312.5602v1.pdf)\n",
      "2. [Deep Reinforcement Learning with Double Q-learning](https://arxiv.org/pdf/1509.06461.pdf)\n",
      "3. [Curriculum learning](https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/15972/Bengio%2C%202009%20Curriculum%20Learning.pdf)\n",
      "4. [Dropout: A Simple Way to Prevent Neural Networks from Overfitting ](http://jmlr.org/papers/v15/srivastava14a.html)\n",
      "\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}