Inspired by Adrian Colyer and Denny Britz and Daniel Seita.

This contains my notes for research papers that I've read. Papers are arranged according to three broad categories and then further numbered on a (1) to (5) scale where a (1) means I have only barely skimmed it, while a (5) means I feel confident that I understand almost everything about the paper. Within a single year, these papers should be organized according to publication date. The links here go to my paper summaries if I have them, otherwise those papers are on my TODO list.

Deep Learning

2017

• Machine Learning as an Adversarial Service/ Learning Black-Box Adversarial Examples, arXiv (1)
• Full Resolution Image Compression with Recurrent Neural Networks, arXiv
• Linear Discriminant Generative Adversarial Networks, arXiv (1)
• Semi-supervised Knowledge Transfer For Deep Learning From Private Training Data ICLR 2017 Openview (1)
• Conditional Image Synthesis with Auxiliary Classifier GANs, arXiv (3)
• FractalNet: Ultra-Deep Neural Networks Without Residuals, ICLR 2017 (1)
• Making Neural Programming Architectures Generalize via Recursion, ICLR 2017 (1)
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer, ICLR 2017 (2)
• Understanding Deep Learning Requires Rethinking Generalization, ICLR 2017 (5)
• The energy landscape of a simple neural network, arXiv (1)
• Improved Training of Wasserstein GANs, arXiv (1)
• The Landscape of Deep Learning Algorithms, arXiv (1)
• ​Learning Chaotic Dynamics using Tensor Recurrent Neural Networks, arXiv (1)
• A Hitting Time Analysis of Stochastic Gradient Langevin Dynamics, arXiv(2)
• Non-Convex Learning via Stochastic Gradient Langevin Dynamics: A Nonasymptotic Analysis, arXiv(2)
• Generating Long-term Trajectories Using Deep Hierarchical Networks, arXiv(1)
• Structured Exploration via Deep Hierarchical Coordination, arXiv (1)

2016

• Super-resolution With Deep Convolutional Sufficient Statistics, ICLR 2016 (1)
• Generating Images with Perceptual Similarity Metrics based on Deep Networks, arXiv (1)
• Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, arXiv (1)
• Deep Unsupervised Clustering With Gaussian Mixture Variational Autoencoders, ICLR 2017, OpenView (2)
• Lossy Image Compression With Compressive Autoencoders, Openview (1)
• Variational Lossy Autoencoder, ICLR 2017, Openview (1)
• Train faster, generalize better: Stability of stochastic gradient descent, arXiv (1)
• NIPS 2016 Tutorial: Generative Adversarial Networks, arXiv (4)
• Using Fast Weights to Attend to the Recent Past, NIPS 2016 (2)
• Improved Techniques for Training GANs, NIPS 2016 (3)
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets, NIPS 2016 (2)
• Deep Residual Learning for Image Recognition, CVPR 2016 (1)
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, ICLR 2016 (2)
• Attention and Augmented Recurrent Neural Networks, Distill (3)
• Visualizing and Understanding Recurrent Networks, ICLR Workshop 2016 (1)

2015

• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, ICCV 2015 (2)
• Training Very Deep Networks, NIPS 2015 (2)
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, ICML 2015 (4)
• DRAW: A Recurrent Neural Network For Image Generation, ICML 2015 (2)
• Going Deeper with Convolutions, CVPR 2015 (1)
• The Loss Surfaces of Multilayer Networks, AISTATS 2015 (3)
• Very Deep Convolutional Networks for Large-Scale Image Recognition, ICLR 2015 (1)
• ADAM: A Method for Stochastic Optimization, ICLR 2015 (2)
• Explaining and Harnessing Adversarial Examples, ICLR 2015 (2)

2014

• Recurrent Neural Network Regularization, arXiv 2014 (1)
• Generative Adversarial Nets, NIPS 2014 (4)
• Recurrent Models of Visual Attention, NIPS 2014 (4)
• Deep Learning in Neural Networks: An Overview, arXiv (1)
• Visualizing and Understanding Convolutional Networks, ECCV 2014 (3)
• Revisiting Natural Gradient for Deep Networks, ICLR 2014 (1)

2013

• Representation Learning: A Review and New Perspectives IEEE transactions on pattern analysis and machine intelligence
• On the Difficulty of Training Recurrent Neural Networks, ICML 2013 (1)
• On the Importance of Initialization and Momentum in Deep Learning, ICML 2013 (2)

2012

• ImageNet Classification with Deep Convolutional Neural Networks, NIPS 2012 (5)
• Large Scale Distributed Deep Networks, NIPS 2012 (1)
• Training Deep and Recurrent Networks With Hessian-Free Optimization, Neural Networks: Tricks of the Trade, 2012 (1)

2011 and Earlier

• Deep Learning via Hessian-Free Optimization, ICML 2010 (2)

Reinforcement Learning

(Also includes imitation learning)

2017

• Bellman Gradient Iteration for Inverse Reinforcement Learning, ArXiv (1)
• Parameter Space Noise for Exploration, ArXiv (1)
• A Distributional Perspective on Reinforcement Learning, ArXiv (1)
• Proximal Policy Optimization Algorithms, OpenAI (1)
• Reinforcement Learning with Unsupervised Auxiliary Tasks, ICLR 2017, Openview , slides, code (2)
• UCB and InfoGain Exploration via Q-Ensembles, arXiv (1)
• Equivalence Between Policy Gradients and Soft Q-Learning, arXiv (1)
• Automatic Goal Generation for Reinforcement Learning Agents, arXiv (2)
• Multi-Level Discovery of Deep Options, arXiv (2)
• Iterative Noise Injection for Scalable Imitation Learning, arXiv (1)
• Bridging the Gap Between Value and Policy Based Reinforcement Learning, arXiv (2)
• Loss is its own Reward: Self-Supervision for Reinforcement Learning, arXiv (2)
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning, arXiv (5)
• One-Shot Imitation Learning, arXiv (3)
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World, arXiv (4)
• The Off-Switch Game, IJCAI 2017 (1)
• Constrained Policy Optimization, ICML 2017 (1)
• Reinforcement Learning with Deep Energy-Based Policies, ICML 2017 (1)
• Modular Multitask Reinforcement Learning with Policy Sketches, ICML 2017 (1)
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks, ICML 2017 (1)
• Curiosity-Driven Exploration by Self-Supervised Prediction, ICML 2017 (3)
• #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning, arXiv (4)
• RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning, arXiv (3)
• Learning to Predict Where to Look in Interactive Environments Using Deep Recurrent Q-Learning, arXiv (3)
• Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data, ICLR 2017 (1)
• Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening, ICLR 2017 (1)
• Q-Prop: Sample-Efficient Policy Gradient with an Off-Policy Critic, ICLR 2017 (2)
• Stochastic Neural Networks for Hierarchical Reinforcement Learning, ICLR 2017 (4)
• Third-Person Imitation Learning, ICLR 2017 (3)
• Deep Visual Foresight for Planning Robot Motion, ICRA 2017 (3)
• Multilateral Surgical Pattern Cutting in 2D Orthotropic Gauze with Deep Reinforcement Learning Policies for Tensioning, ICRA 2017 (5)
• Comparing Human-Centric and Robot-Centric Sampling for Robot Deep Learning from Demonstrations, ICRA 2017 (4)

2016

• Modular Multitask Reinforcement Learning with Policy Sketches, arXiv (1)
• Value Iteration Networks, NIPS 2016 (4)
• Generative Adversarial Imitation Learning, NIPS 2016 (3)
• VIME: Variational Information Maximizing Exploration, NIPS 2016 (3)
• Deep Exploration via Bootstrapped DQN, NIPS 2016 (1)
• Cooperative Inverse Reinforcement Learning, NIPS 2016 (1)
• Unifying Count-Based Exploration and Intrinsic Motivation, NIPS 2016 (1)
• A Connection Between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models, NIPS Workshop 2016 (1)
• Principled Option Learning in Markov Decision Processes, EWRL 2016 (4)
• Robot Grasping in Clutter: Using a Hierarchy of Supervisors for Learning from Demonstrations, CASE 2016 (4)
• Taming the Noise in Reinforcement Learning via Soft Updates, UAI 2016 (4)
• Asynchronous Methods for Deep Reinforcement Learning, ICML 2016 (4)
• Benchmarking Deep Reinforcement Learning for Continuous Control, ICML 2016 (4)
• Model-Free Imitation Learning with Policy Optimization, ICML 2016 (4)
• Graying the Black Box: Understanding DQNs, ICML 2016 (4)
• Control of Memory, Active Perception, and Action in Minecraft, ICML 2016 (2)
• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization, ICML 2016 (1)
• Dueling Network Architectures for Deep Reinforcement Learning, ICML 2016 (1)
• Learning Deep Neural Network Policies with Continuous Memory States, ICRA 2016 (2)
• Prioritized Experience Replay, ICLR 2016 (4)
• High-Dimensional Continuous Control Using Generalized Advantage Estimation, ICLR 2016 (4)
• Continuous Control with Deep Reinforcement Learning, ICLR 2016 (3)
• End-to-End Training of Deep Visuomotor Policies, JMLR 2016 (2)
• Deep Reinforcement Learning with Double Q-learning, AAAI 2016 (2)
• Mastering the Game of Go with Deep Neural Networks and Tree Search, Nature 2016 (1)

2015

• Learning Continuous Control Policies by Stochastic Value Gradients, NIPS 2015 (1)
• Deep Attention Recurrent Q-Network, NIPS Workshop 2015 (3)
• Deep Recurrent Q-Learning for Partially Observable MDPs, AAAI-SDMIA 2015 (5)
• Trust Region Policy Optimization, ICML 2015 (4)
• Probabilistic Inference for Determining Options in Reinforcement Learning, ICML Workshop 2015 (3)
• Massively Parallel Methods for Deep Reinforcement Learning, ICML Workshop 2015 (2)
• Human-Level Control Through Deep Reinforcement Learning, Nature 2015 (5)

2014

• Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning, NIPS 2014 (3)
• Learning Neural Network Policies with Guided Policy Search Under Unknown Dynamics, NIPS 2014 (1)
• Deterministic Policy Gradient Algorithms, ICML 2014 (2)

2013 and Earlier

• Playing Atari with Deep Reinforcement Learning, NIPS Workshop 2013 (5)
• A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning, Foundations and Trends in Machine Learning 2013 (4)
• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning, AISTATS 2011 (3)
• Maximum Entropy Inverse Reinforcement Learning, AAAI 2008 (4)
• Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning, Machine Learning 1992 (1)
• Active Perception and Reinforcement Learning, Neural Computation 1990 (3)

MCMC, Machine Learning, Robotics, and Miscellaneous Papers

2017

• Online Learning and Online Convex Optimization , By Shai Shalev-Shwartz (2)
• Hidden Physics Models Machine Learning of Nonlinear Partial Differential Equations, ArXiv (1)
• Neuroscience-Inspired Artificial Intelligence, DeepMind, Neuron (2)
• Learned Primal-dual Reconstruction, arXiv (2)
• Using dVRK Teleoperation to Facilitate Deep Learning of Automation Tasks for an Industrial Robot, CASE 2017 (4)
• Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics, RSS 2017 (3)
• A Conceptual Introduction to Hamiltonian Monte Carlo, arXiv (1)

2016

• A Multi-Batch L-BFGS Method for Machine Learning, Nips 2016 (3)
• Minimum-Information LQG Control Part I: Memoryless Controllers, CDC 2016 (2)
• Minimum-Information LQG Control Part II: Retentive Controllers, CDC 2016 (1)
• Gradient Descent Converges to Minimizers, COLT 2016 (3)
• Scalable Discrete Sampling as a Multi-Armed Bandit Problem, ICML 2016 (1)
• Dex-Net 1.0: A Cloud-Based Network of 3D Objects for Robust Grasp Planning Using a Multi-Armed Bandit Model with Correlated Rewards, ICRA 2016 (5)
• TSC-DL: Unsupervised Trajectory Segmentation of Multi-Modal Surgical Demonstrations with Deep Learning, ICRA 2016 (3)
• Automating Multi-Throw Multilateral Surgical Suturing with a Mechanical Needle Guide and Sequential Convex Optimization, ICRA 2016 (4)
• On Markov Chain Monte Carlo Methods for Tall Data, JMLR 2016 (I think?) (3)
• Revisiting Active Perception, arXiv (1)

2015

• A Complete Recipe for Stochastic Gradient MCMC, NIPS 2015 (2)
• Large-Scale Distributed Bayesian Matrix Factorization using Stochastic Gradient MCMC, KDD 2015 (1)
• Learning by Observation for Surgical Subtasks: Multilateral Cutting of 3D Viscoelastic and 2D Orthotropic Tissue Phantoms, ICRA 2015 (4)

2014

• Motion Planning with Sequential Convex Optimization and Convex Collision Checking, IJRR 2014 (1)
• Learning Accurate Kinematic Control of Cable-Driven Surgical Robots Using Data Cleaning and Gaussian Process Regression, CASE 2014 (2)
• Firefly Monte Carlo: Exact MCMC with Subsets of Data, UAI 2014 (2)
• Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget, ICML 2014 (4)
• Stochastic Gradient Hamiltonian Monte Carlo, ICML 2014 (3)
• Towards Scaling up Markov Chain Monte Carlo: An Adaptive Subsampling Approach, ICML 2014 (4)
• Planning Locally Optimal, Curvature-Constrained Trajectories in 3D Using Sequential Convex Optimization , ICRA 2014 (1)
• Autonomous Multilateral Debridement with the Raven Surgical Robot, ICRA 2014 (3)
• RRE: A Game-Theoretic Intrusion Response and Recovery Engine, IEEE Transactions on Parallel and Distributed Systems 2014 (4)

2013 and Earlier

• Generalization in Robotic Manipulation Through the Use of Non-Rigid Registration, ISRR 2013 (1)
• A Case Study of Trajectory Transfer Through Non-Rigid Registration for a Simplified Suturing Scenario, IROS 2013 (2)
• Finding Locally Optimal, Collision-Free Trajectories with Sequential Convex Optimization, RSS 2013 (3)
• Bayesian Learning via Stochastic Gradient Langevin Dynaimcs, ICML 2011 (4)
• MCMC Using Hamiltonian Dynamics, Handbook of Markov Chain Monte Carlo 2010 (2)
• Active Perception: Interactive Manipulation for Improving Object Detection, Technical Report 2010 (3)
• An Introduction to the Conjugate Gradient Method Without the Agonizing Pain, Technical Report, 1994 (3)
• Active Perception, Proceedings of the IEEE 1988 (2)