This repository aims to keep an up-to-date list of research on explainable reinforcement learning (XRL). If you find this helpful, you can give this repository a star and share it.
Missing resources, issues, or questions? Please open an issue here.
- Awesome Explainable Reinforcement Learning. Link
| #/Link | Title | Venue/Journal | Year |
|---|---|---|---|
| 1 | Explainable Reinforcement Learning: A Survey and Comparative Review | ACM Comput. Surv. | 2024 |
| 2 | Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunities | ACM Comput. Surv. | 2024 |
| 3 | A Survey of Global Explanations in Reinforcement Learning | Explainable Agency in Artificial Intelligence | 2024 |
| 4 | A survey on interpretable reinforcement learning | Mach. Learn. | 2024 |
| 5 | Explainable reinforcement learning (XRL): a systematic literature review and taxonomy | Mach. Learn. | 2024 |
| 6 | Explainable and Interpretable Reinforcement Learning for Robotics | SLAIML | 2024 |
| 7 | Explainability in Deep Reinforcement Learning, a Review into Current Methods and Applications | ACM Comput. Surv. | 2023 |
| 8 | A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges | CoRR | 2023 |
| 9 | Explainable reinforcement learning for broad-XAI: a conceptual framework and survey | Neural Comput. Appl. | 2023 |
| 10 | Explainable Deep Reinforcement Learning: State of the Art and Challenges | ACM Comput. Surv. | 2022 |
| 11 | Explainability in reinforcement learning: perspective and position | CoRR | 2022 |
| 12 | Explainable AI and Reinforcement Learning - A Systematic Review of Current Approaches and Trends | Frontiers Artif. Intell. | 2021 |
| 13 | Explainability in deep reinforcement learning | Knowl. Based Syst. | 2021 |
| 14 | Explainable Reinforcement Learning: A Survey | CD-MAKE | 2020 |
| 15 | Reinforcement Learning Interpretation Methods: A Survey | IEEE Access | 2020 |
| #/Link | Title | Venue/Journal | Year |
|---|---|---|---|
| 1 | Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomes | AAAI | 2024 |
| 2 | ASAP: Attention-Based State Space Abstraction for Policy Summarization | ACML | 2024 |
| 3 | Causal State Distillation for Explainable Reinforcement Learning | CLeaR | 2024 |
| 4 | Discovering Behavioral Modes in Deep Reinforcement Learning Policies Using Trajectory Clustering in Latent Space | CoRR | 2024 |
| 5 | Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents | CoRR | 2024 |
| 6 | Interpretable and Editable Programmatic Tree Policies for Reinforcement Learning | CoRR | 2024 |
| 7 | Detection of Important States through an Iterative Q-value Algorithm for Explainable Reinforcement Learning | HICSS | 2024 |
| 8 | ''You just can't go around killing people'' Explaining Agent Behavior to a Human Terminator | ICML Workshop MHFAIA | 2024 |
| 9 | Deep Reinforcement Learning Behavioral Mode Switching Using Optimal Control Based on a Latent Space Objective | MED | 2024 |
| 10 | Explaining Deep Reinforcement Learning Policies with SHAP, Decision Trees, and Prototypes | MED | 2024 |
| 11 | Local Explanations for Reinforcement Learning | AAAI | 2023 |
| 12 | Explainable Reinforcement Learning Based on Q-Value Decomposition by Expected State Transitions | AAAI-MAKE | 2023 |
| 13 | GANterfactual-RL: Understanding Reinforcement Learning Agents' Strategies through Visual Counterfactual Explanations | AAMAS | 2023 |
| 14 | Interpreting a deep reinforcement learning model with conceptual embedding and performance analysis | Appl. Intell. | 2023 |
| 15 | Deep Explainable Relational Reinforcement Learning: A Neuro-Symbolic Approach | ECML PKDD | 2023 |
| 16 | Inherently Interpretable Deep Reinforcement Learning Through Online Mimicking | EXTRAAMAS | 2023 |
| 17 | A Closer Look at Reward Decomposition for High-Level Robotic Explanations | ICDL | 2023 |
| 18 | Towards Interpretable Deep Reinforcement Learning with Human-Friendly Prototypes | ICLR | 2023 |
| 19 | Explaining Reinforcement Learning with Shapley Values | ICML | 2023 |
| 20 | Counterfactual Explanation Policies in RL | ICML Workshop on Counterfactuals in Minds and Machines | 2023 |
| 21 | Explaining Black Box Reinforcement Learning Agents Through Counterfactual Policies | IDA | 2023 |
| 22 | Extracting Decision Tree From Trained Deep Reinforcement Learning in Traffic Signal Control | IEEE Trans. Comput. Soc. Syst. | 2023 |
| 23 | Real-Time Counterfactual Explanations For Robotic Systems With Multiple Continuous Outputs | IFAC-PapersOnLine | 2023 |
| 24 | Explainable Multi-Agent Reinforcement Learning for Temporal Queries | IJCAI | 2023 |
| 25 | Explainable Reinforcement Learning via a Causal World Model | IJCAI | 2023 |
| 26 | Unveiling Concepts Learned by a World-Class Chess-Playing Agent | IJCAI | 2023 |
| 27 | Extracting tactics learned from self-play in general games | Inf. Sci. | 2023 |
| 28 | Learning state importance for preference-based reinforcement learning | Mach. Learn. | 2023 |
| 29 | Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction | NeurIPS | 2023 |
| 30 | State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understanding | NeurIPS | 2023 |
| 31 | StateMask: Explaining Deep Reinforcement Learning through State Mask | NeurIPS | 2023 |
| 32 | Comparing explanations in RL | Neural Comput. Appl. | 2023 |
| 33 | Explainable robotic systems: understanding goal-driven actions in a reinforcement learning scenario | Neural Comput. Appl. | 2023 |
| 34 | Hierarchical goals contextualize local reward decomposition explanations | Neural Comput. Appl. | 2023 |
| 35 | Achieving efficient interpretability of reinforcement learning via policy distillation and selective input gradient regularization | Neural Networks | 2023 |
| 36 | Model tree methods for explaining deep reinforcement learning agents in real-time robotic applications | Neurocomputing | 2023 |
| 37 | Integrating Policy Summaries with Reward Decomposition for Explaining Reinforcement Learning Agents | PAAMS | 2023 |
| 38 | Contrastive Visual Explanations for Reinforcement Learning via Counterfactual Rewards | xAI | 2023 |
| 39 | IxDRL: A Novel Explainable Deep Reinforcement Learning Toolkit Based on Analyses of Interestingness | xAI | 2023 |
| 40 | "I Don't Think So": Summarizing Policy Disagreements for Agent Comparison | AAAI | 2022 |
| 41 | CAPS: Comprehensible Abstract Policy Summaries for Explaining Reinforcement Learning Agents | AAMAS | 2022 |
| 42 | Interpretable Preference-based Reinforcement Learning with Tree-Structured Reward Functions | AAMAS | 2022 |
| 43 | Lazy-MDPs: Towards Interpretable RL by Learning When to Act | AAMAS | 2022 |
| 44 | Explaining Online Reinforcement Learning Decisions of Self-Adaptive Systems | ACSOS | 2022 |
| 45 | Analysis of Explainable Goal-Driven Reinforcement Learning in a Continuous Simulated Environment | Algorithms | 2022 |
| 46 | BEERL: Both Ends Explanations for Reinforcement Learning | Applied Sciences | 2022 |
| 47 | Energy-Efficient Driving for Adaptive Traffic Signal Control Environment via Explainable Reinforcement Learning | Applied Sciences | 2022 |
| 48 | Concept Learning for Interpretable Multi-Agent Reinforcement Learning | CoRL | 2022 |
| 49 | Comparing Strategies for Visualizing the High-Dimensional Exploration Behavior of CPS Design Agents | DESTION | 2022 |
| 50 | InAction: Interpretable Action Decision Making for Autonomous Driving | ECCV | 2022 |
| 51 | Enhanced Oblique Decision Tree Enabled Policy Extraction for Deep Reinforcement Learning in Power System Emergency Control | Electric Power Systems Research | 2022 |
| 52 | Attributation Analysis of Reinforcement Learning-Based Highway Driver | Electronics | 2022 |
| 53 | Multi-objective Genetic Programming for Explainable Reinforcement Learning | EuroGP | 2022 |
| 54 | Deep-Learning-based Fuzzy Symbolic Processing with Agents Capable of Knowledge Communication | ICAART | 2022 |
| 55 | Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations | ICLR | 2022 |
| 56 | POETREE: Interpretable Policy Learning with Adaptive Decision Trees | ICLR | 2022 |
| 57 | Programmatic Reinforcement Learning without Oracles | ICLR | 2022 |
| 58 | Explaining Reinforcement Learning Policies through Counterfactual Trajectories | ICML Workshop on HILL | 2022 |
| 59 | Mean-variance Based Risk-sensitive Reinforcement Learning with Interpretable Attention | ICMVA | 2022 |
| 60 | Towards Interpretable Deep Reinforcement Learning Models via Inverse Reinforcement Learning | ICPR | 2022 |
| 61 | Explaining Intelligent Agent's Future Motion on Basis of Vocabulary Learning With Human Goal Inference | IEEE Access | 2022 |
| 62 | Interpretable Autonomous Flight Via Compact Visualizable Neural Circuit Policies | IEEE Robotics Autom. Lett. | 2022 |
| 63 | Explainable AI in Deep Reinforcement Learning Models for Power System Emergency Control | IEEE Trans. Comput. Soc. Syst. | 2022 |
| 64 | Hierarchical Program-Triggered Reinforcement Learning Agents for Automated Driving | IEEE Trans. Intell. Transp. Syst. | 2022 |
| 65 | Interpretable End-to-End Urban Autonomous Driving With Latent Deep Reinforcement Learning | IEEE Trans. Intell. Transp. Syst. | 2022 |
| 66 | Continuous Action Reinforcement Learning From a Mixture of Interpretable Experts | IEEE Trans. Pattern Anal. Mach. Intell. | 2022 |
| 67 | Self-Supervised Discovering of Interpretable Features for Reinforcement Learning | IEEE Trans. Pattern Anal. Mach. Intell. | 2022 |
| 68 | Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learning | IEEE Trans. Pattern Anal. Mach. Intell. | 2022 |
| 69 | Visual Analytics for RNN-Based Deep Reinforcement Learning | IEEE Trans. Vis. Comput. Graph. | 2022 |
| 70 | Toward Interpretable-AI Policies Using Evolutionary Nonlinear Decision Trees for Discrete-Action Systems | IEEE Transactions on Cybernetics | 2022 |
| 71 | Understanding via Exploration: Discovery of Interpretable Features With Deep Reinforcement Learning | IEEE Transactions on Neural Networks and Learning Systems | 2022 |
| 72 | Summarising and Comparing Agent Dynamics with Contrastive Spatiotemporal Abstraction | IJCAI Workshop on XAI | 2022 |
| 73 | ACMViz: a visual analytics approach to understand DRL-based autonomous control model | J. Vis. | 2022 |
| 74 | Incorporating Explanations to Balance the Exploration and Exploitation of Deep Reinforcement Learning | KSEM | 2022 |
| 75 | Towards Explainable Reinforcement Learning Using Scoring Mechanism Augmented Agents | KSEM | 2022 |
| 76 | Explainable Reinforcement Learning via Model Transforms | NeurIPS | 2022 |
| 77 | GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesis | NeurIPS | 2022 |
| 78 | Inherently Explainable Reinforcement Learning in Natural Language | NeurIPS | 2022 |
| 79 | Non-Markovian Reward Modelling from Trajectory Labels via Interpretable Multiple Instance Learning | NeurIPS | 2022 |
| 80 | ProtoX: Explaining a Reinforcement Learning Agent via Prototyping | NeurIPS | 2022 |
| 81 | (When) Are Contrastive Explanations of Reinforcement Learning Helpful? | NeurIPS workshop on HiLL | 2022 |
| 82 | Mo"ET: Mixture of Expert Trees and its application to verifiable reinforcement learning | Neural Networks | 2022 |
| 83 | Analysing deep reinforcement learning agents trained with domain randomisation | Neurocomputing | 2022 |
| 84 | Why? Why not? When? Visual Explanations of Agent Behaviour in Reinforcement Learning | PacificVis | 2022 |
| 85 | Driving behavior explanation with multi-level fusion | Pattern Recognit. | 2022 |
| 86 | Acquisition of chess knowledge in AlphaZero | Proc. Natl. Acad. Sci. U.S.A. | 2022 |
| 87 | Learning Interpretable, High-Performing Policies for Autonomous Driving | Robotics: Science and Systems | 2022 |
| 88 | Event-driven temporal models for explanations - ETeMoX: explaining reinforcement learning | Softw. Syst. Model. | 2022 |
| 89 | Toward a Psychology of Deep Reinforcement Learning Agents Using a Cognitive Architecture | Top. Cogn. Sci. | 2022 |
| 90 | DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning | AAAI | 2021 |
| 91 | Iterative Bounding MDPs: Learning Interpretable Policies via Non-Interpretable Methods | AAAI | 2021 |
| 92 | TripleTree: A Versatile Interpretable Representation of Black Box Agents and their Environments | AAAI | 2021 |
| 93 | Explaining Deep Reinforcement Learning Agents in the Atari Domain through a Surrogate Model | AIIDE | 2021 |
| 94 | A framework of explanation generation toward reliable autonomous robots | Adv. Robotics | 2021 |
| 95 | Explainable Deep Reinforcement Learning for UAV autonomous path planning | Aerospace Science and Technology | 2021 |
| 96 | Explaining robot policies | Applied AI Letters | 2021 |
| 97 | Counterfactual state explanations for reinforcement learning agents via generative deep learning | Artif. Intell. | 2021 |
| 98 | Local and global explanations of agent behavior: Integrating strategy summaries with saliency maps | Artif. Intell. | 2021 |
| 99 | XPM: An Explainable Deep Reinforcement Learning Framework for Portfolio Management | CIKM | 2021 |
| 100 | Interactive Explanations: Diagnosis and Repair of Reinforcement Learning Based Agent Behaviors | CoG | 2021 |
| 101 | CDT: Cascading Decision Trees for Explainable Reinforcement Learning | CoRR | 2021 |
| 102 | Contrastive Explanations for Comparing Preferences of Reinforcement Learning Agents | CoRR | 2021 |
| 103 | Approximating a deep reinforcement learning docking agent using linear model trees | ECC | 2021 |
| 104 | Robotic Lever Manipulation using Hindsight Experience Replay and Shapley Additive Explanations | ECC | 2021 |
| 105 | Off-Policy Differentiable Logic Reinforcement Learning | ECML PKDD | 2021 |
| 106 | Neuro-Symbolic Reinforcement Learning with First-Order Logic | EMNLP | 2021 |
| 107 | Explainable Reinforcement Learning for Longitudinal Control | ICAART | 2021 |
| 108 | Explainable deep reinforcement learning for portfolio management: an empirical approach | ICAIF | 2021 |
| 109 | Explainable Reinforcement Learning for Human-Robot Collaboration | ICAR | 2021 |
| 110 | DRIVE: Deep Reinforced Accident Anticipation with Visual Explanation | ICCV | 2021 |
| 111 | Contrastive Explanations for Reinforcement Learning via Embedded Self Predictions | ICLR | 2021 |
| 112 | Explaining by Imitating: Understanding Decisions by Interpretable Policy Learning | ICLR | 2021 |
| 113 | Learning "What-if" Explanations for Sequential Decision-Making | ICLR | 2021 |
| 114 | Discovering symbolic policies with deep reinforcement learning | ICML | 2021 |
| 115 | Re-understanding Finite-State Representations of Recurrent Policy Networks | ICML | 2021 |
| 116 | Explainable Reinforcement Learning with the Tsetlin Machine | IEA/AIE | 2021 |
| 117 | A Blood Glucose Control Framework Based on Reinforcement Learning With Safety and Interpretability: In Silico Validation | IEEE Access | 2021 |
| 118 | Symbolic Regression Methods for Reinforcement Learning | IEEE Access | 2021 |
| 119 | Efficient Robotic Object Search Via HIEM: Hierarchical Policy Learning With Intrinsic-Extrinsic Modeling | IEEE Robotics Autom. Lett. | 2021 |
| 120 | Learning to Discover Task-Relevant Features for Interpretable Reinforcement Learning | IEEE Robotics Autom. Lett. | 2021 |
| 121 | Explaining Deep Learning Models Through Rule-Based Approximation and Visualization | IEEE Trans. Fuzzy Syst. | 2021 |
| 122 | Interpretable Decision-Making for Autonomous Vehicles at Highway On-Ramps With Latent Space Reinforcement Learning | IEEE Trans. Veh. Technol. | 2021 |
| 123 | Explainable AI methods on a deep reinforcement learning agent for automatic docking | IFAC-PapersOnLine | 2021 |
| 124 | Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning | IJCNN | 2021 |
| 125 | Programmatic Policy Extraction by Iterative Local Search | ILP | 2021 |
| 126 | Explaining the Decisions of Deep Policy Networks for Robotic Manipulations | IROS | 2021 |
| 127 | XAI-N: Sensor-based Robot Navigation using Expert Policies and Decision Trees | IROS | 2021 |
| 128 | Mixed Autonomous Supervision in Traffic Signal Control | ITSC | 2021 |
| 129 | Can You Trust Your Autonomous Car? Interpretable and Verifiably Safe Reinforcement Learning | IV | 2021 |
| 130 | Explaining a Deep Reinforcement Learning Docking Agent Using Linear Model Trees with User Adapted Visualization | Journal of Marine Science and Engineering | 2021 |
| 131 | Visual Analysis of Deep Q-network | KSII Trans. Internet Inf. Syst. | 2021 |
| 132 | Automatic discovery of interpretable planning strategies | Mach. Learn. | 2021 |
| 133 | EDGE: Explaining Deep Reinforcement Learning Policies | NeurIPS | 2021 |
| 134 | Learning Tree Interpretation from Object Representation for Deep Reinforcement Learning | NeurIPS | 2021 |
| 135 | Learning to Synthesize Programs as Interpretable and Generalizable Policies | NeurIPS | 2021 |
| 136 | Machine versus Human Attention in Deep Reinforcement Learning Tasks | NeurIPS | 2021 |
| 137 | Explainable Artificial Intelligence (XAI) for Increasing User Trust in Deep Reinforcement Learning Driven Autonomous Systems | NeurIPS Workshop on Deep RL | 2021 |
| 138 | Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatment | NeurIPS Workshop on Machine Learning for Health | 2021 |
| 139 | Feature-Based Interpretable Reinforcement Learning based on State-Transition Models | SMC | 2021 |
| 140 | A co-evolutionary approach to interpretable reinforcement learning in environments with continuous action spaces | SSCI | 2021 |
| 141 | Interpretable AI Agent Through Nonlinear Decision Trees for Lane Change Problem | SSCI | 2021 |
| 142 | Learning Sparse Evidence- Driven Interpretation to Understand Deep Reinforcement Learning Agents | SSCI | 2021 |
| 143 | Explainable Reinforcement Learning through a Causal Lens | AAAI | 2020 |
| 144 | Attribution-based Salience Method towards Interpretable Reinforcement Learning | AAAI-MAKE | 2020 |
| 145 | Learning an Interpretable Traffic Signal Control Policy | AAMAS | 2020 |
| 146 | Optimization Methods for Interpretable Differentiable Decision Trees Applied to Reinforcement Learning | AISTATS | 2020 |
| 147 | Interestingness elements for explainable reinforcement learning: Understanding agents' capabilities and limitations | Artif. Intell. | 2020 |
| 148 | Model primitives for hierarchical lifelong reinforcement learning | Auton. Agents Multi Agent Syst. | 2020 |
| 149 | Understanding the Behavior of Reinforcement Learning Agents | BIOMA | 2020 |
| 150 | Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control | Big Data | 2020 |
| 151 | Explaining Autonomous Driving by Learning End-to-End Visual Attention | CVPRW | 2020 |
| 152 | Understanding Learned Reward Functions | CoRR | 2020 |
| 153 | Interpretable policy derivation for reinforcement learning based on evolutionary feature synthesis | Complex & Intelligent Systems | 2020 |
| 154 | DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning | Comput. Graph. Forum | 2020 |
| 155 | Understanding RL Vision | Distill | 2020 |
| 156 | Interpretable policies for reinforcement learning by empirical fuzzy sets | Eng. Appl. Artif. Intell. | 2020 |
| 157 | Neuroevolution of self-interpretable agents | GECCO | 2020 |
| 158 | Topological Visualization Method for Understanding the Landscape of Value Functions and Structure of the State Space in Reinforcement Learning | ICAART | 2020 |
| 159 | Identifying Critical States by the Action-Based Variance of Expected Return | ICANN | 2020 |
| 160 | TLdR: Policy Summarization for Factored SSP Problems Using Temporal Abstractions | ICAPS | 2020 |
| 161 | Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution | ICLR | 2020 |
| 162 | Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning | ICLR | 2020 |
| 163 | Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents | ICLR | 2020 |
| 164 | Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions | ICML | 2020 |
| 165 | Deep Reinforcement Learning for Safe Local Planning of a Ground Vehicle in Unknown Rough Terrain | IEEE Robotics Autom. Lett. | 2020 |
| 166 | Towards Interpretable Reinforcement Learning with State Abstraction Driven by External Knowledge | IEICE Trans. Inf. Syst. | 2020 |
| 167 | Improved Policy Extraction via Online Q-Value Distillation | IJCNN | 2020 |
| 168 | Visualization of topographical internal representation of learning robots | IJCNN | 2020 |
| 169 | Explainable navigation system using fuzzy reinforcement learning | IJIDeM | 2020 |
| 170 | Explainability of Intelligent Transportation Systems using Knowledge Compilation: a Traffic Light Controller Case | ITSC | 2020 |
| 171 | xGAIL: Explainable Generative Adversarial Imitation Learning for Explainable Human Decision Analysis | KDD | 2020 |
| 172 | What Did You Think Would Happen? Explaining Agent Behaviour through Intended Outcomes | NeurIPS | 2020 |
| 173 | Explaining Conditions for Reinforcement Learning Behaviors from Real and Imagined Data | NeurIPS Workshop on Challenges of Real-World RL | 2020 |
| 174 | DynamicsExplorer: Visual Analytics for Robot Control Tasks involving Dynamics and LSTM-based Control Policies | PacificVis | 2020 |
| 175 | Combining reinforcement learning with rule-based controllers for transparent and general decision-making in autonomous driving | Robotics Auton. Syst. | 2020 |
| 176 | Modelling Agent Policies with Interpretable Imitation Learning | TAILOR | 2020 |
| 177 | Interpretable, Verifiable, and Robust Reinforcement Learning via Program Synthesis | xxAI - Beyond Explainable AI | 2020 |
| 178 | Generation of Policy-Level Explanations for Reinforcement Learning | AAAI | 2019 |
| 179 | SDRL: Interpretable and Data-Efficient Deep Reinforcement Learning Leveraging Symbolic Planning | AAAI | 2019 |
| 180 | Towards Better Interpretability in Deep Q-Networks | AAAI | 2019 |
| 181 | Toward Robust Policy Summarization | AAMAS | 2019 |
| 182 | Towards Governing Agent's Efficacy: Action-Conditional \textdollar(\beta)\textdollar-VAE for Deep Transparent Reinforcement Learning | ACML | 2019 |
| 183 | Memory-Based Explainable Reinforcement Learning | AI | 2019 |
| 184 | Summarizing agent strategies | Auton. Agents Multi Agent Syst. | 2019 |
| 185 | Enabling robots to communicate their objectives | Auton. Robots | 2019 |
| 186 | Visualization of Deep Reinforcement Learning using Grad-CAM: How AI Plays Atari Games? | CoG | 2019 |
| 187 | Explaining Reward Functions in Markov Decision Processes | FLAIRS | 2019 |
| 188 | Explanation-Based Reward Coaching to Improve Human Performance via Reinforcement Learning | HRI | 2019 |
| 189 | Free-Lunch Saliency via Attention in Atari Agents | ICCVW | 2019 |
| 190 | Deep reinforcement learning with relational inductive biases | ICLR | 2019 |
| 191 | Learning Finite State Representations of Recurrent Policy Networks | ICLR | 2019 |
| 192 | Neural Logic Reinforcement Learning | ICML | 2019 |
| 193 | Interpretable Approximation of a Deep Reinforcement Learning Agent as a Set of If-Then Rules | ICMLA | 2019 |
| 194 | Semantic Predictive Control for Explainable and Efficient Policy Learning | ICRA | 2019 |
| 195 | DQNViz: A Visual Analytics Approach to Understand Deep Q-Networks | IEEE Trans. Vis. Comput. Graph. | 2019 |
| 196 | Visualizing Deep Q-Learning to Understanding Behavior of Swarm Robotic System | IES | 2019 |
| 197 | Exploring Computational User Models for Agent Policy Summarization | IJCA | 2019 |
| 198 | Explaining Reinforcement Learning to Mere Mortals: An Empirical Study | IJCAI | 2019 |
| 199 | Counterfactual States for Atari Agents via Generative Deep Learning | IJCAI Workshop on XAI | 2019 |
| 200 | Distilling Deep Reinforcement Learning Policies in Soft Decision Trees | IJCAI Workshop on XAI | 2019 |
| 201 | Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation | IROS | 2019 |
| 202 | Reinforcement Learning with Explainability for Traffic Signal Control | ITSC | 2019 |
| 203 | Interestingness Elements for Explainable Reinforcement Learning through Introspection | IUI Workshops | 2019 |
| 204 | Explainable Reinforcement Learning via Reward Decomposition | JCAI Workshop on XAI | 2019 |
| 205 | Enhancing Explainability of Deep Reinforcement Learning Through Selective Layer-Wise Relevance Propagation | KI | 2019 |
| 206 | Imitation-Projected Programmatic Reinforcement Learning | NeurIPS | 2019 |
| 207 | Towards Interpretable Reinforcement Learning Using Attention Augmented Agents | NeurIPS | 2019 |
| 208 | Verbal Explanations for Deep Reinforcement Learning Neural Networks with Attention on Extracted Features | RO-MAN | 2019 |
| 209 | A formal methods approach to interpretable reinforcement learning for robotic planning | Sci. Robotics | 2019 |
| 210 | HIGHLIGHTS: Summarizing Agent Behavior to People | AAMAS | 2018 |
| 211 | Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations | AIES | 2018 |
| 212 | Transparency and Explanation in Deep Reinforcement Learning Neural Networks | AIES | 2018 |
| 213 | Visual Rationalizations in Deep Reinforcement Learning for Atari Games | BNAIC | 2018 |
| 214 | Textual Explanations for Self-Driving Vehicles | ECCV | 2018 |
| 215 | Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees | ECML PKDD | 2018 |
| 216 | Interpretable policies for reinforcement learning by genetic programming | Eng. Appl. Artif. Intell. | 2018 |
| 217 | Generating interpretable fuzzy controllers using particle swarm optimization and genetic programming | GECCO | 2018 |
| 218 | Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning | ICLR | 2018 |
| 219 | Programmatically Interpretable Reinforcement Learning | ICML | 2018 |
| 220 | Visualizing and Understanding Atari Agents | ICML | 2018 |
| 221 | Deep Reinforcement Learning Monitor for Snapshot Recording | ICMLA | 2018 |
| 222 | Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences | IJCAI Workshop on XAI | 2018 |
| 223 | Explaining Deep Adaptive Programs via Reward Decomposition | IJCAI/ECAI Workshop XAI | 2018 |
| 224 | Establishing Appropriate Trust via Critical States | IROS | 2018 |
| 225 | Unsupervised Video Object Segmentation for Deep Reinforcement Learning | NeurIPS | 2018 |
| 226 | Verifiable Reinforcement Learning via Policy Extraction | NeurIPS | 2018 |
| 227 | Visual Sparse Bayesian Reinforcement Learning: A Framework for Interpreting What an Agent Has Learned | SSCI | 2018 |
| 228 | Particle swarm optimization for generating interpretable fuzzy reinforcement learning policies | Eng. Appl. Artif. Intell. | 2017 |
| 229 | Autonomous Self-Explanation of Behavior for Interactive Reinforcement Learning Agents | HAI | 2017 |
| 230 | Improving Robot Controller Transparency Through Autonomous Policy Explanation | HRI | 2017 |
| 231 | Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention | ICCV | 2017 |
| 232 | Application of Instruction-Based Behavior Explanation to a Reinforcement Learning Agent with Changing Policy | ICONIP | 2017 |
| 233 | Graying the black box: Understanding DQNs | ICML | 2016 |