Skip to content

Latest commit

 

History

History
74 lines (62 loc) · 3.33 KB

README.md

File metadata and controls

74 lines (62 loc) · 3.33 KB

T2TL

Exploiting Transformer Sparse Reward in Reinforcement Learning for Interpretable Temporal Logic Motion Planning

If any questions, feel free to contact: [email protected].

Installation instructions

We recommend using Python 3.7 or Python 3.8 to run this code.

  1. pip install -r requirements.txt or conda env create -f conda_env.yml (Note that .yml is only effective for Python 3.7)
  2. Install Spot-2.9:
    • For Python 3.7, follow the installation instructions at the link. Spot should be installed in /usr/local/lib/python3.7/site-packages/spot. This step usually takes around 20 mins.
    • For Python 3.8+, conda install -c conda-forge spot (Note that this command needs Python is 3.8+)
  3. To train the agent in ZoneEnv, you will need Mujoco installed, as well as an active license.
    • pip install mujoco-py==2.0.2.9
    • pip install -e src/envs/safety/safety-gym/
    • pip install numpy==1.21.5, if Python 3.7 or pip install numpy==1.23, if Python 3.8
  4. Install dgl:
    • If you only reproduce the results from Transformer-encoded method, just pip install dgl-cu113 -f https://data.dgl.ai/wheels/repo.html
    • If you also want to reproduce the results from GNN-encoded method, we found the dgl team has not maintained dgl_cu111==0.7a210520. You can install it as follows:
      • download the dgl file from google drive
      • unzip dgl.zip and unzip dgl_cu111-0.7a210520.dist-info.zip
      • mv -f dgl/ /home/{your_workspace_name}/anaconda3/envs/T2TL/lib/python3.7/site-packages/
      • mv -f dgl_cu111-0.7a210520.dist-info/ /home/{your_workspace_name}/anaconda3/envs/T2TL/lib/python3.7/site-packages/

Examples

  1. cd dis_src/
  2. To train the agent with GNN method, run:
    • python T1TL.py --gnn RGCN_8x32_ROOT_SHARED
  3. To train the agent with pretrained GNN method, run:
    • python train_PreGNNAgent.py
  4. To train the agent with T1TL method, run:
    • python T1TL.py
  5. To train the agent with pretrained T1TL method, run:
    • python T1TL_pretrain.py
  6. To train the agent with T2TL method, run:
    • python T2TL.py
  7. To train the agent with pretrained T2TL method, run:
    • python T2TL_pretrain..py

WaterWorld

ZoneEnv

Dimension Comparison

Interpretability via Attention

Citation

@ARTICLE{10167731,
  author={Zhang, Hao and Wang, Hao and Kan, Zhen},
  journal={IEEE Robotics and Automation Letters}, 
  title={Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planning}, 
  year={2023},
  volume={8},
  number={8},
  pages={4831-4838},
  doi={10.1109/LRA.2023.3290511}}