Consensus-Driven Distillation for Trustworthy Explanations in Self-Interpretable GNNs

💻 Official implementation of our TPAMI submission (extension of ICML 2025): Consensus-Driven Distillation for Trustworthy Explanations in Self-Interpretable GNNs

🧠 Authors: Wenxin Tai, Fan Zhou, Steve Azzolin, Ting Zhong, Goce Trajcevski, Kunpeng Zhang, Philip S. Yu
📍 Institutions: University of Electronic Science and Technology of China & University of Trento & Iowa State University & University of Maryland, College Park & University of Illinois, Chicago.
🔗 Paper Link 🤖 This repository is maintained by ICDM Lab

---

🧩 Overview

TL;DR: Our ICML paper proposed Explanation Ensemble (EE), which improves the trustworthiness of SI-GNNs by aggregating multiple explanations from independently trained models. While effective, it has high computational cost during inference (limits its deployment) and is incompatible with single-explanation metrics such as FID (limits its evaluation). In this extension, we propose Consensus Distillation (CD), which distills the ensemble’s consensus knowledge into a single model, retaining EE’s capability while addressing its limitations.

---

📦 Repository Structure

├── assets
├── configs         # configuration
├── criterion.py    # loss function
├── dataloader.py   # load data
├── dataset.py      # process data
├── datasets        # raw dataset
├── explainer.py    # explainer in self-interpretable GNNs (MLP)
├── main.py         # entry
├── model.py        # GNN backbone (GIN/GCN)
├── outputs         # checkpoints/logs
├── README.md
├── run.sh 
└── trainer.py      # train/valid/test

---

⚙️ Installation

We recommend creating a fresh Python environment (e.g., with conda):

conda create -n exgnn python=3.9
conda activate exgnn
pip install -r requirements.txt

---

📚 Datasets

We evaluate our method on a variety of datasets:

• Synthetic: BA-2MOTIFS
• Molecular: MUTAGENICITY, 3MR, BENZENE

Datasets can be downloaded from Google Drive, place all datasets (e.g., ba_2motifs, benzene, mr, mutag) in the datasets/ folder.

---

🏃‍♀️ Quick Start

1. Train self-interpretable GNNs

python main.py --run_time 10 --dataset ba_2motifs --method gsat_cd

2. Evaluate redundancy (SHD and AUC)

python main.py --run_time 10 --dataset ba_2motifs --method gsat_cd --calculate_all_metrics

---

📁 Pretrained Checkpoints

We provide pretrained model checkpoints for quick evaluation and reproduction.

You can download them from the Releases tab

To use the checkpoint, place it in the outputs/checkpoints/ folder and run:

python main.py --run_time 10 --dataset ba_2motifs --method gsat_cd --calculate_all_metrics

---

📌 Citation

If you find this work useful, please cite us:

@inproceedings{tai2025redundancy,
  title     = {Redundancy Undermines the Trustworthiness of Self-Interpretable GNNs},
  author    = {Tai, Wenxin and Zhong, Ting and Trajcevski, Goce and Zhou, Fan},
  booktitle = {Proceedings of the 42nd International Conference on Machine Learning (ICML)},
  year      = {2025}
}

---

📬 Contact

If you have questions or suggestions, feel free to reach out via GitHub Issues or email: wxtai [AT] outlook [DOT] com