CL²R: Compatible Lifelong Learning Representations

This repo contains the code of "CL²R: Compatible Lifelong Learning Representations", Niccolò Biondi, Federico Pernici, Matteo Bruni, Daniele Mugnai, and Alberto Del Bimbo, ACM Transactions on Multimedia Computing, Communications, and Applications

Open access version on arxiv

In this paper, we propose a method to partially mimic natural intelligence for the problem of lifelong learning representations that are compatible. We take the perspective of a learning agent that is interested in recognizing object instances in an open dynamic universe in a way in which any update to its internal feature representation does not render the features in the gallery unusable for visual search. We refer to this learning problem as Compatible Lifelong Learning Representations (CL2R) as it considers compatible representation learning within the lifelong learning paradigm. We identify stationarity as the property that the feature representation is required to hold to achieve compatibility and propose a novel training procedure that encourages local and global stationarity on the learned representation. Due to stationarity, the statistical properties of the learned features do not change over time, making them interoperable with previously learned features. Extensive experiments on standard benchmark datasets show that our CL2R training procedure outperforms alternative baselines and state-of-the-art methods. We also provide novel metrics to specifically evaluate compatible representation learning under catastrophic forgetting in various sequential learning tasks.

Install and Train

git clone https://github.com/NiccoBiondi/CompatibleLifelongRepresentation.git
cd CompatibleLifelongRepresentation
pip install -r requirements.txt
pip install -e .

# run experiment
cl2r --config_path config.yml

To modify hyperparameters please modify the config file.

Authors

• Niccolò Biondi <niccolo.biondi (at) unifi.it>
• Federico Pernici <federico.pernici (at) unifi.it>
• Matteo Bruni <matteo.bruni (at) unifi.it>
• Daniele Mugnai <mugnai.mugnai (at) gmail.com>
• Alberto Del Bimbo <alberto.delbimbo (at) unifi.it>

Citation

If you find this code useful for your research, please cite our paper:

@article{biondi2023cl2r,
  title={CL2R: Compatible Lifelong Learning Representations},
  author={Biondi, Niccolo and Pernici, Federico and Bruni, Matteo and Mugnai, Daniele and Bimbo, Alberto Del},
  journal={ACM Transactions on Multimedia Computing, Communications and Applications},
  volume={18},
  number={2s},
  pages={1--22},
  year={2023},
  publisher={ACM New York, NY}
}

TODO

• add creation pairs (at runtime for valiation)
• add model
• add evaluation
• add checkpoints saving
• add train function
• add correct args