Our code heavily relies on the implementation of Energy-based Out-of-distribution Detection

Paper

(Three Factors to Improve Out-of-Distribution Detection)

Prerequisite

Prepare Dataset

We use cifar10, cifar100 as training data with Longtailed, too.

We use auxiliary data as 300K random images following Outlier Exposure

We test on the SC-OOD benchmark ,this should be inserted in data tree which can be downloaded from SC-OOD UDG

We also test on the MOOD benchmark(with synthetic(SYN, generated by code) data). which can be downloaded from MOOD

Three-Factors/data
├── cifar10
├── cifar100
├── data 
│       ├── images
│       └── imglist
|
|──svhn
|──STL10
|──mnist
|──kmnist
|──fashionmnist
|──dtd
|──Places
|──LSUN
|──LSUN_resize
|──iSUN
└── tinyimages80m
    └── 300K_random_images.npy

Install dependencies

The project is based on the pytorch 1.8.1 with python 3.8.

1. create conda env

   $ conda env create -f three.yml

2. install the torch 1.8.1

   $ conda activate three
   # IF cuda version < 11.0
   $ pip install torch==1.8.1+cu102 torchvision==0.9.1+cu102 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
   # IF cuda version >= 11.0 (e.g., 30x or above)
   $ pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

Get the pretrained model and outlier hardness result

We fine tune based on the pretrained model and outlier hardness result(inferred score from pretrained model) which can be downloaded from asset

Prepare checkpoint

cifar10_res_pretrained_epoch_99.pt cifar100_res_pretrained_epoch_99.pt cifar10_res0.01im_pretrained_epoch_99.pt cifar100_res0.01im_pretrained_epoch_99.pt

put the checkpoint in the path : Three-Factors/snapshots/pretrained

Prepare outlier hardness npy file

balance_cifar10.npy balance_cifar100.npy imbalance_cifar10.npy imbalance_cifar100.npy

put the hardness npy in the path : Three-Factors/

Finetuning code run

in the Three-Factors/

mkdir ./snapshots/O_base_con

For running baseline OE

run ResNet18 OE_fine_tune training and testing for cifar10 with trial(seed) 1 to 8

bash OE_run_resc.sh oe_tune 0

run ResNet18 OE_fine_tune training and testing for cifar100 with trial(seed) 1 to 8

bash OE_run_resc.sh oe_tune 1

run ResNet18 OE_fine_tune training and testing for LT-cifar10 with trial(seed) 1 to 8

bash OE_run_im_resc.sh oe_tune 0

run ResNet18 OE_fine_tune training and testing for LT-cifar100 with trial(seed) 1 to 8

bash OE_run_im_resc.sh oe_tune 1

For running OE with all three factor (Ours)

in the Three-Factors/

mkdir ./snapshots/D_all

run ResNet18 our OE_fine_tune training and testing for cifar10 with trial(seed) 1 to 8

bash OE_run_resD.sh oe_tune 0

run ResNet18 our OE_fine_tune training and testing for cifar100 with trial(seed) 1 to 8

bash OE_run_resD.sh oe_tune 1

run ResNet18 our OE_fine_tune training and testing for LT-cifar10 with trial(seed) 1 to 8

bash OE_run_im_resD.sh oe_tune 0

run ResNet18 our OE_fine_tune training and testing for LT-cifar100 with trial(seed) 1 to 8

bash OE_run_im_resD.sh oe_tune 1

For running baseline of OECC, Energy, Balanced Energy

run ResNet18 OECC_fine_tune training and testing for LT-cifar10 with trial(seed) 1 to 8

bash CC_run_im_resc.sh oe_tune 0

run ResNet18 Energy_fine_tune training and testing for LT-cifar10 with trial(seed) 1 to 8

bash E_run_im_resc.sh energy_ft 0

run ResNet18 Balanced Energy_fine_tune training and testing for LT-cifar10 with trial(seed) 1 to 8

bash BE_run_im_resc.sh energy_ft 0

For running ours with OECC, Energy, Balanced Energy

run ResNet18 our OECC_fine_tune training and testing for LT-cifar10 with trial(seed) 1 to 8

bash CC_run_im_resD.sh oe_tune 0

run ResNet18 our Energy_fine_tune training and testing for LT-cifar10 with trial(seed) 1 to 8

bash E_run_im_resD.sh energy_ft 0

run ResNet18 our Balanced Energy_fine_tune training and testing for LT-cifar10 with trial(seed) 1 to 8

bash BE_run_im_resD.sh energy_ft 0