TRIM

Trimmed Rank with Inverse softMax probability.

Universal and reliable, but simple, OOD(Out-of-Distribution) score.

Datasets

Place datasets under ./data/

ImageNet-1k as ID

• ILSVRC-2012/val [download]
• run valprep.sh (from soumith/imagenetloader.torch) after untar the downloaded ILSVRC2012_img_val.tar file:

  wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash

OOD for ImageNet-1k

From deeplearning-wisc/dice

• Describable Textures Dataset (DTD) [download]

  rm -rf dtd/imdb dtd/labels

• iNaturalist [download]
• SUN [download]
• Places [download]

From hendrycks/natural-adv-examples

• ImageNet-O [download]

From ViM

• OpenImage-O: a subset of the OpenImage-V3 testing set. [filelist]

Create Runners

Create runners and save them in ./runners/.

Runners not contained in this repository.

python create_runner.py WEIGHT_NAME

Choose one WEIGHT_NAME in the weights.list.

Evaluate

Evaluate OOD methods and save results in ./results/.

Saved results are contained in this repository.

python eval_runner.py WEIGHT_NAME

Choose one WEIGHT_NAME in the weights.list

• For more details eval.ipynb

resnet50.{A,B,C}

Three weights trained from scratch are contained in ./train/.

• For more details read ./train/README.md