aniketr/mrl-resnet50

Matryoshka Representation Learning🪆

Aditya Kusupati*, Gantavya Bhatt*, Aniket Rege*, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, Ali Farhadi

GitHub: https://github.com/RAIVNLab/MRL

Arxiv: https://arxiv.org/abs/2205.13147

We provide pretrained models trained with FFCV on ImageNet-1K:

1. mrl : ResNet50 mrl models trained with Matryoshka loss (vanilla and efficient) with nesting starting from d=8 (default) and d=16
2. fixed-feature : independently trained ResNet50 baselines at log(d) granularities
3. resnet-family : mrl and ff models trained on ResNet18/34/101

Citation

If you find this project useful in your research, please consider citing:

@inproceedings{kusupati2022matryoshka,
  title     = {Matryoshka Representation Learning},
  author    = {Kusupati, Aditya and Bhatt, Gantavya and Rege, Aniket and Wallingford, Matthew and Sinha, Aditya and Ramanujan, Vivek and Howard-Snyder, William and Chen, Kaifeng and Kakade, Sham and Jain, Prateek and others},
  title     = {Matryoshka Representation Learning.},
  booktitle = {Advances in Neural Information Processing Systems},
  month     = {December},
  year      = {2022},
}