Dataset and Model Credits

Image Classification, Light-weight

Dataset: Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M. S.; Berg, A. C. & Fei-Fei, L. (2015), 'ImageNet Large Scale Visual Recognition Challenge', International Journal of Computer Vision (IJCV).

Model: Howard, A.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andretto, M.; Adam, H. (2017), 'MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications', CoRR abs/1704.04861.

Training model implementation: MLPerf™ team

Inference model implementation: MLPerf team

Advisory board: n/a

Image Classification, Heavy-weight

Model: He, K.; Zhang, X.; Ren, S. & Sun, J. (2015), 'Deep Residual Learning for Image Recognition', CoRR abs/1512.03385.

Training model implementation: MLPerf team

Inference model implementation: MLPerf team

Advisory board: n/a

Object Identification, Light-weight

Dataset: Lin, T.-Y.; Maire, M.; Belongie, S. J.; Bourdev, L. D.; Girshick, R. B.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P. & Zitnick, C. L. (2014), 'Microsoft COCO: Common Objects in Context', CoRR abs/1405.0312.

Model: Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.; and Berg, A. (2016), ‘Ssd: Single shot multibox detector’, European Conference on Computer Vision.

Training model implementation: MLPerf team

Inference model implementation: MLPerf team

Advisory board: n/a

Object Identification, Heavy-weight

Model: He, K.; Gkioxari, G.; Dollár, P. & Girshick, R. B. (2017), 'Mask R-CNN', CoRR abs/1703.06870.

Training model implementation: MLPerf team

Inference model implementation: MLPerf team

Advisory board: n/a

Medical Imaging

Dataset: Multimodal Brain Tumor Segmentation (BraTS) Challenge 2019

Model: Fabian Isensee and Jens Petersen and Simon A. A. Kohl and Paul F. Jäger and Klaus H. Maier-Hein (2019). nnU-Net: Breaking the Spell on Successful Medical Image Segmentation CoRR, abs/1904.08128.

Training model implementation: Pablo Ribalta @ Nvidia, Fabian Isensee @ Division of Medical Image Computing, German Cancer Research Center, Anthony Reina @ Intel, Alexandros Karargyris @ MLPerf

Inference model implementation: Po-Han Huang @ Nvidia, Dmitry Rizshkov @ Intel, Khalique Ahmed @ AMD, Guenther Schmuelling @ Microsoft, Pablo Ribalta @ Nvidia, Fabian Isensee @ Division of Medical Image Computing at German Cancer Research Center, Anthony Reina @ Intel, Alexandros Karargyris @ MLPerf

Advisory Board: Spyros Bakas @ University of Pennsylvania, Akshay Chaudhari @ Stanford University, Alejandro Frangi @ Leeds University, Bogdan Georgescu @ Siemens, Maya Khalifé @ Arterys, Ilya Kovler @ RSIP Vision, Bennett Landman @ Vanderbilt University, Ratna Saripalli @ GE, Yefeng Zheng @ Tencent, Kevin Zhou @ Chinese Academy of Sciences

Translation, Recurrent

Dataset: WMT English-German from Bojar, O.; Buck, C.; Federmann, C.; Haddow, B.; Koehn, P.; Monz, C.; Post, M. & Specia, L., ed. (2014), Proceedings of the Ninth Workshop on Statistical Machine Translation, Association for Computational Linguistics, Baltimore, Maryland, USA.

Model: Wu, Y.; Schuster, M.; Chen, Z.; Le, Q., Norouzi, M.; Macherey, W.; Krikun, M.; Cao, Y.; Gao, Q.; Macherey, K.; et al., (2016), ‘Google’s multilingual neural machine translation system: Enabling zero-shot translation’, CoRR, abs/1611.04558.

Training model implementation: MLPerf team

Inference model implementation: MLPerf team

Advisory board: n/a

Translation, Non-Recurrent

Model: Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L. & Polosukhin, I. (2017), 'Attention Is All You Need', CoRR abs/1706.03762.

Training model implementation: MLPerf team

Inference model implementation: MLPerf team

Advisory board: n/a

NLP

Dataset: Wikipedia dump 1/1/2020

Model: Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova (2019), 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding',
CoRR abs/1810.04805.

Training model implementation: (Dehao Chen, Yuechao Pan, Shibo Wang, Hongkun Yu) @ Google

Inference model implementation: (Christopher Forster, Po-Han Huang, Patrick Judd) @ NVIDIA

Advisory board: n/a

Recommendation

Dataset: Criteo 1-Terabyte Clickthrough Dataset, courtesy of Flavian Vasile and his team @ Criteo

Model: Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherniavskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, Volodymyr Kondratenko, Stephanie Pereira, Xianjie Chen, Wenlin Chen, Vijay Rao, Bill Jia, Liang Xiong, Misha Smelyanskiy (2019), 'Deep Learning Recommendation Model for Personalization and Recommendation Systems', CoRR abs/1906.00091.

Training model implementation: (Maxim Naumov, Mustafa Ozdal) @ Facebook, Tayo Oguntebi @ Google, (Jian Ping Chen, Guokai Ma) @ Intel, (Tomasz Grel, Po-Han Huang, Paulius Micikevicius, Dilip Sequeira) @ NVIDIA

Inference model implementation: (Udit Gupta, Dmitriy Korchev, Maxim Naumov) @ Facebook

Advisory board: (Carole-Jean Wu, Hao Zhang) @ Facebook, Ed H. Chi @ Google, Yves Raimond @ Netflix, Julian McAuley @ UCSD, Robin Burke @ University of Colorado, Joseph A. Konstan @ University of Minnesota

Reinforcement Learning

Dataset: Go

Model: Brian Lee, Andrew Jackson, Tom Madams, Seth Troisi, Derek Jones (2019), 'Minigo: A Case Study in Reproducing Reinforcement Learning Research', https://openreview.net/forum?id=H1eerhIpLV.

Training model implementation: (Andrew Jackson, Brian Lee, Tom Madams, Seth Troisi) @ Google.

Advisory board: n/a

MLPerf Inference Tiny Datasets

Warden, Pete. "Speech commands: A dataset for limited-vocabulary speech recognition." arXiv preprint arXiv:1804.03209 (2018).

Chowdhery, Aakanksha, Pete Warden, Jonathon Shlens, Andrew Howard, and Rocky Rhodes. "Visual wake words dataset." arXiv preprint arXiv:1906.05721 (2019).

Krizhevsky, Alex, and Geoffrey Hinton. "Learning multiple layers of features from tiny images." (2009).

Yuma Koizumi, Shoichiro Saito, Noboru Harada, Hisashi Uematsu, and Keisuke Imoto, "ToyADMOS: A Dataset of Miniature-Machine Operating Sounds for Anomalous Sound Detection," in Proc of Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2019.

Harsh Purohit, Ryo Tanabe, Kenji Ichige, Takashi Endo, Yuki Nikaido, Kaori Suefusa, and Yohei Kawaguchi, “MIMII Dataset: Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspection,” in Proc. 4th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE), 2019.

Yuma Koizumi, Yohei Kawaguchi, Keisuke Imoto, Toshiki Nakamura, Yuki Nikaido, Ryo Tanabe, Harsh Purohit, Kaori Suefusa, Takashi Endo, Masahiro Yasuda, and Noboru Harada, "Description and Discussion on DCASE2020 Challenge Task2: Unsupervised Anomalous Sound Detection for Machine Condition Monitoring," in arXiv e-prints: 2006.05822, 2020.