Distill-to-Label: Weakly Supervised Instance Labeling Using Knowledge Distillation.

Jayaraman J. Thiagarajan10
Estimated H-index: 10
(LLNL: Lawrence Livermore National Laboratory),
Satyananda Kashyap (IBM), Alexandros Karagyris1
Estimated H-index: 1
Weakly supervised instance labeling using only image-level labels, in lieu of expensive fine-grained pixel annotations, is crucial in several applications including medical image analysis. In contrast to conventional instance segmentation scenarios in computer vision, the problems that we consider are characterized by a small number of training images and non-local patterns that lead to the diagnosis. In this paper, we explore the use of multiple instance learning (MIL) to design an instance label generator under this weakly supervised setting. Motivated by the observation that an MIL model can handle bags of varying sizes, we propose to repurpose an MIL model originally trained for bag-level classification to produce reliable predictions for single instances, i.e., bags of size 1 To this end, we introduce a novel regularization strategy based on virtual adversarial training for improving MIL training, and subsequently develop a knowledge distillation technique for repurposing the trained MIL model. Using empirical studies on colon cancer and breast cancer detection from histopathological images, we show that the proposed approach produces high-quality instance-level prediction and significantly outperforms state-of-the MIL methods.
  • References (20)
  • Citations (0)
#1Alexander Ratner (Stanford University)H-Index: 9
#2Stephen H. Bach (Brown University)H-Index: 10
Last.Christopher Ré (Stanford University)H-Index: 40
view all 0 authors...
2018 in ICML (International Conference on Machine Learning)
#1Tommaso Furlanello (SC: University of Southern California)H-Index: 4
#2Zachary C. Lipton (CMU: Carnegie Mellon University)H-Index: 17
Last.Anima Anandkumar ( 10
view all 0 authors...
2018 in CVPR (Computer Vision and Pattern Recognition)
#1Kunpeng Li (NU: Northeastern University)
#2Ziyan Wu (Siemens)H-Index: 10
Last.Yun Fu (NU: Northeastern University)H-Index: 44
view all 0 authors...
2018 in CVPR (Computer Vision and Pattern Recognition)
#1Yanzhao Zhou (CAS: Chinese Academy of Sciences)H-Index: 3
#2Yi Zhu (CAS: Chinese Academy of Sciences)H-Index: 2
Last.Jianbin Jiao (CAS: Chinese Academy of Sciences)H-Index: 16
view all 0 authors...
2018 in ICLR (International Conference on Learning Representations)
#1Rui Shu (Stanford University)H-Index: 8
#2Hung Hai Bui (Google)H-Index: 20
Last.Stefano Ermon (Stanford University)H-Index: 20
view all 0 authors...
#1Adam PaszkeH-Index: 1
#2Sam GrossH-Index: 7
Last.Adam LererH-Index: 12
view all 10 authors...
Jul 1, 2017 in CVPR (Computer Vision and Pattern Recognition)
#1Guanbin Li (HKU: University of Hong Kong)H-Index: 13
#2Yuan Xie (SYSU: Sun Yat-sen University)H-Index: 3
Last.Yizhou Yu (HKU: University of Hong Kong)H-Index: 38
view all 4 authors...
Jul 1, 2017 in CVPR (Computer Vision and Pattern Recognition)
#1Anna Khoreva (MPG: Max Planck Society)H-Index: 12
#2Rodrigo Benenson (MPG: Max Planck Society)H-Index: 23
Last.Bernt Schiele (MPG: Max Planck Society)H-Index: 84
view all 5 authors...
Cited By0
View next paperU-Net adaptation for multiple instance learning