Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks

Abstract

This paper proposes a method to transfer Convolutional Neural Network (CNN) learned representations from large datasets to tasks with limited data, significantly improving object and action classification on the PASCAL VOC dataset.

Related Work

The paper reviews transfer learning and deep learning literature, highlighting the challenge of applying data-hungry CNNs to tasks with limited annotated images and distinguishing its approach by transferring learned representations.

Transferring CNN Weights

The proposed method reuses internal layers of a pre-trained CNN as a feature extractor and trains new adaptation layers on the target task data, addressing label bias and dataset differences through specific training procedures.

Network Training and Classification

The method adapts a pre-trained network for target tasks by using a sliding window strategy to extract and label image patches, addressing dataset biases and re-sampling to balance data before aggregating patch scores for classification.

Experiments and Results

Experiments on PASCAL VOC classification and action recognition demonstrate that transferring CNN weights significantly improves performance, with further gains achieved by augmenting source task data and showing capabilities in localization.

Conclusion

The paper concludes that transferring mid-level features from a CNN pre-trained on ImageNet achieves state-of-the-art results on smaller datasets like PASCAL VOC, indicating the generalizability of learned representations and their potential for localization.