CNN Features off-the-shelf: an Astounding Baseline for Recognition

Abstract

Generic descriptors from convolutional neural networks demonstrate superior performance across various recognition tasks when used with a linear SVM classifier.

Introduction

This paper investigates the effectiveness of using off-the-shelf CNN features from the OverFeat network for a wide range of computer vision tasks, demonstrating strong results with simple classifiers.

Background and Outline

The study utilizes the publicly available OverFeat CNN, trained on ImageNet for object classification, and evaluates its features on diverse recognition tasks including visual classification and instance retrieval.

Visual Classification Method and Datasets

A linear SVM classifier is applied to L2 normalized 4096-dimensional feature vectors from OverFeat's first fully connected layer for image classification on PASCAL VOC and MIT indoor scenes datasets.

Object Detection and Fine-grained Recognition

While not directly experimented on, object detection benefits from off-the-shelf CNN features, and fine-grained recognition tasks like bird and flower classification show strong performance with CNN features, even outperforming methods using segmentation.

Fine-grained Recognition Datasets and Results

CNN features are evaluated on CUB 200-2011 birds and Oxford 102 flowers datasets, outperforming existing methods in fine-grained classification tasks even without using dataset-specific annotations like part landmarks or segmentation.

Attribute Detection Datasets and Results

On UIUC object attributes and H3D human attributes datasets, CNN features demonstrate competitive performance, matching or exceeding state-of-the-art methods like DPD and Poselets, despite using only bounding box information.

Implementation Details and Visual Instance Retrieval

This section details the implementation for CNN-SVM and CNNaug-SVM, including data augmentation techniques, and outlines the setup for visual instance retrieval experiments on datasets like Oxford5k, Paris6k, Sculptures6k, Holidays, and UKbench.

Conclusion

Off-the-shelf CNN features from OverFeat, combined with simple classifiers, prove to be a powerful and general baseline for various visual recognition tasks, suggesting deep learning with CNNs should be the primary candidate for future work.