CNN Features Off-the-Shelf: An Astounding Baseline for Recognition

Abstract

Generic descriptors from convolutional neural networks are powerful for diverse recognition tasks, achieving superior results compared to state-of-the-art systems.

Introduction

Convolutional neural network features can be exploited for a wide variety of vision tasks without task-specific retraining, demonstrating significant performance gains.

Network Architecture and Training Data

The OverFeat network, trained on ImageNet for object classification, is used to extract generic features for various recognition tasks.

Experimental Setup

Experiments utilize features from OverFeat's first fully connected layer combined with linear SVM classifiers, with optional data augmentation.

Image Classification

OverFeat CNN features with linear SVMs significantly outperform previous methods on challenging object and scene classification datasets like Pascal VOC and MIT indoor scenes.

Object Detection and Fine-Grained Recognition

While not directly tested for object detection, OverFeat features show promise, and they excel at fine-grained recognition tasks, capturing subtle differences between subclasses.

Fine-Grained Recognition Datasets

The CNN-SVM approach achieves state-of-the-art performance on fine-grained datasets like CUB 200-2011 birds and Oxford 102 flowers, even without specialized annotations.

Attribute Detection

CNN features demonstrate competitive performance in attribute detection tasks on the UIUC and H3D datasets, outperforming methods that use part-level annotations.

Implementation Details

Experiments employ libsvm and liblinear with data augmentation, including crops, rotations, and power transforms, and sum responses for multiple test-time representations.

Instance Retrieval

CNN features are competitive with established instance retrieval methods on various datasets, outperforming low memory footprint methods after standard processing steps.

Retrieval Results

CNN representations, with or without spatial search and standard processing, achieve strong performance on diverse retrieval benchmarks, particularly against low memory footprint methods.

Conclusion

Off-the-shelf CNN features from OverFeat, combined with simple classifiers, are a powerful and general solution for various visual recognition tasks, establishing a new baseline.