Going Deeper with Convolutions

Abstract

GoogLeNet introduces an Inception module that significantly improves computational resource utilization in deep convolutional neural networks, achieving state-of-the-art results in image classification and detection.

Standard CNNs and Inception's Inspiration

Convolutional neural networks typically use stacked convolutional and fully-connected layers, with GoogLeNet's Inception architecture incorporating multi-scale processing and the Network-in-Network concept for improved performance and reduced dimensionality.

Challenges of Deep Networks

Increasing the depth and width of deep neural networks enhances model quality but raises concerns about overfitting and escalating computational costs, necessitating efficient resource distribution and potentially sparser network structures.

Sparse vs. Dense Computation

Sparse matrix computations face practical challenges on current hardware optimized for dense operations, leading to the proposal of approximating sparse structures using dense components for efficiency in deep learning architectures like Inception.

Inception Architecture Design

The Inception architecture approximates optimal local sparse structures using dense components by combining outputs from parallel convolutional layers of different sizes (1x1, 3x3, 5x5), with a pooling path included for additional benefit.

Dimensionality Reduction in Inception

The Inception architecture reduces computational complexity by judiciously applying 1x1 convolutions for dimension reduction before expensive convolutions, allowing for increased network width and depth without prohibitive computational costs.

GoogLeNet Configuration

GoogLeNet, a 22-layer deep network utilizing the Inception architecture, was configured with rectified linear activations and specific reduction layers, prioritizing computational efficiency for practical, real-world applications.

Network Depth and Auxiliary Classifiers

GoogLeNet's depth is managed with auxiliary classifiers to combat potential vanishing gradients and improve regularization, though their impact on final performance is minor and they are removed during inference.

Training GoogLeNet

GoogLeNet was trained using DistBelief with a combination of model and data parallelism, employing asynchronous stochastic gradient descent and a decaying learning rate schedule, with variations in image sampling and hyperparameter tuning.

ILSVRC 2014 Classification

The ILSVRC 2014 classification challenge involved predicting one of 1000 categories using ensemble predictions from seven independently trained GoogLeNet models and aggressive image cropping strategies.

Classification Performance

The GoogLeNet submission achieved a top-5 error rate of 6.67% in the ILSVRC 2014 classification challenge, demonstrating significant improvement through ensemble methods and effective testing strategies.

ILSVRC 2014 Detection

For the ILSVRC 2014 detection task, GoogLeNet used the Inception model for region classification and enhanced region proposal with selective search and multi-box predictions, achieving competitive results without bounding box regression.

Conclusion and Future Work

Approximating sparse neural network structures with dense components effectively enhances computer vision models, and future work can focus on automating sparser architectures and applying these principles to other domains.