Deep Residual Learning for Image Recognition

Abstract

Residual learning frameworks ease the training of substantially deeper neural networks by learning residual functions, achieving higher accuracy and winning image recognition competitions.

Introduction

Deep convolutional neural networks are crucial for image classification, but increasing depth leads to a degradation problem where accuracy saturates and then declines, not due to overfitting.

Proposed Method

The deep residual learning framework addresses the degradation problem by explicitly letting stacked layers fit a residual mapping F(x) = H(x) - x, which is easier to optimize and can be implemented with shortcut connections.

Related Work

Residual learning draws inspiration from various fields like image recognition (VLAD, Fisher Vector), PDEs, and earlier neural network practices that utilized shortcut connections for improved optimization and performance.

Learning Residual Functions

Reformulating the learning objective to approximate a residual function F(x) = H(x) - x, instead of the direct mapping H(x), is hypothesized to simplify optimization, especially when the optimal function is close to an identity mapping.

Residual Building Block

The residual learning framework utilizes building blocks where the output y is the sum of the input x and a learned residual mapping F(x), implemented with parameter-free identity shortcut connections.

Implementation Details

Plain and residual networks are implemented with 3x3 convolutional filters, following specific design rules for feature map size and filter count, with identity shortcuts used for residual versions.

Training Procedures

Images are augmented, and models are trained using SGD with batch normalization, momentum, and weight decay, with learning rates adjusted as errors plateau.

Experiments on ImageNet

Experiments on ImageNet show that deeper plain networks suffer from higher training error (degradation problem), while residual networks overcome this, demonstrating accuracy gains from increased depth.

Experiments on CIFAR-10

The degradation problem is also observed on CIFAR-10, and residual networks successfully address it, achieving accuracy gains with increased depth, and even demonstrating stable training for networks over 1000 layers.

Analysis of Layer Responses

Analysis of layer responses reveals that residual functions in ResNets generally have smaller magnitudes, supporting the hypothesis that they are closer to zero and thus easier to learn.

Generalization to Other Tasks

Residual networks exhibit strong generalization performance on object detection tasks, providing significant improvements over standard networks and contributing to winning multiple competitions.

Object Detection

Integrating ResNet into the Faster R-CNN framework for object detection significantly improves performance, particularly on the COCO dataset, attributing gains solely to the learned representations.

Further Improvements

Techniques such as box refinement, global context, and multi-scale testing further enhance object detection performance when combined with residual networks.

Architecture Details

Deeper residual networks utilize bottleneck architectures with 1x1, 3x3, and 1x1 convolutions to reduce computational complexity, with identity shortcuts being crucial for efficiency.

Object Detection Improvements and Results

Faster R-CNN with ResNet-101 and ensemble methods achieves state-of-the-art results on MS COCO and PASCAL VOC detection tasks, winning first place in COCO 2015.

ImageNet Detection and Localization Tasks

A modified RPN framework with per-class regression achieves state-of-the-art results in ImageNet detection and localization tasks, winning first place in ILSVRC 2015.

Per-Class RPN and R-CNN for Localization

A per-class RPN and R-CNN approach, trained for class-dependent proposals, significantly reduces localization error and wins first place in the ILSVRC 2015 ImageNet localization task.