MULTI-SCALE CONTEXT AGGREGATION BY DILATED CONVOLUTIONS

Abstract

Dilated convolutions systematically aggregate multi-scale contextual information without losing resolution for dense prediction tasks.

1 Introduction

This work introduces a new convolutional network module for dense prediction that aggregates multi-scale context without losing resolution, addressing limitations of image classification network adaptations.

Dilated Convolutions

Dilated convolutions exponentially expand receptive fields without loss of resolution or coverage, enabling effective multi-scale context aggregation in convolutional networks.

3 Multi-Scale Context Aggregation

A new context module using dilated convolutions aggregates multi-scale contextual information in a plug-and-play fashion, improving dense prediction accuracy with specific initialization strategies.

4 Front End

A simplified front-end prediction module, adapted from VGG-16 by removing counterproductive classification layers and introducing dilation, achieves higher accuracy for dense prediction.

Figure 2

Semantic segmentations produced by different adaptations of the VGG-16 classification network are visualized, showing improved performance of the simplified front-end module.

5 Experiments

The context module, when plugged into various semantic segmentation architectures, consistently improves accuracy, demonstrating synergy with structured prediction methods.

Figure 3

Visual comparison of semantic segmentations from different models highlights the effectiveness of the context module and its combination with structured prediction.

6 Conclusion

Dilated convolutions are well-suited for dense prediction, and a new network structure utilizing them reliably increases accuracy, while simplifying existing networks also improves performance.

Table 4

The context network significantly boosts accuracy on the VOC-2012 test set, outperforming prior methods and demonstrating the value of dedicated dense prediction architectures.

REFERENCES

This section lists academic papers and datasets relevant to semantic segmentation and convolutional neural networks.

Appendix A: Urban Scene Understanding

Experiments were conducted on three urban scene understanding datasets (CamVid, KITTI, Cityscapes) using a convolutional network with front-end and context modules, achieving results without structured prediction.

A.1 CamVid Dataset Experiments

On the CamVid dataset, the Dilation8 model, combining a front-end and an 8-layer context module, outperformed previous work in semantic segmentation after joint training.

A.2 KITTI Dataset Experiments

The Dilation7 model, adapted for the KITTI dataset's resolution with a 7-layer context module, achieved higher accuracy in semantic segmentation compared to prior methods.

A.3 Cityscapes Dataset Experiments

The Dilation10 model, extended with two additional layers for the Cityscapes dataset, outperformed prior work in semantic segmentation after a three-stage training process.

Figure 5 and Tables 7-8: Cityscapes Results

Figure 5 visualizes the performance improvement of the Dilation10 model through different training stages, with detailed quantitative results presented in Tables 7 and 8.