Modeling the Background for Incremental Learning in Semantic Segmentation

Conf. on Computer Vision and Pattern Recognition (CVPR) 2020 /
By Fabio Cermelli, Massimiliano Mancini, Samuel Rota Bulò, Elisa Ricci, Barbara Caputo

Abstract

Despite their effectiveness in a wide range of tasks, deep architectures suffer from some important limitations. In particular, they are vulnerable to catastrophic forgetting, i.e. they perform poorly when they are required to update their model as new classes are available but the original training set is not retained. This paper addresses this problem in the context of semantic segmentation. Current strategies fail on this task because they do not consider a peculiar aspect of semantic segmentation: since each training step provides annotation only for a subset of all possible classes, pixels of the background class (i.e. pixels that do not belong to any other classes) exhibit a semantic distribution shift. In this work we revisit classical incremental learning methods, proposing a new distillation-based framework which explicitly accounts for this shift. Furthermore, we introduce a novel strategy to initialize classifier’s parameters, thus preventing biased predictions toward the background class. We demonstrate the effectiveness of our approach with an extensive evaluation on the Pascal-VOC 2012 and ADE20K datasets, significantly outperforming state of the art incremental learning methods.

Supplementary Video

More publications

Mapillary Planet-Scale Depth Dataset

By Manuel López-Antequera, Pau Gargallo, Markus Hofinger, Samuel Rota Bulò, Yubin Kuang, Peter Kontschieder
European Conf. on Computer Vision (ECCV) 2020 /

Improving Optical Flow on a Pyramid Level

By Markus Hofinger, Samuel Rota Bulò, Lorenzo Porzi, Arno Knapitsch, Thomas Pock, Peter Kontschieder
European Conf. on Computer Vision (ECCV) 2020 /

Towards Generalization Across Depth for Monocular 3D Object Detection

By Andrea Simonelli, Samuel Rota Bulò, Lorenzo Porzi, Elisa Ricci, Peter Kontschieder
European Conf. on Computer Vision (ECCV) 2020 /

The Mapillary Traffic Sign Dataset for Detection and Classification on a Global Scale

By Christian Ertler, Jerneja Mislej, Tobias Ollmann, Lorenzo Porzi, Gerhard Neuhold, Yubin Kuang
European Conf. on Computer Vision (ECCV) 2020 /

Mapillary Street-Level Sequences: A Dataset for Lifelong Place Recognition

By Frederik Warburg, Soren Hauberg, Manuel López-Antequera, Pau Gargallo, Yubin Kuang, Javier Civera
Conf. on Computer Vision and Pattern Recognition (CVPR) 2020 /

Learning Multi-Object Tracking and Segmentation from Automatic Annotations

By Lorenzo Porzi, Markus Hofinger, Idoia Ruiz, Joan Serrat, Samuel Rota Bulò, Peter Kontschieder
Conf. on Computer Vision and Pattern Recognition (CVPR) 2020 /

Disentangling Monocular 3D Object Detection

By Andrea Simonelli, Samuel Rota Bulò, Lorenzo Porzi, Manuel López-Antequera, Peter Kontschieder
International Conf. on Computer Vision (ICCV) 2019 /

Seamless Scene Segmentation

By Lorenzo Porzi, Samuel Rota Bulò, Aleksander Colovic, Peter Kontschieder
Conf. on Computer Vision and Pattern Recognition (CVPR) 2019 /

AdaGraph: Unifying Predictive and Continuous Domain Adaptation through Graphs

By Massimiliano Mancini, Samuel Rota Bulò, Barbara Caputo, Elisa Ricci
Conf. on Computer Vision and Pattern Recognition (CVPR) 2019 /

Unsupervised Domain Adaptation using Feature-Whitening and Consensus Loss

By Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Samuel Rota Bulò, Nicu Sebe, Elisa Ricci
Conf. on Computer Vision and Pattern Recognition (CVPR) 2019 /

Deep Single Image Camera Calibration with Radial Distortion

By Manuel López-Antequera, Roger Marı́, Pau Gargallo, Yubin Kuang, Javier Gonzalez-Jimenez, Gloria Haro
Conf. on Computer Vision and Pattern Recognition (CVPR) 2019 /

In-Place Activated BatchNorm for Memory-Optimized Training of DNNs

By Samuel Rota Bulò, Lorenzo Porzi, Peter Kontschieder
Conf. on Computer Vision and Pattern Recognition (CVPR) 2018 /

Boosting Domain Adaptation by Discovering Latent Domains

By Massimilano Mancini, Lorenzo Porzi, Samuel Rota Bulò, Barbara Caputo, Elisa Ricci
Conf. on Computer Vision and Pattern Recognition (CVPR) 2018 /

Geometry-Aware Network for Non-Rigid Shape Prediction from a Single View

By Albert Pumarola, Antonio Agudo, Lorenzo Porzi, Alberto Sanfeliu, Vincent Lepetit, Francesc Moreno-Noguer
Conf. on Computer Vision and Pattern Recognition (CVPR) 2018 /

The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes

By Gerhard Neuhold, Tobias Ollmann, Samuel Rota Bulò, Peter Kontschieder
International Conf. on Computer Vision (ICCV) 2017 /

AutoDIAL: Automatic DomaIn Alignment Layers

By Fabio Maria Carlucci, Lorenzo Porzi, Barbara Caputo, Elisa Ricci, Samuel Rota Bulò
International Conf. on Computer Vision (ICCV) 2017 /

Loss Max-Pooling for Semantic Image Segmentation

By Samuel Rota Bulò, Gerhard Neuhold, Peter Kontschieder
Conf. on Computer Vision and Pattern Recognition (CVPR) 2017 /

Online Learning with Bayesian Classification Trees

By Samuel Rota Bulò, Peter Kontschieder
Conf. on Computer Vision and Pattern Recognition (CVPR) 2016 /

Dropout Distillation

By Samuel Rota Bulò, Lorenzo Porzi, Peter Kontschieder
Intern. Conf. on Machine Learning (ICML) 2016 /