ECCV 2020 Tutorial: International, City-Scale Computer Vision Benchmarking with Mapillary MegaCities
Mapillary Research is organizing a tutorial entitled
Advancing computer vision (CV) technologies is at the core of every ECCV, CVPR and ICCV conference. Many CV tasks have seen extraordinary performance gains in the past decade, often driven by the ability to rigorously benchmark methods on datasets like ImageNet, Pascal VOC, or the KITTI benchmark suite, to name just a few. Autonomous driving, i.e. a concrete example depending on a number of CV algorithms, can only be safely deployed in real-life conditions after benchmarking existing solutions in rich testing environments and at scale, and with test results made available to the public. Conversely, benchmarking of autonomously driving systems is often tackled by using proprietary datasets not available to the public research community, on datasets with limited geographical coverage or by testing against simulated data.
To overcome this limitation, we introduce a novel, publicly available large-scale dataset called
Cities can be accessed as highly accurate, reconstructed 3D point clouds, geolocated via landmarks captured at land-surveying grade accuracy. Correspondences of objects are provided across images, 3D models, aerial imagery, street-level LiDAR, aerial LiDAR and according to actual elevation information, where available. The dataset dynamically grows with new images uploaded through contributors and all information will be fully accessible through Mapillary’s platform and APIs. MegaCities will empower benchmarking of object recognition (bounding box based, 2D and 3D), object tracking, depth estimation, 3D reconstruction, domain adaptation, object re-identification, optical flow, object property and occlusion estimation, change detection, etc.
Given the complexity of the new MegaCities dataset and the positive impact it might have on the community, the goal of this tutorial is to provide an early introduction to the dataset, related benchmarking tools and pre-trained models in order to facilitate experimentation by interested researchers and engineers.
Provided Course Material
We will make the following material publicly available before the tutorial date: Online registration for data access, code for online access and evaluation of trained models, pre-trained models serving as baselines for all benchmarking tasks, technical report with dataset description. All material will be available from this website.
Tentative Tutorial Agenda
The tutorial will focus on three main topics: i) introducing MegaCities and its novel design philosophy as a living-and-breathing dataset, ii) describing the data gathering and annotation process, and iii) overviewing the dataset's usage, both as a training and benchmarking tool. The tentative tutorial agenda is:
|The MegaCities Dataset||(30 min)|
|Dataset Creation Process||(60 min)|
|Coffee Break||(15 min)|
|Dataset Usage||(60 min)|
|Pre-trained models||(10 min)|
|Questions and Discussions||(15 min)|