IDD-AW: A Benchmark for Safe and Robust Segmentation of Drive Scenes in Unstructured Traffic and Adverse Weather

Some examples from the dataset showing different weather conditions along with paired RGB-NIR images for each rain, fog, lowlight and snow.

Abstract

Large-scale deployment of fully autonomous vehicles requires a very high degree of robustness to unstructured traffic, weather conditions, and should prevent unsafe mispredictions. While there are several datasets and benchmarks focusing on segmentation for drive scenes, they are not specifically focused on safety and robustness issues. We introduce the IDD-AW dataset, which provides 5000 pairs of high-quality images with pixel-level annotations, captured under rain, fog, low light, and snow in unstructured driving conditions. As compared to other adverse weather datasets, we provide i.) more annotated images, ii.) paired Near-Infrared (NIR) image for each frame, iii.) larger label set with a 4-level label hierarchy to capture unstructured traffic conditions. We benchmark state-of-the-art models for semantic segmentation in IDD-AW. We also propose a new metric called “Safe mean Intersection over Union (Safe mIoU)” for hierarchical datasets which penalizes dangerous mispredictions that are not captured in the traditional definition of mean Intersection over Union (mIoU). The results show that IDD-AW is one of the most challenging datasets to date for these tasks. The dataset and code will be available here: http://iddaw.github.io.

IDD-AW Summary

IDD-AW is a groundbreaking dataset designed to address the challenges of autonomous driving in adverse weather conditions and unstructured environments. While existing datasets have primarily focused on well-organized, controlled settings, IDD-AW takes a different approach by capturing the complexities of real-world driving scenarios. The dataset is particularly unique for its focus on adverse weather conditions like rain, fog, snow and lowlight, collected in cities all across India.

Key Features:

Diverse Geographical Coverage: IDD-AW is collected across various states and terrains of India, from the highways of Hyderabad to the snowy hills of Manali and the foggy roads of Delhi and Ooty encompassing a range of road types, traffic densities, and adverse weather conditions like rain, fog, snow and lowlight
Rich Annotations: The dataset includes both NIR and RGB paired components for each image and annotated for semantic and instance segmentation.
High-Quality Data: The dataset is meticulously curated, with high-resolution RGB and NIR camera sensors capturing over 1 million frames. However, through thorough inspection and high quality checks, the final dataset has shortlisted 5000 RGB-NIR image pairs.
Unique Object Categories: Unlike other datasets that often generalize objects into broad categories, IDD-AW provides a more nuanced classification, including unique vehicle types and pedestrian behaviors commonly seen in unstructured environments and adverse weather conditions.

We show the pixel level comparison between IDD and IDD-AW for all labels. We can see that even though IDD-AW was collected in various challenging adverse weather conditions, it still has all the classes and almost comparable to normal SOTA driving datasets

We highlight multiple levels of drivable surfaces including roads, highways, flyovers etc. The data collection vehicles were driven across different parts of the city to capture the diversity in the road layouts and different heights. It is important to note that the objects in the scene are annotated irrespective of their height as long as they are visible in the LiDAR data. This allows for better understanding of geometric layouyt of the traffic scenes.

Poster

BibTeX



@inproceedings{shaik2024idd,
  title={IDD-AW: A Benchmark for Safe and Robust Segmentation of Drive Scenes in Unstructured Traffic and Adverse Weather},
  author={Shaik, Furqan Ahmed and Reddy, Abhishek and Billa, Nikhil Reddy and Chaudhary, Kunal and Manchanda, Sunny and Varma, Girish},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={4614--4623},
  year={2024}
}