[Paper Review] DSOR: A Scalable Statistical Filter for Removing Falling Snow from LiDAR Point Clouds in Severe Winter Weather

Akhil Kurup, Jeremy P. Bos|arXiv (Cornell University)|Sep 15, 2021

Remote Sensing and LiDAR Applications45 citations

TL;DR

The paper introduces the Winter Adverse Driving dataSet (WADS) with dense point-wise labels for severe winter LiDAR data, and proposes the Dynamic Statistical Outlier Removal (DSOR) filter, a PCL-based method that outperforms state-of-the-art snow de-noising in recall and speed, with better scalability.

ABSTRACT

For autonomous vehicles to viably replace human drivers they must contend with inclement weather. Falling rain and snow introduce noise in LiDAR returns resulting in both false positive and false negative object detections. In this article we introduce the Winter Adverse Driving dataSet (WADS) collected in the snow belt region of Michigan's Upper Peninsula. WADS is the first multi-modal dataset featuring dense point-wise labeled sequential LiDAR scans collected in severe winter weather; weather that would cause an experienced driver to alter their driving behavior. We have labelled and will make available over 7 GB or 3.6 billion labelled LiDAR points out of over 26 TB of total LiDAR and camera data collected. We also present the Dynamic Statistical Outlier Removal (DSOR) filter, a statistical PCL-based filter capable or removing snow with a higher recall than the state of the art snow de-noising filter while being 28\\% faster. Further, the DSOR filter is shown to have a lower time complexity compared to the state of the art resulting in an improved scalability. Our labeled dataset and DSOR filter will be made available at https://bitbucket.org/autonomymtu/dsor_filter

Motivation & Objective

• Motivate robust autonomous driving perception under severe winter weather with falling snow.
• Provide a densely annotated winter LiDAR dataset (WADS) with point-wise labels and snow-specific classes.
• Develop and validate a dynamic, scalable snow de-noising filter (DSOR) for LiDAR point clouds.
• Compare DSOR to existing filters (SOR, DROR) in recall, precision, and speed.
• Demonstrate DSOR’s scalability for larger, higher-density LiDAR setups.

Proposed method

• Extend PCL’s Statistical Outlier Removal (SOR) to DSOR by incorporating range-based dynamic thresholds.
• Compute global threshold from k-nearest neighbor distances, then apply a dynamic threshold scaled by distance (T_d = T_g * r * distance).
• Use k-d tree for efficient nearest-neighbor searches (k-NN) with mean and standard deviation of neighbor distances.
• Label and provide 22 classes including active-snow and accumulated-snow to capture snow phenomena.
• Evaluate recall, precision, and runtime on sequences of LiDAR scans from heavy snow conditions.

Experimental results

Research questions

• RQ1 Can DSOR effectively remove falling snow while preserving environmental features in severe winter LiDAR data?
• RQ2 Does DSOR provide improved recall and faster processing compared to the state-of-the-art DROR filter under heavy snowfall?
• RQ3 How does DSOR scale with increasing LiDAR point density and larger point clouds?
• RQ4 What value does the WADS dataset offer for training and evaluating perception systems in adverse weather?

Key findings

• DSOR achieves higher recall (95.6%) than DROR (91.9%) in snow removal.
• DSOR attains 65.1% precision vs. DROR’s 71.5% in snow removal, indicating more non-snow points are removed by DSOR at certain ranges.
• DSOR removes more snow than DROR in the near field (within 20 m) and preserves more features overall than SOR and DROR in various ranges.
• DSOR is approximately 28% faster than DROR on average (369.68 ms vs. 510.55 ms per point cloud).
• DSOR exhibits lower time complexity (O(N log N)) than DROR (roughly O(N^(2/3))) and scales better with larger point clouds.