[Paper Review] EventMix: An Efficient Augmentation Strategy for Event-Based Data

Guobin Shen, Dongcheng Zhao+1 more|arXiv (Cornell University)|May 24, 2022

Advanced Memory and Neural Computing9 citations

TL;DR

This paper proposes EventMix, an efficient data augmentation strategy for event-based data that generates random 3D masks using a Gaussian Mixture Model to enable arbitrary spatio-temporal mixing of event streams. It introduces a novel label weight assignment based on relative distance and event count, significantly improving performance on neuromorphic datasets for both ANNs and SNNs, achieving state-of-the-art results on DVS-CIFAR10, N-Caltech101, N-CARS, and DVS-Gesture.

ABSTRACT

High-quality and challenging event stream datasets play an important role in the design of an efficient event-driven mechanism that mimics the brain. Although event cameras can provide high dynamic range and low-energy event stream data, the scale is smaller and more difficult to obtain than traditional frame-based data, which restricts the development of neuromorphic computing. Data augmentation can improve the quantity and quality of the original data by processing more representations from the original data. This paper proposes an efficient data augmentation strategy for event stream data: EventMix. We carefully design the mixing of different event streams by Gaussian Mixture Model to generate random 3D masks and achieve arbitrary shape mixing of event streams in the spatio-temporal dimension. By computing the relative distances of event streams, we propose a more reasonable way to assign labels to the mixed samples. The experimental results on multiple neuromorphic datasets have shown that our strategy can improve its performance on neuromorphic datasets both for ANNs and SNNs, and we have achieved state-of-the-art performance on DVS-CIFAR10, N-Caltech101, N-CARS, and DVS-Gesture datasets.

Motivation & Objective

• To address the challenge of small-scale, sparse event stream datasets that hinder model generalization and training stability in neuromorphic computing.
• To develop a data augmentation strategy tailored to the spatio-temporal dynamics and sparsity of event stream data, overcoming limitations of existing image-based augmentation methods.
• To improve model performance on event-based datasets for both artificial neural networks (ANNs) and spiking neural networks (SNNs) through more effective data mixing and label assignment.
• To enable efficient and diverse data mixing patterns that enhance model generalization without relying on fixed-shaped masks like in CutMix.

Proposed method

• EventMix generates random 3D masks using a Gaussian Mixture Model to enable flexible, arbitrary-shaped mixing of event streams across spatial and temporal dimensions.
• The method introduces a spatio-temporal mask generation process that respects the dynamic nature of event data, avoiding monotonous rectangular or square masks.
• It proposes two label weight assignment strategies: one based on the relative distance between original and mixed samples, and another based on the number of events in the mixed region.
• The label mixing strategy ensures more accurate and rational label assignment by considering the contribution of events in different regions of the mixed sample.
• The approach is compatible with both ANNs and SNNs, and is evaluated using standard architectures like ResNet34-ANN on multiple benchmark neuromorphic datasets.
• The method is designed to be efficient and scalable, with minimal computational overhead while maximizing data diversity and model robustness.

Experimental results

Research questions

• RQ1 Can a 3D mask generation strategy based on Gaussian Mixture Models improve data mixing diversity and model generalization in event stream data augmentation?
• RQ2 How does relative distance-based label assignment compare to event count-based assignment in preserving semantic accuracy during sample mixing?
• RQ3 To what extent does EventMix improve performance on small-scale neuromorphic datasets compared to existing augmentation baselines like MixUp and CutMix?
• RQ4 Does EventMix maintain consistent performance gains across different model types, including ANNs and SNNs, on diverse event-based benchmarks?
• RQ5 How effective is EventMix in low-data regimes, particularly when only a small fraction of training data is available?

Key findings

• EventMix achieved a 3.01% accuracy improvement over baseline data augmentation on the DVS-CIFAR10 dataset using ResNet34-ANN, with the best component combination.
• On DVS-CIFAR10, EventMix improved model performance by 2.45% compared to CutMix under identical training conditions.
• On N-Caltech101, EventMix delivered a 2.36% performance gain over CutMix, demonstrating consistent gains across datasets.
• On the DVS-Gesture dataset, EventMix achieved a 4.69% improvement over CutMix, highlighting its effectiveness on highly sparse data.
• With only 10% of the N-Caltech101 training set, EventMix achieved 54.06% validation accuracy, a 7.28% improvement over morphological data augmentation.
• EventMix reached state-of-the-art performance on DVS-CIFAR10, N-Caltech101, N-CARS, and DVS-Gesture for both ANNs and SNNs, confirming its broad effectiveness.