[Paper Review] Mixture of Pre-processing Experts Model for Noise Robust Deep Learning on Resource Constrained Platforms

Taesik Na, Minah Lee+4 more|arXiv (Cornell University)|Apr 29, 2019

Advanced Neural Network Applications20 references4 citations

TL;DR

This paper proposes a lightweight Mixture of Pre-processing Experts (MoPE) model to enhance deep learning robustness on resource-constrained edge devices under noisy and low-quality inputs. By using a gating network to dynamically select between identity, average filtering, or adversarially trained autoencoder pre-processing experts, MoPE achieves superior object detection, tracking, and activity classification accuracy on noisy data with only 0.67% memory and 0.17% computation overhead over baseline models.

ABSTRACT

Deep learning on an edge device requires energy efficient operation due to ever diminishing power budget. Intentional low quality data during the data acquisition for longer battery life, and natural noise from the low cost sensor degrade the quality of target output which hinders adoption of deep learning on an edge device. To overcome these problems, we propose simple yet efficient mixture of pre-processing experts (MoPE) model to handle various image distortions including low resolution and noisy images. We also propose to use adversarially trained auto encoder as a pre-processing expert for the noisy images. We evaluate our proposed method for various machine learning tasks including object detection on MS-COCO 2014 dataset, multiple object tracking problem on MOT-Challenge dataset, and human activity classification on UCF 101 dataset. Experimental results show that the proposed method achieves better detection, tracking and activity classification accuracies under noise without sacrificing accuracies for the clean images. The overheads of our proposed MoPE are 0.67% and 0.17% in terms of memory and computation compared to the baseline object detection network.

Motivation & Objective

• To address the challenge of deploying deep learning on power-constrained edge devices where sensor noise and low-resolution inputs degrade model performance.
• To develop a lightweight, efficient pre-processing solution that maintains high accuracy on clean data while improving robustness on noisy or low-quality inputs.
• To minimize computational and memory overhead in edge deployment by using a dynamic gating mechanism to route inputs to appropriate pre-processing experts.
• To demonstrate the effectiveness of MoPE across diverse tasks, including object detection, multiple object tracking, and human activity classification.
• To show that MoPE outperforms data augmentation and fixed pre-processing modules in both clean and noisy data scenarios.

Proposed method

• A gating network is trained to classify input image distributions (clean, noisy, low-resolution) and route them to the appropriate pre-processing expert.
• Three pre-processing experts are used: identity (for clean/low-res), average filtering (for low-res), and an adversarially trained autoencoder with skip connections (for noisy images).
• The adversarially trained autoencoder is optimized to reconstruct clean images from noisy inputs while preserving structural details, reducing noise without introducing spatial distortions.
• The MoPE framework is integrated before the main deep neural network (e.g., Faster R-CNN, Inception V2), with the pre-processed output fed into the backbone network.
• The gating network is trained end-to-end with the main task network using cross-entropy loss to optimize routing decisions based on input quality.
• Model complexity is minimized by using a small architecture for both the gating network and denoising autoencoder to ensure negligible overhead on edge platforms.

Experimental results

Research questions

• RQ1 Can a dynamic pre-processing pipeline improve deep learning performance on noisy and low-quality inputs without degrading accuracy on clean data?
• RQ2 How can a lightweight mixture of experts be designed to minimize memory and computation overhead on edge devices?
• RQ3 Does using an adversarially trained autoencoder as a pre-processing expert outperform traditional filtering methods in preserving detection and classification accuracy under noise?
• RQ4 Can MoPE generalize across different deep learning tasks such as object detection, multi-object tracking, and human activity recognition?
• RQ5 What is the actual memory and computational overhead of integrating MoPE into existing deep learning models on edge platforms?

Key findings

• On the MS-COCO 2014 dataset, MoPE achieved a mean average precision (mAP) of 0.531 at IoU=0.5 under Gaussian noise (σ=0.15), outperforming the baseline and data-augmented models.
• For multiple object tracking on MOT-Challenge, MoPE with the denoising expert achieved a higher MOTA score than all baselines, including data-augmented models, under both clean and noisy conditions.
• On UCF 101 for human activity classification, MoPE improved accuracy to 0.531 on noisy data (σ=0.15), compared to 0.492 for data-augmented models and 0.157 for the baseline.
• The MoPE model introduced only 0.67% additional memory and 0.17% additional computation overhead compared to the baseline Faster R-CNN with Inception V2.
• The gating network successfully routed inputs to appropriate experts, with the denoising autoencoder being selected for noisy inputs and identity for clean/low-res inputs.
• The method demonstrated consistent performance gains across convolutional and recurrent networks, proving its applicability to diverse deep learning tasks.