[Paper Review] EdgeSegNet: A Compact Network for Semantic Segmentation

Zhong Qiu Lin, Brendan Chwyl+1 more|arXiv (Cornell University)|May 10, 2019

Advanced Neural Network Applications28 references4 citations

TL;DR

EdgeSegNet is a compact deep convolutional neural network for semantic segmentation, designed via a human-machine collaborative strategy that combines principled network prototyping with machine-driven architecture search. It achieves 89.7% accuracy on CamVid—within 0.6% of RefineNet—while being over 20× smaller (16.7 MB) and running at 38.5 FPS on an edge device (NVIDIA Jetson AGX Xavier), making it highly efficient for low-power, real-time edge deployment.

ABSTRACT

In this study, we introduce EdgeSegNet, a compact deep convolutional neural network for the task of semantic segmentation. A human-machine collaborative design strategy is leveraged to create EdgeSegNet, where principled network design prototyping is coupled with machine-driven design exploration to create networks with customized module-level macroarchitecture and microarchitecture designs tailored for the task. Experimental results showed that EdgeSegNet can achieve semantic segmentation accuracy comparable with much larger and computationally complex networks (>20x} smaller model size than RefineNet) as well as achieving an inference speed of ~38.5 FPS on an NVidia Jetson AGX Xavier. As such, the proposed EdgeSegNet is well-suited for low-power edge scenarios.

Motivation & Objective

• To develop a compact deep neural network for semantic segmentation tailored for low-power, on-device edge scenarios with strict computational and memory constraints.
• To address the challenge of deploying large, accurate semantic segmentation models (e.g., RefineNet) on resource-limited edge devices such as mobile phones, drones, and vehicles.
• To leverage a human-machine collaborative design strategy that combines human-driven architectural principles with automated search for optimal module-level macro- and microarchitectures.
• To achieve state-of-the-art segmentation accuracy while drastically reducing model size and inference latency compared to existing large models.
• To enable real-time, efficient semantic segmentation on edge hardware by optimizing for both accuracy and computational efficiency.

Proposed method

• Employ a human-machine collaborative design strategy: first, a principled network design prototype is created based on established segmentation principles, such as multi-path refinement and long-range skip connections.
• Use machine-driven design exploration to automatically determine optimal module-level macroarchitecture and microarchitecture configurations based on the dataset and predefined constraints (e.g., accuracy ≥88% on CamVid, model size <20 MB).
• Integrate heterogeneous macroarchitectures: a mix of residual and non-residual bottleneck modules to balance feature representation and computational efficiency.
• Implement selective long-range shortcut connections that refine only specific high-level features, reducing architectural complexity without sacrificing performance.
• Apply aggressive spatial reduction using 8×8 strided convolutions in non-residual bottleneck modules to drastically reduce feature map dimensions and computational load.
• Optimize the final architecture using a constrained search process guided by an indicator function $1_r(\cdot)$ that enforces accuracy and size constraints, resulting in a compact, high-performance model.

Experimental results

Research questions

• RQ1 Can a human-machine collaborative design strategy produce a compact semantic segmentation network that matches the accuracy of larger, state-of-the-art models while being significantly smaller and faster?
• RQ2 How effective is selective long-range shortcut connectivity in reducing architectural complexity without degrading segmentation performance?
• RQ3 To what extent can aggressive spatial reduction via large-stride convolutions (e.g., 8×8) maintain segmentation accuracy in a compact network?
• RQ4 Can a hybrid macroarchitecture of residual and non-residual bottleneck modules improve efficiency and performance in edge-optimized semantic segmentation?
• RQ5 What trade-offs exist between model size, inference speed, and accuracy in edge-deployable semantic segmentation networks, and can they be optimized effectively?

Key findings

• EdgeSegNet achieves 89.7% mean intersection-over-union (mIoU) accuracy on the CamVid dataset, which is within 0.6 percentage points of the much larger RefineNet (90.3%).
• The model size of EdgeSegNet is only 16.7 MB, representing a reduction of over 20× compared to RefineNet (343 MB), making it highly suitable for memory-constrained edge devices.
• EdgeSegNet attains an inference speed of approximately 38.5 FPS on the NVIDIA Jetson AGX Xavier, a low-power edge platform, enabling real-time semantic segmentation.
• RefineNet could not run on the Jetson AGX Xavier due to insufficient memory, highlighting the critical advantage of EdgeSegNet’s compact design.
• The use of selective long-range shortcuts and aggressive 8×8 strided convolutions in non-residual modules significantly reduces computational complexity while preserving high performance.
• The human-machine collaborative design process successfully generated a model that balances accuracy, size, and speed, outperforming larger models in efficiency metrics without major accuracy loss.