[Paper Review] Near-field Sensing Architecture for Low-Speed Vehicle Automation using a Surround-view Fisheye Camera System.

Ciarán Eising, Jonathan Horgan+1 more|arXiv (Cornell University)|Mar 31, 2021

Robotics and Sensor-Based Localization89 references4 citations

TL;DR

This paper proposes a 4R architecture—Recognition, Reconstruction, Relocalization, and Reorganization—for near-field sensing in low-speed automated driving using a surround-view fisheye camera system. The modular framework enables high-accuracy, close-range perception for applications like automated parking and traffic jam assistance by synergistically processing fisheye camera data to enhance spatial understanding and system robustness.

ABSTRACT

Cameras are the primary sensor in automated driving systems. They provide high information density and are optimal for detecting road infrastructure cues laid out for human vision. Surround view cameras typically comprise of four fisheye cameras with 190° field-of-view covering the entire 360° around the vehicle focused on near field sensing. They are the principal sensor for low-speed, high accuracy and close-range sensing applications, such as automated parking, traffic jam assistance and low-speed emergency braking. In this work, we describe our visual perception architecture on surround view cameras designed for a system deployed in commercial vehicles, provide a functional review of the different stages of such a computer vision system, and discuss some of the current technological challenges. We have designed our system into four modular components namely Recognition, Reconstruction, Relocalization and Reorganization. We jointly call this the 4R Architecture. We discuss how each component accomplishes a specific aspect and how they are synergized to form a complete system. Qualitative results are presented in the video at \url{this https URL}.

Motivation & Objective

• Address the need for high-accuracy, low-speed perception in automated driving systems using visual sensors.
• Overcome limitations of conventional perception systems in near-field, close-range environments.
• Develop a modular computer vision pipeline tailored for commercial vehicle deployment.
• Integrate fisheye camera data effectively to support real-time, 360° situational awareness.
• Enable robust performance in automated parking and traffic jam assistance through advanced visual processing.

Proposed method

• Design a modular 4R architecture comprising Recognition, Reconstruction, Relocalization, and Reorganization components.
• Use fisheye cameras with 190° field-of-view to cover 360° around the vehicle for near-field sensing.
• Apply recognition techniques to detect road infrastructure and obstacles in the near field.
• Perform 3D reconstruction from multiple fisheye images to build a spatial representation of the environment.
• Implement relocalization to maintain consistent pose estimation using visual features and temporal consistency.
• Use reorganization to fuse and refine perception outputs across time and views for improved reliability.

Experimental results

Research questions

• RQ1 How can a surround-view fisheye camera system be effectively leveraged for near-field, low-speed automated driving tasks?
• RQ2 What modular architecture enables robust and accurate perception in close-range, dynamic environments?
• RQ3 How do Recognition, Reconstruction, Relocalization, and Reorganization components synergize to improve system performance?
• RQ4 What are the key technical challenges in processing fisheye camera data for real-time vehicle automation?
• RQ5 How does the 4R architecture enhance accuracy and reliability in automated parking and traffic jam assistance?

Key findings

• The 4R architecture enables high-accuracy perception in low-speed, near-field scenarios using only fisheye cameras.
• The system effectively processes 360° fisheye imagery to support real-time automated driving functions.
• Modular design allows for scalable and maintainable implementation in commercial vehicle systems.
• Qualitative results demonstrate robust performance in automated parking and emergency braking scenarios.
• The architecture addresses key challenges in fisheye image processing, such as distortion correction and multi-view fusion.
• The system achieves reliable relocalization and consistent spatial understanding through temporal and geometric consistency.