[Paper Review] FPGA-based ORB Feature Extraction for Real-Time Visual SLAM

Weikang Fang, Yanjun Zhang+2 more|arXiv (Cornell University)|Oct 18, 2017

Robotics and Sensor-Based Localization5 references3 citations

TL;DR

This paper presents an FPGA-based hardware accelerator for ORB feature extraction in real-time visual SLAM, achieving 67 FPS with 83% lower energy consumption than Intel Core i5 and 51% lower latency than ARM Krait by optimizing the orientation computation unit with 8-bit word length and pipelined processing, significantly improving performance and energy efficiency for mobile and IoT devices.

ABSTRACT

Simultaneous Localization And Mapping (SLAM) is the problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location within it. How to enable SLAM robustly and durably on mobile, or even IoT grade devices, is the main challenge faced by the industry today. The main problems we need to address are: 1.) how to accelerate the SLAM pipeline to meet real-time requirements; and 2.) how to reduce SLAM energy consumption to extend battery life. After delving into the problem, we found out that feature extraction is indeed the bottleneck of performance and energy consumption. Hence, in this paper, we design, implement, and evaluate a hardware ORB feature extractor and prove that our design is a great balance between performance and energy consumption compared with ARM Krait and Intel Core i5.

Motivation & Objective

• Address the challenge of high computational and energy costs in visual SLAM on mobile and IoT devices.
• Identify feature extraction as the primary bottleneck in SLAM pipelines, consuming over 50% of CPU resources and energy.
• Design a hardware accelerator for ORB feature extraction to reduce latency and energy consumption.
• Optimize the orientation computation unit for reduced word length (from 21 to 8 bits) while maintaining acceptable accuracy.
• Evaluate the FPGA-based ORB accelerator against ARM Krait and Intel Core i5 in terms of performance, throughput, and energy efficiency.

Proposed method

• Implement an FPGA-based ORB feature extractor using a pipelined architecture with dedicated modules for data access, control, and feature extraction.
• Optimize the orientation computation unit by truncating 21-bit fixed-point values to 8-bit word length using a two-step process: removing overlapping leading zeros and truncating to N bits.
• Apply word-length reduction to minimize LUT and register usage—achieving 65% and 82% reduction in LUTs and registers, respectively.
• Use a steered BRIEF descriptor computation method with rotation-invariant point pairs selected via Gaussian distribution and rotated using the computed orientation.
• Integrate the accelerator into an AXI-based system-on-FPGA architecture with DMA for efficient data transfer between camera and processing units.
• Employ a metric to quantify error from word-length truncation: $ \text{Error} = \sqrt{(x' - x'_N)^2 + (y' - y'_N)^2} $, ensuring minimal impact on feature point coordinates.

Experimental results

Research questions

• RQ1 Can a hardware-accelerated ORB feature extractor reduce latency and energy consumption in real-time visual SLAM compared to general-purpose CPU and mobile processor implementations?
• RQ2 How does reducing the word length of the orientation computation unit from 21 to 8 bits affect accuracy and hardware resource usage?
• RQ3 What is the trade-off between precision and resource efficiency in the orientation computation unit for ORB feature extraction?
• RQ4 To what extent can FPGA-based acceleration improve throughput and energy efficiency in visual SLAM systems?
• RQ5 Can the proposed ORB accelerator maintain sufficient accuracy for robust SLAM performance despite reduced precision in angle computation?

Key findings

• The FPGA-based ORB feature extractor achieves a throughput of 67 FPS on a Stratix V FPGA, significantly outperforming ARM Krait (33 FPS) and Intel Core i5 (40 FPS).
• Latency is reduced to 14.8ms per frame, representing a 51% improvement over ARM Krait and a 41% improvement over Intel Core i5.
• Energy consumption is reduced to 68 mJ/frame, achieving 9% lower energy than ARM Krait and 83% lower than Intel Core i5.
• Word length optimization from 21 to 8 bits reduces LUT usage by 65% and register usage by 82%, with minimal impact on accuracy as shown by error analysis.
• The maximum error introduced by 8-bit word length truncation is acceptably low, with exponential growth only at the farthest patch points (±15, ±15), and remains within tolerable bounds.
• The hardware design consumes 25,648 ALUTs, 21,791 registers, 1.18 MB BRAM, and 8 DSP blocks, demonstrating efficient resource utilization for real-time SLAM acceleration.