[Paper Review] Towards CNN Map Compression for camera relocalisation

Luis Contreras, Walterio Mayol‐Cuevas|arXiv (Cornell University)|Mar 2, 2017

Advanced Vision and Imaging30 references3 citations

TL;DR

This paper proposes a compact CNN-based map representation for camera relocalisation that enables map compression by training a small, fixed-size CNN (CNN-F) with increasing numbers of training trajectories. Despite its minimal architecture, the model achieves relocalisation accuracy comparable to larger networks like PoseNet (0.59m mean error), demonstrating that performance improves with more training data while maintaining constant model size.

ABSTRACT

This paper presents a study on the use of Convolutional Neural Networks for camera relocalisation and its application to map compression. We follow state of the art visual relocalisation results and evaluate response to different data inputs -- namely, depth, grayscale, RGB, spatial position and combinations of these. We use a CNN map representation and introduce the notion of CNN map compression by using a smaller CNN architecture. We evaluate our proposal in a series of publicly available datasets. This formulation allows us to improve relocalisation accuracy by increasing the number of training trajectories while maintaining a constant-size CNN.

Motivation & Objective

• To explore compact CNN architectures for efficient camera relocalisation map representation.
• To investigate how different sensor inputs (RGB, depth, spatial position) affect relocalisation accuracy.
• To evaluate whether increasing training trajectories improves accuracy without increasing model size.
• To demonstrate that map compression can be achieved by maintaining a fixed CNN architecture while retraining with new data.
• To compare the performance of a lightweight CNN-F (8 layers) against more complex models like PoseNet (23 layers)

Proposed method

• Uses a lightweight 8-layer CNN-F architecture with 5 convolutional and 3 fully connected layers, adapted to input dimensionality (n) based on data type.
• Employs a PoseNet-inspired loss function combining L2 distance for translation and normalized quaternion error for orientation.
• Processes inputs including RGB, grayscale, depth, 3D spatial position, and their combinations, with 224×224 spatial resolution after cropping and resizing.
• Trains the same CNN-F model incrementally by adding new trajectories from the 7-Scenes dataset, updating only the weights while keeping architecture fixed.
• Evaluates performance on public datasets (TUM, 7-Scenes) using mean relocalisation error in meters as the primary metric.
• Compares results across input modalities and with PoseNet baseline to assess relative performance

Experimental results

Research questions

• RQ1 How does the performance of a compact CNN (CNN-F) compare to larger models like PoseNet in camera relocalisation tasks?
• RQ2 Which input modality (RGB, depth, grayscale, spatial position, or combinations) yields the best relocalisation accuracy?
• RQ3 Can relocalisation accuracy be improved by increasing the number of training trajectories without increasing the model size?
• RQ4 To what extent can a fixed-size CNN represent increasingly complex map information through incremental retraining?
• RQ5 Is it possible to achieve map compression by using a minimal CNN architecture that maintains high accuracy across diverse inputs?

Key findings

• The CNN-F model achieves a mean relocalisation error of 0.589 meters on the Red Kitchen sequence using RGB input, matching the 0.59m error reported by PoseNet.
• RGB input yields the best performance (0.589m error), outperforming depth (1.326m), grayscale (0.795m), and point cloud (0.791m) inputs.
• Adding more training trajectories consistently reduces relocalisation error without increasing model size, indicating scalability and potential for compression.
• The model shows no sign of saturation in performance even after training on multiple trajectories, suggesting further improvements are possible with more data.
• The CNN-F architecture achieves comparable accuracy to PoseNet despite having only 8 layers, demonstrating that model size can be drastically reduced without performance loss.
• The map representation remains constant in size (fixed architecture) while information content increases through weight updates from new trajectories, enabling effective map compression.