[Paper Review] On Machine Learning and Structure for Mobile Robots

Markus Wulfmeier|arXiv (Cornell University)|Jun 15, 2018

Robotic Path Planning Algorithms109 references4 citations

TL;DR

This paper surveys the integration of machine learning and structured prior knowledge in mobile robotics, arguing that combining learned models with geometric, kinematic, and dynamic priors enables more robust, efficient, and safe autonomous systems. It demonstrates that hybrid approaches—where learning enhances or corrects traditional modules—yield superior performance compared to pure learning or handcrafted systems alone.

ABSTRACT

Due to recent advances - compute, data, models - the role of learning in autonomous systems has expanded significantly, rendering new applications possible for the first time. While some of the most significant benefits are obtained in the perception modules of the software stack, other aspects continue to rely on known manual procedures based on prior knowledge on geometry, dynamics, kinematics etc. Nonetheless, learning gains relevance in these modules when data collection and curation become easier than manual rule design. Building on this coarse and broad survey of current research, the final sections aim to provide insights into future potentials and challenges as well as the necessity of structure in current practical applications.

Motivation & Objective

• To analyze the current role of machine learning in perception, localization, mapping, planning, and control for mobile robots.
• To investigate how structured prior knowledge (e.g., geometry, dynamics) complements and enhances learning-based approaches.
• To evaluate the trade-offs between end-to-end learning and modular, structured systems in practical robotics applications.
• To identify key challenges and future directions in merging learning with prior knowledge for robust and safe autonomous systems.
• To advocate for a hybrid paradigm—'learning with structure'—as the most effective path for near-term deployable robotics.

Proposed method

• Surveying state-of-the-art learning-based methods across perception, localization, mapping, tracking, planning, and control modules in mobile robotics.
• Analyzing hybrid architectures that combine differentiable, learnable components with traditional, handcrafted modules (e.g., pose correction for visual odometry, SLAM-informed RL).
• Examining self-supervised learning techniques that leverage geometric and temporal structure (e.g., differentiable image warping for depth and pose prediction).
• Evaluating data augmentation and inductive biases (e.g., translation invariance, objectness, temporal coherence) as forms of structured knowledge in deep learning.
• Reviewing approaches that use prior knowledge to improve generalization, even when models are inaccurate, through structured supervision or architectural constraints.
• Comparing end-to-end learning with structured, modular systems in terms of data efficiency, interpretability, safety, and real-world deployability.

Experimental results

Research questions

• RQ1 How can machine learning be effectively combined with prior knowledge (e.g., geometry, dynamics) to improve performance in mobile robot perception and control?
• RQ2 In what ways do structured priors enhance the sample efficiency, robustness, and safety of learned models in robotics?
• RQ3 What are the practical trade-offs between fully learned systems and hybrid systems that integrate traditional programming with learning?
• RQ4 To what extent can self-supervised learning based on geometric structure (e.g., reprojection errors) reduce reliance on annotated data?
• RQ5 How can the integration of SLAM and map information improve the training and generalization of reinforcement learning policies?

Key findings

• Deep learning has achieved dominant performance in perception tasks such as object detection, semantic segmentation, and depth estimation, largely due to large-scale datasets like ImageNet.
• Despite advances in perception, localization, mapping, and planning modules still rely heavily on handcrafted rules and geometric priors due to safety, interpretability, and data efficiency constraints.
• Hybrid systems—where learning corrects or enhances traditional modules (e.g., pose refinement, image enhancement for visual odometry)—achieve better performance and robustness than pure learning or pure rule-based systems.
• Self-supervised learning using geometric structure (e.g., differentiable image warping) enables training without explicit labels, reducing annotation burden while maintaining accuracy.
• Incorporating inductive biases such as translation invariance, temporal coherence, and objectness improves generalization and sample efficiency in deep learning for robotics.
• Even when prior models are inaccurate, embedding structural knowledge into learning frameworks can still improve performance, demonstrating the resilience and utility of structured priors.