[Paper Review] Predicting Terrain Mechanical Properties in Sight for Planetary Rovers with Semantic Clues

Ruyi Zhou, Wenhao Feng+5 more|arXiv (Cornell University)|Nov 3, 2020

Robotic Locomotion and Control50 references4 citations

TL;DR

This paper proposes a vision-based method for predicting terrain mechanical properties (bearing and shearing resistance) in advance using semantic clues from rover-mounted cameras, enabling proactive hazard avoidance. It uses a lightweight semantic segmentation network followed by human-like inference to predict properties with 12.5% and 10.8% full-scale error on test routes, significantly improving planning safety for planetary rovers.

ABSTRACT

Non-geometric mobility hazards such as rover slippage and sinkage posing great challenges to costly planetary missions are closely related to the mechanical properties of terrain. In-situ proprioceptive processes for rovers to estimate terrain mechanical properties need to experience different slip as well as sinkage and are helpless to untraversed regions. This paper proposes to predict terrain mechanical properties with vision in the distance, which expands the sensing range to the whole view and can partly halt potential slippage and sinkage hazards in the planning stage. A semantic-based method is designed to predict bearing and shearing properties of terrain in two stages connected with semantic clues. The former segmentation phase segments terrain with a light-weighted network promising to be applied onboard with competitive 93% accuracy and high recall rate over 96%, while the latter inference phase predicts terrain properties in a quantitative manner based on human-like inference principles. The prediction results in several test routes are 12.5% and 10.8% in full-scale error and help to plan appropriate strategies to avoid suffering non-geometric hazards.

Motivation & Objective

• To address the challenge of non-geometric mobility hazards like slippage and sinkage in planetary rover missions.
• To extend sensing range beyond in-situ proprioception by predicting terrain mechanical properties from visual data in the distance.
• To develop a method that leverages semantic clues to infer quantitative mechanical properties of terrain before traversal.
• To enable proactive path planning by estimating terrain properties before the rover reaches hazardous regions.
• To achieve high accuracy and recall for terrain segmentation while supporting onboard deployment.

Proposed method

• A lightweight semantic segmentation network is used to classify terrain types in real-time from onboard camera images.
• The method employs a two-stage pipeline: first segmenting terrain into semantic categories, then inferring mechanical properties based on learned human-like reasoning principles.
• Mechanical property prediction is performed quantitatively using semantic clues derived from segmented terrain regions.
• The inference phase models terrain behavior using analogies to human perception, assigning mechanical properties based on visual texture, color, and structure.
• The system is designed for on-board deployment with high efficiency and low computational cost.
• The approach integrates semantic segmentation with physical property estimation to expand sensing range beyond direct contact.

Experimental results

Research questions

• RQ1 Can terrain mechanical properties be predicted from visual data in the distance to anticipate non-geometric hazards?
• RQ2 How accurately can semantic segmentation and inference models estimate bearing and shearing resistance of planetary terrain?
• RQ3 To what extent does using semantic clues improve prediction accuracy compared to purely geometric or physical models?
• RQ4 Can the proposed method be deployed onboard a rover with acceptable computational overhead?
• RQ5 How effective is the method in reducing slippage and sinkage risks during path planning?

Key findings

• The semantic segmentation model achieved 93% accuracy and a recall rate exceeding 96%, demonstrating strong performance for onboard deployment.
• The prediction method reduced full-scale error to 12.5% for bearing resistance and 10.8% for shearing resistance on test routes.
• The system successfully expanded sensing range beyond in-situ measurements, enabling hazard anticipation during mission planning.
• The two-stage approach—segmentation followed by semantic-informed inference—proved effective in estimating mechanical properties without direct physical contact.
• The method enables proactive avoidance of non-geometric hazards such as slippage and sinkage before traversal.
• The results demonstrate the feasibility of using vision and semantic cues to predict terrain mechanics in real-time for planetary rovers.