[Paper Review] Sequential Topological Representations for Predictive Models of Deformable Objects

Rika Antonova, Anastasiia Varava+3 more|arXiv (Cornell University)|Nov 23, 2020

3D Shape Modeling and Analysis30 references4 citations

TL;DR

This paper proposes a sequential topological representation method using persistent homology to track and predict the evolving topological state of highly deformable objects from point clouds. It introduces the sequential persistent homology (seqPH) algorithm and a learnable neural network model that predicts future topological features over a 60-step horizon, achieving faster and more robust predictions than standalone computational topology methods.

ABSTRACT

Deformable objects present a formidable challenge for robotic manipulation due to the lack of canonical low-dimensional representations and the difficulty of capturing, predicting, and controlling such objects. We construct compact topological representations to capture the state of highly deformable objects that are topologically nontrivial. We develop an approach that tracks the evolution of this topological state through time. Under several mild assumptions, we prove that the topology of the scene and its evolution can be recovered from point clouds representing the scene. Our further contribution is a method to learn predictive models that take a sequence of past point cloud observations as input and predict a sequence of topological states, conditioned on target/future control actions. Our experiments with highly deformable objects in simulation show that the proposed multistep predictive models yield more precise results than those obtained from computational topology libraries. These models can leverage patterns inferred across various objects and offer fast multistep predictions suitable for real-time applications.

Motivation & Objective

• Address the challenge of modeling and controlling highly deformable objects in robotics, which lack canonical low-dimensional representations.
• Develop a topological state representation that captures global shape features like holes, loops, and connectivity, invariant under continuous deformations.
• Enable real-time planning and control by learning multistep predictive models of topological evolution from sequences of point cloud observations.
• Provide a theoretically grounded method to recover topology from point clouds under mild assumptions about dynamics and observation quality.
• Integrate interpretable topological invariants like the linking number into task-specific cost functions for manipulation tasks such as hanging or knot tying.

Proposed method

• Use sequential persistent homology (seqPH) to extract and track topological features—specifically loops and holes—from time-series point cloud data of deformable objects.
• Represent each topological feature as a Gaussian mixture model (GMM) over the spatial locations of its associated simplices, with parameters derived from vertex positions and persistence values.
• Train a deep neural network (NN) to predict future topological states by conditioning on a sequence of past point clouds (over 16 timesteps) and a sequence of future control actions (up to 60 steps ahead).
• Leverage the NN as a regularizer to improve robustness to noise and trajectory-specific anomalies that may affect direct seqPH inference.
• Use the linking number as a topological invariant to define task-relevant cost functions, enabling planning for tasks like hooking or entanglement control.
• Apply PointNet and fully connected architectures to evaluate model performance, with training on 23,000 simulated trajectories across diverse deformable objects (clothing, bags).

Experimental results

Research questions

• RQ1 Can topological features such as loops and holes in deformable objects be reliably extracted and tracked from raw point cloud data without semantic labeling?
• RQ2 Under what assumptions can the evolution of topological states over time be theoretically recovered from noisy, sequential point cloud observations?
• RQ3 Can a learned neural network model predict future topological states over a multi-step horizon more accurately and robustly than direct computational topology methods?
• RQ4 How do different neural network architectures (e.g., PointNet vs. fully connected) affect the accuracy and generalization of topological state prediction?
• RQ5 To what extent can interpretable topological invariants like the linking number be used to define effective, task-specific cost functions for robotic manipulation?

Key findings

• The proposed predictive model with a PointNet architecture outperformed both the seqPH method and a standard fully connected NN in terms of accuracy on loop localization, achieving superior generalization.
• The neural network predictions produced tighter Gaussian mixture representations of loops compared to seqPH, suggesting regularization effects from learning across diverse trajectories.
• For a subset of 800 test trajectories with manually marked ground truth loop areas, the mean of the predicted mixture with the longest lifetime fell within the convex hull of the true loop area in a comparable or higher fraction of cases than seqPH.
• The model achieved fast, multistep predictions (up to 60 steps ahead) suitable for real-time control, with inference speed significantly outpacing standalone computational topology pipelines.
• The method successfully captured non-trivial topological features such as apron straps and bag handles in simulation, even under varying material properties and dynamic motions.
• The integration of topological invariants like the linking number enables the formulation of interpretable, high-level task objectives for complex manipulation tasks such as hanging or knot tying.