[Paper Review] Latest Datasets and Technologies Presented in the Workshop on Grasping and Manipulation Datasets

Matteo Bianchi, Jeannette Bohg+1 more|arXiv (Cornell University)|Sep 8, 2016

Robot Manipulation and Learning17 references3 citations

TL;DR

This paper summarizes the 2016 Workshop on Grasping and Manipulation Datasets, highlighting recent advances in open-access datasets for robotic grasping and manipulation. It presents key datasets like YCB, Dex-Net, and HandCorpus, and emphasizes the need for standardized, high-quality, multimodal datasets to enable benchmarking, improve learning, and accelerate progress in robotics.

ABSTRACT

This paper reports the activities and outcomes in the Workshop on Grasping and Manipulation Datasets that was organized under the International Conference on Robotics and Automation (ICRA) 2016. The half day workshop was packed with nine invited talks, 12 interactive presentations, and one panel discussion with ten panelists. This paper summarizes all the talks and presentations and recaps what has been discussed in the panels session. This summary servers as a review of recent developments in data collection in grasping and manipulation. Many of the presentations describe ongoing efforts or explorations that could be achieved and fully available in a year or two. The panel discussion not only commented on the current approaches, but also indicates new directions and focuses. The workshop clearly displayed the importance of quality datasets in robotics and robotic grasping and manipulation field. Hopefully the workshop could motivate larger efforts to create big datasets that are comparable with big datasets in other communities such as computer vision.

Motivation & Objective

• To review and consolidate the latest developments in open-access datasets for robotic grasping and manipulation research.
• To identify critical needs in data collection, standardization, and benchmarking to improve reproducibility and comparability across robotics research.
• To promote the creation of large-scale, multimodal, and coherent datasets comparable to those in computer vision.
• To address challenges in data accessibility, hardware cost, and the integration of perception, control, and learning modules in robotic systems.
• To stimulate community-wide coordination in dataset development and foster long-term data-sharing practices in robotics.

Proposed method

• Compilation and synthesis of nine invited talks and twelve interactive presentations from the ICRA 2016 workshop on grasping and manipulation datasets.
• Curation of datasets from diverse domains including object geometry, tactile sensing, human motion, instrument interaction, and robotic manipulation.
• Presentation and analysis of key datasets such as YCB (Yale-CMU-Berkeley), Dex-Net (cloud-based 3D object network), and HandCorpus (open-access repository for hand motion and grasp data).
• Evaluation of sensing modalities including RGBD, force/torque, haptic feedback, and motion capture for data collection and benchmarking.
• Discussion of standardized test environments and protocols to enable reproducible evaluation and reduce overfitting in benchmarking.
• Proposal of a framework for modular system evaluation, including metrics for speed, learning efficiency, and failure reporting to improve transparency and progress tracking.

Experimental results

Research questions

• RQ1 How can open-access, high-quality datasets improve benchmarking and reproducibility in robotic grasping and manipulation research?
• RQ2 What are the key challenges in collecting large-scale, multimodal datasets for robotic manipulation, especially involving tactile, force, and motion data?
• RQ3 How can standardized test environments and protocols reduce overfitting and improve generalization in robotic system evaluation?
• RQ4 What role do cloud-based datasets and shared repositories (e.g., HandCorpus, Dex-Net) play in democratizing access to robotic learning data?
• RQ5 How can modular system evaluation and baseline architectures help attribute progress to specific components in complex robotic systems?

Key findings

• The YCB Object and Model Set provides a standardized benchmark with 85 objects, including RGBD scans, physical properties, and geometric models for use in planning and control research.
• Dex-Net offers a cloud-based network of 3D objects with over 100,000 synthetic grasp configurations, enabling robust grasp planning through deep learning.
• HandCorpus is an open-access, login-free repository hosting diverse human and robot hand motion data, unifying datasets with varying formats and modalities.
• The KIT Whole-Body Human Motion Database provides high-fidelity, multi-view RGBD recordings of human motion, supporting studies in human-robot interaction and imitation learning.
• There is a growing trend toward automated data collection and the integration of tactile, force, and friction data to improve grasp stability assessment and robot learning.
• Despite progress, challenges remain in creating large-scale, coherent, and multimodal datasets, and the community still lacks a unified standard for data collection, sharing, and evaluation.