[Paper Review] Human-centered manipulation and navigation with Robot DE NIRO

Fabian Falck, Sagar Doshi+4 more|arXiv (Cornell University)|Oct 23, 2018

Social Robot Interaction and HRIPsychology15 references3 citations

TL;DR

This paper presents DE NIRO, a human-centered mobile manipulation robot designed to assist geriatric caregivers by autonomously retrieving and delivering objects like medicine. Equipped with Baxter arms, LIDAR-based navigation, and perception via speech and face recognition, DE NIRO enables safe, intuitive human-robot interaction through collision avoidance and dynamic path planning, demonstrating reliable performance in real-world scenarios with minimal human intervention.

ABSTRACT

Social assistance robots in health and elderly care have the potential to support and ease human lives. Given the macrosocial trends of aging and long-lived populations, robotics-based care research mainly focused on helping the elderly live independently. In this paper, we introduce Robot DE NIRO, a research platform that aims to support the supporter (the caregiver) and also offers direct human-robot interaction for the care recipient. Augmented by several sensors, DE NIRO is capable of complex manipulation tasks. It reliably interacts with humans and can autonomously and swiftly navigate through dynamically changing environments. We describe preliminary experiments in a demonstrative scenario and discuss DE NIRO's design and capabilities. We put particular emphases on safe, human-centered interaction procedures implemented in both hardware and software, including collision avoidance in manipulation and navigation as well as an intuitive perception stack through speech and face recognition.

Motivation & Objective

• To develop a robot assistant that supports geriatric caregivers by handling routine, repetitive tasks such as medicine delivery, thereby freeing time for more empathetic care.
• To enable safe, natural, and reliable human-robot interaction in dynamic, real-world care environments through integrated perception and collision avoidance.
• To create a modular, open-source research platform for mobile manipulation that emphasizes human-centered design and real-world deployability.
• To address ethical concerns in elderly care robotics by focusing on caregiver support rather than replacing human interaction.
• To demonstrate robustness in object manipulation and navigation using fiducial markers, inverse kinematics, and real-time path replanning.

Proposed method

• DE NIRO integrates Baxter dual arms with a QUICKIE electric wheelchair base, enabling mobile manipulation with passive compliance for safe human interaction.
• It uses a LIDAR-based SLAM system for real-time mapping and localization, with dynamic costmaps to maintain safe distances from obstacles and humans.
• Trajectory planning employs a timed elastic band algorithm to optimize for travel time and obstacle proximity, ensuring smooth and safe motion.
• Object recognition relies on 2D fiducial markers attached to target objects, enabling consistent 3D localization for manipulation tasks.
• A custom PID controller manages motor actuation, translating planned velocities into precise wheel movements for navigation.
• The system implements a three-stage grasping procedure: approach, grasp, and handover mode, with dynamic awareness to avoid collisions during motion.

Experimental results

Research questions

• RQ1 How can a robot be designed to safely and reliably assist caregivers in elderly care without replacing human interaction?
• RQ2 What perception and control architecture enables robust, real-time navigation and manipulation in dynamic, unstructured environments?
• RQ3 How can human-robot interaction be made intuitive and safe through speech, face recognition, and collision avoidance?
• RQ4 To what extent can a robot perform repeated, well-defined tasks like medicine delivery with minimal human supervision?
• RQ5 What are the limitations of current sensor and actuation systems in mobile manipulation for care robotics?

Key findings

• DE NIRO successfully demonstrated autonomous navigation and object retrieval in a dynamic environment using LIDAR-based SLAM and real-time path replanning.
• The use of 2D fiducial markers enabled consistent and robust object localization, outperforming generic object recognition methods in the tested scenarios.
• The robot achieved reliable grasping and handover through a three-stage procedure with dynamic awareness and collision avoidance.
• The system maintained safe distances from humans and obstacles using a dynamic costmap and real-time trajectory optimization.
• DE NIRO’s design supports natural interaction via speech and face recognition, enhancing usability in care settings.
• The platform is open-sourced with extensive documentation, enabling reproducibility and further research in human-centered robotics.