[Paper Review] Lighter-Than-Air Autonomous Ball Capture and Scoring Robot -- Design, Development, and Deployment

Joseph Prince Mathew, Dinesh Karri+7 more|arXiv (Cornell University)|Sep 12, 2023

Robotic Path Planning AlgorithmsComputer Science3 citations

TL;DR

This paper presents a lightweight, autonomous lighter-than-air robot designed for the Defend The Republic (DTR) robotics competition, using a monocular RGB camera for perception, four propellers for 3D maneuvering, and a PD control system to capture and score neutrally buoyant balls. The system achieved reliable autonomous ball capture and goal scoring in a real competition, demonstrating a minimal viable design for LTA robotics with strong performance under real-world constraints.

ABSTRACT

This paper describes the full end-to-end design of our primary scoring agent in an aerial autonomous robotics competition from April 2023. As open-ended robotics competitions become more popular, we wish to begin documenting successful team designs and approaches. The intended audience of this paper is not only any future or potential participant in this particular national Defend The Republic (DTR) competition, but rather anyone thinking about designing their first robot or system to be entered in a competition with clear goals. Future DTR participants can and should either build on the ideas here, or find new alternate strategies that can defeat the most successful design last time. For non-DTR participants but students interested in robotics competitions, identifying the minimum viable system needed to be competitive is still important in helping manage time and prioritizing tasks that are crucial to competition success first.

Motivation & Objective

• To develop a minimal viable autonomous robot capable of capturing and scoring neutrally buoyant balls in the LTA-based DTR competition.
• To enable rapid deployment of a homogeneous fleet of lightweight, air-borne robots with minimal hardware and software complexity.
• To identify and implement critical subsystems—sensing, actuation, propulsion, and control—necessary for autonomous operation in a 3D aerial environment.
• To demonstrate a functional, end-to-end system that can operate autonomously for 30 minutes in a real competition setting.
• To provide a documented, reusable design framework for future robotics teams entering similar competitions.

Proposed method

• A monocular USB camera (OV5640) was used as the sole perception sensor for detecting colored balls and goals based on color and shape.
• Four propellers mounted on a custom gondola provided 3D maneuverability, enabling the robot to navigate to target locations.
• A PD (proportional-derivative) control system was implemented to guide the robot toward targets with a 5–10 cm positional error tolerance.
• The robot’s control system used in-situ labeling and retraining of object detection models during competition week to adapt to venue-specific lighting and color conditions.
• A monolithic controller board (72g) integrated all processing, minimizing size and weight while enabling real-time perception and control.
• The robot’s envelope was designed to hold 200 cu.ft of helium, providing sufficient lift for the total system weight, with careful component weight balancing to maximize team capacity.

Figure 1: Defend The Republic (DTR) game scenario showing two teams (Red versus Blue) with green and purple balls scattered in the environment and goals (one triangular, circular and square per side) in fixed locations. The agents must capture these green balls and score by completely passing the ba

Experimental results

Research questions

• RQ1 What minimal hardware and software configuration enables a fully autonomous, lighter-than-air robot to capture and score balls in a complex 3D environment?
• RQ2 How effective is a single RGB camera for real-time perception of color-coded game objects under variable lighting conditions?
• RQ3 To what extent can a PD control system with limited perception accuracy achieve reliable ball capture and goal scoring?
• RQ4 What are the key system-level challenges in deploying a fleet of homogeneous, lightweight LTA robots in a real competition?
• RQ5 How can perception and control be adapted in real time to improve performance during competition without prior environmental knowledge?

Key findings

• The robot successfully captured and scored balls autonomously during the April 2023 DTR competition, demonstrating full end-to-end functionality of the minimal viable system.
• In-situ labeling and retraining of the object detection model significantly improved perception performance under venue-specific lighting and color conditions.
• The system achieved stable flight and control with a total weight of 72g for the main controller board, enabling efficient integration of processing and control.
• Despite reliance on a single RGB camera, the robot achieved sufficient detection accuracy for ball capture and goal scoring with a 5–10 cm positional tolerance.
• The team deployed a fleet of agents, but setup and tuning took over two days due to mechanical idiosyncrasies and lack of standardization.
• Perception performance was highly sensitive to lighting conditions, highlighting the need for multi-sensor fusion (e.g., IR for retro-reflective goals) in future designs.

Figure 2: Simplified blimp diagram with motors and cage.