[Paper Review] AprilTags 3D: Dynamic Fiducial Markers for Robust Pose Estimation in Highly Reflective Environments and Indirect Communication in Swarm Robotics

Luis A. Mateos|arXiv (Cornell University)|Jan 11, 2020

Robotics and Sensor-Based Localization15 references10 citations

TL;DR

This paper introduces AprilTags3D, a 3D fiducial marker system that enhances pose estimation accuracy in highly reflective outdoor environments by using multiple AprilTags mounted on non-coplanar, rotated planes. By fusing pose estimates from these spatially diverse tags in real time, the method reduces noise and improves detection reliability—achieving 95% tag detection in outdoor aquatic conditions where standard AprilTags failed 40% of the time—while enabling dynamic, indirect communication in swarm robotics via changing tag IDs on LCD screens.

ABSTRACT

Although fiducial markers give an accurate pose estimation in laboratory conditions, where the noisy factors are controlled, using them in field robotic applications remains a challenge. This is constrained to the fiducial maker systems, since they only work within the RGB image space. As a result, noises in the image produce large pose estimation errors. In robotic applications, fiducial markers have been mainly used in its original and simple form, as a plane in a printed paper sheet. This setup is sufficient for basic visual servoing and augmented reality applications, but not for complex swarm robotic applications in which the setup consists of multiple dynamic markers (tags displayed on LCD screen). This paper describes a novel methodology, called AprilTags3D, that improves pose estimation accuracy of AprilTags in field robotics with only RGB sensor by adding a third dimension to the marker detector. Also, presents experimental results from applying the proposed methodology to swarm autonomous robotic boats for latching between them and for creating robotic formations.

Motivation & Objective

• Address the challenge of unreliable AprilTag detection and pose estimation in real-world, highly reflective environments such as urban rivers with sunlight and water reflections.
• Overcome the limitations of planar AprilTags, which suffer from perspective ambiguity and reflection-induced noise in field robotics.
• Enable robust, real-time 6-DOF pose estimation using only RGB cameras by introducing a 3D arrangement of multiple tags on non-coplanar planes.
• Develop a scalable method for indirect communication in swarm robotics by dynamically changing tag IDs on LCD screens to signal robot states without additional hardware.
• Demonstrate the framework’s effectiveness in autonomous robotic boat latching and formation control under real-world environmental disturbances.

Proposed method

• Mount multiple AprilTags on separate, non-parallel planes (e.g., rotated 15–30° from each other) to form a 3D object resembling a half-disco ball, reducing the impact of reflections on pose estimation.
• Use a multi-tag pose fusion algorithm that combines corner detections from all visible tags (4n points for n tags) to compute a single, more robust 6-DOF pose estimate, mimicking Kalman filter-based sensor fusion.
• Apply a weighted least-squares optimization to estimate the global pose by minimizing reprojection error across all detected tag corners, with uncertainty-based weighting to favor more reliable detections.
• Integrate dynamic tag ID updates on LCD screens to encode robot state (e.g., latching status), enabling indirect, vision-based communication in swarm robotic systems.
• Deploy the system on autonomous robotic boats using only RGB cameras, with no additional sensors like IMUs or depth cameras.
• Use a camera-to-target principle for latching, where the follower robot estimates the leader’s pose and navigates to a target position with sub-40mm accuracy.

Experimental results

Research questions

• RQ1 Can 3D arrangement of AprilTags on non-coplanar planes significantly improve pose estimation robustness in highly reflective outdoor environments compared to planar markers?
• RQ2 To what extent does the AprilTags3D framework reduce detection failure rates under real-world conditions such as water reflections, wind, and moving platforms?
• RQ3 Can dynamic tag ID changes on LCD screens enable reliable, indirect communication between robots in a swarm without additional communication hardware?
• RQ4 How does the fusion of multiple non-coplanar AprilTags improve pose estimation accuracy and reduce noise in noisy, dynamic environments?
• RQ5 Can the AprilTags3D framework be effectively scaled to support N-robot formations and latching tasks in real-time field applications?

Key findings

• The AprilTags3D system achieved a 95% tag detection rate in outdoor aquatic environments with intense sunlight reflections, compared to only 60% for standard printed AprilTags.
• In outdoor latching tasks, the AprilTags3D system reduced the average number of attempts per successful latching from five to fewer than two, due to improved pose estimation stability.
• The 3D tag arrangement significantly reduced pose estimation errors caused by reflections and perspective ambiguity, enabling reliable 6-DOF pose estimation even when individual tags were partially obscured.
• Dynamic tag updates on LCD screens successfully communicated robot states (e.g., latching readiness) across a swarm, enabling scalable, vision-based formation control in a telephone-game-like communication pattern.
• The system enabled autonomous robotic boats to latch with sub-40mm positional accuracy in dynamic, wave-affected conditions, meeting the required precision for mechanical latching.
• The AprilTags3D framework demonstrated scalability to N-robot formations, as shown in a train-link formation with four boats, where each robot passed the state information forward via tag updates.