[Paper Review] Breaking the Barriers to True Augmented Reality

Christian Sandor, Martin Fuchs+5 more|arXiv (Cornell University)|Dec 17, 2015

Augmented Reality Applications21 references18 citations

TL;DR

This paper proposes 'True Augmented Reality' (True AR) as a vision of AR indistinguishable from reality, where virtual content seamlessly integrates with the physical world. It introduces the concept of an AR Turing Test to objectively evaluate when AR achieves perceptual indistinguishability, emphasizing technical, ethical, and societal challenges in achieving this goal.

ABSTRACT

In recent years, Augmented Reality (AR) and Virtual Reality (VR) have gained considerable commercial traction, with Facebook acquiring Oculus VR for \$2 billion, Magic Leap attracting more than \$500 million of funding, and Microsoft announcing their HoloLens head-worn computer. Where is humanity headed: a brave new dystopia-or a paradise come true? In this article, we present discussions, which started at the symposium "Making Augmented Reality Real", held at Nara Institute of Science and Technology in August 2014. Ten scientists were invited to this three-day event, which started with a full day of public presentations and panel discussions (video recordings are available at the event web page), followed by two days of roundtable discussions addressing the future of AR and VR.

Motivation & Objective

• To define and establish a clear benchmark for when augmented reality becomes indistinguishable from reality, termed 'True AR'.
• To address the technical and perceptual challenges of fusing virtual content coherently with the real world in real time.
• To examine the ethical and societal implications of pervasive, realistic AR, including privacy, surveillance asymmetry, and information transparency.
• To advocate for an interdisciplinary research agenda that includes ethics, philosophy, and social sciences alongside core AR technologies.
• To propose the AR Turing Test as a measurable, protocol-based standard for evaluating the achievement of True AR.

Proposed method

• Proposes an AR Turing Test inspired by Alan Turing’s original test, focusing on perceptual indistinguishability across multiple sensory modalities.
• Defines test parameters based on sensory modalities (e.g., vision, touch), consistency with physical laws (e.g., object interaction, time passage), and transition mechanisms between real and augmented states.
• Emphasizes that a successful True AR system must pass tests where users cannot detect virtual content through physical interaction, such as touching, cooking, or using tools like microscopes or CT scanners.
• Suggests that the test protocol must be flexible and application-specific, avoiding one-size-fits-all benchmarks.
• Introduces the concept of 'AR Firewalls' and symmetric AR use to mitigate ethical risks such as surveillance asymmetry and privacy violations.
• Advocates for integrating insights from ethics, art, and social sciences into AR development to guide responsible innovation.

Experimental results

Research questions

• RQ1 What criteria define a system as 'True AR'—a form of AR that is indistinguishable from reality in perception?
• RQ2 Can a standardized, objective test—modeled after the Turing Test—be developed to verify when AR has achieved perceptual indistinguishability?
• RQ3 What are the key technical challenges in achieving seamless, real-time fusion of virtual content with the physical environment?
• RQ4 How do ethical concerns such as surveillance asymmetry, privacy, and information transparency emerge in the context of True AR?
• RQ5 What role should interdisciplinary fields like ethics, philosophy, and social science play in shaping the development and deployment of True AR?

Key findings

• True AR is defined as a modification of perception that cannot be detected by the user, even through physical interaction or long-term observation.
• An AR Turing Test is proposed as a viable, protocol-based method to objectively assess whether AR has achieved perceptual indistinguishability from reality.
• The test must be flexible and context-sensitive, accounting for sensory modalities, physical consistency (e.g., object interaction, time), and transition mechanisms.
• Current AR systems fall short of True AR because they fail to maintain perceptual coherence across touch, time, and environmental dynamics.
• Ethical risks such as asymmetric surveillance and loss of privacy are significant, especially when users can perceive others without being perceived in return.
• The development of True AR requires not only advances in computer graphics, vision, and optics, but also integration with ethics, philosophy, and social sciences to guide responsible deployment.