[論文レビュー] All Reality: Virtual, Augmented, Mixed (X), Mediated (X,Y), and Multimediated Reality

Steve Mann, Thomas A. Furness+3 more|arXiv (Cornell University)|Apr 20, 2018

Virtual Reality Applications and Impacts参考文献 23被引用数 126

ひとこと要約

tldr: 本論文は現実性の包括的な分類法と連続体を提案し（VR/AR/MR/XR/mediated/phenomenological）、Multimediated Reality を、多次元で多感覚のフレームワークとして導入し、追加感覚および社会的/監視を含む考慮事項を含む。

ABSTRACT

The contributions of this paper are: (1) a taxonomy of the "Realities" (Virtual, Augmented, Mixed, Mediated, etc.), and (2) some new kinds of "reality" that come from nature itself, i.e. that expand our notion beyond synthetic realities to include also phenomenological realities. VR (Virtual Reality) replaces the real world with a simulated experience (virtual world). AR (Augmented Reality) allows a virtual world to be experienced while also experiencing the real world at the same time. Mixed Reality provides blends that interpolate between real and virtual worlds in various proportions, along a "Virtuality" axis, and extrapolate to an "X-axis". Mediated Reality goes a step further by mixing/blending and also modifying reality. This modifying of reality introduces a second axis. Mediated Reality is useful as a seeing aid (e.g. modifying reality to make it easier to understand), and for psychology experiments like Stratton's 1896 upside-down eyeglasses experiment. We propose Multimediated Reality as a multidimensional multisensory mediated reality that includes not just interactive multimedia-based reality for our five senses, but also includes additional senses (like sensory sonar, sensory radar, etc.), as well as our human actions/actuators. These extra senses are mapped to our human senses using synthetic synesthesia. This allows us to directly experience real (but otherwise invisible) phenomena, such as wave propagation and wave interference patterns, so that we can see radio waves and sound waves and how they interact with objects and each other. Multimediated reality is multidimensional, multimodal, multisensory, and multiscale. It is also multidisciplinary, in that we must consider not just the user, but also how the technology affects others, e.g. how its physical appearance affects social situations.

研究の動機と目的

• Reality を含む VR, AR, MR, XR, mediated reality, and multimediated reality の分類学とフレームワークを定義する。
• 従来の現実概念を拡張し、合成感覚を超えた現象論的および多感覚的経験を含める。
• Multimediated Reality を、 extra senses と人間の行動を人間の知覚へマッピングする多次元システムとして導入する。
• multimediated reality の社会的、倫理的、インフラ的な考慮事項（例：privacy and veillance）を強調する。

提案手法

• VR/AR/MR/XR および mediated reality の歴史的・現代的概念をレビューし統合する。
• Emerging realities を分類するための多次元連続体（Reality–Virtuality, Mediality, Phenomenality）を提案する。
• 現象的な可視化、SWIM-like sensing などのハードウェア/ソフトウェア構築を導入し、物理現象（波、干渉）を直接知覚可能にする。
• Multimediated Reality を、 multisensory synesthesia と human-in-the-loop intelligence (HI) の統合として、 multiveillant と multiscale の生態系で説明する。
• Reality–Virtuality、Mediality、Phenomenality、Veillance、Wearability など複数軸を持つ連続体モデルを提示し、設計と倫理への含意を論じる。

実験結果

リサーチクエスチョン

• RQ1 What taxonomy and continuum best capture VR, AR, MR, XR, mediated reality, and phenomenological realities?
• RQ2 How can multimediated reality be realized as a multidimensional, multisensory framework that includes extra senses and human actions?
• RQ3 What are the social, privacy, and ethical implications of deploying multimediated reality across wearables and smart environments?
• RQ4 How can mediated/phenomenological approaches complement traditional measurement and education tools?

主な発見

• A formal taxonomy and a multidimensional continuum are proposed to unify Virtual Reality, Augmented Reality, Mixed Reality, XR, and Mediated Reality.
• Multimediated Reality extends beyond synthetic senses to include extra senses (e.g., radio, sonar) via synthetic synesthesia, enabling direct visualization of physical phenomena.
• The framework emphasizes multiscale, multisensory experiences and incorporates social and veillance considerations (surveillance and sousveillance).
• Phenomenological Reality is introduced as a way to visualize real physical quantities directly through nature-based signals (e.g., wave propagation).
• HI (Humanistic Intelligence) and a feedback-loop approach are proposed as core to intelligent, responsible multimediated systems.
• The continuum integrates sensory attenuation, environmental context (smart buildings/cities), and ethical dimensions for future design.