[Paper Review] Text Entry in Immersive Head-Mounted Display-based Virtual Reality using Standard Keyboards

Jens Grubert, Lukas Witzani+4 more|arXiv (Cornell University)|Feb 2, 2018

Interactive and Immersive Displays4 citations

TL;DR

This study evaluates desktop and touchscreen keyboards for text entry in immersive VR using real-time hand and keyboard tracking. It finds that users retain ~60% of desktop keyboard typing speed and ~40–45% of touchscreen keyboard speed in VR, with no significant learning effect, and repositioning the keyboard improves performance—especially for touchscreens—by enhancing visual feedback and reducing cognitive load.

ABSTRACT

We study the performance and user experience of two popular mainstream text entry devices, desktop keyboards and touchscreen keyboards, for use in Virtual Reality (VR) applications. We discuss the limitations arising from limited visual feedback, and examine the efficiency of different strategies of use. We analyze a total of 24 hours of typing data in VR from 24 participants and find that novice users are able to retain about 60% of their typing speed on a desktop keyboard and about 40-45\\% of their typing speed on a touchscreen keyboard. We also find no significant learning effects, indicating that users can transfer their typing skills fast into VR. Besides investigating baseline performances, we study the position in which keyboards and hands are rendered in space. We find that this does not adversely affect performance for desktop keyboard typing and results in a performance trade-off for touchscreen keyboard typing.

Motivation & Objective

• To evaluate the performance and user experience of standard desktop and touchscreen keyboards in immersive VR environments.
• To investigate how visual feedback and keyboard repositioning affect typing efficiency and user comfort in VR.
• To determine whether users can transfer existing typing skills to VR without significant learning curves.
• To explore the impact of hand and keyboard positioning on typing performance and perceived immersion.
• To assess the feasibility of using standard physical keyboards as robust text input tools in VR for extended productivity tasks.

Proposed method

• Conducted a controlled user study with 24 participants performing extensive text entry tasks in a VR environment.
• Used a high-precision optical tracking system to capture hand and finger movements in real time.
• Rendered virtual representations of desktop and touchscreen keyboards in the VR scene, with and without repositioning toward the user’s view.
• Displayed finger tips in the virtual space to improve visual feedback and reduce cognitive load.
• Measured typing speed (words per minute), character error rate, and user-reported comfort and immersion.
• Applied auto-correction via the VelociTap decoder to reduce error rates and improve perceived performance.

Experimental results

Research questions

• RQ1 How does text entry performance on standard desktop and touchscreen keyboards compare in immersive VR?
• RQ2 What is the impact of repositioning the virtual keyboard and hand representation toward the user’s center of view on typing performance?
• RQ3 To what extent can users transfer their existing typing skills to VR without significant learning effects?
• RQ4 How does visual feedback from rendered finger movements affect typing accuracy and perceived usability?
• RQ5 What are the trade-offs between performance, comfort, and immersion when using different keyboard types and placements in VR?

Key findings

• Novice users retained approximately 60% of their desktop keyboard typing speed in VR, indicating strong transferability of typing skills.
• Users retained 40–45% of their touchscreen keyboard typing speed in VR, with no significant learning effect over time.
• Repositioning the keyboard and hands toward the user’s center of view significantly improved performance for touchscreen keyboards, though it had minimal impact on desktop keyboard performance.
• The use of real-time finger tip rendering enhanced visual feedback and reduced performance degradation, particularly for touchscreen input.
• Character error rates were reduced to 2.6–4.0% with auto-correction, demonstrating the effectiveness of decoding algorithms in mitigating input errors.
• Users reported higher comfort and perceived immersion when the keyboard and hands were repositioned to face the user, suggesting improved embodiment and reduced cognitive load.