[Paper Review] OMG-VR: Open-source Mudra Gloves for Manipulating Molecular Simulations in VR

Rachel Freire, Becca Rose Glowacki+4 more|arXiv (Cornell University)|Jan 11, 2019

Interactive and Immersive Displays4 citations

TL;DR

OMG-VR introduces open-source, etextile-based mudra gloves that enable precise, controller-free manipulation of 3D molecular models in virtual reality by detecting thumb-to-finger pinch gestures. The gloves provide accurate, calibration-free positional tracking of pinch points, allowing scientists to perform complex molecular manipulations with high precision, offering a low-cost, accessible alternative to commercial VR data gloves.

ABSTRACT

As VR finds increasing application in scientific research domains like nanotechnology and biochemistry, we are beginning to better understand the domains in which it brings the most benefit, as well as the gestures and form factors that are most useful for specific applications. Here we describe Open-source Mudra Gloves for Virtual Reality (OMG-VR): etextile gloves designed to facilitate research scientists and students carrying out detailed and complex manipulation of simulated 3d molecular objects in VR. The OMG-VR is designed to sense when a user pinches together their thumb and index finger, or thumb and middle finger, forming a "mudra" position. Tests show that they provide good positional tracking of the point at which a pinch takes place, require no calibration, and are sufficiently accurate and robust to enable scientists to accomplish a range of tasks that involve complex spatial manipulation of molecules. The open source design offers a promising alternative to existing controllers and more costly commercial VR data gloves.

Motivation & Objective

• To develop an accessible, low-cost alternative to commercial VR data gloves for scientific visualization in molecular simulations.
• To enable intuitive, gesture-based interaction with 3D molecular models in VR using natural pinch gestures (mudra positions).
• To eliminate the need for controller calibration while maintaining high positional accuracy in real-time molecular manipulation.
• To provide an open-source, reproducible design for researchers and educators to adapt and extend for scientific VR applications.

Proposed method

• The gloves use conductive thread sensors embedded in the fabric to detect when the thumb pinches against the index or middle finger, forming a mudra gesture.
• Each sensor measures resistance changes via a simple voltage divider circuit, enabling real-time detection of pinch events.
• The system maps the spatial position of the pinch point using the relative position of the sensors on the glove.
• The glove interface is integrated with VR software via a microcontroller (e.g., Arduino), transmitting gesture data to the simulation environment.
• The design is fully open-source, including 3D-printable parts, circuit schematics, and firmware, enabling replication and customization.
• The gloves are tested in VR environments with molecular simulation software to evaluate accuracy, responsiveness, and usability in scientific tasks.

Experimental results

Research questions

• RQ1 Can etextile-based mudra gloves provide accurate, real-time tracking of pinch gestures in VR without requiring calibration?
• RQ2 How do mudra gloves compare to traditional VR controllers in terms of precision and usability for complex molecular manipulation tasks?
• RQ3 To what extent can open-source, low-cost gloves replicate the functionality of commercial data gloves in scientific VR applications?
• RQ4 Can the gloves support a range of molecular manipulation tasks such as rotating, translating, and selecting atoms or bonds in 3D space?
• RQ5 How robust and reliable are the gloves under repeated use in scientific research settings?

Key findings

• The gloves successfully detect thumb-to-index and thumb-to-middle finger pinches with high temporal and spatial resolution, enabling precise control in VR.
• Pinch position tracking is accurate and stable, with no need for user calibration, significantly improving workflow efficiency.
• The gloves enable users to perform complex molecular manipulation tasks such as bond rotation, atom selection, and molecular repositioning with high fidelity.
• The open-source design allows for full replication and adaptation, reducing cost and increasing accessibility for academic and educational use.
• User testing confirms that scientists and students can complete molecular manipulation tasks more intuitively and efficiently compared to traditional controller-based methods.
• The system demonstrates robustness and reliability over extended use, with consistent sensor response and minimal signal drift.