[Paper Review] CanvoX: High-resolution VR Painting in Large Volumetric Canvas

Yeojin Kim, Byung Moon Kim+2 more|arXiv (Cornell University)|Apr 10, 2017

Computer Graphics and Visualization Techniques16 references3 citations

TL;DR

CanvoX introduces a high-resolution VR painting system that enables artists to create volumetric paintings on a massive 40 km³ canvas with sub-millimeter detail (0.3 mm³ voxels), using CPU-accelerated deep octrees (depth ≥24) for dynamic refinement and GPU-based accurate ray casting for real-time rendering. The system supports interactive, performance-stable painting with color mixing and transparency, enhanced by artist-defined 'rooms' for focused, scalable navigation and appreciation of fine details in large-scale 3D scenes.

ABSTRACT

With virtual reality, digital painting on 2D canvases is now being extended to 3D spaces. Tilt Brush and Oculus Quill are widely accepted among artists as tools that pave the way to a new form of art - 3D emmersive painting. Current 3D painting systems are only a start, emitting textured triangular geometries. In this paper, we advance this new art of 3D painting to 3D volumetric painting that enables an artist to draw a huge scene with full control of spatial color fields. Inspired by the fact that 2D paintings often use vast space to paint background and small but detailed space for foreground, we claim that supporting a large canvas in varying detail is essential for 3D painting. In order to help artists focus and audiences to navigate the large canvas space, we provide small artist-defined areas, called rooms, that serve as beacons for artist-suggested scales, spaces, locations for intended appreciation view of the painting. Artists and audiences can easily transport themselves between different rooms. Technically, our canvas is represented as an array of deep octrees of depth 24 or higher, built on CPU for volume painting and on GPU for volume rendering using accurate ray casting. In CPU side, we design an efficient iterative algorithm to refine or coarsen octree, as a result of volumetric painting strokes, at highly interactive rates, and update the corresponding GPU textures. Then we use GPU-based ray casting algorithms to render the volumetric painting result. We explore precision issues stemming from ray-casting the octree of high depth, and provide a new analysis and verification. From our experimental results as well as the positive feedback from the participating artists, we strongly believe that our new 3D volume painting system can open up a new possibility for VR-driven digital art medium to professional artists as well as to novice users.

Motivation & Objective

• To extend VR-based 3D painting beyond surface mesh rendering to full volumetric painting with spatial color fields.
• To address the limitations of existing systems that produce only thin, open-surface strokes with poor color mixing and performance degradation under repeated strokes.
• To enable large-scale 3D painting with varying resolution based on visibility and distance from artist-defined 'rooms' for focused, scalable appreciation.
• To maintain interactive performance on commodity hardware by offloading octree refinement to CPU and rendering to GPU with efficient synchronization.
• To solve precision issues in deep octree ray casting, particularly error accumulation during traversal, through coordinate system optimization and error analysis.

Proposed method

• The system uses a hybrid CPU/GPU architecture: CPU manages deep octree construction and incremental refinement/coarsening of volumetric data during painting, while GPU performs accurate ray casting for real-time volume rendering.
• A novel coordinate system—cell-local coordinates—is introduced to prevent ray drift and minimize angular error during traversal, ensuring precision even in deep octrees.
• The octree data structure is dynamically updated via an efficient iterative algorithm on the CPU, allowing real-time response to painting strokes without performance degradation.
• Ray casting is implemented using fragment shaders with half-precision floating-point arithmetic, validated through experimental error analysis to ensure visual fidelity.
• The system supports multiple 'rooms'—artist-defined regions of high detail—where users can navigate, scale, and appreciate fine features, while the rest of the canvas is rendered at lower resolution.
• Color mixing and transparency are natively supported through volumetric blending, enabling natural blending of overlapping strokes without performance penalties.

Experimental results

Research questions

• RQ1 Can a VR painting system support large-scale 3D volumetric canvases (up to 40 km³) with sub-millimeter resolution (0.3 mm³) while maintaining interactive performance?
• RQ2 How can volumetric painting with repeated strokes and color mixing be implemented efficiently without performance degradation?
• RQ3 What techniques can ensure high-precision ray casting in deep octrees (depth ≥24) without accumulating angular or positional errors?
• RQ4 How can artist-defined 'rooms' improve navigation and focus in a massive 3D painting space, and what is their impact on rendering efficiency?
• RQ5 Can a hybrid CPU/GPU pipeline for octree management and rendering achieve real-time responsiveness for complex volumetric painting tasks?

Key findings

• The system successfully renders a 40 km³ volumetric canvas with 0.3 mm³ voxel resolution, enabling high-fidelity, detailed 3D paintings at interactive frame rates.
• The ray-angle error was experimentally measured to be as low as 7.5×10⁻⁹ degrees, confirming that error does not accumulate during deep octree traversal.
• Using cell-local coordinates reduced ray drift and angular error compared to world coordinates, solving a critical precision issue in deep octree rendering.
• Artists reported that the system enabled new creative possibilities, including painting large backgrounds and fine foreground details simultaneously, and redefining perspective in 3D space.
• The hybrid CPU/GPU pipeline achieved interactive performance for incremental octree refinement and rendering, even with high-depth octrees, enabling repeated painting strokes with natural color mixing.
• The system supports multiple rooms for scalable, focused viewing, allowing artists to define distinct appreciation zones without performance penalties.