[Paper Review] OpenEDS: Open Eye Dataset

Stephan J. Garbin, Yiru Shen+4 more|arXiv (Cornell University)|Apr 30, 2019

Retinal Imaging and Analysis45 references68 citations

TL;DR

OpenEDS provides a large-scale, VR-focused eye image dataset with multi-class eye-region annotations (iris, pupil, sclera, background), unlabelled images, video sequences, and corneal topography. A baseline semantic segmentation experiment achieves high mIoU, showcasing OpenEDS’ utility for eye-tracking research in VR.

ABSTRACT

We present a large scale data set, OpenEDS: Open Eye Dataset, of eye-images captured using a virtual-reality (VR) head mounted display mounted with two synchronized eyefacing cameras at a frame rate of 200 Hz under controlled illumination. This dataset is compiled from video capture of the eye-region collected from 152 individual participants and is divided into four subsets: (i) 12,759 images with pixel-level annotations for key eye-regions: iris, pupil and sclera (ii) 252,690 unlabelled eye-images, (iii) 91,200 frames from randomly selected video sequence of 1.5 seconds in duration and (iv) 143 pairs of left and right point cloud data compiled from corneal topography of eye regions collected from a subset, 143 out of 152, participants in the study. A baseline experiment has been evaluated on OpenEDS for the task of semantic segmentation of pupil, iris, sclera and background, with the mean intersectionover-union (mIoU) of 98.3 %. We anticipate that OpenEDS will create opportunities to researchers in the eye tracking community and the broader machine learning and computer vision community to advance the state of eye-tracking for VR applications. The dataset is available for download upon request at https://research.fb.com/programs/openeds-challenge

Motivation & Objective

• Address the need for a large, high-resolution, VR eye image dataset with detailed eye-region annotations.
• Provide controlled-capture data (VR HMD with synchronized eye cameras at 200 Hz) across many participants.
• Offer multi-faceted data (images, masks, videos, corneal topography) to support eye-tracking and gaze estimation research.
• Demonstrate baseline semantic segmentation performance on pupil/iris/sclera/background segmentation.

Proposed method

• Capture eye imagery using a custom VR HMD with two synchronized cameras at 200 Hz under controlled illumination.
• Annotate 12,759 images with masks for eyelid, iris (ellipse and boundary points), pupil (ellipse and boundary), enabling multi-class segmentation.
• Provide additional unlabelled images (252,690) and 91,200 video frames for sequence-based tasks.
• Record corneal topography as 143 left/right eye point clouds using Scheimpflug imaging, mapped to full eye-region data.
• Evaluate baseline segmentation using modified SegNet-based architectures (mSegnet) with boundary refinement and separable convolutions.

Experimental results

Research questions

• RQ1 Can a large-scale, high-resolution eye image dataset with per-eye masks improve eye-region segmentation for VR gaze tracking?
• RQ2 How well do neural networks segment pupil, iris, sclera, and background in controlled VR eye imagery?
• RQ3 What is the impact of boundary-aware and efficient architectures on segmentation accuracy and model size for eye data?

Key findings

• OpenEDS includes 12,759 annotated images, 252,690 unlabelled images, 91,200 video frames, and 286 corneal topography point clouds from 152 participants.
• Baseline semantic segmentation achieved mIoU up to 91.4% with SegNet-based BR model on pupil/iris/sclera/background.
• The best performing model (SegNet with boundary refinement) attained pixel accuracy 98.3, mean accuracy 97.5, F1 98.3, IoU 91.4, with 3.5 MB parameters.
• SegNet variants without BR underperform in IoU and F1 compared to BR variant, and more complex models (SC) have smaller footprint but reduced accuracy.
• Annotation protocol includes ellipse and polygon annotations for iris and pupil, plus 18-point eyelid annotations, enabling precise eye-region segmentation.