[Paper Review] Design of a Mobile Face Recognition System for Visually Impaired Persons

Shonal Chaudhry, Rohitash Chandra|arXiv (Cornell University)|Feb 3, 2015

Tactile and Sensory Interactions57 references21 citations

TL;DR

This paper presents a mobile face recognition system for visually impaired users, leveraging Android-based computer vision with OpenCV for face detection and recognition, integrated with Google TalkBack for audio feedback. The system achieves 93% face detection accuracy and 70% recognition accuracy under well-lit conditions, using a client-server architecture to support enrolment and identity management while prioritizing privacy through temporary image handling and encrypted storage.

ABSTRACT

It is estimated that 285 million people globally are visually impaired. A majority of these people live in developing countries and are among the elderly population. One of the most difficult tasks faced by the visually impaired is identification of people. While naturally, voice recognition is a common method of identification, it is an intuitive and difficult process. The rise of computation capability of mobile devices gives motivation to develop applications that can assist visually impaired persons. With the availability of mobile devices, these people can be assisted by an additional method of identification through intelligent software based on computer vision techniques. In this paper, we present the design and implementation of a face detection and recognition system for the visually impaired through the use of mobile computing. This mobile system is assisted by a server-based support system. The system was tested on a custom video database. Experiment results show high face detection accuracy and promising face recognition accuracy in suitable conditions. The challenges of the system lie in better recognition techniques for difficult situations in terms of lighting and weather.

Motivation & Objective

• To address the challenge of social identification faced by visually impaired individuals, who often rely on unreliable or cognitively taxing methods like voice recognition.
• To develop a portable, mobile-based solution using computer vision to assist in face detection and recognition for improved social interaction.
• To ensure accessibility and usability for visually impaired users through integration with Android's TalkBack screen reader.
• To maintain privacy and security by storing face images temporarily and encrypting identity data.
• To evaluate system performance in terms of accuracy, memory usage, and computational efficiency on real-world video data.

Proposed method

• The system uses a mobile device’s camera for real-time video acquisition of faces.
• Face detection is performed using a cascade classifier implemented in OpenCV’s C++ module, trained on a custom dataset.
• For recognition, a temporary image of the detected face is captured and processed using a server-assisted recognition pipeline.
• The application integrates Google TalkBack to provide audio feedback, enabling hands-free and accessible interaction.
• A server-based component supports user enrolment and identity lookup, reducing on-device computation.
• Face images are stored temporarily and deleted after processing; identities are encrypted to ensure privacy.

Experimental results

Research questions

• RQ1 Can a mobile-based face recognition system achieve acceptable accuracy for visually impaired users in real-world lighting conditions?
• RQ2 How effective is the integration of OpenCV-based face detection and recognition with Android’s TalkBack for accessibility?
• RQ3 What are the performance trade-offs in terms of memory, data usage, and battery consumption for on-device computer vision?
• RQ4 How can privacy and data security be maintained when storing and processing biometric face data on mobile devices?
• RQ5 To what extent does lighting affect the detection and recognition accuracy of the system?

Key findings

• The system achieved a face detection accuracy of 93% under well-lit conditions, demonstrating strong performance in controlled environments.
• Face recognition accuracy reached 70% in well-lit conditions, indicating moderate but promising results for real-world deployment.
• Memory usage was low, with the application consuming minimal system resources, as confirmed by on-device testing and monitoring.
• Data usage was also low, indicating the system is efficient in bandwidth consumption, suitable for mobile networks.
• The system’s performance degraded significantly in poorly-lit conditions, highlighting a key limitation in detection robustness.
• The use of temporary image storage and encrypted identity management effectively protected user privacy and reduced long-term data exposure.