[Paper Review] Swar The Voice Operated PC

Kamlesh Sharma, T. Durga Prasad|arXiv (Cornell University)|Jan 1, 2009

Speech and Audio Processing1 references3 citations

TL;DR

This paper proposes a voice-operated PC system using Hidden Markov Models (HMMs) for speech recognition, enabling hands-free control via spoken commands. The system processes audio input through feature extraction (MFCCs and energy), matches it against trained HMM models, and executes commands, offering a viable solution for physically disabled users and low-literacy populations.

ABSTRACT

Keyboard, although a popular medium, is not very convenient as it requires a certain amount of skill for effective usage. A mouse on the other hand requires a good hand-eye co-ordination. Also current computer interfaces also assume a certain level of literacy from the user. It also expect the user to have certain level of proficiency in English. In our country where the literacy level is as low as 50% in some states, if information technology has to reach the grass root level; these constraints have to be eliminated. As a solution for these, Speech Recognition and hence the concept of Voice operated computer system comes into picture. In this paper we propose a technique to develop a voice recognition system which will be used for controlling computer via speech input from any user i.e. without the use of mouse and / or keyboard. Once developed this system would be of great benefit to physically handicapped people as Instead of scrolling through written procedures on a laptop or handheld computer, they can wear a headset and have their hands and eyes free.

Motivation & Objective

• To develop a voice-controlled PC system that eliminates reliance on keyboards and mice.
• To address usability barriers for physically disabled and low-literacy users in India, where literacy rates are as low as 50%.
• To implement a speech recognition system using HMMs that enables command-based interaction without requiring English proficiency or technical keyboard skills.
• To design a system that supports continuous speech input and robust command recognition in real-world environments.

Proposed method

• Audio input is captured via a microphone and processed through a word detection module using energy and zero-crossing rate.
• Feature extraction computes Mel Frequency Cepstrum Coefficients (MFCCs) and normalized energy for each speech frame.
• A continuous, multi-dimensional Hidden Markov Model (HMM) is used for recognition, with state transitions guided by transition probabilities and observation likelihoods.
• Each recognized word is associated with a unique HMM model trained on multiple audio samples, with initial model parameters set using the first N feature vectors from training data.
• The system compares incoming audio buffers with stored .wav or .au files in a database using a comparator module to validate commands.
• Implementation is based on .NET API to simplify development, reduce code size, and support object-oriented programming and hardware interaction.

Experimental results

Research questions

• RQ1 How can a speech recognition system be designed to function effectively without requiring keyboard or mouse input?
• RQ2 What techniques can improve accessibility for physically disabled users and low-literacy populations in India?
• RQ3 How can HMM-based models be trained and applied to achieve accurate word recognition from spoken input?
• RQ4 What role does feature extraction (MFCC and energy) play in improving recognition accuracy?
• RQ5 How can the system distinguish valid voice commands from irrelevant speech or noise?

Key findings

• The system successfully implements a voice-operated PC using HMM-based speech recognition, enabling control via spoken commands without keyboard or mouse.
• Feature extraction using MFCCs and normalized energy effectively captures phonetic characteristics for reliable model training.
• HMM models are trained using Euclidean distance to assign observations to states and estimate transition probabilities from consecutive observations.
• The system uses a file-based database of pre-recorded voice samples in .au or .wav format for comparison with real-time input.
• The use of .NET API simplifies development, reduces code complexity, and supports efficient hardware interaction for real-time processing.
• The system demonstrates feasibility for real-world deployment, particularly for disabled users and low-literacy populations, though robustness and continuous speech handling remain challenges.