[Paper Review] Phonetic Word Embeddings

Rahul Sharma, Kunal Dhawan+1 more|arXiv (Cornell University)|Sep 30, 2021

Music and Audio Processing24 references4 citations

TL;DR

This paper proposes a novel phonetic word embedding method that models human-like perception of sound similarity using phonetic features, edit distance with non-diagonal penalties, and end-of-word weighting. It achieves state-of-the-art performance on phonetic similarity tasks in English and introduces the first reported results for Hindi, outperforming prior methods on both benchmark and heterographic pun datasets.

ABSTRACT

This work presents a novel methodology for calculating the phonetic similarity between words taking motivation from the human perception of sounds. This metric is employed to learn a continuous vector embedding space that groups similar sounding words together and can be used for various downstream computational phonology tasks. The efficacy of the method is presented for two different languages (English, Hindi) and performance gains over previous reported works are discussed on established tests for predicting phonetic similarity. To address limited benchmarking mechanisms in this field, we also introduce a heterographic pun dataset based evaluation methodology to compare the effectiveness of acoustic similarity algorithms. Further, a visualization of the embedding space is presented with a discussion on the various possible use-cases of this novel algorithm. An open-source implementation is also shared to aid reproducibility and enable adoption in related tasks.

Motivation & Objective

• To develop a phonetic similarity metric that reflects human perception of sound similarity in spoken language.
• To learn continuous vector embeddings that group phonetically similar words together, enabling downstream computational phonology tasks.
• To address the lack of benchmarking mechanisms in phonetic similarity by introducing a heterographic pun dataset for evaluation.
• To extend the method to low-resource languages, particularly Hindi, with high grapheme-to-phoneme correspondence.
• To enable zero-shot comparison of unseen words by leveraging a learnable phonetic similarity metric.

Proposed method

• The method computes phonetic similarity between two phonemes using a set of well-defined distinctive phonetic features, inspired by phonetic theory.
• It applies a Wagner-Fischer edit distance algorithm with a non-diagonal penalty to align phonetic sequences, capturing contextual acoustic effects.
• A novel end-of-word phone weightage factor is introduced to model human perceptual sensitivity to word-final sounds.
• The phonetic similarity metric is used to train continuous word embeddings via a contrastive learning objective, grouping similar-sounding words in vector space.
• The model is trained and evaluated on CMU Pronouncing Dictionary for English and a Hindi phonetic lexicon, with t-SNE visualizations to validate clustering.
• A heterographic pun dataset from SemEval-2017 Task 7 is used as a benchmark to compare cosine similarity distributions across models.

Experimental results

Research questions

• RQ1 Can a phonetic similarity metric based on human perception of sound improve word embedding quality for phonetic tasks?
• RQ2 How does the proposed method compare to prior state-of-the-art approaches like PSSVec on standard phonetic similarity benchmarks?
• RQ3 Can the method generalize to low-resource languages like Hindi, where prior work is limited?
• RQ4 Does the embedding space effectively group phonetically similar words, as evidenced by t-SNE visualizations?
• RQ5 To what extent does the model capture acoustic similarity in heterographic puns, where words sound alike but have different meanings?

Key findings

• The proposed method achieves state-of-the-art performance on English phonetic similarity benchmarks, outperforming previously reported results.
• For the first time, the paper reports performance on Hindi phonetic similarity tasks, demonstrating the method's applicability to low-resource Indian languages.
• On the heterographic pun detection task, the cosine similarity distribution of the proposed method forms a sharp Gaussian with a mean near 1, indicating strong capture of acoustic similarity.
• The method outperforms PSSVec on 778 heterographic pun pairs, with a mean cosine similarity of 0.899 for 'mutter' and 'mother', while PSSVec assigns a negative similarity of -0.0123.
• The t-SNE visualization confirms that phonetically similar words cluster together in the embedding space for both English and Hindi.
• The model demonstrates robustness to out-of-vocabulary words, as it can compute similarity without requiring prior exposure to the words in training.