[Paper Review] A Deep Generative Model of Vowel Formant Typology

Ryan Cotterell, Jason Eisner|arXiv (Cornell University)|Jul 8, 2018

Speech Recognition and Synthesis19 references4 citations

TL;DR

This paper proposes a deep generative model that learns the acoustic typology of vowel systems by directly modeling formant frequencies (F1, F2) across 233 languages, using a differentiable probabilistic framework combining deep neural networks and determinantal point processes (DPPs) to capture vowel dispersion and focalization. The model outperforms baselines by explicitly modeling acoustic variability and demonstrates that non-linear transformation of a Gaussian prior improves modeling of dispersed, well-separated vowel systems.

ABSTRACT

What makes some types of languages more probable than others? For instance, we know that almost all spoken languages contain the vowel phoneme /i/; why should that be? The field of linguistic typology seeks to answer these questions and, thereby, divine the mechanisms that underlie human language. In our work, we tackle the problem of vowel system typology, i.e., we propose a generative probability model of which vowels a language contains. In contrast to previous work, we work directly with the acoustic information -- the first two formant values -- rather than modeling discrete sets of phonemic symbols (IPA). We develop a novel generative probability model and report results based on a corpus of 233 languages.

Motivation & Objective

• To develop a probabilistic generative model of vowel inventories that operates directly on acoustic formant values (F1, F2) rather than discrete IPA symbols.
• To investigate how principles like vowel dispersion and focalization shape the distribution of vowel systems across languages.
• To improve upon prior models by learning a non-linear transformation of a Gaussian prior through deep neural networks, capturing complex acoustic patterns.
• To demonstrate that modeling formant values directly yields better performance than IPA-based or simpler generative models.
• To provide a universal prior over potential vowel systems that reflects cross-linguistic regularities in acoustic phonetics.

Proposed method

• The model uses a deep neural network to learn a non-linear transformation of a Gaussian prior, mapping latent variables to (F1, F2) formant pairs.
• A determinantal point process (DPP) is used to model the dependency structure among vowels, enforcing diversity and dispersion in the predicted vowel sets.
• The framework employs a differentiable diffeomorphism constraint, ensuring each network layer has exactly two hidden units matching the two formants.
• The model is trained via an EM algorithm with 100 iterations, using 5 Monte Carlo samples per E-step and 50 SGD steps per M-step.
• Hyperparameters (σ², ρ) are tuned via cross-entropy minimization on a held-out development set over a grid of 25 values.
• Reversible-jump MCMC is used to allow the number of phones (N) to vary during inference, enabling flexible modeling of vowel inventory size.

Experimental results

Research questions

• RQ1 How do acoustic formant values (F1, F2) vary across the world’s 233 languages, and what underlying principles shape this variation?
• RQ2 To what extent do principles like vowel dispersion and focalization explain the structure of real vowel inventories?
• RQ3 Can a deep generative model that learns from raw (F1, F2) data outperform models based on discrete IPA symbols or simpler probabilistic assumptions?
• RQ4 Does a non-linear transformation of a Gaussian prior improve the modeling of dispersed, well-separated vowel systems?
• RQ5 How does the performance of the model change with increasing vowel inventory size?

Key findings

• The DPP-based model significantly outperforms baseline models, demonstrating that modeling dispersion through a DPP improves predictive performance.
• The model's performance improves with larger vowel inventories, indicating that dispersion becomes more critical as the number of vowels increases.
• The model learns a non-linear transformation of a Gaussian prior that better captures the distribution of real-world (F1, F2) vowel pairs than linear or fixed-structure alternatives.
• The number of induced phones in the model is close to the number of IPA phonemes in the data, but the phoneme-supervised baseline performs worse, suggesting field linguists’ IPA choices may be pragmatically biased.
• Despite modeling acoustic phonetics, the model does not fully explain the evolutionary drivers of vowel systems, indicating that dispersion alone is insufficient to capture the full selection mechanism.
• The results support the hypothesis that both dispersion and focalization are key drivers of vowel system typology, with dispersion playing a more prominent role in larger systems.