[Paper Review] Identification of potential Music Information Retrieval technologies for computer-aided jingju singing training

Rong Gong, Xavier Serra|arXiv (Cornell University)|Nov 2, 2017

Music and Audio Processing4 references3 citations

TL;DR

This paper identifies Music Information Retrieval (MIR) technologies tailored for computer-aided training in jingju (Beijing opera) singing by analyzing classroom practices and trainee feedback. It determines the relative importance of five musical dimensions—intonation, rhythm, loudness, tone quality, and pronunciation—and proposes targeted MIR solutions to support the traditional mouth/heart teaching method with data-driven feedback.

ABSTRACT

Music Information Retrieval (MIR) technologies have been proven useful in assisting western classical singing training. Jingju (also known as Beijing or Peking opera) singing is different from western singing in terms of most of the perceptual dimensions, and the trainees are taught by using mouth/heart method. In this paper, we first present the training method used in the professional jingju training classroom scenario and show the potential benefits of introducing the MIR technologies into the training process. The main part of this paper dedicates to identify the potential MIR technologies for jingju singing training. To this intent, we answer the question: how the jingju singing tutors and trainees value the importance of each jingju musical dimension-intonation, rhythm, loudness, tone quality and pronunciation? This is done by (i) classifying the classroom singing practices, tutor's verbal feedbacks into these 5 dimensions, (ii) surveying the trainees. Then, with the help of the music signal analysis, a finer inspection on the classroom practice recording examples reveals the detailed elements in the training process. Finally, based on the above analysis, several potential MIR technologies are identified and would be useful for the jingju singing training.

Motivation & Objective

• To analyze the structure and pedagogical practices of professional jingju singing training classrooms.
• To assess the relative importance of five jingju musical dimensions—intonation, rhythm, loudness, tone quality, and pronunciation—from the perspective of tutors and trainees.
• To identify specific Music Information Retrieval (MIR) technologies that can support jingju singing training by aligning with pedagogical priorities.
• To bridge the gap between traditional jingju vocal pedagogy and modern signal processing by mapping training needs to technical solutions.
• To provide a foundation for developing intelligent systems that offer real-time, dimension-specific feedback in jingju singing education.

Proposed method

• Classified classroom singing practices and tutor verbal feedbacks into five musical dimensions: intonation, rhythm, loudness, tone quality, and pronunciation.
• Conducted a survey among jingju trainees to evaluate the perceived importance of each musical dimension in training.
• Performed detailed signal analysis on recorded classroom sessions to extract and inspect specific vocal elements related to each dimension.
• Mapped identified training needs to existing MIR technologies capable of analyzing pitch, timing, dynamics, spectral envelope, and phonetic content.
• Evaluated the feasibility of applying MIR techniques such as pitch tracking, onset detection, spectral centroid analysis, and phoneme recognition to jingju vocal data.
• Synthesized findings into a framework of potential MIR tools tailored for jingju pedagogy, emphasizing real-time feedback and alignment with traditional teaching goals.

Experimental results

Research questions

• RQ1 How do jingju singing tutors and trainees prioritize the five core musical dimensions—intonation, rhythm, loudness, tone quality, and pronunciation—in training?
• RQ2 What specific vocal elements within each musical dimension are most frequently addressed in classroom feedback?
• RQ3 Which Music Information Retrieval (MIR) technologies can be effectively adapted to analyze and provide feedback on these jingju vocal dimensions?
• RQ4 How can MIR systems be designed to complement the traditional mouth/heart teaching method in jingju training?
• RQ5 What signal processing techniques are most suitable for capturing the perceptual nuances of jingju singing, particularly in pitch accuracy, rhythmic precision, and phonetic clarity?

Key findings

• Intonation was ranked as the most critical dimension by both tutors and trainees, followed by pronunciation, rhythm, loudness, and tone quality.
• Tutors frequently provided feedback on pitch accuracy and phonetic clarity, indicating that these are central to vocal mastery in jingju.
• Rhythm and loudness were emphasized in ensemble and performance contexts, suggesting a need for timing and dynamic control in training.
• Signal analysis revealed that pitch tracking and formant tracking are feasible for jingju due to its characteristic vocal production and vocal range.
• Phoneme recognition and spectral envelope analysis show promise for assessing pronunciation and tone quality, though challenges remain due to the non-Western phonetic and timbral characteristics.
• The integration of MIR tools could enable real-time, dimension-specific feedback, enhancing the scalability and consistency of traditional jingju training methods.