[Paper Review] Joint Modeling of Content and Discourse Relations in Dialogues

Kechen Qin, Lu Wang+1 more|arXiv (Cornell University)|May 14, 2017

Topic Modeling30 references3 citations

TL;DR

This paper proposes a joint modeling framework that simultaneously identifies salient content phrases and predicts discourse relations in spoken meetings using argumentative discourse structures. By leveraging interactions between content and discourse, the model achieves state-of-the-art performance on phrase selection (63.2 accuracy) and discourse relation prediction (59.2 accuracy), and significantly improves consistency-of-understanding prediction (F1: 63.1) over prior work.

ABSTRACT

We present a joint modeling approach to identify salient discussion points in spoken meetings as well as to label the discourse relations between speaker turns. A variation of our model is also discussed when discourse relations are treated as latent variables. Experimental results on two popular meeting corpora show that our joint model can outperform state-of-the-art approaches for both phrase-based content selection and discourse relation prediction tasks. We also evaluate our model on predicting the consistency among team members' understanding of their group decisions. Classifiers trained with features constructed from our model achieve significant better predictive performance than the state-of-the-art.

Motivation & Objective

• To jointly model salient content phrases and discourse relations in spoken meetings to improve both content selection and discourse understanding.
• To address the challenge of limited and expensive human-annotated discourse relations by jointly learning content and discourse signals.
• To evaluate whether model-predicted discourse and content features can enhance downstream tasks like consistency-of-understanding prediction.
• To demonstrate that joint modeling yields better performance than separate modeling or strong baselines on multiple meeting corpus benchmarks.

Proposed method

• The model uses a joint inference framework that jointly predicts salient phrases (content) and discourse relations (e.g., argumentative relations from the Twente Argumentation Schema) between speaker turns.
• It employs a structured prediction approach with a conditional random field (CRF)-like formulation to model dependencies between content and discourse labels.
• A variant of the model treats discourse relations as latent variables during training when gold-standard discourse annotations are unavailable.
• Features for consistency-of-understanding prediction are constructed from model outputs: consistency probability difference, discourse relation patterns (length-1 and length-2), and word entrainment scores based on salient phrases.
• SVM classifiers are trained on these features to predict whether team members share consistent understanding of group decisions.
• The model is evaluated on two meeting corpora: AMI and ICSI, using leave-one-out cross-validation.

Experimental results

Research questions

• RQ1 Can joint modeling of content and discourse relations improve performance on phrase-based content selection in spoken meetings?
• RQ2 Does joint learning enhance discourse relation prediction accuracy compared to state-of-the-art models?
• RQ3 Can features derived from the joint model’s predictions improve consistency-of-understanding (COU) prediction in group discussions?
• RQ4 How does the model perform when discourse relations are treated as latent variables during training?

Key findings

• The joint model achieved 63.2% accuracy on phrase selection, significantly outperforming an SVM baseline.
• The discourse relation prediction accuracy reached 59.2%, surpassing a state-of-the-art neural network baseline (54.2%).
• When discourse relations were treated as latent variables, the model still outperformed SVMs on both AMI and ICSI for phrase selection.
• Extractive summaries generated using the model’s selected phrases achieved a ROUGE-SU4 F1 score of 21.3 on the AMI corpus, outperforming baselines and keyword selection methods.
• For consistency-of-understanding prediction, an SVM classifier trained with features from the model achieved an F1 score of 63.1, significantly outperforming the prior state-of-the-art (50.5).
• The combination of discourse relation patterns, consistency probability difference, and word entrainment features yielded the best COU prediction performance (F1: 63.1).