[Paper Review] A Deep Sequential Model for Discourse Parsing on Multi-Party Dialogues

Zhouxing Shi, Minlie Huang|arXiv (Cornell University)|Dec 1, 2018

Topic Modeling30 references4 citations

TL;DR

This paper proposes a deep sequential model that jointly predicts discourse dependency relations and incrementally constructs discourse structures in multi-party dialogues by leveraging both local EDU features and global structural context through a structured encoder. The model outperforms state-of-the-art baselines, achieving an F1 score of 55.7% on relation classification, with ablation studies confirming the effectiveness of structured representations and a speaker highlighting mechanism.

ABSTRACT

Discourse structures are beneficial for various NLP tasks such as dialogue understanding, question answering, sentiment analysis, and so on. This paper presents a deep sequential model for parsing discourse dependency structures of multi-party dialogues. The proposed model aims to construct a discourse dependency tree by predicting dependency relations and constructing the discourse structure jointly and alternately. It makes a sequential scan of the Elementary Discourse Units (EDUs) in a dialogue. For each EDU, the model decides to which previous EDU the current one should link and what the corresponding relation type is. The predicted link and relation type are then used to build the discourse structure incrementally with a structured encoder. During link prediction and relation classification, the model utilizes not only local information that represents the concerned EDUs, but also global information that encodes the EDU sequence and the discourse structure that is already built at the current step. Experiments show that the proposed model outperforms all the state-of-the-art baselines.

Motivation & Objective

• To address the limitations of pipeline-based discourse parsing methods that separate dependency prediction from structure construction.
• To model non-projective discourse structures in multi-party dialogues, which often involve non-adjacent relations.
• To improve discourse parsing by jointly and alternately predicting dependency links and constructing discourse structures using global structural context.
• To incorporate speaker information into discourse representation via a speaker highlighting mechanism to enhance dialogue understanding.
• To demonstrate that structured representations encoding the current discourse structure significantly improve dependency prediction performance.

Proposed method

• The model performs a sequential scan over Elementary Discourse Units (EDUs), predicting for each EDU its parent EDU and relation type based on local and global context.
• It uses a structured encoder to represent the current discourse structure, encoding both the EDU sequence and the already-predicted dependencies.
• Local representations are derived from the current EDU and its candidate parent, while global representations include the full EDU sequence and the evolving discourse structure.
• A speaker highlighting mechanism enhances the structured representation by emphasizing EDUs from the same speaker as the current EDU, improving context awareness.
• The model alternates between predicting links and updating the discourse structure, enabling iterative refinement through feedback from the current structure.
• The framework uses a structured encoder to maintain and update the discourse tree incrementally, with attention mechanisms to weigh relevant structural and sequential information.

Experimental results

Research questions

• RQ1 Can a sequential model that jointly predicts dependency links and constructs discourse structures improve discourse parsing performance in multi-party dialogues?
• RQ2 Does incorporating global structural context—beyond just local EDU features—enhance the accuracy of dependency relation prediction?
• RQ3 How effective is a speaker highlighting mechanism in improving discourse structure prediction for dialogues with multiple speakers?
• RQ4 To what extent do structured representations encoding the predicted discourse structure contribute to better link and relation prediction?
• RQ5 How does the proposed model compare to pipeline-based approaches that decouple dependency prediction from structure construction?

Key findings

• The full model (Deep Sequential) achieves an F1 score of 55.7% on relation classification, significantly outperforming the best baseline (Deep+Greedy) at 51.9%.
• The model with structured representations (Deep Sequential) outperforms both the non-structured baseline (71.0% link F1) and the random-structure variant (71.8% link F1), demonstrating the value of meaningful structural encoding.
• Disabling the speaker highlighting mechanism results in a drop in F1 from 55.7% to 54.5%, confirming its positive impact on relation prediction.
• The model with only non-structured representations (Deep Sequential (NS)) still outperforms the pipeline baseline (Deep+Greedy), indicating that sequential modeling with global context improves performance even without explicit structure encoding.
• Case studies show that structured representations help resolve ambiguity in dependency links by encoding prior relations, such as a question being answered, which guides correct parent selection.
• The model correctly predicts the parent of EDU $u_4$ as $u_3$ by leveraging the previously predicted $u_1 \rightarrow u_3$ relation, while baselines fail due to lack of structural context.