[Paper Review] On Tracking Dialogue State by Inheriting Slot Values in Mentioned Slot Pools

Zhoujian Sun, Zhengxing Huang+1 more|arXiv (Cornell University)|Feb 15, 2022

Speech and dialogue systems4 citations

TL;DR

This paper proposes a novel dialogue state tracking (DST) model that uses a Mentioned Slot Pool (MSP) to improve slot value inheritance and tracking. By maintaining a slot-specific memory of all previously mentioned values, the model dynamically decides whether to inherit the prior value or re-extract from current context, correcting errors and handling indirectly mentioned slots. The MSP model achieves state-of-the-art performance on MultiWOZ 2.1 and 2.2 without external data.

ABSTRACT

Dialogue state tracking (DST) is a component of the task-oriented dialogue system. It is responsible for extracting and managing slot values according to dialogue utterances, where each slot represents an essential part of the information to accomplish a task, and slot value is updated recurrently in each dialogue turn. However, many DST models cannot update slot values appropriately. These models may repeatedly inherit wrong slot values extracted in previous turns, resulting in the fail of the entire DST task. They cannot update indirectly mentioned slots well, either. This study designed a model with a mentioned slot pool (MSP) to tackle the update problem. The MSP is a slot-specific memory that records all mentioned slot values that may be inherited, and our model updates slot values according to the MSP and the dialogue context. Our model rejects inheriting the previous slot value when it predicates the value is wrong. Then, it re-extracts the slot value from the current dialogue context. As the contextual information accumulates with the dialogue progress, the new value is more likely to be correct. It also can track the indirectly mentioned slot by picking a value from the MSP. Experimental results showed our model reached state-of-the-art DST performance on MultiWOZ 2.1 and 2.2 datasets.

Motivation & Objective

• To address the problem of incorrect slot value inheritance in dialogue state tracking, where wrong values are repeatedly carried forward across turns.
• To improve tracking of indirectly mentioned slots, where values are referenced from other slots rather than explicitly stated.
• To design a more robust dialogue state update strategy that can reject incorrect prior values and re-extract based on current context.
• To evaluate the effectiveness of the MSP mechanism in enhancing DST performance without relying on external datasets.

Proposed method

• The model introduces a Mentioned Slot Pool (MSP), a slot-specific memory that stores all previously mentioned slot values for potential inheritance.
• For each turn, the model evaluates whether to inherit the previous slot value or re-extract a new value based on current dialogue context and MSP content.
• The model uses attention mechanisms to compare current utterance with MSP entries and determine if the previous value is likely incorrect.
• When a value is deemed incorrect, the model discards it and re-extracts the slot value from the current dialogue context using span prediction.
• The MSP enables indirect slot tracking by allowing the model to retrieve values from related slots stored in the pool.
• The model is trained end-to-end on MultiWOZ datasets using cross-entropy loss for slot value prediction.

Experimental results

Research questions

• RQ1 Can a memory-based mechanism like MSP improve the accuracy of slot value inheritance in dialogue state tracking?
• RQ2 How effectively can the MSP model correct previously wrong slot values through re-extraction from current context?
• RQ3 To what extent can the MSP model track indirectly mentioned slot values not explicitly stated in the current turn?
• RQ4 Does the MSP-based update strategy outperform standard inheritance strategies in terms of overall DST performance?

Key findings

• The MSP model achieved state-of-the-art performance on MultiWOZ 2.1 and 2.2 datasets among models not trained with external data.
• The MSP-full model, which includes information on indirectly mentioned slots, outperformed the MSP-self model, indicating improved handling of indirect references.
• The model corrected incorrect slot values by rejecting prior values and re-extracting from current context, especially when contextual evidence accumulated over turns.
• The model successfully tracked 91% of indirectly mentioned slot cases by retrieving values from the MSP, demonstrating strong capability in indirect reference resolution.
• Despite high overall accuracy, the model showed high PLFP (predicted label false positive) rates—over 30% in some slots—indicating difficulty in distinguishing correct from incorrect candidate values in complex contexts.
• Error analysis revealed that the main failure modes were FP and FN predictions, with PLFP being a major bottleneck, particularly in time-related slots like 'train-leaveat'.