[Paper Review] Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems

Chien-Sheng Wu, Andrea Madotto+4 more|arXiv (Cornell University)|May 21, 2019

Speech and dialogue systemsComputer Science43 references42 citations

TL;DR

TRADE generates transferable dialogue states across domains without a predefined ontology, enabling zero-shot and few-shot domain adaptation while achieving strong joint goal and slot accuracy on MultiWOZ.

ABSTRACT

Over-dependence on domain ontology and lack of knowledge sharing across domains are two practical and yet less studied problems of dialogue state tracking. Existing approaches generally fall short in tracking unknown slot values during inference and often have difficulties in adapting to new domains. In this paper, we propose a Transferable Dialogue State Generator (TRADE) that generates dialogue states from utterances using a copy mechanism, facilitating knowledge transfer when predicting (domain, slot, value) triplets not encountered during training. Our model is composed of an utterance encoder, a slot gate, and a state generator, which are shared across domains. Empirical results demonstrate that TRADE achieves state-of-the-art joint goal accuracy of 48.62% for the five domains of MultiWOZ, a human-human dialogue dataset. In addition, we show its transferring ability by simulating zero-shot and few-shot dialogue state tracking for unseen domains. TRADE achieves 60.58% joint goal accuracy in one of the zero-shot domains, and is able to adapt to few-shot cases without forgetting already trained domains.

Motivation & Objective

• Address difficulties of DST with fixed ontologies and lack of cross-domain knowledge sharing.
• Enable tracking of unknown slot values and transfer knowledge across domains.
• Develop a shared, ontology-free model for multi-domain DST that supports zero-shot and few-shot adaptation.

Proposed method

• Three-part architecture shared across domains: utterance encoder, context-enhanced slot gate, and state generator.
• Soft-gated copy mechanism (pointer-generator) enabling generation of slot values beyond fixed vocabularies.
• Joint decoding over all (domain, slot) pairs with a start token per pair to generate values.
• Three-way slot gate (triggered, none, dontcare) to decide whether a (domain, slot) pair yields a value.
• End-to-end optimization with a weighted sum of slot-gate loss and value-generation loss.

Experimental results

Research questions

• RQ1 Can a shared, ontology-free model accurately track dialogue states across multiple domains?
• RQ2 To what extent can the model transfer knowledge to unseen domains (zero-shot) and adapt with limited data (few-shot) without catastrophically forgetting learned domains?
• RQ3 How does cross-domain parameter sharing affect joint goal accuracy and slot accuracy in multi-domain DST?

Key findings

• TRADE achieves state-of-the-art joint goal accuracy (48.62%) and slot accuracy (96.92%) on MultiWOZ across five domains.
• TRADE reaches 60.58% joint goal accuracy in zero-shot taxi domain, illustrating strong cross-domain transfer.
• The model supports zero-shot DST for unseen slots and domains, and adapts to few-shot domains without forgetting previously learned domains.
• In domain expansion, GEM-based fine-tuning preserves performance on original domains better than naive fine-tuning or EWC, while enabling better adaptation to new domains.
• On restaurant domain within MultiWOZ, TRADE's joint accuracy (65.35%) and slot accuracy (93.28%) surpass SpanPtr and other baselines when using a full ontology-free setup.