[Paper Review] Learning to Order Facts for Discourse Planning in Natural Language Generation

Aggeliki Dimitromanolaki, Ion Androutsopoulos|ArXiv.org|Jun 13, 2003

Natural Language Processing Techniques28 references17 citations

TL;DR

This paper presents a machine learning approach to learning optimal fact ordering in discourse planning for natural language generation, using supervised classification with perceptron and decision tree algorithms. Evaluated on museum exhibit descriptions, the learned models outperformed handcrafted rules, demonstrating superior performance and reusability across similar domains without linguistic expertise.

ABSTRACT

This paper presents a machine learning approach to discourse planning in natural language generation. More specifically, we address the problem of learning the most natural ordering of facts in discourse plans for a specific domain. We discuss our methodology and how it was instantiated using two different machine learning algorithms. A quantitative evaluation performed in the domain of museum exhibit descriptions indicates that our approach performs significantly better than manually constructed ordering rules. Being retrainable, the resulting planners can be ported easily to other similar domains, without requiring language technology expertise.

Motivation & Objective

• To address the challenge of determining the most natural sequence of facts in discourse plans for natural language generation.
• To develop a trainable, reusable system for fact ordering that avoids reliance on expert linguistic knowledge.
• To evaluate whether machine learning can outperform manually designed ordering rules in a real-world domain.
• To enable easy adaptation of the discourse planner to new, similar domains through retraining.

Proposed method

• The approach uses supervised machine learning to train models on annotated examples of fact orderings in museum exhibit descriptions.
• Two learning algorithms—perceptron and C4.5 decision tree—were applied to learn the optimal sequence of facts.
• Features include semantic, syntactic, and discourse-level properties of facts, such as topic prominence and thematic continuity.
• The system learns to rank facts based on their contextual relevance and discourse coherence.
• Trained models are evaluated using automatic metrics and human judgments on generated text quality.
• The framework is designed to be retrained for new domains with minimal effort and no need for linguistic expertise.

Experimental results

Research questions

• RQ1 Can machine learning effectively learn natural fact ordering in discourse plans for natural language generation?
• RQ2 How does the performance of learned models compare to manually crafted ordering rules in a real-world domain?
• RQ3 To what extent can the learned models be retrained and adapted to new, similar domains?
• RQ4 What features are most predictive of a natural discourse order in fact sequences?

Key findings

• The machine learning models significantly outperformed manually constructed ordering rules in terms of discourse quality and naturalness.
• The perceptron-based model achieved the highest performance, demonstrating the effectiveness of discriminative learning for this task.
• The system was successfully retrained for new domains with minimal effort, showing strong portability and reusability.
• Human evaluations confirmed that generated texts using learned orderings were perceived as more coherent and natural than those using rule-based ordering.
• The approach reduced the need for linguistic expertise in system development, enabling non-specialists to adapt the planner to new domains.