[Paper Review] Integrating Multiple Knowledge Sources for Robust Semantic Parsing

Jordi Atserias, Lluís Padró+1 more|ArXiv.org|Sep 17, 2001

Natural Language Processing Techniques8 references3 citations

TL;DR

This paper proposes a robust, flexible semantic parsing approach called PARDON that frames semantic parsing as a Consistent Labelling Problem (CLP), integrating linguistic subcategorization frames and statistical lexical attraction models via chunk parsing. It achieves 95% precision in verbal model identification and 72% precision in case-role filling, demonstrating effective integration of syntactic and semantic knowledge for unrestricted text.

ABSTRACT

This work explores a new robust approach for Semantic Parsing of unrestricted texts. Our approach considers Semantic Parsing as a Consistent Labelling Problem (CLP), allowing the integration of several knowledge types (syntactic and semantic) obtained from different sources (linguistic and statistic). The current implementation obtains 95% accuracy in model identification and 72% in case-role filling.

Motivation & Objective

• To develop a robust and flexible semantic parsing system capable of handling unrestricted natural language text.
• To address the limitations of sequential parsing models by integrating syntax and semantics in a unified framework.
• To improve case-role assignment accuracy by combining hand-crafted subcategorization frames with statistical lexical attraction models.
• To minimize overgeneration, undergeneration, and substitution errors in semantic role labeling through knowledge integration.
• To enable extensibility and portability by formalizing semantic parsing as a Consistent Labelling Problem (CLP).

Proposed method

• Formalize semantic parsing as a Consistent Labelling Problem (CLP), treating case-role assignment as a labeling task over syntactic chunks.
• Use a two-stage architecture: Sentence Analyzer for PoS tagging, chunking, and semantic annotation, followed by a Selection module to resolve role assignments.
• Integrate linguistic knowledge via hand-built subcategorization frames from the LEXPIR project, specifying argument structure, prepositions, and semantic roles for 61 verbs.
• Complement linguistic models with statistical lexical attraction models derived from corpora to improve robustness and handle ambiguity.
• Apply MUC evaluation metrics to assess precision, recall, undergeneration, overgeneration, and substitution in case-role filling.
• Use verb-specific models (e.g., impersonal model for 'hablar') and noun modifier models to guide consistent role assignment across chunks.

Experimental results

Research questions

• RQ1 How can syntactic and semantic knowledge be effectively integrated to improve robustness in semantic parsing of unrestricted text?
• RQ2 To what extent can a hybrid approach combining linguistic subcategorization and statistical models reduce errors in case-role assignment?
• RQ3 Can formalizing semantic parsing as a Consistent Labelling Problem (CLP) lead to better control over overgeneration, undergeneration, and substitution errors?
• RQ4 How do missing or spurious roles arise in case-role filling, and can they be mitigated through improved model coverage?
• RQ5 Can the system’s output be used to iteratively refine and extend verbal subcategorization models?

Key findings

• The system achieved 95% precision and 91% recall in verbal model identification, indicating high accuracy in selecting correct verb argument structures.
• Case-role filling reached 72% precision and 70% recall, with low error rates: 20% undergeneration, 18% overgeneration, and 12% substitution.
• The low values for undergeneration, overgeneration, and substitution suggest the system makes informed, knowledge-driven decisions rather than arbitrary assignments.
• Most errors in model identification were traced to preprocessing issues, particularly in ambiguous or poorly segmented input.
• Missing and spurious roles were primarily due to insufficient semantic information and lack of models for adjuncts, such as temporal phrases incorrectly assigned as arguments.
• The system demonstrated feasibility for non-domain-specific semantic parsing, with potential for extension through feedback loops and expanded statistical knowledge.