[论文解读] Bootstrapping Structure into Language: Alignment-Based Learning

Van Zaanen, Menno Matthias|arXiv (Cornell University)|Sep 1, 2001

Natural Language Processing Techniques参考文献 102被引用 78

一句话总结

本文提出了一种无监督框架——对齐学习（Alignment-Based Learning, ABL），通过句子对齐与可替换片段检测，识别句法成分，进而构建结构化、带括号的语料库。该方法在无需监督的情况下成功学习了递归句法结构，实验验证了其在英语、荷兰语及《华尔街日报》语料上的有效性。

ABSTRACT

refined and abstract meanings largely grow out of more concrete meanings. Bloomfield (1933) This thesis introduces a new unsupervised learning framework, called Alignment-Based Learning, which is based on the alignment of sentences and Harris's (1951) notion of substitutability . Instances of the framework can be applied to an untagged, unstructured corpus of natural language sentences, resulting in a labelled, bracketed version of that corpus. Firstly, the framework aligns all sentences in the corpus in pairs, resulting in a partition of the sentences consisting of parts of the sentences that are equal in both sentences and parts that are unequal. Unequal parts of sen tences can be seen as being substitutable for each other, since substituting one unequal part for the other results in another valid sentence. The unequal parts of the sentences are thus considered to be possible (possibly overlapping) constituents, called hypotheses. Secondly , the selection learning phase considers all hypotheses found by the alignment learning phase and selects the best of these. The hypotheses are selected based on the order in which they were found, or based on a probabilistic function. The framework can be extended with a grammar extraction phase. This extended framework is called parseABL. Instead of returning a structured version of the unstructured input corpus, like the ABL system, this system also returns a stochastic context-free or tree substitution grammar. Different instances of the framework have been tested on the English ATIS corpus, the Dutch OVIS corpus and the Wall Street Journal corpus. One of the interesting results, apart from the encouraging numerical results, is that all instances can (and do) learn recursive structures.

研究动机与目标

• 开发一种无监督学习框架，从无标注、无结构的文本中发现句法结构，无需预先的语言学标注。
• 解决在缺乏显式监督或预定义语法的情况下，诱导句法成分的挑战。
• 通过可替换性原则（受Harris, 1951启发）建模句法结构，识别可互换的句子片段。
• 将框架扩展至提取随机上下文无关或树替换语法，以实现更广泛的句法泛化能力。
• 展示该框架从多样化语料（包括ATIS、OVIS及《华尔街日报》语料）中学习递归句法结构的能力。

提出的方法

• 在全语料范围内执行成对句子对齐，以识别句子之间的匹配与差异片段。
• 将对齐句子对中不匹配的片段视为候选句法成分（即‘假设’），依据其可替换性进行判定。
• 通过排序与选择阶段，基于时间顺序或概率函数，对最合理的假设进行筛选。
• 该框架支持扩展为parseABL，从选定的句法成分中提取随机上下文无关或树替换语法。
• 该方法依赖于以下原则：在有效句子中，用一个不等长片段替换另一个不等长片段，若结果仍为有效句子，则表明两片段具有句法等价性。
• 该框架完全基于原始无标注语料运行，无需外部语言资源或预标注结构。

实验结果

研究问题

• RQ1 能否通过句子对齐与可替换性分析，在无标注文本中可靠地发现句法成分？
• RQ2 在缺乏显式监督的情况下，无监督框架在多大程度上能学习递归句法结构？
• RQ3 基于对齐的假设选择机制在识别有意义句法成分方面的有效性如何？
• RQ4 该框架能否在多样化语言领域（如ATIS、OVIS及《华尔街日报》语料）中实现泛化？
• RQ5 扩展至语法提取（parseABL）是否能从原始文本中生成可解释且有用的句法语法？

主要发现

• 该框架成功从无标注语料中学习了递归句法结构，证明仅通过对齐与可替换性分析即可实现递归结构的涌现。
• 在英语ATIS、荷兰语OVIS及《华尔街日报》语料上的所有测试实例中，该框架在句法成分发现任务上均取得了令人鼓舞的数值结果。
• 对齐过程能一致识别出对应于有意义句法成分的可替换片段，即使存在重叠也成立。
• 无论基于时间顺序还是概率函数，选择阶段均能有效从全部候选假设中筛选出合理结果。
• parseABL扩展成功从学习到的句法成分中生成了随机上下文无关或树替换语法。
• 该方法在多种语言与领域中表现出鲁棒性，表明其在无监督句法结构归纳任务中具有广泛适用性。