Tokyo Institute of Technology · 컴퓨터과학
Youmi Ma 교수의 연구실은 문서 수준의 관계 추출(DocRE)과 이름 있는 엔터티 및 관계 추출 분야에서 핵심적인 연구를 수행하고 있습니다. 특히, 증거 문장 추론을 통한 메모리 효율성 향상과 함께, 테이블 기반 표현을 활용한 간결하고 강력한 모델 설계를 통해 관계 추출의 정확성과 효율성을 동시에 향상시키는 데 초점을 맞추고 있습니다. 비영어 언어, 특히 일본어를 포함한 다국어 환경에서도 유의미한 성능을 내기 위한 자동 번역 기반 데이터셋 구축 및 고도화 연구도 진행 중입니다.
표시된 성과는 수집된 데이터 기준으로 산출되며, 일부 차이가 있을 수 있습니다.
Document-level relation extraction (DocRE) is the task of identifying all relations between each entity pair in a document. Evidence, defined as sentences containing clues for the relationship between an entity pair, has been shown to help DocRE systems focus on relevant texts, thus improving relation extraction. However, evidence retrieval (ER) in DocRE faces two major issues: high memory consumption and limited availability of annotations. This work aims at addressing these issues to improve t
In this study, we propose a method designed to extract named entities and relations from unstructured text based on table representations. To extract named entities, the proposed method computes representations for entity mentions and long-range dependencies using contextualized representations without hand-crafted features or complex neural network architectures. To extract relations, it applies a tensor dot product to predict all relation labels simultaneously without considering dependencies
This study introduces a novel approach to the joint extraction of entities and relations by stacking convolutional neural networks (CNNs) on pretrained language models. We adopt table representations to model the entities and relations, casting the entity and relation extraction as a table-labeling problem. Regarding each table as an image and each cell in a table as an image pixel, we apply two-dimensional CNNs to the tables to capture local dependencies and predict the cell labels. The experim
Document-level relation extraction (DocRE) is the task of identifying all relations between each entity pair in a document. Evidence, defined as sentences containing clues for the relationship between an entity pair, has been shown to help DocRE systems focus on relevant texts, thus improving relation extraction. However, evidence retrieval (ER) in DocRE faces two major issues: high memory consumption and limited availability of annotations. This work aims at addressing these issues to improve t
In this study, a novel method for extracting named entities and relations from unstructured text based on the table representation is presented. By using contextualized word embeddings, the proposed method computes representations for entity mentions and long-range dependencies without complicated hand-crafted features or neural-network architectures. We also adapt a tensor dot-product to predict relation labels all at once without resorting to history-based predictions or search strategies. The
Advances in mechanistic interpretability have identified special attention heads, known as retrieval heads, that are responsible for retrieving information from the context. However, the role of these retrieval heads in improving model performance remains unexplored. This work investigates whether retrieval heads can be leveraged to enhance the long-context capabilities of LLMs. Specifically, we propose RetMask, a method that generates training signals by contrasting normal model outputs with th
Advances in mechanistic interpretability have identified special attention heads, known as retrieval heads, that are responsible for retrieving information from the context. However, the role of these retrieval heads in improving model performance remains unexplored. This work investigates whether retrieval heads can be leveraged to enhance the long-context capabilities of LLMs. Specifically, we propose RetMask, a method that generates training signals by contrasting normal model outputs with th
Document-level Relation Extraction (DocRE) is the task of extracting all semantic relationships from a document. While studies have been conducted on English DocRE, limited attention has been given to DocRE in non-English languages. This work delves into effectively utilizing existing English resources to promote DocRE studies in non-English languages, with Japanese as the representative case. As an initial attempt, we construct a dataset by transferring an English dataset to Japanese. However,
文書レベル関係抽出 (DocRE) は文書中のすべてのエンティティの組の関係を推定するタスクである．エンティティ組の関係推定に十分な手掛かりを含む文の集合を根拠と呼ぶ．根拠は関係抽出の性能を改善できるが，既存研究では DocRE と根拠認識を別々のタスクとしてモデル化していた．本稿では，根拠認識を関係抽出のモデルに統合する手法を提案する．具体的には，エンティティ組のエンコード過程において，根拠に高い重みを配分するように自己注意機構を誘導することにより，根拠に注目した分散表現を得る．さらに，根拠のアノテーションが付与されていないデータに根拠の疑似的な教師信号を付与し，大量の自動ラベル付けデータを活用する方法を提案する．実験結果から，提案手法は文書レベル関係抽出のベンチマーク DocRED 及び Re-DocRED において，関係抽出と根拠認識の両方で現時点の世界最良性能を達成した．
Open Information Extraction (OpenIE) structures information from natural language text in the form of (subject, predicate, object) triples.Supervised OpenIE is, in principle, only possible for English, for which plenty of labeled data exists.Recent research efforts tackled multilingual OpenIE by means of zero-shot transfer from English, with massively multilingual language models as vehicles of transfer.Given that OpenIE is a highly syntactic task, such transfer tends to fail for languages that
