[Paper Review] Incorporating Domain Knowledge To Improve Topic Segmentation Of Long MOOC Lecture Videos

Ananda Kumar Das, Partha Pratim Das|arXiv (Cornell University)|Dec 8, 2020

Video Analysis and Summarization4 citations

TL;DR

This paper proposes a hybrid approach for topic segmentation in long MOOC lecture videos by combining a state-of-the-art language model (LSTM with attention) and a domain knowledge graph to improve segmentation accuracy, especially for longer topics. The method leverages local semantic understanding from the language model and global conceptual relationships from the knowledge graph, achieving higher accuracy than baseline methods, particularly on topics exceeding 15 minutes in duration.

ABSTRACT

Topical Segmentation poses a great role in reducing search space of the topics taught in a lecture video specially when the video metadata lacks topic wise segmentation information. This segmentation information eases user efforts of searching, locating and browsing a topic inside a lecture video. In this work we propose an algorithm, that combines state-of-the art language model and domain knowledge graph for automatically detecting different coherent topics present inside a long lecture video. We use the language model on speech-to-text transcription to capture the implicit meaning of the whole video while the knowledge graph provides us the domain specific dependencies between different concepts of that subjects. Also leveraging the domain knowledge we can capture the way instructor binds and connects different concepts while teaching, which helps us in achieving better segmentation accuracy. We tested our approach on NPTEL lecture videos and holistic evaluation shows that it out performs the other methods described in the literature.

Motivation & Objective

• To address the challenge of topic segmentation in long MOOC lecture videos where metadata lacks topic boundaries.
• To improve segmentation accuracy for longer topics (>15 minutes) that traditional local-context methods fail to capture.
• To integrate domain-specific knowledge from knowledge graphs to model global semantic relationships between concepts in lecture transcripts.
• To reduce user effort in browsing and searching specific topics within lengthy lecture videos.
• To develop a hybrid framework that combines local linguistic features with global structural knowledge for robust topic boundary detection.

Proposed method

• A bidirectional LSTM with attention mechanism is used to model local semantic context in transcribed lecture text.
• The language model captures local syntactic and semantic patterns to detect potential topic boundaries.
• A domain-specific knowledge graph is constructed to encode relationships between educational concepts and their hierarchical dependencies.
• The knowledge graph is leveraged during segmentation to enhance global context understanding, especially for longer topics.
• Segmentation decisions are refined by combining predictions from the language model and knowledge graph, with thresholding based on topic duration.
• The system uses BERT to capture overall semantics of the transcript, improving representation learning for topic detection.

Experimental results

Research questions

• RQ1 How can local semantic modeling in lecture transcripts be enhanced with global domain knowledge to improve topic boundary detection?
• RQ2 What is the optimal context size for local sentence-level classification in topic segmentation?
• RQ3 How does topic duration affect the performance of structural (local-context) vs. semantic (knowledge-augmented) segmentation methods?
• RQ4 To what extent does incorporating a domain knowledge graph improve segmentation accuracy for long-duration topics?
• RQ5 Can a hybrid approach combining language models and knowledge graphs outperform purely similarity-based or language-model-only methods?

Key findings

• The proposed method achieved an OTR (F1-score) of 0.83 on the holistic evaluation, outperforming existing methods in the literature.
• For topics longer than 15 minutes, the semantic segmentation approach using the knowledge graph maintained consistent performance, while the structural method's accuracy dropped significantly.
• The optimal context size for sentence-level classification was determined to be K=10 sentences with 20 words per sentence, based on OTR score maximization.
• Among 225 topics from software engineering lectures, 164 were shorter than 15 minutes and 91 were longer, with the knowledge graph-based method showing superior performance on the longer ones.
• The method demonstrated steady performance across all duration categories, with the semantic analysis approach maintaining high accuracy regardless of topic length.
• The integration of domain knowledge significantly improved boundary detection for larger topics, addressing the limitation of local-context models in capturing global semantics.