[论文解读] MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos

Amir Zadeh, Rowan Zellers+2 more|arXiv (Cornell University)|Jun 20, 2016

Sentiment Analysis and Opinion Mining参考文献 33被引用 344

一句话总结

论文引入 MOSI：首个对在线视频中情感强度与主观性进行意见级标注的多模态语料库，包含每帧视觉特征和每毫秒音频特征，以及基线与多模态融合模型。

ABSTRACT

People are sharing their opinions, stories and reviews through online video sharing websites every day. Studying sentiment and subjectivity in these opinion videos is experiencing a growing attention from academia and industry. While sentiment analysis has been successful for text, it is an understudied research question for videos and multimedia content. The biggest setbacks for studies in this direction are lack of a proper dataset, methodology, baselines and statistical analysis of how information from different modality sources relate to each other. This paper introduces to the scientific community the first opinion-level annotated corpus of sentiment and subjectivity analysis in online videos called Multimodal Opinion-level Sentiment Intensity dataset (MOSI). The dataset is rigorously annotated with labels for subjectivity, sentiment intensity, per-frame and per-opinion annotated visual features, and per-milliseconds annotated audio features. Furthermore, we present baselines for future studies in this direction as well as a new multimodal fusion approach that jointly models spoken words and visual gestures.

研究动机与目标

• 为在线视频中的情感与主观性缺乏适当的多模态数据集提供动机与解决方案。
• 提供一个带有丰富模态标注（视觉、音频和说话内容）的意见级标注语料。
• 建立用于视频数据的多模态情感分析与主观性检测的基线。
• 提出一个能够联合建模说话词语与视觉手势的多模态融合方法。

提出的方法

• 将 MOSI 作为首个面向在线视频中情感与主观性的意见级标注语料库。
• 用主观性标签、情感强度、每帧视觉特征、每个意见的标注以及每毫秒音频特征对数据进行标注。
• 为未来的多模态情感分析研究提供基线模型。
• 提出一个新的多模态融合方法，联合建模说话词语与视觉手势。

实验结果

研究问题

• RQ1 如何在在线视频中以意见级别对情感强度与主观性进行有效标注与测量？
• RQ2 在视频数据上结合文本、音频与视觉线索的多模态情感分析，哪些基线适用？
• RQ3 一个联合使用说话词语与视觉手势的融合模型能否在情感与主观性分析上优于单模态方法？

主要发现

• MOSI 提供用于在线视频中意见级情感与主观性的严格标注语料库。
• 数据集包含每帧视觉特征和每毫秒音频特征，以支持细粒度分析。
• 提出基线模型与一种新的多模态融合方法，用于联合建模口语内容与视觉手势。
• 该研究为视频数据上的多模态情感分析研究奠定基础。