[Paper Review] Projecting Embeddings for Domain Adaption: Joint Modeling of Sentiment Analysis in Diverse Domains

Jeremy Barnes, Roman Klinger+1 more|arXiv (Cornell University)|Aug 1, 2018

Sentiment Analysis and Opinion Mining27 references19 citations

TL;DR

This paper proposes a novel domain adaptation method for sentiment analysis by projecting mono-domain embeddings into a shared bi-domain space through joint optimization for cross-domain alignment and sentiment prediction. It achieves state-of-the-art performance on 11 of 20 source-target domain pairs, especially excelling in highly divergent domains compared to existing methods.

ABSTRACT

Domain adaptation for sentiment analysis is challenging due to the fact that supervised classifiers are very sensitive to changes in domain. The two most prominent approaches to this problem are structural correspondence learning and autoencoders. However, they either require long training times or suffer greatly on highly divergent domains. Inspired by recent advances in cross-lingual sentiment analysis, we provide a novel perspective and cast the domain adaptation problem as an embedding projection task. Our model takes as input two mono-domain embedding spaces and learns to project them to a bi-domain space, which is jointly optimized to (1) project across domains and to (2) predict sentiment. We perform domain adaptation experiments on 20 source-target domain pairs for sentiment classification and report novel state-of-the-art results on 11 domain pairs, including the Amazon domain adaptation datasets and SemEval 2013 and 2016 datasets. Our analysis shows that our model performs comparably to state-of-the-art approaches on domains that are similar, while performing significantly better on highly divergent domains. Our code is available at https://github.com/jbarnesspain/domain_blse

Motivation & Objective

• To address the challenge of sentiment classifier performance degradation when applied across domains with differing linguistic styles and vocabularies.
• To overcome limitations of existing methods like structural correspondence learning and autoencoders, which suffer from long training times or poor performance on highly divergent domains.
• To introduce a new perspective on domain adaptation by framing it as an embedding projection task between mono-domain spaces.
• To jointly optimize domain alignment and sentiment prediction in a shared bi-domain embedding space.
• To achieve state-of-the-art results on diverse benchmark datasets, including Amazon, SemEval 2013, and SemEval 2016.

Proposed method

• The model takes two separate mono-domain embedding spaces (source and target) as input.
• It learns a projection matrix to map both source and target embeddings into a shared bi-domain space.
• The projection is jointly optimized to minimize domain shift while maximizing sentiment classification accuracy.
• The joint objective combines a contrastive loss for domain alignment and a cross-entropy loss for sentiment prediction.
• The method is trained end-to-end using backpropagation with gradient descent.
• The model leverages insights from cross-lingual transfer to improve generalization across semantically and lexically divergent domains.

Experimental results

Research questions

• RQ1 Can embedding projection between mono-domain spaces outperform traditional domain adaptation methods on highly divergent domains?
• RQ2 Does joint optimization of domain alignment and sentiment prediction improve transfer performance compared to sequential or independent training?
• RQ3 How does the proposed method compare to state-of-the-art approaches on standard sentiment analysis benchmarks like SemEval 2013 and 2016?
• RQ4 In what scenarios does the model show significant gains over existing methods, particularly in terms of domain divergence?
• RQ5 Can the model generalize across domains with minimal labeled data in the target domain?

Key findings

• The proposed method achieves state-of-the-art performance on 11 out of 20 source-target domain pairs in sentiment classification.
• It performs significantly better than prior approaches on highly divergent domains, where structural correspondence learning and autoencoders struggle.
• On similar domains, the model performs comparably to state-of-the-art methods, indicating robustness across domain similarity levels.
• The joint optimization of embedding projection and sentiment prediction leads to improved alignment and classification accuracy.
• The model demonstrates strong generalization on benchmark datasets, including Amazon and SemEval 2013/2016, with publicly available code.
• The results validate the effectiveness of framing domain adaptation as an embedding projection task with joint learning.