[Paper Review] Tracking Sentiment in Mail: How Genders Differ on Emotional Axes

Saif M. Mohammad, T.Y. Yang+1 more|arXiv (Cornell University)|Sep 24, 2013

Sentiment Analysis and Opinion MiningComputer Science32 references172 citations

TL;DR

This paper proposes a crowdsourced word–emotion lexicon to analyze gender differences in emotional expression across email types. Using sentiment analysis and visualizations, it finds that women use more joy–sadness words, while men favor fear–trust terms in workplace emails, offering insights for emotion-aware email tools and mental health monitoring.

ABSTRACT

With the widespread use of email, we now have access to unprecedented amounts of text that we ourselves have written. In this paper, we show how sentiment analysis can be used in tandem with effective visualizations to quantify and track emotions in many types of mail. We create a large word--emotion association lexicon by crowdsourcing, and use it to compare emotions in love letters, hate mail, and suicide notes. We show that there are marked differences across genders in how they use emotion words in work-place email. For example, women use many words from the joy--sadness axis, whereas men prefer terms from the fear--trust axis. Finally, we show visualizations that can help people track emotions in their emails.

Motivation & Objective

• To develop a large-scale, high-coverage word–emotion association lexicon using crowdsourcing for sentiment and emotion analysis in written text.
• To investigate how emotional language varies across distinct email types—love letters, hate mail, and suicide notes—using the lexicon.
• To examine gender-based differences in emotional expression within workplace emails, focusing on specific emotion axes.
• To design and demonstrate visualizations that enable individuals to track emotional content in their own email correspondence.
• To support practical applications such as mental health risk detection, affect-based email search, and improved emotional communication in digital correspondence.

Proposed method

• Constructed a large word–emotion lexicon by crowdsourcing annotations on Amazon Mechanical Turk, using the Roget’s Thesaurus as a source for word senses.
• Applied quality control measures to ensure reliability and consistency in emotion labeling across multiple annotators.
• Mapped words to eight basic emotions (joy, sadness, anger, fear, trust, disgust, surprise, anticipation) based on Ekman’s theory of basic emotions.
• Used the Google N-gram Corpus to filter words based on frequency, focusing only on high-frequency terms to ensure robustness.
• Applied the lexicon to analyze emotional content in three distinct email corpora: love letters, hate mail, and suicide notes.
• Integrated emotion analysis with email services (e.g., Gmail) to enable real-time emotional tone tracking through visual feedback.

Experimental results

Research questions

• RQ1 How do emotional word distributions differ across love letters, hate mail, and suicide notes?
• RQ2 What are the gendered patterns in emotional expression within workplace emails, particularly across the joy–sadness and fear–trust axes?
• RQ3 Can a crowdsourced emotion lexicon effectively capture nuanced emotional content in informal written communication?
• RQ4 To what extent can visualizations of emotional tone in email help users monitor and manage their emotional expression?
• RQ5 Can emotion analysis in email contribute to early detection of psychological distress or improve interpersonal communication?

Key findings

• Women’s workplace emails contain significantly more words from the joy–sadness emotional axis compared to men’s emails.
• Men’s workplace emails show a higher frequency of terms associated with the fear–trust axis, indicating a distinct emotional register.
• The emotion lexicon successfully captured meaningful differences in emotional tone across the three email types, with suicide notes showing high levels of sadness and fear.
• Love letters were characterized by elevated use of joy and trust, while hate mail featured high levels of anger, disgust, and fear.
• Visualizations of emotional trends over time revealed detectable shifts in emotional tone, supporting their use in personal emotional self-monitoring.
• The method demonstrated that emotion analysis in email can be both reliable and actionable, especially when combined with user-friendly visual feedback.