[Paper Review] Emojis Decoded: Leveraging ChatGPT for Enhanced Understanding in Social Media Communications

Yuhang Zhou, Paiheng Xu+4 more|arXiv (Cornell University)|Jan 22, 2024

Artificial Intelligence in Healthcare and EducationMedicine3 citations

TL;DR

This study evaluates ChatGPT’s capability to interpret emoji semantics, sentiments, and usage intentions, demonstrating strong alignment with human annotations. It finds that ChatGPT can effectively replace human annotators in emoji research and enhance communication clarity by explaining emoji meanings, with performance comparable to human experts across multiple tasks and cultural contexts.

ABSTRACT

Emojis, which encapsulate semantics beyond mere words or phrases, have become prevalent in social network communications. This has spurred increasing scholarly interest in exploring their attributes and functionalities. However, emoji-related research and application face two primary challenges. First, researchers typically rely on crowd-sourcing to annotate emojis in order to understand their sentiments, usage intentions, and semantic meanings. Second, subjective interpretations by users can often lead to misunderstandings of emojis and cause the communication barrier. Large Language Models (LLMs) have achieved significant success in various annotation tasks, with ChatGPT demonstrating expertise across multiple domains. In our study, we assess ChatGPT's effectiveness in handling previously annotated and downstream tasks. Our objective is to validate the hypothesis that ChatGPT can serve as a viable alternative to human annotators in emoji research and that its ability to explain emoji meanings can enhance clarity and transparency in online communications. Our findings indicate that ChatGPT has extensive knowledge of emojis. It is adept at elucidating the meaning of emojis across various application scenarios and demonstrates the potential to replace human annotators in a range of tasks.

Motivation & Objective

• To assess whether ChatGPT can serve as a viable alternative to human annotators in emoji research.
• To evaluate ChatGPT’s understanding of emoji usage patterns across different communities (e.g., gender, platform, culture).
• To test ChatGPT’s performance in downstream tasks such as irony detection and emoji prediction.
• To explore the potential of GPT-4V for interpreting non-Unicode emojis through visual understanding.
• To examine the impact of prompt design and model temperature on response consistency.

Proposed method

• ChatGPT was prompted to explain emoji semantics, sentiments, and usage intentions both with and without textual context.
• The model was evaluated on its ability to identify emoji usage patterns across communities using platform, gender, hashtag, and cultural cues.
• Downstream tasks included irony annotation and emoji prediction, with performance measured against human-annotated benchmarks.
• For non-Unicode emojis, GPT-4V was used with image inputs and contextual prompts to assess visual-semantic understanding.
• Temperature hyperparameters were set to 0.7 for qualitative tasks and 0 for quantitative tasks to balance creativity and consistency.
• Prompts were standardized across experiments, with additional context provided for culturally specific emojis to reduce hallucination.

Experimental results

Research questions

• RQ1 RQ1: Does ChatGPT generate explanations for emoji semantics, sentiments, and usage intentions that are consistent with human annotations?
• RQ2 RQ2: Does ChatGPT encode knowledge about emoji usage patterns associated with different communities (e.g., gender, platform, culture)?
• RQ3 RQ3: What is the performance of ChatGPT in emoji-related downstream tasks such as irony annotation and emoji prediction?
• RQ4 RQ4: How do GPT-4V’s visual-semantic capabilities compare to GPT-4 in interpreting non-Unicode emojis?
• RQ5 RQ5: How sensitive are ChatGPT’s emoji interpretations to prompt design and temperature settings?

Key findings

• ChatGPT’s explanations for emoji semantics, sentiments, and intentions closely align with human-annotated labels in most cases, indicating strong semantic understanding.
• ChatGPT demonstrated consistent knowledge of emoji usage patterns across different communities, including gender, platform, and cultural contexts.
• In irony annotation and emoji prediction tasks, ChatGPT achieved performance levels comparable to human baselines, showing strong downstream applicability.
• GPT-4V successfully interpreted non-Unicode emojis like (broken) and (onlooker) when provided with cultural context, reducing hallucination risks.
• For the emoji (pleading face), GPT-4 and GPT-4V produced divergent sentiment labels—GPT-4 labeled it positive (empathy), while GPT-4V labeled it negative (frowning), highlighting context-dependent interpretation.
• Model responses were sensitive to prompt design, with culturally specific context significantly improving accuracy for non-standard emojis.