[Paper Review] Event Detection and Retrieval on Social Media

Manos Schinas, Symeon Papadopoulos+2 more|arXiv (Cornell University)|Jul 10, 2018

Web Data Mining and Analysis54 references4 citations

TL;DR

This paper presents a comprehensive survey of event detection and retrieval methods in social media, focusing on feature-pivot, document-pivot, and topic modeling approaches for identifying real-world events from multimedia streams. It highlights key components like multimodal feature representation, semantic similarity, clustering, and content ranking, and identifies critical challenges including lack of public evaluation datasets and cross-platform consistency.

ABSTRACT

In the recent years, we have witnessed the rapid adoption of social media platforms, such as Twitter, Facebook and YouTube, and their use as part of the everyday life of billions of people worldwide. Given the habit of people to use these platforms to share thoughts, daily activities and experiences it is not surprising that the amount of user generated content has reached unprecedented levels, with a substantial part of that content being related to real-world events, i.e. actions or occurrences taking place at a certain time and location. Given the key role of events in our life, the task of annotating and organizing social media content around them is of crucial importance for ensuring real-time and future access to multimedia content about an event of interest. In this chapter, we present several research efforts from recent years that tackle two main problems: a) event detection and b) event-based media retrieval and summarization. Given archived collections or live streams of social media items, the purpose of event detection methods is to identify previously unknown events in the form of sets of items that describe them. In general, the events could be of any type, but there are also approaches aiming at events of specific type. Given a target event the goal of event summarization is first to identify relevant content and then to represent it in a concise way, selecting the most appealing and representative content.

Motivation & Objective

• To address the challenge of detecting real-world events in large-scale, noisy social media content from platforms like Twitter, Flickr, and YouTube.
• To organize and retrieve multimedia content (images, videos, text) related to specific events in real time or from archives.
• To develop robust methods for event detection and summarization that handle multimodal data and inconsistent metadata.
• To identify key methodological components such as feature representation, similarity computation, clustering, and content ranking for effective system design.
• To highlight the scarcity of public evaluation datasets and the need for cross-platform, persistent benchmarks for comparative research.

Proposed method

• Uses feature-pivot methods that detect abnormal frequency spikes in visual or textual features (e.g., words, tags) to identify emerging events.
• Employs document-pivot clustering using similarity measures (e.g., cosine, Jaccard) to group related multimedia items into event clusters.
• Applies topic modeling techniques such as LDA to discover latent event topics from text and metadata in social media streams.
• Integrates multimodal features including Bag-of-Words, Bag-of-Visual-Words, and deep learning features (e.g., CNN embeddings) for improved representation.
• Utilizes clustering algorithms like k-means, DBSCAN, and spectral clustering to assign items to event-specific groups, with selective clustering options for noise handling.
• Employs content ranking models that prioritize relevance, diversity, and quality to summarize events effectively, with potential for personalization and credibility assessment.

Experimental results

Research questions

• RQ1 How can real-world events be automatically detected in large-scale, noisy social media streams using multimodal data?
• RQ2 What are the most effective feature representations and similarity measures for grouping multimedia items into coherent event clusters?
• RQ3 How do different clustering strategies (e.g., partitioning vs. selective clustering) impact event detection performance?
• RQ4 What role do semantic and learned similarity models play in improving event detection and retrieval accuracy?
• RQ5 What are the key challenges in evaluating event detection and retrieval systems, and how can public, persistent, and cross-platform datasets be developed?

Key findings

• Feature-pivot methods detect events by identifying abnormal spikes in feature frequencies, such as hashtags or visual words, indicating sudden real-world occurrences.
• Document-pivot clustering using cosine or Jaccard similarity effectively groups related content, especially when enriched with named entities and tags.
• Topic modeling approaches like LDA help uncover latent event topics in textual content, improving detection in text-heavy platforms like Twitter.
• Deep learning features (e.g., CNN embeddings) have become increasingly important for visual content representation, outperforming traditional BoVW in many cases.
• The lack of public, persistent, and cross-platform evaluation datasets remains a major bottleneck, requiring significant manual effort and suffering from content decay.
• There is a notable gap in research on personalization, credibility assessment, and aesthetic quality in event summarization, despite their importance for user relevance.