[Paper Review] On Using Synthetic Social Media Stimuli in an Emergency Preparedness Functional Exercise

Andrew Hampton, Shreyansh Bhatt+6 more|arXiv (Cornell University)|Mar 2, 2015

Public Relations and Crisis Communication10 references3 citations

TL;DR

This paper presents a large-scale synthetic Twitter-like microblog stream (32,000+ messages) designed for a regional emergency preparedness functional exercise. By integrating valid, misleading, and irrelevant social media content, the study evaluates how public information officers (PIOs) detect events, verify information, and disseminate alerts—demonstrating that social media can enhance situational awareness and inter-agency coordination when supported by training and filtering tools.

ABSTRACT

This paper details the creation and use of a massive (over 32,000 messages) artificially constructed 'Twitter' microblog stream for a regional emergency preparedness functional exercise. By combining microblog conversion, manual production, and a control set, we created a web based information stream providing valid, misleading, and irrelevant information to public information officers (PIOs) representing hospitals, fire departments, the local Red Cross, and city and county government officials. PIOs searched, monitored, and (through conventional channels) verified potentially acionable information that could then be redistributed through a personalized screen name. Our case study of a key PIO reveals several capabilities that social media can support, including event detection, the distribution of information between functions within the emergency response community, and the distribution of messages to the public. We suggest that training as well as information filtering tools are necessary to realize the potential of social media in both emergencies and exercises.

Motivation & Objective

• To develop a realistic, large-scale synthetic social media stream to simulate real-time emergency information flows during a functional exercise.
• To assess how public information officers (PIOs) detect, verify, and disseminate actionable information from mixed-quality social media content.
• To evaluate the role of social media in supporting event detection, inter-agency communication, and public messaging during emergency response scenarios.
• To identify gaps in PIOs' ability to manage information overload and misinformation, and to recommend training and filtering tools to improve performance.

Proposed method

• A synthetic microblog stream of over 32,000 messages was generated using a combination of automated microblog conversion, manual content creation, and a control set of neutral messages.
• Messages were categorized as valid (actionable emergency events), misleading (false but plausible claims), or irrelevant (off-topic or non-urgent content) to simulate real-world social media complexity.
• The synthetic stream was delivered via a web-based interface, allowing PIOs to monitor, search, and verify information through conventional verification channels.
• PIOs were assigned roles representing hospitals, fire departments, Red Cross, and local government, and were required to redistribute verified information using personalized screen names to simulate real-time public communication.
• The exercise was designed as a functional drill, focusing on information flow and decision-making rather than physical response.
• A case study of one key PIO was conducted to analyze behavior, information processing, and communication patterns during the exercise.

Experimental results

Research questions

• RQ1 How do public information officers (PIOs) detect and verify actionable emergency information from a synthetic social media stream containing valid, misleading, and irrelevant content?
• RQ2 To what extent can social media support event detection and inter-agency coordination during emergency preparedness exercises?
• RQ3 What challenges do PIOs face in managing information overload and misinformation in real-time social media feeds during emergency simulations?
• RQ4 How effective are training and information filtering tools in improving PIOs’ ability to identify and disseminate credible information?

Key findings

• The synthetic social media stream successfully simulated realistic information dynamics, including valid emergency alerts, misleading claims, and irrelevant content, providing a challenging yet authentic exercise environment.
• Public information officers demonstrated the ability to detect key emergency events and coordinate with other agencies using the synthetic stream, indicating social media's potential for enhancing situational awareness.
• Despite their efforts, PIOs struggled with information overload and misclassification of misleading content, highlighting the need for improved filtering and verification tools.
• The case study revealed that PIOs relied heavily on conventional verification channels and personalized messaging, suggesting that social media can serve as a critical conduit for inter-agency and public communication when properly managed.
• Training in social media monitoring and information filtering was identified as essential for PIOs to effectively leverage social media in real emergencies.
• The study confirms that social media can support event detection, inter-agency coordination, and public messaging in emergency response, but only when supported by structured training and technological aids.