[Paper Review] The Effect of Social Feedback in a Reddit Weight Loss Community

Tiago Oliveira Cunha, Ingmar Weber+2 more|arXiv (Cornell University)|Feb 25, 2016

Social Media and Politics15 references4 citations

TL;DR

This study uses Mahalanobis Distance Matching on 37,278 initial posts from the Reddit r/loseit community to show that receiving social feedback—such as upvotes or positive comments—on a user's first post increases their likelihood of returning by approximately 18%, even after controlling for linguistic differences. The effect diminishes with later posts, indicating feedback is most critical during initial engagement.

ABSTRACT

It is generally accepted as common wisdom that receiving social feedback is helpful to (i) keep an individual engaged with a community and to (ii) facilitate an individual's positive behavior change. However, quantitative data on the effect of social feedback on continued engagement in an online health community is scarce. In this work we apply Mahalanobis Distance Matching (MDM) to demonstrate the importance of receiving feedback in the "loseit" weight loss community on Reddit. Concretely we show that (i) even when correcting for differences in word choice, users receiving more positive feedback on their initial post are more likely to return in the future, and that (ii) there are diminishing returns and social feedback on later posts is less important than for the first post. We also give a description of the type of initial posts that are more likely to attract this valuable social feedback. Though we cannot yet argue about ultimate weight loss success or failure, we believe that understanding the social dynamics underlying online health communities is an important step to devise more effective interventions.

Motivation & Objective

• To investigate whether social feedback on initial posts in an online weight loss community increases user retention.
• To determine whether the impact of social feedback diminishes over time, particularly across multiple posts.
• To identify characteristics of initial posts that attract higher levels of social feedback from the community.
• To understand the role of linguistic and emotional cues in driving community engagement in health-focused online forums.

Proposed method

• Applied Mahalanobis Distance Matching (MDM) to create balanced comparison groups of users with similar initial post characteristics but differing levels of received feedback.
• Used VADER sentiment analysis to classify comments as supportive or discouraging, though the study notes potential limitations in labeling accuracy.
• Tracked user return behavior by measuring whether users posted again in the subreddit after their first post.
• Conducted qualitative analysis on top and bottom 10% of posts by number of comments to identify linguistic and content patterns associated with higher feedback.
• Analyzed feedback types separately: upvotes and positive comments, to assess their individual impact on return probability.
• Used statistical matching to reduce confounding effects from differences in word choice, vocabulary, and post content.

Experimental results

Research questions

• RQ1 Does receiving social feedback on a user's first post in the r/loseit community increase their probability of returning to the subreddit?
• RQ2 Is social feedback on later posts as impactful as feedback on the initial post in sustaining user engagement?
• RQ3 What characteristics of initial posts are associated with higher levels of social feedback from the community?

Key findings

• Users who received more social feedback on their first post were approximately 18% more likely to return to the r/loseit subreddit in the future, even after controlling for differences in linguistic content.
• The positive effect of social feedback on return probability diminishes significantly with subsequent posts, indicating 'diminishing returns' over time.
• Posts that included images from imgur—often before-and-after body transformation photos—were significantly more likely to attract social feedback, suggesting visual content enhances engagement.
• Despite the use of automated sentiment analysis, the study observed that even negative-sounding comments (e.g., 'You f***ing skinny son of a b*tch') were often supportive in tone, indicating a generally positive community culture.
• The study found no evidence of widespread 'fat shaming' or harmful comments, suggesting the community environment is largely supportive despite the potential for such behavior.
• The return probability gains from feedback plateaued after receiving around 4–5 feedback instances, indicating a saturation point beyond which additional feedback yields little extra benefit.