[Paper Review] Verifying the Medical Specialty from User Profile of Online Community for Health-Related Advices

Соломія Федушко, Nataliya Shakhovska+1 more|arXiv (Cornell University)|Dec 15, 2018

Information Systems and Technology Applications10 references9 citations

TL;DR

This paper proposes a computer-linguistic method to verify medical specialties from user profiles in online health communities by analyzing linguistic and communicative indicators in user posts. Using a training sample of over 3,000 Ukrainian-speaking medical forum users, the approach constructs a weighted indicator matrix to classify and verify user specialties, achieving 25.75% verified medical specialties in real-world testing on the 'Ukrainian Doctors Forum'.

ABSTRACT

The paper describes the verifying methods of medical specialty from user profile of online community for health-related advices. To avoid critical situations with the proliferation of unverified and inaccurate information in medical online community, it is necessary to develop a comprehensive software solution for verifying the user medical specialty of online community for health-related advices. The algorithm for forming the information profile of a medical online community user is designed. The scheme systems of formation of indicators of user specialization in the profession based on a training sample is presented. The method of forming the user information profile of online community for healthrelated advices by computer-linguistic analysis of the information content is suggested. The system of indicators based on a training sample of users in medical online communities is formed. The matrix of medical specialties indicators and method of determining weight coefficients these indicators is investigated. The proposed method of verifying the medical specialty from user profile is tested in online medical community.

Motivation & Objective

• To address the critical problem of unverified and potentially harmful medical advice in online health communities.
• To develop a practical, software-based method for verifying the medical specialty of users based on their profile and post content.
• To formalize a system of linguistic and communicative indicators that reliably reflect a user's medical specialization.
• To test the method in a real-world online medical community to validate its effectiveness and reliability.
• To support community administrators in identifying competent, verified medical contributors and filtering out unqualified advice.

Proposed method

• Constructs a training sample of verified users from trusted sources, including administrators and moderators with long-standing reputations.
• Identifies linguistic and communicative markers—grammatical, lexical-semantic, and lexical-syntactic features—associated with specific medical specialties through automated analysis of user information tracks.
• Forms indicator sets for each medical specialty by analyzing content across thematic sections of two Ukrainian-language medical forums.
• Develops a matrix of medical specialty indicators and assigns weight coefficients to each indicator based on frequency and relevance using a multi-level computer monitoring system.
• Applies computer-linguistic analysis to build user information profiles from post content, tagging profiles as 'verified' or 'unverified' based on consistency and completeness of specialty data.
• Employs a formalized algorithm to process user information tracks, verify personal data, and classify users into verified or unverified specialty categories.

Experimental results

Research questions

• RQ1 Can linguistic and communicative features in user posts reliably indicate a user's medical specialty in online health communities?
• RQ2 How can a weighted system of indicators be constructed to differentiate between medical specialties based on user content?
• RQ3 What is the real-world verification rate of medical specialties when applying this method in an active online medical community?
• RQ4 To what extent can automated computer-linguistic analysis reduce the risk of unqualified medical advice in online forums?
• RQ5 How effective is the method in distinguishing verified medical professionals from users with incorrect or missing specialty information?

Key findings

• The method achieved a 25.75% verification rate for medical specialties among users who explicitly listed their specialty in their profile on the 'Ukrainian Doctors Forum'.
• 49.16% of users in the community had medical specialties listed, indicating a significant base of potential verified contributors.
• 5.69% of users falsely claimed non-medical specialties, highlighting the need for automated verification to prevent misinformation.
• 25.96% of user profiles lacked any data about medical specialty, making them candidates for automated verification.
• 4.65% of users provided incorrect specialty information, demonstrating the risk of unverified profiles in online medical communities.
• The system successfully classified user information profiles using computer-linguistic analysis, enabling administrators to flag unverified or misleading profiles.