[Paper Review] Social Network Extraction: Superficial Method and Information Retrieval

Mahyuddin K. M. Nasution, Shahrul Azman Mohd Noah+1 more|arXiv (Cornell University)|Jan 12, 2016

Information Retrieval and Data Mining9 references15 citations

TL;DR

This paper proposes an unsupervised, information retrieval-driven method for extracting social networks from web documents by leveraging URL structures and TF-IDF-weighted keywords to improve relation extraction beyond simple co-occurrence. The approach achieves 70% recall and 9% precision using URL-based similarity (USR), outperforming keyword-only methods, and demonstrates that URL analysis reduces ambiguity in identifying social relations from unstructured web data.

ABSTRACT

Social network has become one of the themes of government issues, mainly dealing with the chaos. The use of web is steadily gaining ground in these issues. However, most of the web documents are unstructured and lack of semantic. In this paper we proposed an Information Retrieval driven method for dealing with heterogeneity of features in the web. The proposed solution is to compare some approaches have shown the capacity to extract social relation: strength relations and relations based on online academic database.

Motivation & Objective

• To address the challenge of extracting social networks from unstructured, semantically ambiguous web documents.
• To improve upon traditional co-occurrence-based methods by incorporating URL structure and TF-IDF to reduce synonymy and ambiguity.
• To evaluate the effectiveness of combining keyword similarity and URL-based matching for relation extraction in academic social networks.
• To benchmark performance against a manually curated DBLP-based social network using standard IR metrics.
• To explore the feasibility of using URL patterns as a proxy for semantic relations in unsupervised network extraction.

Proposed method

• Uses TF-IDF to select relevant keywords from web snippets, filtering those with values above 30% of the maximum TF-IDF score.
• Applies a threshold-based relation extraction method (SRS) using co-occurrence and similarity measures on keyword clusters.
• Introduces a URL-based similarity method (USR) that computes similarity between URLs of web pages to infer social relations.
• Employs Jaccard coefficient to measure similarity between extracted and benchmark networks, using recall, precision, and F-measure as evaluation metrics.
• Uses a benchmark graph derived from DBLP’s author-coauthor relationships to validate results.
• Applies association rule mining (ARS) to generate the ground truth network for comparison.

Experimental results

Research questions

• RQ1 Can URL structure be effectively used to enhance social relation extraction beyond keyword co-occurrence?
• RQ2 How does the performance of URL-based similarity compare to keyword-based similarity in extracting academic social networks?
• RQ3 To what extent does incorporating TF-IDF and URL analysis improve recall and precision in unsupervised social network extraction?
• RQ4 Does the proposed method reduce ambiguity caused by synonymy and polysemy in web-based relation extraction?
• RQ5 How does the method scale when applied to a large set of actor names and potential relations?

Key findings

• The URL-based similarity method (USR) achieved 70% recall and 9% precision, significantly outperforming the keyword-only method (SRS), which achieved 47% recall and 10% precision.
• USR identified 19,513 relations (86% of potential relations) at α=0.01, compared to 12,621 relations (53%) for SRS at α=0.0001.
• The Jaccard similarity between USR and the benchmark network was 0.009, slightly higher than SRS’s 0.0094, indicating better alignment with ground truth.
• Limiting the number of keywords per cluster improved F-measure by reducing noise from low-quality or irrelevant terms.
• The method demonstrated that URLs can serve as a stable, unambiguous signal for relation inference, reducing issues of synonymy and ambiguity in keyword-based methods.
• The results suggest that URL structure provides a valuable, underutilized signal for unsupervised social network extraction from web documents.