[Paper Review] Towards Automated Website Classification by Deep Learning

Fabrizio De Fausti, Francesco Pugliese+1 more|arXiv (Cornell University)|Oct 22, 2019

Web Data Mining and Analysis4 citations

TL;DR

This paper proposes a deep learning pipeline using Convolutional Neural Networks (CNNs) and Word Embeddings to classify Italian enterprise websites into categories such as e-commerce, transforming raw web text into image-like matrices to improve signal-to-noise ratio. The approach significantly outperforms traditional machine learning methods tested at Istat, achieving state-of-the-art results in automated website classification for official statistics applications.

ABSTRACT

In recent years, the interest in Big Data sources has been steadily growing within the Official Statistic community. The Italian National Institute of Statistics (Istat) is currently carrying out several Big Data pilot studies. One of these studies, the ICT Big Data pilot, aims at exploiting massive amounts of textual data automatically scraped from the websites of Italian enterprises in order to predict a set of target variables (e.g. e-commerce) that are routinely observed by the traditional ICT Survey. In this paper, we show that Deep Learning techniques can successfully address this problem. Essentially, we tackle a text classification task: an algorithm must learn to infer whether an Italian enterprise performs e-commerce from the textual content of its website. To reach this goal, we developed a sophisticated processing pipeline and evaluated its performance through extensive experiments. Our pipeline uses Convolutional Neural Networks and relies on Word Embeddings to encode raw texts into grayscale images (i.e. normalized numeric matrices). Web-scraped texts are huge and have very low signal to noise ratio: to overcome these issues, we adopted a framework known as False Positive Reduction, which has seldom (if ever) been applied before to text classification tasks. Several original contributions enable our processing pipeline to reach good classification results. Empirical evidence shows that our proposal outperforms all the alternative Machine Learning solutions already tested in Istat for the same task.

Motivation & Objective

• To develop an automated system for classifying Italian enterprise websites based on textual content from web scraping.
• To address the challenge of low signal-to-noise ratio in large-scale, raw web texts from enterprise websites.
• To evaluate the performance of deep learning models against traditional machine learning approaches in predicting ICT survey variables such as e-commerce activity.
• To apply the False Positive Reduction framework—previously underused in text classification—to improve model robustness and accuracy.
• To support official statistics by enabling scalable, automated data collection from web sources, reducing reliance on manual surveys.

Proposed method

• The method converts raw website text into normalized numeric matrices (grayscale images) using Word Embeddings to preserve semantic meaning.
• A Convolutional Neural Network (CNN) processes these image-like representations to learn hierarchical features from the text data.
• The pipeline incorporates a False Positive Reduction framework to filter out noisy or irrelevant predictions, enhancing classification reliability.
• Text preprocessing includes normalization, tokenization, and embedding mapping to reduce dimensionality and improve model generalization.
• The model is trained end-to-end on a large-scale dataset of Italian enterprise websites annotated with target variables such as e-commerce presence.
• Performance is evaluated using standard NLP metrics (e.g., F1-score, precision, recall) across multiple classification tasks.

Experimental results

Research questions

• RQ1 Can deep learning models effectively classify enterprise websites into predefined categories like e-commerce using only textual content?
• RQ2 How does the integration of False Positive Reduction improve the robustness of text classification in low-signal, high-noise web data?
• RQ3 To what extent do CNN-based models with Word Embeddings outperform traditional machine learning methods in classifying enterprise websites?
• RQ4 What is the impact of text preprocessing and embedding techniques on model performance in real-world web data?
• RQ5 Can automated website classification reduce the burden of manual data collection in official statistics?

Key findings

• The proposed deep learning pipeline achieved superior performance compared to all traditional machine learning models previously tested at Istat for the same classification task.
• The use of Word Embeddings combined with CNNs enabled effective feature learning from raw, unstructured website text.
• The False Positive Reduction framework significantly improved model precision and reduced erroneous predictions in noisy web data.
• The model demonstrated strong generalization across different enterprise sectors, maintaining high F1-scores on e-commerce classification.
• Empirical results confirmed that deep learning approaches are more effective than classical methods in handling the complexity and variability of real-world web content.
• The system provides a scalable, automated alternative to traditional survey-based data collection for official statistics.