[Paper Review] Automatically Extracting Web API Specifications from HTML Documentation

Jinqiu Yang, Erik Wittern+3 more|arXiv (Cornell University)|Jan 26, 2018

Web Data Mining and Analysis21 references3 citations

TL;DR

This paper presents D2Spec, a machine learning-based tool that automatically extracts web API endpoint specifications—including base URLs, path templates, and HTTP methods—from HTML-based API documentation. Evaluated on 120 APIs, D2Spec achieved 87.5% precision for base URLs, 81.3% precision and 80.6% recall for path templates, and 84.4% precision and 76.2% recall for HTTP methods, demonstrating its effectiveness in mining specifications from unstructured documentation.

ABSTRACT

Web API specifications are machine-readable descriptions of APIs. These specifications, in combination with related tooling, simplify and support the consumption of APIs. However, despite the increased distribution of web APIs, specifications are rare and their creation and maintenance heavily relies on manual efforts by third parties. In this paper, we propose an automatic approach and an associated tool called D2Spec for extracting specifications from web API documentation pages. Given a seed online documentation page on an API, D2Spec first crawls all documentation pages on the API, and then uses a set of machine learning techniques to extract the base URL, path templates, and HTTP methods, which collectively describe the endpoints of an API. We evaluated whether D2Spec can accurately extract endpoints from documentation on 120 web APIs. The results showed that D2Spec achieved a precision of 87.5% in identifying base URLs, a precision of 81.3% and a recall of 80.6% in generating path templates, and a precision of 84.4% and a recall of 76.2% in extracting HTTP methods. In addition, we found that D2Spec was useful when applied to APIs with pre-existing API specifications: D2Spec revealed many inconsistencies between web API documentation and their corresponding publicly available specifications. Thus, D2Spec can be used by web API providers to keep documentation and specifications in synchronization.

Motivation & Objective

• To address the scarcity of machine-readable web API specifications despite the widespread use of web APIs.
• To reduce the manual effort required to create and maintain API specifications by automating extraction from existing HTML documentation.
• To improve consistency between web API documentation and published specifications by detecting discrepancies.
• To enable broader tooling support for API consumption by generating OpenAPI-compliant specifications from public documentation.

Proposed method

• D2Spec begins by crawling all documentation pages associated with a seed API documentation page.
• It applies a set of machine learning models to identify and extract base URLs from free-form text and HTML structures.
• Path templates are extracted using pattern recognition and inference from multiple example URLs, leveraging syntactic and structural cues.
• HTTP methods are identified by analyzing textual context and proximity to path templates, using NLP and rule-based filtering.
• The approach handles two distinct documentation styles: example-based (e.g., GitHub) and reference-based (e.g., Instagram).
• The system uses heuristics and training data to generalize across diverse API providers and documentation layouts.

Experimental results

Research questions

• RQ1 Can machine learning techniques accurately extract base URLs from semi-structured HTML API documentation?
• RQ2 To what extent can path templates be inferred from example URLs and surrounding text in documentation?
• RQ3 How accurately can HTTP methods be identified from their textual context near path descriptions?
• RQ4 Can D2Spec detect inconsistencies between existing public API specifications and their corresponding documentation?
• RQ5 How generalizable is the approach across diverse API providers and documentation styles?

Key findings

• D2Spec achieved 87.5% precision in identifying base URLs across 120 web APIs, indicating strong reliability in URL extraction.
• The tool achieved 81.3% precision and 80.6% recall in generating path templates, showing effective inference from example URLs and structural patterns.
• For HTTP method extraction, D2Spec reached 84.4% precision and 76.2% recall, demonstrating robust detection from contextual clues.
• The evaluation revealed significant inconsistencies between existing public API specifications and their corresponding documentation, highlighting the value of D2Spec for validation.
• D2Spec successfully generalized across 98 different API providers, showing adaptability to diverse documentation structures and styles.