[Paper Review] General-purpose Declarative Inductive Programming with Domain-Specific Background Knowledge for Data Wrangling Automation

Lidia Contreras-Ochando, Cèsar Ferri+4 more|arXiv (Cornell University)|Sep 26, 2018

Logic, Reasoning, and Knowledge20 references3 citations

TL;DR

This paper proposes a general-purpose declarative inductive programming approach enhanced with domain-specific background knowledge to automate data wrangling tasks from minimal examples. By integrating a functional programming language with a domain-specific library and an IP system (MagicHaskeller), it achieves high accuracy across diverse data formats—outperforming FlashFill—while enabling extensibility through user-defined domain files.

ABSTRACT

Given one or two examples, humans are good at understanding how to solve a problem independently of its domain, because they are able to detect what the problem is and to choose the appropriate background knowledge according to the context. For instance, presented with the string "8/17/2017" to be transformed to "17th of August of 2017", humans will process this in two steps: (1) they recognise that it is a date and (2) they map the date to the 17th of August of 2017. Inductive Programming (IP) aims at learning declarative (functional or logic) programs from examples. Two key advantages of IP are the use of background knowledge and the ability to synthesise programs from a few input/output examples (as humans do). In this paper we propose to use IP as a means for automating repetitive data manipulation tasks, frequently presented during the process of {\em data wrangling} in many data manipulation problems. Here we show that with the use of general-purpose declarative (programming) languages jointly with generic IP systems and the definition of domain-specific knowledge, many specific data wrangling problems from different application domains can be automatically solved from very few examples. We also propose an integrated benchmark for data wrangling, which we share publicly for the community.

Motivation & Objective

• Address the high manual effort and lack of systematic automation in data wrangling tasks across diverse domains.
• Overcome limitations of existing tools like FlashFill, which fail on complex or ambiguous data formats due to insufficient background knowledge.
• Enable automatic program synthesis from just one or two input-output examples using declarative, functional programming and domain-specific knowledge.
• Provide a publicly available benchmark dataset to ensure reproducibility and facilitate evaluation of future data wrangling tools.
• Demonstrate that combining general-purpose functional programming with domain-specific background knowledge enables scalable, extensible, and accurate data transformation

Proposed method

• Use a general-purpose functional programming language (Haskell) as the target language for program synthesis.
• Integrate a domain-specific background knowledge (DSBK) library containing functions and constraints relevant to specific data types (e.g., dates, addresses, names).
• Leverage an inductive programming system (MagicHaskeller) to infer correct transformation programs from one or two input-output examples.
• Define domain-specific libraries using a functional language, allowing users to extend or customize the system for new data types.
• Apply a hypothesis-driven search strategy guided by background knowledge to reduce search space and improve synthesis efficiency.
• Evaluate the system on a newly created public benchmark of 18 datasets across six domains, including dates, emails, phone numbers, and names.

Experimental results

Research questions

• RQ1 Can a general-purpose inductive programming system, enhanced with domain-specific background knowledge, automate data wrangling tasks from minimal examples?
• RQ2 How does the integration of domain-specific knowledge improve the accuracy and robustness of program synthesis compared to DSL-based systems like FlashFill?
• RQ3 To what extent can a single, extensible framework handle diverse data wrangling problems across multiple domains without requiring reconfiguration or retraining?
• RQ4 Can a publicly shared benchmark with diverse, real-world data wrangling problems enable reproducible evaluation and foster tool development?
• RQ5 How does the depth of the synthesized program (number of primitives) and the size of the background knowledge library affect synthesis performance?

Key findings

• The proposed DSI-based approach outperformed FlashFill on complex data wrangling tasks, particularly with dates in multiple formats, where FlashFill failed due to lack of contextual knowledge.
• The system successfully synthesized correct programs for 17 out of 18 datasets in the benchmark, including challenging cases like ambiguous date formats and non-standard punctuation.
• Using only one example, the system achieved high accuracy in all domains tested, demonstrating the effectiveness of domain-specific background knowledge in guiding synthesis.
• The integration of functional programming with domain-specific libraries enabled concise, human-readable programs with low depth (d), indicating efficient and compact solutions.
• The public benchmark repository, available at https://github.com/liconoc/DataWrangling-DSI, provides reusable, well-documented datasets for evaluating future data wrangling tools.
• The approach is extensible: users can define new domain files in a general-purpose functional language, allowing adaptation to new data types without system redesign.