[Paper Review] Probabilistic Relational Model Benchmark Generation

Mouna Ben Ishak, Rajani Chulyadyo+1 more|arXiv (Cornell University)|Mar 2, 2016

Bayesian Modeling and Causal Inference33 references3 citations

TL;DR

This paper presents a novel algorithmic framework for generating synthetic probabilistic relational models (PRMs) and corresponding relational database instances from randomly generated schemas and probabilistic dependencies. By combining random DAG generation with probabilistic dependency modeling, it enables reproducible benchmarking for machine learning and database systems, supporting both SRL research and DBMS evaluation with scalable, uncertainty-aware data.

ABSTRACT

The validation of any database mining methodology goes through an evaluation process where benchmarks availability is essential. In this paper, we aim to randomly generate relational database benchmarks that allow to check probabilistic dependencies among the attributes. We are particularly interested in Probabilistic Relational Models (PRMs), which extend Bayesian Networks (BNs) to a relational data mining context and enable effective and robust reasoning over relational data. Even though a panoply of works have focused, separately , on the generation of random Bayesian networks and relational databases, no work has been identified for PRMs on that track. This paper provides an algorithmic approach for generating random PRMs from scratch to fill this gap. The proposed method allows to generate PRMs as well as synthetic relational data from a randomly generated relational schema and a random set of probabilistic dependencies. This can be of interest not only for machine learning researchers to evaluate their proposals in a common framework, but also for databases designers to evaluate the effectiveness of the components of a database management system.

Motivation & Objective

• To address the lack of publicly available benchmarks for Probabilistic Relational Models (PRMs) in machine learning and database research.
• To enable systematic evaluation of SRL learning algorithms and DBMS components under controlled, synthetic conditions.
• To generate PRMs and associated relational data that capture both functional and probabilistic dependencies.
• To support reproducible experimentation by providing a repeatable, algorithmic process for benchmark generation.

Proposed method

• Use the PMMixed algorithm to generate uniformly distributed, connected directed acyclic graphs (DAGs) for relational schema structures.
• Apply rejection sampling to ensure the generated DAGs are connected, preserving uniformity in the sample space.
• Randomly assign attributes and cardinalities per relation using Poisson distributions with λ=1 for attribute count and λ=1 for value states.
• Construct intra-class dependency structures via a modified PMMixed algorithm to model conditional probability distributions within each class.
• Generate inter-class dependencies by restricting edge creation to variables across different classes, ensuring relational integrity.
• Determine slot chains (paths) up to a maximum length K_max using recursive exploration of the schema graph to support complex probabilistic queries.

Experimental results

Research questions

• RQ1 How can a fully random, yet valid, probabilistic relational model be generated from scratch with controlled structural and probabilistic properties?
• RQ2 What algorithmic strategies ensure the generation of connected, scalable, and diverse relational schemas suitable for benchmarking?
• RQ3 How can probabilistic dependencies be systematically encoded across multiple classes while preserving model expressiveness and correctness?
• RQ4 To what extent can the proposed method generate synthetic data that reflects both functional and probabilistic constraints found in real-world databases?
• RQ5 Can the framework serve as a reusable benchmarking tool for both SRL and DBMS evaluation?

Key findings

• The proposed method successfully generates PRMs with both relational schema and probabilistic dependencies in a single, coherent pipeline.
• The use of rejection sampling after PMMixed-based DAG generation ensures connectedness while maintaining uniform distribution over the space of valid DAGs.
• Attribute cardinalities and value states are generated using Poisson distributions with λ=1, resulting in an average of 2 attributes per relation.
• The time complexity for relational schema generation is O(T * N * lg N), where T is the expected number of PMMixed calls needed to produce a connected DAG.
• The dependency structure generation has a time complexity of O(N * 2 * lg 2), reflecting the average of two attributes per class.
• The slot chain generation process has a time complexity of O(N^{K_max}), which captures the exponential cost of path enumeration in complex schemas.