[Paper Review] A general framework for blockchain analytics

Massimo Bartoletti, Andrea Bracciali+2 more|arXiv (Cornell University)|Jan 1, 2017

Blockchain Technology Applications and Security17 references10 citations

TL;DR

This paper presents a general-purpose, open-source Scala library framework for blockchain analytics that unifies data from Bitcoin and Ethereum with external sources into either SQL or NoSQL databases, enabling flexible, reusable, and scalable analytics. The key contribution is a modular, extensible abstraction layer that supports diverse use cases and outperforms ad-hoc tools in expressiveness and maintainability.

ABSTRACT

Modern cryptocurrencies exploit decentralised blockchains to record a public and unalterable history of transactions. Besides transactions, further information is stored for different, and often undisclosed, purposes, making the blockchains a rich and increasingly growing source of valuable information, in part of difficult interpretation. Many data analytics have been developed, mostly based on specifically designed and ad-hoc engineered approaches. We propose a general-purpose framework, seamlessly supporting data analytics on both Bitcoin and Ethereum - currently the two most prominent cryptocurrencies. Such a framework allows us to integrate relevant blockchain data with data from other sources, and to organise them in a database, either SQL or NoSQL. Our framework is released as an open-source Scala library. We illustrate the distinguishing features of our approach on a set of significant use cases, which allow us to empirically compare ours to other competing proposals, and evaluate the impact of the database choice on scalability.

Motivation & Objective

• Address the lack of reusable, general-purpose tools for blockchain analytics that integrate internal blockchain data with external sources.
• Overcome the limitations of ad-hoc, one-off analytics tools that duplicate functionality and lack extensibility.
• Provide a unified, modular framework supporting both Bitcoin and Ethereum, with flexible database backends (SQL and NoSQL).
• Enable researchers to build, validate, and extend analytics using standard query languages and community-driven development.
• Evaluate the performance trade-offs between SQL and NoSQL databases for blockchain analytics workloads.

Proposed method

• Design a two-step workflow: (1) construct a unified blockchain view by ingesting raw blockchain data and enriching it with external data (e.g., exchange rates, address tags, metadata), and (2) store the view in a SQL or NoSQL database.
• Implement a domain-specific Scala library that abstracts blockchain data parsing and external data integration, supporting both Bitcoin and Ethereum.
• Support schema extensibility and dynamic data linking to external sources (e.g., blockchain explorers, APIs, forums) via pluggable modules.
• Use standard SQL or NoSQL query languages (e.g., MySQL, MongoDB) for analysis, enabling complex queries without low-level programming.
• Optimize for consumer hardware by using disk-based storage instead of in-memory processing, improving accessibility.
• Provide open-source release with example use cases to demonstrate functionality and performance across different database backends.

Experimental results

Research questions

• RQ1 Can a general-purpose, composable framework unify diverse blockchain analytics tasks that currently rely on ad-hoc, non-reusable tools?
• RQ2 How does the integration of external data (e.g., exchange rates, address tags) impact the expressiveness and scalability of blockchain analytics?
• RQ3 What are the performance characteristics of SQL versus NoSQL databases when storing and querying blockchain analytics views?
• RQ4 To what extent does the framework outperform existing tools in terms of flexibility, maintainability, and support for extensibility?
• RQ5 Can the framework support real-world use cases such as transaction fee analysis, metadata tracking, and address tagging with comparable or better performance?

Key findings

• The framework enables the construction of blockchain views that are more than twice the size of the native Bitcoin blockchain due to the inclusion of textual script representations and external data.
• Query performance in both SQL and NoSQL databases is comparable in terms of execution time, with the main performance bottleneck in SQL arising from complex join operations.
• The NoSQL approach simplifies Scala scripts due to its schema-less nature, reducing boilerplate and increasing development agility.
• The framework supports a wide range of use cases—metadata analysis, exchange rate tracking, transaction fees, and address tagging—demonstrating its generality and reusability.
• Unlike RAM-based tools like BlockSci and blockparser, the disk-based approach enables execution on consumer-grade hardware, increasing accessibility.
• The framework’s open-source release under a permissive license facilitates community validation, extension, and long-term maintainability, addressing a key limitation of proprietary or monolithic tools.