[Paper Review] Software Libraries and Their Reuse: Entropy, Kolmogorov Complexity, and Zipf's Law

Todd L. Veldhuizen|arXiv (Cornell University)|Aug 3, 2005

Spreadsheets and End-User Computing23 references20 citations

TL;DR

This paper introduces a theoretical model linking software reuse to information theory, defining an entropy parameter $ H \in [0,1] $ that quantifies the intrinsic diversity of a problem domain. Using Kolmogorov complexity and Zipf's law, it shows that the maximum proportion of reusable code is $ 1 - H $, with $ H \to 0 $ enabling large-scale component reuse and $ H \to 1 $ limiting reuse to minimal code fragments.

ABSTRACT

We analyze software reuse from the perspective of information theory and Kolmogorov complexity, assessing our ability to ``compress'' programs by expressing them in terms of software components reused from libraries. A common theme in the software reuse literature is that if we can only get the right environment in place-- the right tools, the right generalizations, economic incentives, a ``culture of reuse'' -- then reuse of software will soar, with consequent improvements in productivity and software quality. The analysis developed in this paper paints a different picture: the extent to which software reuse can occur is an intrinsic property of a problem domain, and better tools and culture can have only marginal impact on reuse rates if the domain is inherently resistant to reuse. We define an entropy parameter $H \in [0,1]$ of problem domains that measures program diversity, and deduce from this upper bounds on code reuse and the scale of components with which we may work. For ``low entropy'' domains with $H$ near 0, programs are highly similar to one another and the domain is amenable to the Component-Based Software Engineering (CBSE) dream of programming by composing large-scale components. For problem domains with $H$ near 1, programs require substantial quantities of new code, with only a modest proportion of an application comprised of reused, small-scale components. Preliminary empirical results from Unix platforms support some of the predictions of our model.

Motivation & Objective

• To investigate whether software reuse is fundamentally constrained by the intrinsic diversity of problem domains, rather than by tools or culture.
• To develop a theoretical framework using information theory and Kolmogorov complexity to quantify the limits of code reuse.
• To define an entropy parameter $ H \in [0,1] $ that measures program diversity within a problem domain and predicts reuse potential.
• To demonstrate that high-entropy domains inherently resist large-scale reuse, regardless of improved tools or development practices.
• To validate the model empirically using data from Unix platforms, showing alignment with theoretical predictions on component reuse patterns.

Proposed method

• Define a domain entropy parameter $ H \in [0,1] $ based on the distribution of program similarities and component usage frequencies.
• Apply Kolmogorov complexity to assess the compressibility of programs by expressing them as compositions of reusable components.
• Use information-theoretic entropy to model the average description length of programs in a domain, linking it to component reuse potential.
• Model component reuse using Zipf’s law, assuming that common components are used frequently and follow a power-law distribution.
• Derive an upper bound on reusable code as $ 1 - H $, where $ H $ is the entropy rate of the domain.
• Validate predictions using empirical data from Unix systems, analyzing component counts and distribution patterns in real software projects.

Experimental results

Research questions

• RQ1 To what extent is software reuse limited by the intrinsic diversity of a problem domain, independent of tools or culture?
• RQ2 How can information theory and Kolmogorov complexity be used to model the theoretical upper bounds of code reuse?
• RQ3 What role does Zipf’s law play in characterizing the frequency distribution of reusable components in software libraries?
• RQ4 Can the entropy parameter $ H $ accurately predict the proportion of reusable code in real-world software projects?
• RQ5 How does the model explain the observed gap between idealized component-based software engineering and real-world reuse rates?

Key findings

• The maximum proportion of code that can be reused from libraries is bounded by $ 1 - H $, where $ H \in [0,1] $ is the entropy of the problem domain.
• For domains with $ H \to 0 $, programs are highly similar, enabling large-scale reuse of high-level components.
• For domains with $ H \to 1 $, programs are highly diverse, and the proportion of reusable code approaches zero, even with optimal tools.
• Empirical data from Unix systems show component usage patterns consistent with Zipf’s law and support the model’s predictions on reuse distribution.
• Libraries can compress incompressible programs by exploiting domain-wide commonalities, but they remain incomplete due to inherent domain diversity.
• The model implies that improving tools or culture cannot significantly increase reuse rates in high-entropy domains, as reuse is fundamentally constrained by $ H $.