[Paper Review] Towards an interdisciplinary, socio-technical analysis of software ecosystem health

Tom Mens, Bram Adams+1 more|arXiv (Cornell University)|Nov 13, 2017

Green IT and Sustainability21 references16 citations

TL;DR

This paper proposes an interdisciplinary, socio-technical methodology to assess and improve software ecosystem health by integrating ecological and toxicological principles with software engineering metrics. It introduces a framework for measuring technical and social health indicators, modeling event impacts, and predicting ecosystem resilience through mixed-methods analysis of open source ecosystems like npm and RubyGems, with key findings on fragility, developer turnover, and dependency network dynamics.

ABSTRACT

This extended abstract presents the research goals and preliminary research results of the interdisciplinary research project SECOHealth, an ongoing collaboration between research teams of Polytechnique Montreal (Canada), the University of Mons (Belgium) and Laval University (Canada). SECOHealth aims to contribute to research and practice in software engineering by delivering a validated interdisciplinary scientific methodology and a catalog of guidelines and recommendation tools for improving software ecosystem health.

Motivation & Objective

• Address the lack of systematic tools and guidelines for monitoring and improving software ecosystem (SECO) health, which is critical for societal stability given SECOs' pervasive role in modern digital infrastructure.
• Develop a validated, interdisciplinary methodology combining software engineering, ecology, and toxicology to analyze SECO health at both micro (individual projects) and macro (ecosystem-wide) levels.
• Identify and operationalize measurable indicators of technical and social health, including fragility, toxicity, diversity, and resilience, to support proactive ecosystem management.
• Build predictive models for how specific events—such as package unpublication or developer departure—affect SECO stability and sustainability.
• Create a catalog of actionable guidelines and recommendations for SECO maintainers to enhance ecosystem health, sustainability, and long-term survival.

Proposed method

• Apply a mixed-methods approach combining bottom-up data mining of version control, bug tracking, code review, and mailing list data with top-down qualitative interviews of 17 SECO practitioners at the Linux Foundation’s summit.
• Use survival analysis to model developer abandonment probabilities based on technical and social factors in npm and RubyGems ecosystems.
• Develop metrics to quantify dependency network evolution, including growth, changeability, reusability, and fragility, analyzing seven package distributions of varying size and age.
• Adopt ecological concepts such as biodiversity and resilience to study how diversity (geographical, temporal, gender-based, activity-based) affects ecosystem robustness.
• Model social toxicity using the concept of 'social debt'—strained interactions due to time pressure or inattention that may lead to community friction or contributor loss.
• Operationalize metrics using GrimoireLab, a CHAOSS-supported toolchain for open source software development analytics, enabling large-scale, reproducible ecosystem monitoring.

Experimental results

Research questions

• RQ1 What are the key technical and social indicators that reflect the health of a software ecosystem, and how can they be operationalized into measurable metrics?
• RQ2 How do specific events—such as package unpublication or key contributor departure—impact the overall health and stability of a software ecosystem?
• RQ3 To what extent does diversity (e.g., gender, geography, activity patterns) contribute to the resilience and sustainability of software ecosystems?
• RQ4 How do health problems propagate through dependency networks, and what metrics best capture fragility and reusability in these networks?
• RQ5 What social and technical factors predict the likelihood of developer abandonment in open source ecosystems?

Key findings

• A high proportion of packages in npm and RubyGems exhibit 'fragile' characteristics due to a growing number of transitive dependencies, increasing the risk of cascading failures.
• Survival analysis revealed that certain social and technical factors—such as low contributor engagement or high code complexity—significantly increase the probability of developer abandonment.
• Dependency networks in major ecosystems like npm and CRAN show consistent growth over time, with a small subset of packages responsible for the majority of updates and reverse dependencies.
• The number of reverse dependencies per package is highly skewed, indicating that a few central packages play a disproportionately large role in ecosystem stability.
• Empirical analysis confirmed that ecosystem resilience decreases with declining diversity, particularly in terms of contributor demographics and activity patterns.
• Social debt, measured through strained communication patterns, was identified as a key precursor to community conflict and contributor turnover in open source projects.