[Paper Review] What makes a good Node.js package? Investigating Users, Contributors, and Runnability

Bodin Chinthanet, Brittany Reid+5 more|arXiv (Cornell University)|Jun 23, 2021

Software Engineering Research62 references4 citations

TL;DR

This study investigates what makes a good Node.js package by analyzing developer perspectives as users and contributors. It identifies shared and distinct quality indicators through a survey of 33 developers and analyzes 104,364 npm packages, revealing that runnability—especially executable code snippets and test success—strongly correlates with package quality and can be predicted using features like file count and commit history.

ABSTRACT

The Node.js Package Manager (i.e., npm) archive repository serves as a critical part of the JavaScript community and helps support one of the largest developer ecosystems in the world. However, as a developer, selecting an appropriate npm package to use or contribute to can be difficult. To understand what features users and contributors consider important when searching for a good npm package, we conduct a survey asking Node.js developers to evaluate the importance of 30 features derived from existing work, including GitHub activity, software usability, and properties of the repository and documentation. We identify that both user and contributor perspectives share similar views on which features they use to assess package quality. We then extract the 30 features from 104,364 npm packages and analyse the correlations between them, including three software features that measure package ``runnability"; ability to install, build, and execute a unit test. We identify which features are negatively correlated with runnability-related features and find that predicting the runnability of packages is viable. Our study lays the groundwork for future work on understanding how users and contributors select appropriate npm packages.

Motivation & Objective

• To understand the criteria developers use when selecting or contributing to npm packages.
• To compare user and contributor perspectives on package quality features.
• To evaluate the runnability of npm packages through installability, buildability, and test execution.
• To identify predictive features for package runnability using empirical data from 104,364 packages.
• To provide a foundation for future work on package recommendation and quality metrics that consider both user and contributor needs.

Proposed method

• Conducted an online survey with 33 Node.js developers to assess the importance of 30 package quality features.
• Categorized features into four types: documentation, repository, software (runnability), and GitHub activity.
• Collected and analyzed data from 104,364 npm packages using the GH-Node.js dataset to measure runnability and feature correlations.
• Evaluated runnability through three metrics: install success, build success, and test execution success of code snippets.
• Used statistical correlation analysis to identify relationships between features and runnability.
• Built predictive models for runnability based on feature importance, showing varying predictive power across different runnability metrics.

Experimental results

Research questions

• RQ1 Which features of an npm package do practitioners consider relevant for assessing package quality?
• RQ2 How do user and contributor perspectives on package quality differ or align?
• RQ3 What is the correlation between package features and runnability (install, build, test execution)?
• RQ4 Can runnability of npm packages be predicted using static and activity-based features?
• RQ5 Which features are most predictive of successful installation, building, and test execution in npm packages?

Key findings

• Users and contributors share similar views on package quality, with 50% of the 30 features relevant to both perspectives.
• Documentation and software runnability features are considered highly important by both user and contributor groups.
• Repository features such as file count and commit history are not seen as indicators of quality by developers.
• Of 104,364 npm packages, 92.95% could be successfully installed, 61.59% contained at least one code snippet in README, and only 15.20% of 220,324 code snippets were executable.
• Among 11,127 packages updated in 2020, 73.68% could be built and only 7.40% passed all test cases, indicating low test runnability.
• Predicting runnability is feasible, with file count and commit history strongly correlating with installability, but less so with buildability.