[Paper Review] Nmag micromagnetic simulation tool - software engineering lessons learned

Hans Fangohr, Maximilian Albert+1 more|ePrints Soton (University of Southampton)|Jan 27, 2016

Magnetic properties of thin films13 references3 citations

TL;DR

This paper reviews the software engineering practices behind Nmag, an open-source micromagnetic simulation tool built with Python for user accessibility and Objective Caml for performance. Key contributions include lessons on using Python for flexible, user-friendly interfaces, robust from-source installations, and the importance of comprehensive documentation and testing—offering actionable insights for future computational science software projects.

ABSTRACT

We review design and development decisions and their impact for the open source code Nmag from a software engineering in computational science point of view. We summarise lessons learned and recommendations for future computational science projects. Key lessons include that encapsulating the simulation functionality in a library of a general purpose language, here Python, provides great flexibility in using the software. The choice of Python for the top-level user interface was very well received by users from the science and engineering community. The from-source installation in which required external libraries and dependencies are compiled from a tarball was remarkably robust. In places, the code is a lot more ambitious than necessary, which introduces unnecessary complexity and reduces main- tainability. Tests distributed with the package are useful, although more unit tests and continuous integration would have been desirable. The detailed documentation, together with a tutorial for the usage of the system, was perceived as one of its main strengths by the community.

Motivation & Objective

• To document and analyze software engineering decisions made during the development of Nmag, a micromagnetic simulation tool.
• To identify and share lessons learned that improve maintainability, usability, and long-term sustainability of computational science software.
• To guide future projects in choosing appropriate technologies, testing strategies, and documentation practices.
• To evaluate the impact of specific design choices—such as using Python for the interface and OCaml for performance—on user adoption and developer contribution.
• To assess the effectiveness of from-source installation, version control, testing, and continuous integration in ensuring software robustness and reproducibility.

Proposed method

• The authors conducted a retrospective analysis of Nmag’s development lifecycle from 2005 to 2012, focusing on design and implementation decisions.
• They evaluated the use of Python as a high-level interface for scientific users, enabling flexibility and ease of use.
• The system leveraged Objective Caml for performance-critical components and finite element methods to solve time-dependent PDEs.
• A from-source installation approach compiled all dependencies (e.g., PETSc, HDF5, SUNDIALS) from source to ensure reproducibility and long-term stability.
• The project used Mercurial for version control and maintained separate repositories for code, tests, documentation, and web content.
• Testing included system tests, regression tests, and limited unit tests; continuous integration was not systematically used.

Experimental results

Research questions

• RQ1 How did the choice of Python as the top-level interface impact user adoption and extensibility in Nmag?
• RQ2 What were the trade-offs and benefits of using a from-source installation with compiled dependencies for long-term software stability?
• RQ3 How effective were the existing tests and documentation in supporting user adoption and software maintainability?
• RQ4 What role did version control and modular repository structure play in managing a complex scientific software project?
• RQ5 How could continuous integration and more extensive unit testing have improved the development process?

Key findings

• The use of Python as the top-level interface significantly enhanced usability and flexibility, with users able to modify and extend simulations easily.
• The from-source installation, though large (91 MB tarball, 1.4 GB disk usage), proved remarkably robust, failing only once in four years due to a GCC version change.
• Comprehensive documentation and tutorials were consistently cited by users as a major strength, contributing to the tool’s widespread adoption.
• System tests were highly effective for validating installations and detecting regressions, though more unit tests would have improved maintainability.
• Version control with distributed systems like Mercurial improved collaboration and long-term project sustainability.
• The lack of continuous integration limited release frequency and made it harder to detect integration issues early, despite the overall stability of the codebase.