[Paper Review] External Use of TOPCAT's Plotting Library

Mark Taylor|arXiv (Cornell University)|Oct 30, 2014

Scientific Research and Discoveries1 references3 citations

TL;DR

This paper presents a method for external use of TOPCAT’s custom Java plotting library, enabling high-performance, interactive, and static visualizations of large datasets (up to billions of points) through command-line (STILTS), Jython (JyStilts), Java API, or inter-process communication. The library supports scalable, configurable 2D and 3D plots with features like hybrid scatter/density rendering and efficient memory usage, achieving 30-minute rendering of a 2-billion-row all-sky density map.

ABSTRACT

The table analysis application TOPCAT uses a custom Java plotting library for highly configurable high-performance interactive or exported visualisations in two and three dimensions. We present here a variety of ways for end users or application developers to make use of this library outside of the TOPCAT application: via the command-line suite STILTS or its Jython variant JyStilts, via a traditional Java API, or by programmatically assigning values to a set of parameters in java code or using some form of inter-process communication. The library has been built with large datasets in mind; interactive plots scale well up to several million points, and static output to standard graphics formats is possible for unlimited sized input data.

Motivation & Objective

• To enable scientists and developers to use TOPCAT’s high-performance plotting library outside the TOPCAT GUI environment.
• To support interactive and static visualization of large datasets (up to billions of rows) with minimal memory overhead.
• To provide flexible integration methods—command-line, Jython, Java API, and inter-process communication—for diverse application contexts.
• To ensure extensibility and configurability of plot types, layers, and parameters without requiring UI code changes.
• To allow efficient rendering of complex visualizations such as density maps and spectrograms from massive datasets.

Proposed method

• Exposing the TOPCAT plotting library via STILTS, a command-line tool with a parameter-based interface for specifying plots using key-value pairs.
• Providing a Jython-compatible interface (JyStilts) to allow Python scripting with full access to plotting parameters and control structures.
• Enabling direct Java API integration by passing parameter name-value pairs via a Map object, supporting both string values and native Java objects (e.g., StarTable implementations).
• Supporting low-level programmatic control through a detailed Java API documented in Javadoc, allowing full customization with compile-time type checking.
• Using a pluggable, object-oriented interface design that allows dynamic discovery of plot types, layers, and configuration options for extensibility.
• Generating both interactive Swing-based GUI components and static output in PNG, GIF, JPEG, PDF, or PostScript formats with animation support.

Experimental results

Research questions

• RQ1 How can TOPCAT’s high-performance plotting library be exposed for use outside its native GUI environment?
• RQ2 What are the most effective integration patterns for enabling external use of the plotting library in scientific applications?
• RQ3 Can the library scale to visualizing datasets of unlimited size with efficient memory usage and acceptable performance?
• RQ4 How can complex, interactive, and static visualizations be generated using a consistent, extensible, and discoverable API?
• RQ5 What performance characteristics can be achieved when rendering very large datasets, such as 2 billion rows, in static output formats?

Key findings

• The plotting library successfully enables interactive visualization of datasets with up to several million points, maintaining high responsiveness.
• Static output generation supports datasets of unlimited size, with a 2-billion-row all-sky density map rendered in approximately 30 minutes.
• Hybrid scatter/density plots provide meaningful visual representations for extremely large datasets where standard scatter plots would be infeasible.
• The command-line interface (STILTS) allows simple plots to be generated with as few as four parameters, while supporting full configurability for complex visualizations.
• The Jython interface (JyStilts) enables seamless integration with Python workflows, supporting control structures and dynamic parameter assignment.
• The low-level Java API provides full programmatic control and compile-time type safety, though it requires significantly more code (50–100 lines) for basic plots.