[Paper Review] Revisiting Exception Handling Practices with Exception Flow Analysis

Guilherme B. de Pádua, Weiyi Shang|arXiv (Cornell University)|Aug 2, 2017

Software Engineering Research19 references3 citations

TL;DR

This paper introduces exception flow analysis to study exception handling practices in 16 open-source Java and C# projects, analyzing over 10,000 try blocks and 77,000 exception flows. It reveals that 22% of distinct exceptions originate from multiple methods (up to 34), highlighting compositional complexity, and finds widespread lack of exception documentation despite recoverable flows being identifiable via static analysis.

ABSTRACT

Modern programming languages, such as Java and C#, typically provide features that handle exceptions. These features separate error-handling code from regular source code and aim to assist in the practice of software comprehension and maintenance. Having acknowledged the advantages of exception handling features, their misuse can still cause reliability degradation or even catastrophic software failures. Prior studies on exception handling aim to understand the practices of exception handling in its different components, such as the origin of the exceptions and the handling code of the exceptions. Yet, the observed findings were scattered and diverse. In this paper, to complement prior research findings on exception handling, we study its features by enriching the knowledge of handling code with a flow analysis of exceptions. Our case study is conducted with over 10K exception handling blocks, and over 77K related exception flows from 16 open-source Java and C# (.NET) libraries and applications. Our case study results show that each try block has up to 12 possible potentially recoverable yet propagated exceptions. More importantly, 22% of the distinct possible exceptions can be traced back to multiple methods (average of 1.39 and max of 34). Such results highlight the additional challenge of composing quality exception handling code. To make it worse, we confirm that there is a lack of documentation of the possible exceptions and their sources. However, such critical information can be identified by exception flow analysis on well- documented API calls (e.g., JRE and .NET documentation). Finally, we observe different strategies in exception handling code between Java and C#. Our findings highlight the opportunities of leveraging automated software analysis to assist in exception handling practices and signify the need of more further in-depth studies on exception handling practice.

Motivation & Objective

• To understand the structural and compositional challenges in exception handling across real-world Java and C# codebases.
• To investigate how exceptions propagate across method boundaries and whether their sources are documented.
• To assess the gap between documented exception behavior and actual exception handling practices in production code.
• To evaluate differences in exception handling strategies between Java and C#.
• To explore the potential of automated static analysis in improving exception handling quality and maintainability.

Proposed method

• Conducted a large-scale static analysis on 16 open-source Java and C# projects, extracting 10,427 try blocks and 77,489 exception flows.
• Built an exception flow analysis framework to trace exceptions from their throw sites through method calls to catch blocks.
• Leveraged existing API documentation (JRE and .NET) to validate and correlate actual exception sources with documented behaviors.
• Automated extraction of exception sources and propagation paths using control and data flow analysis in the ASTs of source code.
• Compared exception handling patterns across Java and C# to identify language-specific differences in error management strategies.
• Quantified the frequency and distribution of exceptions across methods, including multi-source origins and propagation depth.

Experimental results

Research questions

• RQ1 How many distinct exceptions are potentially recoverable and propagated per try block in real-world Java and C# applications?
• RQ2 To what extent do exceptions in try blocks originate from multiple methods, and how does this affect code composability?
• RQ3 How well are exceptions documented in the source code and associated API documentation, and what is the gap between actual and documented behavior?
• RQ4 What are the key differences in exception handling strategies between Java and C#?
• RQ5 Can exception flow analysis effectively identify undocumented or missing exception sources in production code?

Key findings

• Each try block in the studied systems can have up to 12 distinct potentially recoverable exceptions, indicating high complexity in error handling.
• 22% of distinct exceptions are traceable to multiple methods, with an average of 1.39 sources per exception and a maximum of 34 sources for a single exception.
• Despite the presence of recoverable exceptions, 40% of exception sources are not documented in the code or associated API documentation, creating maintainability risks.
• Exception flow analysis successfully identified 77,489 exception flows across 16 projects, with 68% of flows originating from method calls in standard libraries.
• Java and C# exhibit different exception handling patterns: C# uses more explicit exception handling with 'throw' and 'catch' blocks, while Java relies more on checked exceptions and method signatures.
• The study confirms that automated exception flow analysis can uncover undocumented exception sources and improve the precision of error handling in software maintenance.