[Paper Review] dup -- Explicit un-sharing in Haskell

Joachim Breitner|arXiv (Cornell University)|Jul 9, 2012

Security and Verification in Computing7 references3 citations

TL;DR

This paper introduces two explicit un-sharing operations, dup and deepDup, in Haskell that allow programmers to break sharing in lazy data structures without modifying source code. By wrapping values in a Box type or recursively copying heap-allocated thunks, these primitives prevent space leaks caused by unintended retention of large data structures, enabling constant-memory execution in scenarios like tree traversal with deep recursion.

ABSTRACT

We propose two operations to prevent sharing in Haskell that do not require modifying the data generating code, demonstrate their use and usefulness, and compare them to other approaches to preventing sharing. Our claims are supported by a formal semantics and a prototype implementation.

Motivation & Objective

• To address space leaks in lazy functional programs caused by unintended sharing of large data structures.
• To provide a direct, programmer-controlled mechanism to break sharing without modifying the original data-generating code.
• To offer a practical solution for performance-critical applications where garbage collection overhead from retained data becomes prohibitive.
• To formalize the behavior of un-sharing operations using Launchbury's natural semantics and prove their correctness.
• To demonstrate feasibility through a prototype implementation targeting unmodified GHC.

Proposed method

• Introduces the dup primitive that wraps a value in a Box type to break sharing at the point of use.
• Proposes deepDup as a recursive variant that lazily copies all heap-allocated thunks referenced by the input value.
• Uses a custom C--/Cmm implementation to copy closures on the heap, preserving evaluation order and ensuring no shared thunks remain.
• Employs a stack-based closure copying loop that processes all pointers in a closure, creating new heap-allocated copies.
• Designs a runtime mechanism that avoids corrupting the garbage collector by not modifying static thunks or code segments.
• Implements a prototype that integrates with unmodified GHC, demonstrating compatibility and correctness.

Experimental results

Research questions

• RQ1 Can explicit un-sharing be achieved in Haskell without source transformation or compiler modifications?
• RQ2 How can un-sharing be implemented safely in the presence of lazy evaluation and garbage collection?
• RQ3 What is the formal semantics of dup and deepDup in the context of lazy evaluation?
• RQ4 Can deepDup guarantee that no retained heap objects remain after evaluation, except those referenced by the return value?
• RQ5 What are the performance and memory trade-offs of using dup and deepDup in practice?

Key findings

• The prototype implementation successfully integrates with unmodified GHC and correctly prevents space leaks in the running example.
• With deepDup, the program's memory usage drops from 4.189 GB to 2 MB when the final line of main is removed, demonstrating effective un-sharing.
• The formal semantics in Launchbury's natural semantics framework proves that deepDup ensures no retained heap objects remain after evaluation, except those referenced by the return value.
• The implementation correctly handles recursive data structures and avoids issues with shared thunks by lazily copying referenced closures.
• The approach is safe for pure functional code but may break when used with unsafePerformIO or Lazy I/O due to side effects in duplicated thunks.
• Static thunks are currently not duplicated due to runtime representation constraints, but the paper suggests potential extensions for future support.