[Paper Review] Not So Fast: Analyzing the Performance of WebAssembly vs. Native Code

Abhinav Jangda, Bobby Powers+2 more|arXiv (Cornell University)|Jan 25, 2019

Security and Verification in Computing19 references47 citations

TL;DR

The paper builds Browsix-Wasm and Browsix-SPEC to run unmodified Unix/WebAssembly apps in the browser and conducts a large SPEC CPU benchmark evaluation, finding WebAssembly is 1.55x slower on Chrome and 1.45x slower on Firefox than native, with substantial gaps and several efficiency issues identified.

ABSTRACT

All major web browsers now support WebAssembly, a low-level bytecode intended to serve as a compilation target for code written in languages like C and C++. A key goal of WebAssembly is performance parity with native code; previous work reports near parity, with many applications compiled to WebAssembly running on average 10% slower than native code. However, this evaluation was limited to a suite of scientific kernels, each consisting of roughly 100 lines of code. Running more substantial applications was not possible because compiling code to WebAssembly is only part of the puzzle: standard Unix APIs are not available in the web browser environment. To address this challenge, we build Browsix-Wasm, a significant extension to Browsix that, for the first time, makes it possible to run unmodified WebAssembly-compiled Unix applications directly inside the browser. We then use Browsix-Wasm to conduct the first large-scale evaluation of the performance of WebAssembly vs. native. Across the SPEC CPU suite of benchmarks, we find a substantial performance gap: applications compiled to WebAssembly run slower by an average of 45% (Firefox) to 55% (Chrome), with peak slowdowns of 2.08x (Firefox) and 2.5x (Chrome). We identify the causes of this performance degradation, some of which are due to missing optimizations and code generation issues, while others are inherent to the WebAssembly platform.

Motivation & Objective

• Motivate and enable large-scale, fair benchmarks of WebAssembly in the browser using unmodified Unix applications.
• Develop Browsix-Wasm to run WebAssembly-compiled Unix programs in-browser with minimal overhead.
• Create Browsix-SPEC to automate SPEC CPU2006/2017 and PolyBenchC benchmarking in browsers and collect detailed timing and hardware counter data.
• Quantitatively compare WebAssembly against native and analyze root causes of performance gaps.
• Provide implementers with actionable guidance to close the WebAssembly performance gap.

Proposed method

• Extend Browsix to Browsix-Wasm to run unmodified WebAssembly Unix programs in the browser with optimized kernel integration.
• Develop Browsix-SPEC to orchestrate browser instances, benchmark assets, and collect performance counters for SPEC and PolyBenchC in-browser benchmarks.
• Compile benchmarks to native (Clang) and WebAssembly (Browsix-Wasm) and measure across Chrome and Firefox without modifying browser code.
• Use performance counters to analyze causes of WebAssembly slowdown, including register pressure, instruction mix, and branching behavior.
• Compare WebAssembly against asm.js to quantify relative improvements and validate prior claims of speedups over asm.js.
• Provide a matrix of execution times and counter-based explanations to guide optimization of WebAssembly backends.

Experimental results

Research questions

• RQ1 How does WebAssembly performance in browsers compare to native code across large, real-world benchmark suites (SPEC CPU2006/2017) and PolyBenchC?
• RQ2 What are the dominant contributors to any observed WebAssembly slowdown relative to native (e.g., register allocation, instruction counts, branching, memory behavior)?
• RQ3 What is the overhead of the Browsix-Wasm/Browsix-SPEC infrastructure itself, and does it invalidate performance conclusions?
• RQ4 Does WebAssembly consistently outperform asm.js, and by how much, across Chrome and Firefox?
• RQ5 What practical guidance can be offered to implementers to reduce WebAssembly-Native performance gaps?

Key findings

• WebAssembly runs slower than native by 1.55x in Chrome and 1.45x in Firefox on average across SPEC benchmarks, with peak slowdowns up to 2.5x (Chrome) and 2.08x (Firefox).
• Seven of the 15 SPEC benchmarks run within 1.5x of native in Chrome, and seven of 15 do so in Firefox.
• WebAssembly yields a mean speedup of 1.54x over asm.js in Chrome and 1.39x in Firefox; best-case WebAssembly vs. best-case asm.js shows about 1.3x speedup.
• Compared to native, WebAssembly exhibits increased loads and stores (Chrome: 2.02x loads, 2.30x stores; Firefox: 1.92x loads, 2.16x stores), more branches, and more instruction cache misses, contributing to slower execution.
• In matrix multiplication, WebAssembly is 2x–3.4x slower than native, highlighting inefficiencies in code generation and register usage.
• Performance counter analysis reveals increased register pressure, poorer register allocation (Chrome/Firefox use linear scan vs. Clang’s greedy graph-coloring), and underutilization of x86 addressing modes as key degradation factors.