[Paper Review] Analyzing Query Performance and Attributing Blame for Contentions in a Cluster Computing Framework

Prajakta Kalmegh, Shivnath Babu+1 more|arXiv (Cornell University)|Aug 28, 2017

Cloud Computing and Resource Management19 references3 citations

TL;DR

ProtoXplore is a novel system that enables administrators to diagnose query performance slowdowns in shared cluster environments by attributing blame for resource contention to concurrent queries using a new metric, Resource Acquire Time Penalty (RATP). It employs a multi-level Proto-Graph to visualize and explore contention sources, significantly improving diagnostic accuracy and reducing manual effort in identifying performance bottlenecks in mixed workloads on Apache Spark.

ABSTRACT

There are many approaches is use today to either prevent or minimize the impact of inter-query interactions on a shared cluster. Despite these measures, performance issues due to concurrent executions of mixed workloads still prevail causing undue waiting times for queries. Analyzing these resource interferences is thus critical in order to answer time sensitive questions like 'who is causing my query to slowdown' in a multi-tenant environment. More importantly, dignosing whether the slowdown of a query is a result of resource contentions caused by other queries or some other external factor can help an admin narrow down the many possibilities of performance degradation. This process of investigating the symptoms of resource contentions and attributing blame to concurrent queries is non-trivial and tedious, and involves hours of manually debugging through a cycle of query interactions. In this paper, we present ProtoXplore - a Proto or first system to eXplore contentions, that helps administrators determine whether the blame for resource bottlenecks can be attributed to concurrent queries, and uses a methodology called Resource Acquire Time Penalty (RATP) to quantify this blame towards contentious sources accurately. Further, ProtoXplore builds on the theory of explanations and enables a step-wise deep exploration of various levels of performance bottlenecks faced by a query during its execution using a multi-level directed acyclic graph called ProtoGraph. Our experimental evaluation uses ProtoXplore to analyze the interactions between TPC-DS queries on Apache Spark to show how ProtoXplore provides explanations that help in diagnosing contention related issues and better managing a changing mixed workload in a shared cluster.

Motivation & Objective

• To address the challenge of diagnosing performance slowdowns in multi-tenant cluster environments where resource contention from concurrent queries is a major but often misdiagnosed cause.
• To provide a systematic framework for blaming specific concurrent queries as the root cause of performance degradation, rather than relying on manual, time-consuming debugging.
• To enable administrators to distinguish between contention-induced slowdowns and other external causes, supporting both diagnosis and targeted mitigation.
• To build a scalable, web-based interface that allows users of varying expertise to explore contention sources through visualized blame attribution and performance metrics.

Proposed method

• Introduces Resource Acquire Time Penalty (RATP), a metric that quantifies the blame for resource contention by measuring blocked times across multiple resources (CPU, network, IO, memory, scheduling queue).
• Constructs a multi-level Directed Acyclic Graph (Proto-Graph) to consolidate RATP and blame values at different granularities, from task-level to query-level interactions.
• Uses time-series monitoring data from cluster workloads to compute edge weights (IF) and node weights (DOR) in the Proto-Graph, enabling parallel construction of subgraphs.
• Employs a web-based UI that visualizes resource-to-host RATP heatmaps and impact scores, allowing users to explore contention contributors step-by-step.
• Applies the theory of explanations to guide users through a hierarchical exploration of performance bottlenecks, starting from low-level resource waits to high-level query interactions.
• Validates the system using TPC-DS workloads on Apache Spark, demonstrating its ability to accurately attribute blame and improve diagnostic efficiency.

Experimental results

Research questions

• RQ1 How can we accurately attribute blame for query performance slowdowns to specific concurrent queries in a shared cluster?
• RQ2 What metrics can systematically quantify the contribution of one query to the resource contention affecting another?
• RQ3 How can we model and visualize complex inter-query interactions to support hierarchical, multi-level debugging of performance bottlenecks?
• RQ4 To what extent does a visual, explanation-driven interface improve diagnostic accuracy and efficiency compared to manual analysis?

Key findings

• Users achieved 69% accuracy in identifying performance causes using ProtoXplore, compared to only 37% with manual analysis using existing monitoring tools.
• The system reduced user abandonment from 54% to 2% in diagnostic tasks, indicating a significant improvement in usability and task completion.
• Experts accepted ProtoXplore’s output as correct in 90% of cases where their manual answers differed, validating the system’s diagnostic reliability.
• The time spent on manual analysis was 19% less than on ProtoXplore, but this was offset by higher task completion and accuracy, indicating that the system’s guidance reduced cognitive load.
• The Graph Constructor module dominated overall latency, highlighting the need for optimization in large-scale deployment.
• The Proto-Graph structure enabled parallel construction of subgraphs across levels, improving scalability for complex workloads.