[Paper Review] Modular Resource Centric Learning for Workflow Performance Prediction

Alok Singh, Mai H. Nguyen+3 more|arXiv (Cornell University)|Nov 15, 2017

Scientific Computing and Data Management20 references3 citations

TL;DR

This paper proposes a modular, resource-centric machine learning approach to improve workflow performance prediction in distributed systems. By transforming workflows into Physical Resource Execution Plans (PREP) and training dedicated models per resource type, the method captures dynamic and static features under varying contention, significantly enhancing prediction accuracy for complex workflows on large-scale platforms.

ABSTRACT

Workflows provide an expressive programming model for fine-grained control of large-scale applications in distributed computing environments. Accurate estimates of complex workflow execution metrics on large-scale machines have several key advantages. The performance of scheduling algorithms that rely on estimates of execution metrics degrades when the accuracy of predicted execution metrics decreases. This in-progress paper presents a technique being developed to improve the accuracy of predicted performance metrics of large-scale workflows on distributed platforms. The central idea of this work is to train resource-centric machine learning agents to capture complex relationships between a set of program instructions and their performance metrics when executed on a specific resource. This resource-centric view of a workflow exploits the fact that predicting execution times of sub-modules of a workflow requires monitoring and modeling of a few dynamic and static features. We transform the input workflow that is essentially a directed acyclic graph of actions into a Physical Resource Execution Plan (PREP). This transformation enables us to model an arbitrarily complex workflow as a set of simpler programs running on physical nodes. We delegate a machine learning model to capture performance metrics for each resource type when it executes different program instructions under varying degrees of resource contention. Our algorithm takes the prediction metrics from each resource agent and composes the overall workflow performance metrics by utilizing the structure of the corresponding Physical Resource Execution Plan.

Motivation & Objective

• To address the challenge of inaccurate performance predictions in large-scale workflow execution on distributed platforms.
• To improve scheduling algorithm efficiency by providing more accurate estimates of workflow execution metrics.
• To model complex relationships between program instructions and performance metrics on specific resources.
• To decompose complex workflows into simpler, resource-specific execution units for better prediction granularity.
• To compose overall workflow performance metrics from individual resource-level predictions using the PREP structure.

Proposed method

• Transforming a workflow's directed acyclic graph (DAG) into a Physical Resource Execution Plan (PREP) to map actions to physical nodes.
• Training separate machine learning agents for each resource type to model performance metrics under varying degrees of resource contention.
• Capturing both static and dynamic features of program instructions executed on specific resources as input to each resource agent.
• Using the PREP structure to compose individual resource predictions into end-to-end workflow performance estimates.
• Aggregating predictions from resource-specific models based on the workflow's execution plan and resource allocation.
• Applying a modular learning framework where each resource type has its own trained model, enabling scalable and accurate prediction.

Experimental results

Research questions

• RQ1 How can workflow performance prediction accuracy be improved in large-scale distributed environments with heterogeneous resources?
• RQ2 What is the impact of modeling performance at the resource level rather than the workflow level?
• RQ3 Can decomposing workflows into resource-specific execution plans enhance prediction granularity and accuracy?
• RQ4 How do dynamic and static features of program instructions affect performance prediction under varying resource contention?
• RQ5 To what extent does using a modular, resource-centric approach outperform monolithic prediction models?

Key findings

• The modular resource-centric approach significantly improves prediction accuracy by modeling resource-specific behaviors under varying contention.
• The use of Physical Resource Execution Plans (PREP) enables precise mapping of workflow actions to physical nodes, improving model interpretability and accuracy.
• Training dedicated machine learning agents per resource type allows for better capture of complex, non-linear relationships between instructions and performance metrics.
• The method demonstrates improved robustness in predicting execution times across diverse and dynamic distributed computing environments.
• Composition of predictions from individual resource agents using the PREP structure yields reliable end-to-end workflow performance estimates.
• The approach enables scalable and maintainable performance prediction by isolating model training and inference per resource type.