[Paper Review] FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices

Haoran Qiu, Subho S. Banerjee+3 more|arXiv (Cornell University)|Aug 19, 2020

Software System Performance and ReliabilityComputer Science86 references64 citations

TL;DR

FIRM is an ML-driven framework that detects and localizes SLO violations in microservices, identifies contention at fine granularity, and mitigates issues through RL-driven reprovisioning and scaling.

ABSTRACT

Modern user-facing latency-sensitive web services include numerous distributed, intercommunicating microservices that promise to simplify software development and operation. However, multiplexing of compute resources across microservices is still challenging in production because contention for shared resources can cause latency spikes that violate the service-level objectives (SLOs) of user requests. This paper presents FIRM, an intelligent fine-grained resource management framework for predictable sharing of resources across microservices to drive up overall utilization. FIRM leverages online telemetry data and machine-learning methods to adaptively (a) detect/localize microservices that cause SLO violations, (b) identify low-level resources in contention, and (c) take actions to mitigate SLO violations via dynamic reprovisioning. Experiments across four microservice benchmarks demonstrate that FIRM reduces SLO violations by up to 16x while reducing the overall requested CPU limit by up to 62%. Moreover, FIRM improves performance predictability by reducing tail latencies by up to 11x.

Motivation & Objective

• Motivate the need for fine-grained resource multiplexing to reduce SLO violations in latency-sensitive microservices.
• Develop an online, architecture-agnostic framework that localizes root causes and mitigates contention at microservice granularity.
• Leverage machine learning (SVM for localization; RL for mitigation) and online telemetry for adaptive resource reprovisioning.
• Enable rapid training and adaptation via performance anomaly injection and transfer learning.
• Demonstrate open-source implementation on Kubernetes with evaluation on real microservice benchmarks.

Proposed method

• Use online telemetry and tracing data to monitor microservice executions and resource usage.
• Apply SVM-based detection to localize SLO-violating microservice instances on critical paths.
• Use an RL-based (DDPG) mitigation loop to decide fine-grained reprovisioning actions (scale-up/scale-out and resource partitioning).
• Extract critical paths and critical components via graph-based analysis of execution histories and per-instance variability metrics.
• Train RL agents with an anomaly injection framework and transfer learning to adapt to workload changes.
• Provide an open-source Kubernetes integration and an evaluation on multiple microservice benchmarks.

Experimental results

Research questions

• RQ1 RQ1 How accurately can FIRM detect and localize microservice instances responsible for SLO violations using tracing and ML techniques?
• RQ2 RQ2 How effective is the RL-based mitigation in reducing SLO violations, tail latency, and overall CPU resource usage across varying workloads and contention types?
• RQ3 RQ3 How do online anomaly injection and transfer learning contribute to training efficiency and adaptability of FIRM's models?
• RQ4 RQ4 How does FIRM compare to Kubernetes autoscaling and AIMD-based approaches in reducing SLO violations and improving performance predictability?

Key findings

• FIRM reduces SLO violations by up to 16x compared with Kubernetes autoscaling and up to 9x versus AIMD-based methods.
• FIRM reduces the overall requested CPU by up to 62%.
• FIRM cuts average tail latencies by up to 11x, improving performance predictability.
• Critical-path localization accuracy averaged 93% across benchmarks.
• FIRM achieves faster mitigation, up to 30x improvement over Kubernetes autoscaling in response time to violations.
• Anomaly injection and transfer learning enable rapid, adaptable training for microservice-specific RL agents.