Virtual Machine Allocation with Lifetime Predictions - Barbalho, Hugo; Kovaleski, Patricia; Li, Beibin; Marshall, Luke; Molinaro, Marco; Pan, Abhisek; Cortez, Eli; Leao, Matheus; Patwari, Harsh; Tang, Zuzu; Santos, Tamires; Gonçalves, Larissa Rozales; et al. (MLSys, 2023)
The emergence of machine learning technology has motivated the use of ML-based predictors in computer systems to improve their efficiency and robustness. However, there are still numerous algorithmic and systems challenges in effectively utilizing ML models in large-scale resource management services that require high throughput and response latency of milliseconds. In this paper, we describe the design and implementation of a VM allocation service that uses ML predictions of the VM lifetime to improve packing efficiencies. We design lifetime-aware placement algorithms that are provably robust to prediction errors and demonstrate their merits in extensive real-trace simulations. We significantly upgraded the VM allocation infrastructure of Microsoft Azure to support such algorithms that require ML inference in the critical path. A robust version of our algorithms has been recently deployed in production and obtains efficiency improvements expected from simulations.