WG211/M15Grebhahn

Performance-Influence Models: Prediction, Optimization, Debugging

Configuration options allow users to optimize the performance of configurable software systems for a specific use case. However, the flexibility gained from configuration gives rise to new challenges for optimization. For example, finding the performance-optimal configuration is a complex task, because users typically do not know the influence of all individual configuration options on performance and when and how options interact with each other. So, understanding the performance characteristics of a complex configurable system is often infeasible. To address this problem, we use a unique combination of machine-learning and sampling techniques, in particular, forward feature selection and linear regression, to learn models describing the influence of configuration options and their interactions on performance. Based on these models, we can predict performance of all variants of a configurable software system with a high accuracy and improve understanding of the system, for example, by using performance-influence models in interviews to validate the expectations of domain experts with the actual influences of the options. In a series of experiments, we analyzed the prediction accuracy of our approach for a number of subject systems from different domains, including configurable multigrid solvers (e.g., the Dune framework), the x264 video encoder, and the LLVM compiler. We are able to predict performance of all variants with an accuracy of more than 80% after measuring only a small number of all variants. Furthermore, we conducted interviews with domain experts on the usefulness and desired properties of performance-influence models.