Treffer: SAES: A Python Library for Statistical Evaluation of Stochastic Artificial Ingelligence Algorithms

SAES (Stochastic Algorithm Evaluation Suite) is a Python library for the rigorous statistical analysis and comparison of stochastic algorithms in artificial intelligence. Stochastic methods—such as metaheuristics and some machine learning techniques—are widely applied in optimization, learning, and simulation for their ability to explore complex solution spaces and avoid local optima. However, their inherent randomness complicates reliable evaluation and fair benchmarking. SAES addresses this challenge by offering a unified framework that combines robust statistical methods with intuitive visualizations. The library includes non-parametric tests (Friedman, Friedman aligned-rank, Quade, and Wilcoxon signed-rank), parametric tests (t-test and ANOVA), and appropriate post hoc analyses, such as the Nemenyi test. It simplifies experimental workflows by automating CSV data handling, statistical inference, customizable visualizations (e.g., boxplots, critical distance diagrams, and Bayesian posterior plots), and LaTeX report generation for reproducible research. Case studies demonstrate its effectiveness in benchmarking multi-objective metaheuristics and comparing machine learning models. By standardizing evaluation practices for stochastic AI algorithms, SAES promotes research reproducibility, enables fair comparisons, and provides an accessible platform for both researchers and practitioners.