Treffer: An empirical investigation into the capabilities of anomaly detection approaches for test smell detection.

Test smells are symptoms of sub-optimal design choices adopted when developing test cases. Previous research has demonstrated their harmfulness for test code maintainability and effectiveness, showing their impact on test code quality. As such, the quality of test cases affected by test smells is likely to deviate significantly from the quality of test cases not affected by any smell and might be classified as anomalies. In this paper, we challenge this observation by experimenting with three anomaly detection approaches based on machine learning, cluster analysis, and statistics to understand their effectiveness for the detection of four test smells, i.e., Eager Test , Mystery Guest , Resource Optimism , and Test Redundancy on 66 open-source Java projects. In addition, we compare our results with state-of-the-art heuristic-based and machine learning-based baselines. Our ultimate goal is not to prove that anomaly detection methods are better than existing approaches, but to objectively assess their effectiveness in this domain. The key findings of the study show that the F-Measure of anomaly detectors never exceeds 47%, obtained in the Eager Test detection using the statistical approach, while the Recall is generally higher for the statistical and clustering approaches. Nevertheless, the anomaly detection approaches have a higher Recall than the heuristic and machine learning-based techniques for all test smells. The low F-Measure values we observed for anomaly detectors provide valuable insights into the current limitations of anomaly detection in this context. We conclude our study by elaborating on and discussing the reasons behind these negative results through qualitative investigations. Our analysis shows that the detection of test smells could depend on the approach exploited, suggesting the feasibility of developing a meta-approach. • Study on test smells detection using anomaly detection approaches. • A comparison with the heuristic-based approaches. • The study highlights that anomaly detection works better than heuristic-based approaches. • The test smell detection could depend on the approach exploited. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Systems & Software is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)