Treffer: In the Search for Truth: Navigating Variability in Neuroimaging Software Pipelines
Weitere Informationen
International audience ; Neuroimaging pipelines -software-driven analysis workflows of brain images -are characterized by a wide range of tools, parameters, and configuration choices. Such flexibility, while enabling diverse scientific inquiries, gives rise to analytical variability: different pipeline variants can lead to different outcomes. In practice, each neuroimaging pipeline variant produces a statistic map -a complex, structured 3D output whose relevance can only be assessed with specific domain expertise, unlike simple metrics such as execution time. And, in most cases, there is no ground truth against which to judge these outputs, making it unclear which variant yields the best result. In this paper, we introduce a "sampling, variant scoring, learning" methodology to study variability in the absence of a quantitative target -i.e. ground truth. We report our experience in developing an Universal Variability Language (UVL) feature model of 90 features representing the configuration space of a well-established open source neuroimaging analysis software (SPM). We sample and run 1000 valid configurations generating 1000 statistic maps as outputs. We studied various candidate proxy ground truths and computed Spearman correlations as a quantitative metric of the performance of each pipeline. We tested the following 12 proxy ground truths: the average statistic map (across variants), the output of an expertderived configuration, and a set of randomly selected outputs (as baseline). Then, we used a decision tree learning approach to inspect variability. We evaluated the sensitivity of our method to the choice of (proxy) ground truth, both in terms of predictive accuracy and in the identification of important features. These first results outline the challenge of choosing and validating a referential to assess our understanding of variability in the absence of ground truth.