Treffer: Methodological insights on building and evaluating models for early warning of hypotension during surgery

Submitted to Anesthesiology. Copyright may be transferred without notice.
Submitted to Anesthesiology. Copyright may be transferred without notice. Background: Early warning of hypotension could help prevent severe complications during surgery. Several data-driven models have been proposed, but framing, data selection, and evaluation metrics differ widely in the literature. Methods: Using datasets from non-cardiac and cardiac surgery, we assess how data selection affects model performance. We compare models trained and tested with or without segments containing ongoing hypotension at prediction time or medical interventions, and evaluate both standard metrics and an operational metric proposed in recent studies. Results: Across both datasets, data selection strongly influenced performance under a sliding window framing, particularly during testing, where model rankings changed with the selection strategy. Excluding interventions and ongoing hypotension better reflected the intended clinical use-case. The operational metric highlighted differences more clearly than ROC or precision-recall curves. In training, removing intervention on training data improve performances while exclusion of ongoing hypotension do not change the performances. Conclusion: Data selection is critical when building and evaluating hypotension prediction models. We recommend training models on datasets excluding interventions, testing on datasets excluding interventions and ongoing hypotension, and evaluating with operational metrics and calibration curves.