TY  - JOUR
T1  - Revealing the Reality: A Comprehensive Framework for Evaluating and Selecting Score-Based Predictive Models
A1  - Kenji Tanaka
A1  - Yui Kobayashi
A1  - Takeshi Nakamura
JF  - Journal of Medical Sciences and Interdisciplinary Research
JO  - J Med Sci Interdiscip Res
SN  - 3108-4826
Y1  - 2023
VL  - 3
IS  - 2
DO  - 10.51847/0slEf5eEe4
SP  - 159
EP  - 170
N2  - Choosing appropriate machine learning (ML) models in biomedical research typically depends on overall performance indicators. However, when these indicators yield very similar results across candidate models, determining the best option for practical clinical use becomes difficult. We developed an innovative combined approach that integrates visual examination of the Model Scoring Distribution Analysis (MSDA) with a newly introduced scoring measure (MSDscore). This technique was integrated into the Digital Phenomics platform via the MSDanalyser tool. It was evaluated by creating and assessing 27 predictive models for prognosis in breast, lung, and renal cancers. The method facilitated an in-depth review of the distributions of true positives, false positives, true negatives, and false negatives throughout the scoring range. This uncovered localized performance trends that standard global metrics often miss. While traditional metrics showed only slight differences among the models, the MSDA framework highlighted notable variations in behavior across specific score regions, enabling clearer differentiation. Combining this composite framework with conventional performance measures offers a valuable supplementary perspective, providing greater insight into model selection in clinical and biomedical contexts.
UR  - https://smerpub.com/article/revealing-the-reality-a-comprehensive-framework-for-evaluating-and-selecting-score-based-predictive-jj9rknencflsm0x
ER  -