Evaluating Reliability in Machine Learning Models for Early Chronic Kidney Disease Prediction: A Systematic Review of Data Leakage and Predictor Stability

The early detection of Chronic Kidney Disease using machine learning has attracted significant interest in healthcare-related computer science.

Despite rapid advancements in this field, many reported studies remain inconsistent and potentially misleading.

A significant drawback is the lack of organized evaluation regarding methodological concerns.

Key issues include data leakage, limited access to temporal patient records and inconsistency in reported clinical indicators.

This research offers a systematic literature review of existing CKD prediction studies using interpretable machine learning techniques, where nineteen relevant studies were selected via systematic searches across major academic databases.

To assess methodological reliability, this study introduces a structured taxonomy of information leakage and a quantitative leakage scoring framework to systematically evaluate reliability across CKD prediction studies.

The analysis reveals a strong relationship between leakage and inflated performance.

Here, High leakage-studies report an average accuracy of 95.48%, compared to 80.2% for leakage-free studies, reflecting an increase of approximately 15.28%.

Furthermore, a cross-study feature stability analysis shows that only a small subset of predictors is consistently reproducible, with over 80% lacking reliability.

Overall, the findings suggest that many reported performance improvements stem from methodological limitations rather than true predictive capability.