Cross-Dataset CKD Prediction Delivers High Accuracy but Misses Too Many Non-CKD Cases
Chronic kidney disease remains a major clinical burden, and the appeal of machine learning in this setting is obvious: identify risk early, before symptoms harden into irreversible loss of function. In this PLOS ONE study, investigators built an explainable machine learning framework designed to predict chronic kidney disease across two public datasets with different distributions, a problem that often breaks models that look strong in one setting but collapse in another.
The work is centered on a practical problem rather than a theoretical one. Clinical data rarely arrive in balanced form, and CKD datasets are often dominated by positive cases, leaving the minority non-CKD class poorly represented. In this analysis, the target dataset contained 1,524 CKD samples and 135 non-CKD samples, an imbalance severe enough to steer conventional classifiers toward the majority label unless correction methods are used.
One sentence captures the core tension: the best-performing model was accurate, but it was not clinically balanced.
How the HXAI-ML Framework Was Built
The study used two public datasets: the UCI CKD Risk Factor Prediction dataset as the source set and a Kaggle CKD dataset as the target set. After preprocessing, label binarization, imputation, and standard scaling, the investigators tested seven balancing methods and six classifiers. They then applied a target-adaptation strategy in which 10% of the target dataset was used for calibration and the remaining 90% was reserved for blind testing.
This design matters because it reflects the central weakness of many predictive models in medicine: they are trained under one data distribution and then asked to perform in another. Cross-dataset transfer can expose instability in feature relationships, sampling patterns, and class structure. To address interpretability, the framework also incorporated SHAP, LIME, and Permutation Importance Analysis, giving clinicians a way to see which variables were driving model output instead of accepting a black-box prediction.
The selection criterion was accuracy-based, and among the evaluated adaptation configurations, RO + TL + ETC emerged as the leading combination. The abstract does not expand the full acronym set in the body excerpt provided, so the safest interpretation is limited to the reported label and its measured performance.
Accuracy Was Strong, But Specificity Collapsed
On the 1,494-sample blind holdout, the selected model achieved 91.97% accuracy, 92.24% precision, 99.64% recall, 95.80% F1-score, and 67.52% ROC-AUC. Those numbers can create an impression of a robust classifier at first glance, particularly because recall was extremely high, suggesting that most CKD cases were captured.
The rest of the metrics tell a more restrained story. Specificity was only 5.74%, and balanced accuracy reached 52.69%. In clinical terms, the model was far better at labeling CKD than at correctly identifying non-CKD cases. A classifier with that profile may look impressive on aggregate accuracy while still generating a large number of false positives for patients who do not have CKD.
This is the central limitation of imbalanced prediction work. Accuracy can be inflated when one class dominates, while balanced accuracy, specificity, and ROC-AUC reveal whether the model can separate both groups in a way that holds up in practice. Here, the gap between recall and specificity is large enough to raise caution about how the tool would behave in screening or triage workflows.
Interpretability Helps, But It Does Not Fix Weak Discrimination
The inclusion of SHAP, LIME, and permutation importance is clinically sensible. Explainability tools can show which features push a prediction toward CKD or away from it, helping clinicians interrogate model behavior and spot unstable decision rules. In kidney disease, where blood tests, filtration status, and broader patient history can interact in non-linear ways, transparent feature attribution is especially valuable.
Still, interpretability cannot compensate for poor class-wise discrimination. A model can be easy to explain and still be wrong in systematic ways. Here, the low specificity suggests that even a transparent explanation layer would sit on top of a classifier that overcalls disease in the non-CKD group. In bedside terms, that could translate into unnecessary follow-up, added testing, and avoidable anxiety if deployed without safeguards.
The pathophysiology of CKD makes early recognition important, but early recognition only helps if the signal is reliable. A tool that over-identifies CKD may increase sensitivity at the expense of practical utility, especially when the clinical question is not simply “Is disease present?” but “Can this patient be confidently sorted into the right branch of care?”
Why This Study Still Matters for Kidney-Risk Modeling
The value of the paper lies in its method rather than in a deployable clinical threshold. It shows that cross-dataset adaptation is feasible when target data are limited and that a hybrid explainable framework can be assembled from public datasets, balancing methods, and standard classifiers. It also exposes how fragile high headline performance can be when the underlying data are heavily skewed.
For CKD screening, the result is sobering. A model with 99.64% recall but 5.74% specificity is not ready to function as a standalone clinical decision-support system. It may still serve as a research platform for refining imbalance handling, improving transfer learning across institutions, and testing whether richer feature sets or larger target cohorts can lift the non-CKD signal without sacrificing sensitivity.
The authors explicitly frame the framework as exploratory rather than clinically deployable, and that boundary is appropriate. The study does not settle whether a cross-dataset explainable model can be trusted in real-world screening. It shows instead that the next obstacle is not merely better accuracy, but better separation between classes under the conditions where kidney risk models are most likely to be used: sparse target data, imbalanced labels, and clinical demand for explanations that hold up under scrutiny.
Technical Constraints Remain the Real Bottleneck
The article identifies four recurring problems in CKD machine learning: severe class imbalance, oversampling artifacts and intersecting samples, difficulty adapting a source-trained model to a new target distribution, and poor interpretability in black-box systems. Each of those issues showed up directly in the evaluation strategy.
In practical terms, hybrid balancing can help a model learn more from sparse minority examples, but it can also introduce noise near class boundaries. Target-domain calibration can improve transfer, but only if the target set is large enough and representative enough to anchor the adaptation. Explainable AI can make predictions intelligible, but it does not guarantee that the decision boundary itself is clinically sound.
What this study leaves open is whether a model with stronger specificity can be built without giving up the high sensitivity that makes screening attractive in the first place. The reported ROC-AUC of 67.52% suggests room for improvement in discriminative power, and the low balanced accuracy shows that future work will need to focus on class equilibrium rather than headline accuracy alone.