¹

1 Introduction↩︎

Chronic kidney disease (CKD) is a progressive condition characterized by sustained loss of kidney function or persistent kidney damage. If undetected, it can lead to end-stage renal disease requiring costly and lifelong treatment [1]. Globally, CKD affects approximately 850 million individuals, with a disproportionate burden in low- and middle-income countries, where access to early diagnosis and preventive care remains limited [2]. In South Asia, particularly in Bangladesh, community-based studies report high CKD prevalence and low awareness in rural and peri-urban populations [3], [4]. Early-stage CKD (stages 1–3) is often asymptomatic, yet timely detection can significantly slow disease progression [1].

Standard CKD screening relies on laboratory markers such as serum creatinine, estimated glomerular filtration rate (eGFR), and urine albumin-to-creatinine ratio, which require well-equipped facilities and trained personnel [1]. In many parts of South Asia, particularly rural and peri-urban areas, limited diagnostic infrastructure and financial constraints restrict access to these tests [2]. While conventional machine learning (ML) models have shown promising performance for CKD detection, they depend on large labeled datasets, predefined feature spaces, and retraining for new populations [5]. Such requirements reduce adaptability in heterogeneous, real-world screening contexts.

Recent advances in large language models (LLMs) have shown strong zero-shot generalization across diverse classification and reasoning tasks [6]. In nephrology, LLMs have been explored for various applications [7], including patient education and personalized dietary advice [8], diagnostic support [9], and medication management [5]. However, their use for systematic CKD screening – particularly in zero-shot settings and based on non-diagnostic or readily accessible community-level features – remains largely unexplored.

Motivated by this gap, we investigate whether LLMs can achieve clinically meaningful zero-shot CKD detection without any dataset-specific training or finetuning. This study makes three primary contributions: (i) we evaluate the zero-shot screening performance of four state-of-the-art LLMs (LLaMA-3, Qwen-3, Mistral, and GPT-4o-mini); (ii) we systematically compare the use of comprehensive feature sets with a compact subset of clinically meaningful variables, examining whether feature reduction improves or maintains screening performance; (iii) we assess generalizability by evaluating the approach on a community-based population and validating it on two additional datasets from different countries. Although the datasets do not share identical feature spaces, a harmonized subset of selected features was available across datasets. Across models and datasets, the selected feature subset achieved comparable or improved performance relative to using all available features. These findings suggest that meaningful feature selection can enhance zero-shot LLM-based CKD screening, which is critical for resource-constrained settings, and demonstrate cross-population robustness.

2 Methods↩︎

Fig. 1 provides an overview of the proposed zero-shot LLM framework for CKD screening. The workflow consists of three main steps: (i) feature harmonization and selection, (ii) prompt construction, and (iii) inference using large language models.

2.1 Datasets↩︎

We evaluated the proposed zero-shot LLM framework using three heterogeneous CKD datasets, summarized in Table ¿tbl:table:datasets?, representing both community-based (Dataset-1) and hospital-based populations (Dataset-2 and Dataset-3) across three countries: Bangladesh, India, and the UAE. To align with our research objective, gold-standard pathology tests used for direct CKD diagnosis, such as serum creatinine, urinary albumin, urinary creatinine, eGFR, and uACR, were excluded from all datasets. These markers are direct clinical identifiers of CKD. Therefore, including them would trivialize the prediction task. Moreover, such pathology tests are costly and not always accessible in low-resource or community settings. By excluding these features, we assess whether LLMs can detect CKD using more accessible and non-gold-standard clinical information.

Dataset-1 [4] is a community-based CKD survey conducted in the Mirzapur, sub-district of Tangail, Bangladesh. This dataset was collected through age-stratified random sampling of adults aged 18 years and above under the Mirzapur demographic surveillance system. CKD was diagnosed based on reduced kidney function (eGFR) and/or elevated urine albumin-to-creatinine ratio (uACR) according to standard clinical guidelines. After removing incomplete records, the final dataset included 284 participants, consisting of 112 early-stage CKD cases (stages 1–3) and 172 non-CKD individuals. The dataset contains 24 variables covering socio-demographic (SD), lifestyle habits (LH), medical history (MH), clinical examination (CE), and pathology test (Path) features.

Dataset-2 [10] is the UCI 2015 CKD dataset, collected from a hospital in India and widely used as a benchmark for CKD detection research. It contains clinical records from 400 patients admitted over two months, including 250 CKD cases and 150 non-CKD controls. Following the removal of gold-standard pathology markers, 23 variables were retained across socio-demographic, lifestyle, medical history, clinical examination, and non-gold-standard pathology features. CKD stages were not reported in this dataset.

Dataset-3 [11] is the UAE CKD dataset derived from hospital records at Tawam Hospital in Al-Ain City, Abu Dhabi, UAE. It includes 56 patients diagnosed with moderate-to-advanced CKD (stages 3–5). CKD, with stages 1–2, was not clearly separated from healthy individuals in the original data; therefore, only confirmed CKD cases were used in our analysis. After excluding gold-standard tests and redundant features, 14 features were retained across demographic, lifestyle, medical history, clinical, and pathology.

2.2 Features Harmonization and Selection↩︎

To ensure consistency across datasets, we performed systematic feature harmonization before model evaluation. Variables representing the same clinical concepts were harmonized across three datasets by standardizing both feature names and value encodings into a unified, semantically interpretable format to facilitate zero-shot LLM inference. This harmonization enabled consistent prompt construction and comparable evaluation across datasets. The complete list of harmonized features considered in this study is presented in Table 1.

Feature selection was informed by our prior machine learning analysis on the community-based Dataset-1 [12], which includes diverse and accessible variables spanning socio-demographic characteristics, lifestyle factors, medical history, clinical examination findings, and selected pathology measures. In [12], ten complementary feature selection methods (including recursive feature elimination with multiple ML estimators) were applied to identify a compact subset of clinically meaningful and readily obtainable variables most predictive of early-stage CKD. The selected features are underlined in Table 1. The selected features from Dataset-1 were subsequently mapped to Dataset-2 and Dataset-3 by identifying variables with equivalent clinical interpretation. As the datasets did not share identical feature spaces, only variables matching the selected features in Dataset-1 were retained as selected features in Datasets 2 and 3.

2.3 Prompt Construction↩︎

We represent each patient record (tabular instance) by serializing feature-value pairs into a structured list format (Appendix 6). This design follows prior findings demonstrating that list-based serialization preserves feature boundaries, reduces ambiguity, and performs comparably to free-text or sentence-style representations when applying LLMs to tabular inference [13]. The serialized representation enables the model to reason over individual attributes while preserving the original tabular semantics.

We adopt an instruction-style, role-conditioned prompting strategy (chat-style for GPT-4o-mini) in which each serialized record is embedded within a structured instruction–input–output format [14]. The prompt (Appendix 6) explicitly specifies the task objective, assigns the model a functional role, defines the expected binary output space, and presents the serialized input record. This design is consistent with instruction-tuning paradigms used in modern LLM training, which have been shown to enhance zero-shot generalization [15]. Unlike native chat-style templates, instruction-style prompts avoid reliance on model-specific control tokens, improving robustness and cross-model generalizability. Furthermore, explicit output constraints reduce generation variability and support more deterministic predictions, which is advantageous for zero-shot binary classification of structured clinical data.

The prompt text was formulated by adapting instruction templates aligned with model instruction-tuning paradigms and iteratively refined through empirical validation, consistent with prior work on instruction-based prompt optimization in in-context learning settings [16].

2.4 Inference using LLMs↩︎

We evaluated four large language models, LLaMA-3 8B, Qwen-3 8B, Mistral 7B, and GPT-4o-mini, selected to represent architectural diversity, variation in parameter scale, and differences in training paradigms (open-weight vs. proprietary). These models have been pretrained on large-scale, heterogeneous corpora that include biomedical and clinical knowledge, making them suitable for zero-shot reasoning in healthcare-related tasks such as CKD screening. Additionally, all selected models expose token-level log-probabilities, enabling probabilistic classification based on likelihood comparisons over predefined label tokens—an important capability not consistently available in many LLMs.

Inference was conducted in a zero-shot setting without task-specific training or parameter updates. For each patient record, the constructed instruction-style prompt was provided to the model, and binary predictions (CKD vs. non-CKD) were derived using log-probability scores over the constrained output space. Deterministic decoding was applied where supported to ensure reproducibility. Each record was processed independently, and no in-context examples were provided.

3 Results↩︎

Fig. 2 compares the use of all available features versus the selected feature subset across LLMs and datasets using balanced accuracy, with statistical significance assessed using a paired permutation test on per-sample Brier loss to provide a distribution-free, paired comparison of probabilistic predictions. Table 2 reports the detailed zero-shot performance metrics, including balanced accuracy, macro AUROC, macro F1-score, and Brier score, along with 95% confidence intervals estimated using nonparametric bootstrap resampling at the sample level using percentile intervals to quantify metric uncertainty.

For Dataset-1, the selected feature set consistently yields higher balanced accuracy across all models, with statistically significant improvements (\(p<0.001\)). The largest gains are observed for Qwen-3 and Mistral, where balanced accuracy increases from 0.62 to 0.78 and from 0.56 to 0.77, respectively. GPT-4o-mini improves from 0.70 to 0.77, while LLaMA-3 shows a modest but significant increase from 0.50 to 0.54. These improvements are accompanied by higher macro AUROC and macro F1-scores and lower Brier loss, indicating enhanced discrimination and calibration. For example, Qwen-3’s AUROC increases from 0.735 to 0.836, macro F1-score from 55.90% to 78.63%, and Brier loss decreases from 0.253 to 0.161. The bootstrap-derived 95% confidence intervals remain well separated in settings with large performance gains, reinforcing the statistical stability of the observed improvements. Similar patterns are observed across the other models, suggesting that restricting inputs to clinically selected features improves both predictive performance and probabilistic reliability.

For Dataset-2, selected features again yield improved or comparable balanced accuracy across all models, with statistically significant gains observed in several cases. The most substantial improvement is seen for Mistral, where balanced accuracy increases from 0.53 to 0.81 (\(p<0.001\)). LLaMA-3 also demonstrates a marked improvement from 0.50 to 0.76. Qwen-3 maintains strong performance under both feature configurations, indicating robustness to feature redundancy. While GPT-4o-mini shows a slight increase, that difference is not statistically significant. These performance patterns are consistent with changes in AUROC, F1-score, and Brier loss. For example, Mistral’s AUROC increases from 0.822 to 0.932, and its Brier loss decreases from 0.341 to 0.179, indicating improved discrimination and probabilistic calibration when using selected features.

For Dataset-3, a different pattern is observed, likely reflecting its focus on moderate-to-advanced CKD cases. LLaMA-3 yields a balanced accuracy of 0 under both feature configurations, indicating an inability to identify CKD cases, although the difference between all and selected features remains statistically significant in terms of Brier loss. The most notable improvement is observed for Qwen-3, where balanced accuracy increases from 0.93 to 0.97 with statistical significance (\(p<0.001\)). In contrast, Mistral and GPT-4o-mini show slight decreases when using selected features, from 1.00 to 0.96 and from 0.96 to 0.95, respectively; however, overall performance remains high. These patterns are also reflected in AUROC and Brier scores. For example, Qwen-3’s Brier loss decreases from 0.084 to 0.036, indicating improved probabilistic calibration in selected features, while Mistral and GPT-4o-mini maintain strong discrimination despite marginal reductions in balanced accuracy.

Overall, the results consistently indicate that the selected feature set improves or maintains balanced accuracy, AUROC, F1-score, and probabilistic calibration across datasets, despite differences in feature space composition. This consistency across heterogeneous cohorts supports the robustness of the approach and suggests that zero-shot LLM-based CKD screening can generalize across populations using a compact, clinically harmonized set of features.

4 Discussions↩︎

The most pronounced gains were observed in early-stage community screening (Dataset-1), where reducing redundant or less informative inputs substantially improved predictive stability and calibration. This finding suggests that in settings where disease signals are subtle, compact, and clinically meaningful features may enhance zero-shot reasoning by reducing noise and guiding the model toward salient risk factors. In hospital-based cohorts (Dataset-2), feature selection yielded particularly great improvements for smaller or open-weight models, whereas higher-capacity models such as GPT-4o-mini demonstrated relative robustness to feature redundancy. In the advanced-stage CKD cohort (Dataset-3), performance remained high under both feature configurations, indicating that feature reduction does not compromise discrimination when disease signals are more pronounced.

Our previous supervised ML study using the same selected feature subset reported a best balanced accuracy of 90.4% on Dataset-1 using 10-fold cross-validation and 79.33% when trained on Dataset-1 and externally tested on Dataset-2 [12]. In comparison, the zero-shot LLM approach achieved up to 78.32% on Dataset-1 (Qwen-3) and 88% on Dataset-2 (GPT-4o-mini) without any dataset-specific training, demonstrating competitive performance with strong practical applicability.

Collectively, these results suggest that feature-guided prompting enhances the reliability and stability of zero-shot LLM-based screening, particularly in early-stage and resource-constrained settings. While larger models appear less sensitive to redundant inputs, a compact and clinically grounded feature set provides clearer and more consistent performance gains across diverse model architectures without sacrificing predictive quality.

Several limitations should be acknowledged. First, the datasets are moderate in size and limited to three geographic regions, which may restrict generalizability. Second, zero-shot prompting relies on prompt design and instruction tuning characteristics, which may vary across future model releases. Third, Dataset-3 contains only CKD-positive cases, limiting full binary evaluation in that context.

5 Conclusions and Future Work↩︎

This study demonstrates that LLMs can support clinically meaningful zero-shot CKD screening using a compact, harmonized feature set without dataset-specific training or retraining. Across heterogeneous community- and hospital-based cohorts, restricting inputs to clinically selected features consistently improved or preserved accuracy, discrimination, and probabilistic calibration.

Future work should evaluate larger and more diverse cohorts, explore structured uncertainty estimation for screening thresholds, and investigate few-shot or retrieval-augmented extensions to further enhance robustness. Integration into community screening workflows and prospective validation in real-world settings will be critical to assess clinical impact.

6 Zero-Shot Prompt Template↩︎

References↩︎