NeoJaundice-AI: Smartphone-Based Neonatal Jaundice Detection
Using Dual-Input Deep Learning and Synthetic Augmentation
June 14, 2026
Abstract
Neonatal jaundice (hyperbilirubinemia) is one of the most common conditions affecting newborns worldwide, with India alone recording roughly 15 million cases per year. Early detection is life-saving, yet standard diagnosis requires blood tests that are impractical in rural clinics where equipment is scarce and laboratory access is limited. This paper presents NeoJaundice-AI, a smartphone-based screening system that uses photographs of a baby’s skin and eye whites to estimate jaundice severity and predict the serum bilirubin level — all in under three seconds, with no internet connection required.
The system is built around a dual-branch EfficientNet-B0 architecture that independently analyses a skin image and a sclera (eye-white) image, fuses the deep features with handcrafted YCbCr colour statistics, and simultaneously performs four-class severity classification and continuous bilirubin regression. A key innovation is a synthetic jaundice generator that simulates bilirubin-induced yellowing via controlled YCbCr channel shifts on normal neonatal skin images. This technique—extended from the authors’ prior work on synthetic augmentation [1]—specifically addresses data scarcity for severe cases and for darker Indian skin tones (Fitzpatrick Types IV–VI). Skin-tone normalisation, originally motivated by the authors’ infrared blood-group detection work [2], ensures consistent predictions across the wide range of Indian neonatal complexions.
The model achieves an overall accuracy of 91.8%, a clinical sensitivity of 93.5%, and a bilirubin Mean Absolute Error of 1.4 mg/dL. After INT8 quantisation to ONNX format, the model occupies only 8.3 MB and runs comfortably within the three-second target on commodity Android hardware. To the best of our knowledge, this is the first India-focused neonatal jaundice AI system that jointly addresses multi-modal fusion, skin-tone adaptation, and offline mobile deployment.
neonatal jaundice, hyperbilirubinemia, deep learning, EfficientNet-B0, smartphone diagnostics, bilirubin estimation, YCbCr colour space, synthetic data augmentation, Fitzpatrick skin tone, multi-task learning, rural healthcare India, ONNX mobile deployment
1 Introduction↩︎
1.1 The Clinical Problem↩︎
A healthy baby’s liver takes several days to start clearing bilirubin efficiently. During this window, bilirubin accumulates in the blood and stains the skin and eyes yellow—a condition clinicians call neonatal hyperbilirubinemia, more commonly known as jaundice. While mild jaundice resolves on its own, moderate-to-severe cases can cause permanent brain damage (kernicterus) or even death if left untreated [3].
The numbers in India are striking. With approximately 25 million births annually, and jaundice affecting around 60% of term newborns and 80% of preterm newborns [4], India sees roughly 15–20 million neonatal jaundice cases every year. An Indian study found significant jaundice requiring intervention in 13.4% of cases [4]. Yet the standard diagnostic tool—serum bilirubin (SB) measurement—requires a blood prick, laboratory equipment, and a trained technician. Transcutaneous bilirubinometers (TcB) are non-invasive but cost $2,000–$8,000 USD, placing them well beyond the budget of most rural primary health centres (PHCs) in India [5].
The result: many cases go undetected until symptoms are severe, particularly in rural areas where a newborn may never be seen by a physician.
1.2 Opportunity: The Smartphone in Every Clinic↩︎
India has over 750 million smartphone users [6]. Almost every rural health worker carries one. High-quality cameras are now standard, and the yellow colouration of jaundice is visually distinguishable in both skin and eye-white images—especially when the analysis is aided by machine learning that can see colour shifts invisible to the human eye.
The opportunity, therefore, is clear: a free app running on any Android phone could bring reliable jaundice screening to every corner of rural India.
1.3 Why Existing Solutions Fall Short↩︎
Despite growing research interest, current AI-based systems have three important weaknesses:
Single-modality input. Most approaches analyse either skin colour or sclera images, but not both. Yet scleral icterus (yellowing of the eye whites) is detectable earlier than skin yellowing, and is more visible on dark skin—making the two modalities complementary.
Skin-tone bias. Public datasets are dominated by light-skinned Caucasian infants. Models trained on such data perform poorly on the brown and dark complexions common among Indian newborns [7].
Data scarcity for severe cases. Severe jaundice is a medical emergency, meaning images are collected less frequently. This creates a severe class imbalance that causes models to underperform exactly when accuracy matters most.
1.4 Our Approach and Contributions↩︎
NeoJaundice-AI addresses all three weaknesses. The main contributions are:
Dual-input fusion model. A two-branch EfficientNet-B0 network processes skin and sclera images in parallel, combining their features for a richer representation of bilirubin accumulation.
India-specific skin-tone normalisation. A preprocessing module calibrated for Fitzpatrick Types III–VI ensures reliable predictions regardless of the baby’s complexion. This extends the normalisation methodology first developed in our blood-group detection work [2].
Synthetic jaundice progression generator. We simulate bilirubin-induced yellowing at four severity levels by shifting the Cb and Cr channels in YCbCr space. This directly extends the synthetic augmentation framework introduced in [1] to the medical imaging domain, and complements the EfficientNet-based non-invasive detection methodology of [8].
Multi-task learning. A single model simultaneously predicts the four-class severity label and estimates the bilirubin level in mg/dL.
Offline mobile deployment. INT8 quantisation reduces the model to 8.3 MB with sub-3-second CPU inference on Android devices, enabling fully offline operation.
2 Related Work↩︎
2.1 Clinical Background↩︎
Bhutani et al.[3] established the widely used hour-specific bilirubin nomogram that defines clinical thresholds for intervention. Their work forms the bedrock of jaundice management globally and is the basis for the age-aware thresholds in our app.
2.2 Smartphone-Based Systems↩︎
BiliScreen [9] was a landmark study showing that sclera photographs from a smartphone could estimate bilirubin with a Pearson correlation of 0.84 in adults. Taylor et al.[10] extended this to newborns, achieving 87% binary accuracy. Neither system addressed multi-class severity grading or Indian skin tones.
2.3 Deep Learning Approaches↩︎
Tayade and Patil [11] applied ResNet-50 to palm images, reaching 89% binary accuracy but with significant degradation on dark skin tones (71.6%). Namba et al.[12] showed strong correlation between RGB forehead values and SB, but used linear regression rather than deep learning.
2.4 Synthetic Augmentation↩︎
The author’s prior work [1] demonstrated that carefully designed synthetic image pipelines—without complex generative models—can substantially reduce overfitting under data-scarce conditions, improving test accuracy on CIFAR-10 with only 50 real images per class. We apply the same philosophy here: instead of GANs, we use physics-informed YCbCr channel manipulation to generate plausible jaundiced appearances. Frid-Adar et al.[13] used GANs for liver lesion augmentation; Baur et al.[14] for skin lesions. Our approach is simpler, faster, and interpretable.
2.5 Non-Invasive Screening via EfficientNet↩︎
The author’s AnemiaVision system [8] demonstrated that EfficientNet-B3 with TrivialAugmentWide and Mixup augmentation can reliably detect anaemia from conjunctival smartphone images, achieving strong performance while maintaining a small model footprint. NeoJaundice-AI follows the same design philosophy—lightweight EfficientNet backbone, smartphone images, non-invasive screening—but adapts it for neonatal jaundice with dual-modal input and continuous bilirubin regression.
2.6 Research Gap↩︎
No published system simultaneously addresses: dual-modality input (skin + sclera), India-specific skin-tone adaptation across Fitzpatrick Types I–VI, four-class severity grading (vs.binary), continuous bilirubin regression in mg/dL, and offline mobile deployment under 20 MB. NeoJaundice-AI fills all five gaps in a single unified system, achieving state-of-the-art accuracy (91.8%) while remaining deployable on a $50 Android phone with no laboratory equipment and no internet connection.
3 System Overview↩︎
Figure 1 shows the complete NeoJaundice-AI pipeline from image capture to clinical output.
4 Model Architecture↩︎
4.1 Backbone Selection↩︎
We evaluated four backbone candidates on the accuracy-versus-parameter-count trade-off (Fig. 2). EfficientNet-B0 [15] achieves the best ImageNet accuracy among lightweight models while requiring only 4.01 M parameters per branch. This is critical: our final ONNX-INT8 model must fit under 20 MB and run in under 3 seconds on a mid-range Android CPU.
4.2 Dual-Branch Design↩︎
Each branch is an EfficientNet-B0 with its original classifier replaced by a learnable projection layer, producing a 256-dimensional embedding. The two branch embeddings are concatenated with a 16-dimensional handcrafted YCbCr feature vector, giving a 528-dimensional fused vector. Two fully connected layers compress this to 128 dimensions, after which the network splits into two task heads (Table 1).
| Component | Input | Output | Params |
|---|---|---|---|
| Skin Branch (EffNet-B0) | \(224{\times}224{\times}3\) | 1280-d | 4.01 M |
| Sclera Branch (EffNet-B0) | \(224{\times}224{\times}3\) | 1280-d | 4.01 M |
| Projection \(\times\)2 | 1280-d | 256-d | 0.33 M |
| YCbCr Feature Vector | — | 16-d | — |
| Fusion MLP (2 layers) | 528-d | 128-d | 0.07 M |
| Classification Head | 128-d | 4-d | 0.5 K |
| Regression Head | 128-d | 1-d | 8.3 K |
| Total | 8.42 M |
4.3 Task Heads↩︎
Classification head: A linear layer followed by softmax outputs class probabilities over four severity levels: Normal (\(<\)5 mg/dL), Mild (5–12 mg/dL), Moderate (12–20 mg/dL), and Severe (\(>\)20 mg/dL). These thresholds follow the Bhutani nomogram [3].
Regression head: A two-layer MLP with Softplus activation outputs a non-negative bilirubin estimate in mg/dL. Softplus (\(\ln(1+e^x)\)) avoids the hard zero boundary of ReLU while still enforcing non-negativity.
5 Preprocessing & Feature Engineering↩︎
5.1 Skin-Tone Normalisation↩︎
Indian neonatal skin spans Fitzpatrick Types III–VI. Without correction, the same bilirubin concentration produces visually different yellowing depending on the underlying skin tone. Our normalisation pipeline applies three steps:
White balance using the grey-world assumption to remove clinic-room lighting colour casts.
Skin-tone classification via the Individual Typology Angle (ITA): \[\text{ITA} = \arctan\!\left(\frac{L^* - 50}{b^*}\right) \cdot \frac{180}{\pi} \label{eq:ita}\tag{1}\] where \(L^*\) and \(b^*\) are CIELab components.
Tone-adaptive channel scaling: a skin-tone-specific affine transform on the Cb and Cr channels so that a given bilirubin level maps to the same target colour regardless of baseline complexion.
5.2 YCbCr Features↩︎
Bilirubin selectively shifts the Cb (blue-difference) and Cr (red-difference) channels while leaving luminance (Y) relatively unchanged [12]. We extract six statistics per region (mean and std of Y, Cb, Cr) for both the skin and sclera region, plus two ratio features (Cb/Y and Cr–Cb) per region, yielding a 16-dimensional feature vector that is concatenated into the model’s fusion layer.
5.3 Region Detection↩︎
Region boundaries are detected fully offline using OpenCV:
Skin (forehead/chest): Haar cascade face detector \(\rightarrow\) upper-third bounding box crop.
Sclera: Eye-region detector \(\rightarrow\) colour segmentation to isolate white pixels, excluding iris and pupil.
If region quality falls below a confidence threshold (area \(<\)1000 px or local contrast \(<\)0.3), the app prompts the user to retake the photo.
6 Synthetic Data Generation↩︎
6.1 Motivation↩︎
Even with the Kaggle dataset (1,847 images), severe cases number only 83—far too few to train a reliable model for the most dangerous scenario. Our solution, extending the methodology from [1], is to generate labelled synthetic images by manipulating the colour channels of normal neonatal skin photographs.
6.2 The Algorithm↩︎
Figure 3 shows the Cb channel shifts at each severity level. The full procedure is summarised in Algorithm [alg:synth].
Normal neonatal image \(I\); severity \(s \in \{0,1,2,3\}\); ITA angle Synthetic jaundiced image \(I'\) with label \(s\) Convert \(I\) from BGR to YCbCr \(\Delta Cb \leftarrow [0, 8, 16, 22][s]\); \(\Delta Cr \leftarrow [0, 3, 6, 9][s]\) Compute spatial weight map \(W\) Scale shifts: \(\Delta Cb \leftarrow \Delta Cb \cdot (1 + \alpha_\text{ITA} \cdot \frac{90 - \text{ITA}}{90})\) \(Cb' \leftarrow \text{clip}(Cb - W \cdot \Delta Cb,\; 0, 255)\) \(Cr' \leftarrow \text{clip}(Cr + W \cdot \Delta Cr,\; 0, 255)\) Add texture noise \(\mathcal{N}(0, 0.5)\) to \(Cb', Cr'\) \(I' \leftarrow \text{YCbCr2BGR}([Y, Cb', Cr'])\) \(I'\)
The ITA-based scaling in Step 4 ensures that the visual magnitude of yellowing is perceptually consistent across skin tones, so the model learns bilirubin-level features rather than skin-tone features.
7 Training Methodology↩︎
7.1 Loss Function↩︎
We jointly optimise classification and regression with a weighted sum: \[\mathcal{L} = \lambda_c \,\mathcal{L}_\text{CE} + \lambda_r \,\mathcal{L}_\text{MSE}, \quad \lambda_c = 0.7,\;\lambda_r = 0.3 \label{eq:loss}\tag{2}\] where \(\mathcal{L}_\text{CE}\) is cross-entropy loss and \(\mathcal{L}_\text{MSE}\) is mean squared error on bilirubin estimates. The weights \(\lambda_c = 0.7, \lambda_r = 0.3\) were selected by grid search on the validation set.
7.2 Training Setup↩︎
Optimiser: Adam, \(\eta = 10^{-4}\), weight decay \(10^{-4}\).
Scheduler: ReduceLROnPlateau (patience = 5, factor = 0.5).
Epochs: 50 with early stopping at patience = 10.
Batch: 32 (paired skin + sclera).
Split: 70/15/15 train/val/test.
Class imbalance:
WeightedRandomSampler(inverse-frequency weights).Augmentation: horizontal flip, \(\pm15^\circ\) rotation, brightness/contrast jitter \(\pm\)0.2, Gaussian noise \(\sigma\!=\!0.01\).
7.3 Training Curves↩︎
Figure 4 and Fig. 5 show the loss and accuracy trajectories over 50 epochs. Convergence is smooth; the gap between training and validation curves remains small, confirming that synthetic augmentation effectively prevents overfitting—a finding consistent with [1].
8 Experiments & Results↩︎
8.1 Dataset↩︎
Table 2 describes the data composition. Original images came from the public Kaggle neonatal jaundice dataset [16] plus 240 images from PHCs under IRB approval. Synthetic images were generated using our algorithm. The severe class grew from 83 to 883 samples—a 10\(\times\) increase that is critical for clinical utility.
| Class | Threshold | Original | Synthetic | Total |
|---|---|---|---|---|
| Normal | \(<\)5 mg/dL | 612 | 400 | 1012 |
| Mild | 5–12 mg/dL | 754 | 900 | 1654 |
| Moderate | 12–20 mg/dL | 398 | 1100 | 1498 |
| Severe | \(>\)20 mg/dL | 83 | 800 | 883 |
| Total | 1847 | 3200 | 5047 |
8.2 Per-Class Metrics↩︎
| Class | Precision | Recall | F1 | \(n\) |
|---|---|---|---|---|
| Normal | 0.948 | 0.961 | 0.954 | 152 |
| Mild | 0.914 | 0.927 | 0.920 | 248 |
| Moderate | 0.903 | 0.893 | 0.898 | 225 |
| Severe | 0.871 | 0.842 | 0.856 | 133 |
| Macro Avg | 0.909 | 0.906 | 0.907 | 758 |
| Weighted Avg | 0.917 | 0.918 | 0.917 | 758 |
8.3 Clinical Metrics↩︎
From a clinical perspective, the most important numbers are sensitivity (the system should not miss jaundiced babies) and specificity (it should not over-refer healthy babies). Table 4 shows these metrics.
| Metric | Value |
|---|---|
| Overall Accuracy | 91.8% |
| Sensitivity (Jaundice vs Normal) | 93.5% |
| Specificity | 95.1% |
| Positive Predictive Value | 96.3% |
| Negative Predictive Value | 91.7% |
| AUC-ROC (macro) | 0.971 |
| Bilirubin MAE | 1.4 mg/dL |
| Bilirubin RMSE | 1.9 mg/dL |
A sensitivity of 93.5% means that for every 100 jaundiced babies, the system correctly flags about 93 of them—a clinically meaningful performance level for a free screening tool.
8.4 Confusion Matrix↩︎
Figure 6 visualises prediction patterns on the test set. Most errors occur between adjacent severity classes (e.g.Mild predicted as Moderate), which is clinically less harmful than confusing Normal with Severe.
8.5 ROC Curves↩︎
8.6 Regression Performance↩︎
8.7 Ablation Study↩︎
Table 5 and Fig. 9 together tell a clear story: synthetic augmentation helps everyone, but it helps dark-skinned babies the most—exactly the population most underrepresented in the original dataset.
| Configuration | Overall | Dark Skin | Severe |
|---|---|---|---|
| Without synthetic data | 81.2% | 72.3% | 65.2% |
| With synthetic (ours) | 91.8% | 87.6% | 83.4% |
| Improvement \(\Delta\) | +10.6% | +15.3% | +18.2% |
8.8 Skin-Tone Analysis↩︎
8.9 Comparison with Prior Work↩︎
| Method | Modality | Classes | Acc. | Sens. | Dark Skin | Bil.Est. | Offline | India |
|---|---|---|---|---|---|---|---|---|
| BiliScreen [9] | Sclera only | 2 | 82.1% | 84.3% | — | |||
| Taylor et al.[10] | Skin only | 2 | 87.0% | 89.1% | — | |||
| Tayade et al.[11] | Skin only | 2 | 89.3% | 91.0% | 71.6% | |||
| NeoJaundice-AI (ours) | Skin+Sclera | 4+Reg. | 91.8% | 93.5% | 87.6% | ✔ | ✔ | ✔ |
| Bil.Est.= Continuous bilirubin regression (mg/dL). Offline = no internet required. India = India skin-tone adapted. | ||||||||
NeoJaundice-AI surpasses all prior systems on overall accuracy (91.8% vs.% best prior) and sensitivity (93.5% vs.%), while simultaneously solving three problems that no existing system has addressed:
Dual-modality: combining skin and sclera gives richer bilirubin signals than either alone—scleral icterus is detectable earlier and remains visible on dark skin.
India-specific skin-tone adaptation: dark-skin accuracy of 87.6% vs.% for the best competing system—a +16% gain on the population that needs it most.
Continuous bilirubin estimation: no prior smartphone system provides a numerical mg/dL estimate. Ours achieves MAE of 1.4 mg/dL, enabling quantitative clinical tracking.
Offline + India-ready: all prior systems require internet or hospital-grade hardware. NeoJaundice-AI runs fully offline on any Android phone.
The accuracy advantage exists because of our technical innovations, not despite them. Multi-task learning (jointly training classification and regression) acts as a regulariser that improves generalisation. Dual-modal fusion gives the model complementary evidence. Synthetic augmentation fills the severe-case data gap that limits all competing systems.
9 Mobile Deployment↩︎
9.1 Model Compression↩︎
Table 7 shows the progressive size and latency reduction from PyTorch FP32 through ONNX INT8 quantisation. The final model is 8.3 MB—less than half the 20 MB target—and runs in 2.1 seconds at the 95th percentile on a mid-range Android CPU.
| Format | Size | p50 Latency | p95 Latency |
|---|---|---|---|
| PyTorch FP32 | 32.1 MB | 4.2 s | 6.8 s |
| ONNX FP32 | 31.8 MB | 2.9 s | 4.5 s |
| ONNX INT8 | 8.3 MB | 1.4 s | 2.1 s |
| Target | \(<\)20 MB | — | \(<\)3 s |
9.2 Web Application↩︎
The app was designed from first principles for low-resource deployment:
Fully offline: inference, region detection, and SQLite patient records all run locally. No internet connection needed.
Age-aware thresholds: results are compared against the Bhutani hour-specific nomogram [3] to determine the appropriate action level.
Risk meter: a colour-coded bar (green \(\to\) amber \(\to\) red) gives an immediate visual indication of urgency.
GradCAM overlay: highlights skin/sclera regions driving the prediction, giving the clinician visual evidence.
Sync queue: records created offline are queued and synchronised when WiFi becomes available.
Hindi localisation: UI labels are available in Hindi for rural workers.
PDF export: one-tap PDF report for the referring physician.
10 Clinical Considerations↩︎
NeoJaundice-AI is a screening tool only. All positive screen results must be confirmed with serum bilirubin measurement before clinical action. The system should not replace professional clinical judgement or established treatment protocols.
Prior to deployment in real clinical settings, the following validation steps are required: enrolment of newborns with concurrent serum bilirubin measurements; stratification by age in hours, gestational age, birth weight, sex, lighting condition, and smartphone model; independent evaluation of sensitivity and specificity for clinically significant jaundice; analysis of false-negative rates for severe cases; ethics committee approval; and data privacy compliance under applicable Indian health-data regulations.
11 Conclusion↩︎
We presented NeoJaundice-AI, a smartphone-based non-invasive screening system designed for rural Indian healthcare. Motivated by the enormous unmet need—15 million neonatal jaundice cases per year in a country where most rural clinics lack a bilirubinometer—we built a system that is accurate, lightweight, offline-capable, and sensitive to the specific skin-tone diversity of Indian newborns.
Our three core technical contributions—dual-branch fusion of skin and sclera images, India-specific skin-tone normalisation, and synthetic YCbCr-space jaundice generation—work together to achieve 91.8% overall accuracy, 93.5% clinical sensitivity, and 1.4 mg/dL bilirubin estimation error. The synthetic augmentation contribution directly extends the augmentation methodology from our prior work [1], the non-invasive health-screening approach from [8], and the skin normalisation ideas from [2] — forming a coherent research programme in affordable AI-powered medical screening for resource-constrained Indian healthcare settings.
Future work includes prospective clinical validation across multiple Indian states, integration of a learnable sclera-segmentation module to replace heuristic region detection, and expansion of the UI to additional regional languages.