1 Introduction↩︎

The DHCD dataset [1] is the standard benchmark for 46-class handwritten Devanagari character recognition, and a widely used reference point for Indic script recognition more broadly; its samples were collected from school students. Top-1 accuracies have crowded into a narrow band above 99.7 %: MallaNet [2] reaches 99.71 % with a 17.32 M-parameter convolutional architecture; Mishra et al. [3] independently report 99.72 % with a similarly large model. At this level successive gains are measured in hundredths of a percent, yet models still carry parameter budgets far beyond what an on-device recognizer can sustain.

Existing DHCD papers have not applied the scrutiny that 99%-range results demand. Point accuracy dominates; when McNemar testing appears at all, it is applied selectively against weaker baselines without exact \(p\)-values. Especially near 99.7 %, with \(n{=}13{,}800\) test samples, raw accuracy rankings carry little inferential weight.

The decisive issue is paired. Do the two models fail on different examples? The exact McNemar test across all student-teacher pairs answers directly, with Wilson confidence interval bounding the accuracy of each model. Once this discipline is applied, the leaderboard will look different.

A 1.11 M-parameter student reaches statistical parity with the prior state-of-the-art [2] (McNemar \(p=0.345\)). That holds even without knowledge distillation. Neither the distillation target (TTA-softened versus clean) nor the ensemble size (9 versus 15 teachers) makes a statistically significant difference. Every configuration tested converges to the same 11-error intrinsic floor; progress now requires a harder evaluation set or substantially different modelling assumptions.

Beyond accuracy numbers, a compact convolution network trained on one recognition task transfers. The pipeline can be reproduced for other character recognition problems.

Contributions.

• CDN—1.11 M parameters—achieves 99.73 % on DHCD, the highest reported on the standard 46-class split, at 15.6\(\times\) smaller footprint than prior state-of-the-art. McNemar (\(p=0.345\)) confirms parity even without distillation.

• DHCD is saturated. Exact McNemar tests and Wilson CIs applied across all configurations reveal a shared 11-error intrinsic floor that no configuration escapes.

• The ablations settle which choices matter: distillation target cleanliness, ensemble size (9 vs. teachers), and TTA each produce no statistically significant gain once noise is properly accounted for.

• At 15.6\(\times\) smaller and 9.5\(\times\) faster on CPU than prior state-of-the-art, CDN transfers to CMATERdb digits (zero-shot 76.6%; fine-tuned 97.8%) and degrades far less under corruptions (mCA 75.7% vs.38.7%).

• Full reproduction materials are at https://anonymous.4open.science/r/cdn-dhcd.

2 Related Work↩︎

2.0.0.1 Indic and Devanagari handwritten character recognition.

The DHCD dataset [1] has been the standard 46-class benchmark since 2015. MallaNet [2] couples residual branch merging with homogeneous filter capsule (HFC) layers to reach 99.71 % at 17.32 M parameters; Mishra et al. [3] independently report 99.72 % with a comparably large architecture; for practical purposes, indistinguishable. A concurrent preprint puts 99.80 % on the 10-class digit subset [4]; digit-only results do not transfer to the 46-class task.

2.0.0.2 Knowledge distillation and compact model design.

Hinton et al. [5] showed that soft probability outputs (“dark knowledge”) transfer accuracy from a large teacher to a smaller student at substantially lower capacity. He et al. [6] introduced residual identity mappings; Hu et al. [7] added squeeze-and-excitation channel re-weighting. Both are now routine components in compact architectures that trade parameter count for retained discriminative power. No prior work has applied knowledge distillation to the 46-class DHCD task. Stress-testing distillation on a saturated benchmark, where the performance gap sits inside sampling variation, is likewise unexplored.

2.0.0.3 Statistical significance and benchmark saturation.

Raw accuracy rankings on DHCD can flip between runs with no change in model quality, once the best models err on fewer than 0.3 % of examples. McNemar’s test [8] handles this: by conditioning on disagreement examples, it detects genuine error-pattern shifts without inflating sensitivity from the large pool of shared correct predictions. Wilson score intervals [9] bound single-model accuracy; [10] links confidence to empirical accuracy, which matters when soft distillation targets are being interpreted. [2] applies McNemar against four weaker baselines but omits its nearest competitor (Mishra et al., 99.72 %) and reports no exact \(p\)-values. CDN closes this gap by reporting exact McNemar \(p\)-values and Wilson CIs across every comparison and showing the benchmark is saturated.

3 Method↩︎

3.1 Compact Student Architecture↩︎

CDN is a pre-activation squeeze-and-excitation residual network [6], [7] designed for single-channel \(32{\times}32\) images. Channel widths run \((40, 80, 160)\) across three stages of depth \((2, 2, 2)\); Stages 2 and 3 downsample by stride 2 (Figure 1). The total trainable parameter count is 1,109,116.

3.2 Teacher Ensemble↩︎

The ensemble consists of 15 teachers drawn from three architectural configurations intersected with five independent random seeds (seeds 0 to 4). Two of those configurations share widths \((96, 192, 384)\) and depths \((3, 3, 3)\) but differ in augmentation intensity (medium vs.heavy; see Section 3.4); the third uses widths \((64, 128, 256)\) with depths \((4, 4, 4)\) under medium augmentation. The resulting parameter counts are 5.97 M and 9.89 M per teacher. The spread across configurations is intentional to discourage mode collapse in the soft targets. All teachers share the same CDN codebase (DevNet class) and are trained without distillation. Their logits are dumped once into a single NumPy archive reused across all student runs.

3.3 Ensemble Knowledge Distillation↩︎

We follow the knowledge distillation framework of Hinton et al. [5]. The ensemble target for each training image is the mean softmax of nine teacher checkpoints (configs A, B, C \(\times\) seeds 0–2), where each teacher’s softmax is computed with horizontal-flip test-time averaging over the training image. The distillation objective combines a temperature-softened KL divergence and a hard cross-entropy term: \[\mathcal{L} = \alpha \cdot T^{2}\, \mathrm{KL}\!\left( \sigma\!\left(\tfrac{z_{t}}{T}\right) \,\Big\|\, \sigma\!\left(\tfrac{z_{s}}{T}\right) \right) + (1-\alpha)\cdot\mathrm{CE}(z_{s}, y), \label{eq:kdloss}\tag{1}\] where \(z_{s}\) and \(z_{t}\) are student and (mean) teacher logits, \(\sigma\) denotes the softmax, \(T = 4.0\) is the temperature, and \(\alpha = 0.7\) weights the soft target. Label smoothing (\(\varepsilon = 0.1\)) is applied to the hard cross-entropy term. KL is computed in the teacher-to-student direction. The choice of target-generation scheme (clean no-TTA vs.flip-averaged) and ensemble size (9 vs. teachers) are examined in Section 4.5.

3.4 Script-Aware Training↩︎

Devanagari Script is not horizontally symmetric. Reflecting glyphs produces invalid characters. Horizontal flips are completely excluded. Instead, the training transformation applies affine perturbations (rotation, translation, scale, shear) followed by elastic deformations. The strength is controlled by an augmentation tier (light/medium/heavy) selected per configuration. Random erasing (\(p = 0.25\), area 2–10%) rounds out the per-image regularisation.

The mixing extensions, mixup [11] (\(\alpha = 0.2\)) and CutMix [12] (\(\alpha = 1.0\)), are selected with equal probability and each is activated for each batch with an overall probability of 0.5. Both are turned off while distillation is running. The KD loss (formula 1 ) always shows a clean, unenhanced teacher prediction, because the per-image teacher logits cannot be linearly combined without compromising the soft target distribution.

We optimise with AdamW using a cosine learning-rate schedule preceded by a 5-epoch linear warm-up. A weight exponential moving average (EMA, decay 0.999) tracks a shadow copy; validation and the final checkpoint use the EMA weights.

3.5 Evaluation Protocol↩︎

Reported accuracies come from a single forward pass over the DHCD held-out test split; no test-time augmentation is applied. Model selection relies on a 10 % stratified validation set carved from the training split before augmentation.

For statistical comparisons between CDN and the baseline [2], we use the exact two-sided McNemar test [8] on per-sample binary correctness indicators. Confidence intervals for accuracy and error rate are Wilson score intervals at 95% [9]. ECE [10] is computed using 15 equal width bins on the held-out test set.

4 Experiments↩︎

4.1 Experimental Setup↩︎

4.1.0.1 Dataset.

We use DHCD [1], the standard 46-class Devanagari handwriting benchmark (36 consonants, 10 digits): 78,200 training images and 13,800 test images, each \(32{\times}32\) grayscale.

4.1.0.2 Reproducing the baseline.

To enable sample-level McNemar comparisons, we reimplemented the architecture of [2] from the published description, including exact normalization (\(\text{mean}{=}0.5,\;\text{std}{=}0.5\)) and the class-index permutation implied by the data loader. Seed 0 produces 40 errors (99.7101%), which matches the reported number.

4.2 Main Parity Result↩︎

Five seeds per configuration; Table 1.

A 1.11 M-parameter student sits inside the noise floor of a 17.32 M-parameter baseline. The distilled, TTA-targets variant (9 teacher checkpoints, flip-averaged soft labels; see §3.3) reaches 36.6\(\pm\)​2.1 errors (99.735%), with McNemar \(p = 0.345\): no detectable gap at \(\alpha = 0.05\). Best seed: 34 errors (99.754%), Wilson 95% CI [99.656%, 99.824%], at 15.6\(\times\) fewer parameters.

Clean single-pass logits from a 15-teacher pool give \(p = 1.000\); the supervised control, trained with no teacher signal, ties at \(p = 0.701\). Distillation is not the source of parity.

4.3 Comparison with Prior Work↩︎

0.07 % separates the top four, well within sampling noise on a 13,800-image test set. No other entry simultaneously clears 99.5 % accuracy and stays below the 2 M parameter mark. The next lightest model above that threshold is MallaNet at 17.32 M parameters [2], a 15.6\(\times\) gap.

4.4 Efficiency↩︎

This is 9.5\(\times\) lower CPU latency and 14.8\(\times\) fewer MACs, with no quantization or pruning and no loss in accuracy.

4.5 Distillation and Ensemble Ablation↩︎

4.5.0.1 Effect of knowledge distillation and target cleanliness.

Table 1 separates two questions: whether the student benefits from a teacher at all, and whether the way we generate teacher targets matters. In the primary configuration, distilled (TTA) targets, the student learns from soft labels generated by averaging each teacher’s predictions over a horizontal-flip pair of each training image (see Section 3.3); it ends at 36.6\(\pm\)​2.1 errors. With distilled (clean) targets, the soft labels come from a single clean forward pass through a larger 15-teacher pool with no flip averaging, and the student reaches 38.2\(\pm\)​2.2 errors. The supervised control, trained with no teacher signal at all, obtains 39.8\(\pm\)​2.8 errors.

Three errors separate the best distilled student and the supervised baseline – moderate, and none of the three McNemar \(p\) values approach \(0.05\) (Table 1). Distillation helps, but only a little. What’s more, is how unimportant the target recipe is. Students on the clean target and those on the TTA target are statistically indistinguishable (38.2 vs. 36.6, with substantial seed overlap in the error distributions). A flipped-average target cannot beat a clean single forward pass through the same teacher. The useful signal is the teacher itself, not the extra target cleanup. For simplicity, we keep a small 9-teacher pool with a flipped average target as the primary configuration, but the results do not depend on that choice.

4.5.0.2 Teacher ensemble size.

Natural follow-up: Is the plateau of 30 errors an upper limit imposed by having only 9 teachers? To find out, we ran the ensemble as direct predictor on the test set, rather than as the source of training time soft targets, and compared 9 teachers to 15 teachers. The answer from Table 4 is straightforward. More teachers do not move the ceiling.

Individual teachers disagree enough to make the test meaningful – the number of clean errors per model ranges from 27 to 43. Yet both majority pools plateau at 30 errors (99.783%) with an identical McNemar \(p = 0.0755\). Ensemble diversity is not the bottleneck: the remaining errors sit in an irreducible floor where additional models vote the same wrong answer.

4.6 Flip-TTA at Test Time↩︎

If we average each teacher’s logits over the original and horizontally-flipped test image, the error drops to 28 (\(p = 0.029\)). This is the only result in our study that crosses the 0.05 threshold. This result is not treated as a positive result. Horizontal reflection of Devanagari produces characters that are not in the script. In the training distribution, such inversions are intentionally excluded (Section 3.4). Gain is also vulnerable. When applied to the individual teacher, the flip TTA causes errors in 88 and 102 at the most affected checkpoints, increasing from 30-43 in the clean baseline. It is only by chance at this split that the ensemble masks this difference. The student training-time targets (Section 3.3) also use flipped averaging, but use the training image as the target smoothing device. This is a clear application that is not affected by the invalidity of flip TTA as a test time predictor. Therefore, we exclude the results for \(p = 0.029\) and report a clean ensemble of 30 errors as an upper criterion for teacher performance.

4.7 DHCD to CMATERdb Digit Transfer↩︎

Do distilled students learn the structure of numbers, or just the visual habits of one collection pipeline? We probe this on CMATERdb 3.2.1 [15], an independently collected set of handwritten Devanagari numerals (digits 0-9, 10 classes). The two datasets have different scanner hardware, writer populations, paper stocks, and binarization pipelines, so the duplicate labels clearly arrive from different source distributions.

With Hamming distance \(\leq\)​5 bits (minimum observed: 6 bits), no near-overlapping pairs were detected and contamination was ruled out. Before evaluation, each CMATERdb image is polarity-inverted to match the white-on-black convention of DHCD.

4.7.0.1 Results.

Table 5 (upper rows) summarises the transfer results. The DHCD-trained distilled student zero-shots at 76.6\(\pm\)​3.7% on CMATERdb using the same 10-class head, versus 62.7% for a supervised baseline trained on DHCD digit images only—+14 percentage points under domain shift despite being tied on in-distribution DHCD. Freezing the backbone and fitting a fresh 10-class head (linear probe) reaches 85.4%; 30 epochs of fine-tuning at a low learning rate reaches 97.8%.

There is one caveat: this experiment covers only the digit subset (10 of 46 DHCD classes). Nothing is claimed about transferring the full character inventory. Within that scope, after contamination and polarity control, the distilled student preserves more structure under dataset shift than the supervised DHCD-only baseline.

4.8 Corruption Robustness↩︎

Three corruption families are evaluated: additive Gaussian noise, Gaussian blur, and contrast reductions. The five severity levels follow the ImageNet-C protocol adapted for \(32{\times}32\) grayscale images. We report mean accuracy for each family and overall severity 1-5 (mCA). To maintain preprocessing symmetry, all models apply their own training-time normalisation to the same corrupted \([0,1]\) input image.

4.8.0.1 Results.

CDN gives about 2\(\times\) better results overall (mCA 75.7% vs. %). The gap in blur and contrast is significant (97.0% vs. % for blur and 75.5% vs. % for contrast). This likely reflects that the baseline HFC design amplifies blur and its fixed \((0.5, 0.5)\) normalization is sensitive to global intensity changes caused by contrast corruption. There is one obvious reversal. For additive noise, the baseline is better (mCA 66.9% vs.%). Practitioners who target noisy deployment environments should note that the noise rankings are reversed.

4.9 Cross-Dataset Generalisation↩︎

Both models are evaluated in zero-shot on two independently collected Devanagari datasets, each with separate writer and scanner workflows from different collection years, without applying any fine-tuning.

4.9.0.1 NHCD (Pant 2012).

The Nepali Handwritten Character Dataset [16] covers the same 46-class inventory in \(28{\times}28\) JPEG images; we polarity-invert and bilinearly resize to \(32{\times}32\) before evaluation. CDN achieves 78.92% zero-shot (95.80% on digits, 72.33% on consonants). The baseline [2] collapses to 23.99% (58.85% on digits, 10.38% on consonants), a 55 pp gap despite identical in-distribution performance on DHCD, with the +62 pp consonant disparity being the most striking number.

The dental/retroflex pair is the most obvious failure mode. Each of the four pairs (ta/ṭa, tha/ṭha, da/ḍa, dha/ḍha) collapses to near-zero accuracy, caused by interclass confusion between strokes with different rendering rules between Indian and Nepali handwriting traditions. This is a character-level domain gap that neither model can resolve. Outside this cluster, CDN and the baseline diverge sharply.

4.9.0.2 Prashanth et al..

After resizing to \(32{\times}32\), CDN achieves 79.31% zero-shot [17] on this 22,500 image digit dataset. The baseline achieves 72.60%. These numbers track the NHCD digit results and the CMATERdb result from Section 4.7. The high rate of confusion of digit 5 with digit 7 is due to pure visual similarity in the Devanagari script and is not a training artifact.

Matching accuracy within the distribution hides where the gaps are. Both out-of-distribution evaluations favor CDN. The corruption results in Section 4.8 show a similar pattern. This pattern across the three withheld assessments suggests that drawing from a diverse teacher ensemble transfers better across writer and scanner variations than the raw DHCD scores indicate.

5 Analysis: Why DHCD is Saturated↩︎

5.1 The Shared Error Floor↩︎

Both the 15 student models and the reproduced baseline [2] fail on the same 11 images. No configuration classifies any of them correctly, regardless of architecture, training strategy, or random seed.

If the same 11 image floors capture both a compact Student of 1.11 M parameters and a baseline 15 times its size, the problem lies in the data. Outside this floor, the error is model-specific (average pairwise Jaccard similarity \(\mathbf{0.48}\)). Seeds shuffle which images fail, but the addition of ensembles or seeds reduces variance rather than bias imposed by the floor.

5.2 Confusable Character Pairs↩︎

Script-level conflicts account for most of the remaining mistakes. Table 7 lists the top distillation-student confusion pairs aggregated across the 10 distillation seeds. We are not entirely sure what drives the ba/waw pair specifically (17 mutual confusions across all seeds); at \(32{\times}32\) resolution both glyphs collapse to nearly identical stroke profiles and only a handful of pixels separate one from the other. waw / tabala (16) is second, tra / ba (15) third, with dha / gha (13) and da / dhaa (9) rounding out the top five.

In \(32{\times}32\) pixels, the characteristic strokes of these pairs span just a few pixels, sometimes less. DHCD provides neither high-resolution input nor stroke-level annotations, so the information needed to resolve them is not present in the image.

5.3 Significance Frontier↩︎

Moving from statistical parity to a clear win over the reproduced baseline (40 errors) under an exact two-sided McNemar test [8] at \(\alpha = 0.05\) requires at most 25 errors (\(\geq\)​99.82% accuracy). Fewer than 15 additional correct classifications separate an inconclusive result from a significant one. A comfortable margin means \(\leq\)20 errors (\(\geq\)​99.855%), which would demand resolving almost every ambiguous glyph pair in Table 7.

Every configuration hits the wall. The single best result is 28 errors, from the 9-teacher ensemble using (semantically invalid) flip-TTA at test time—excluded on methodological grounds (Section 4.6) because horizontal reflection produces characters that do not exist in Devanagari script and the gain does not survive when applied to individual teachers. The most defensible result is the clean ensemble at 30 errors (99.783%); the most defensible single seed reaches 34 errors. Both sit inside the statistical parity region.

No lever tested in this study achieved a statistically significant improvement over the baseline. Scaling from 9 to 15 teachers changes nothing. Five independent seeds with TTA-softened targets change nothing. This is not a failure of experimental design. It is what a saturated benchmark looks like. The frontier is a wall built from mislabeled images and inherently ambiguous glyphs, not a line that more computation can asymptotically approach.

5.4 Calibration↩︎

Expected Calibration Error (ECE) [10] falls within the range of 0.13–0.16 across all CDN configurations. It is not noticeable for softmax classifiers without post-hoc temperature scaling. ECE does not covary with accuracy across seeds. A better seed will not calibrate better. The 11 floor images are confidently wrong, not merely uncertain, at every seed. The model assigns a high probability to the incorrect class, regardless of how the student is trained.

6 Discussion↩︎

6.0.0.1 Scope of transfer results.

The transfer experiment (Section 4.7) is deliberately narrow: it uses only the digit subset (10 of 46 DHCD classes). That choice reflects the available data, not a design preference. Extending the test to the full 36-consonant inventory would require a genuinely independent target set with DHCD’s label taxonomy; the apparent public candidates—principally the Acharya [1] and Pant datasets—are same-provenance splits of DHCD itself. Transferability claims here are therefore limited to digits, not consonants.

6.0.0.2 Single-benchmark evaluation.

The saturation claim is tied to DHCD’s particular conditions—its \(32{\times}32\) resolution, concentrated label noise, and writer pool. A cleaner or more diverse consonant benchmark might expose capacity differences that DHCD no longer reveals.

6.0.0.3 Efficiency versus accuracy.

Compactness is the appropriate response to benchmark saturation: when remaining errors are governed by data rather than model capacity, a larger network has little room to justify its cost.

7 Conclusion↩︎

We have effectively reached the saturation point on DHCD. What remains can be attributed to label noise and handwriting ambiguity in the data itself, not model capacity. Throwing a bigger recognizer at it does not buy a better recognizer. At that point, compactness is no longer a compromise; it becomes the right operating point.

CDN reaches 99.73 % accuracy with just 1.11 M parameters and achieves statistical equivalence with the leading large-scale baselines (McNemar \(p = 0.345\)) even without knowledge distillation. This means a parameter reduction of 15.6\(\times\) and a CPU latency reduction of 9.5\(\times\) without sacrificing accuracy. Why can’t we move forward? The same 11-error intrinsic floor breaks all tested configurations, teacher ensembles included. Ablation closes off every obvious escape route: distillation target cleanliness makes no difference. Ensemble size makes no difference. Test-time augmentation makes no difference. The floor is the ceiling.

Two limitations are worth noting. The saturation claim is tied to the specific conditions of DHCD, namely \(32{\times}32\) resolution, concentrated label noise, and its particular pool of writers. Cleaner or higher-resolution benchmarks may expose capacity differences that DHCD cannot. The transfer results only cover digit subset (10 out of 46 classes). Consonant-level transferability is out of scope here, as there are no independent consonant target sets that share the label taxonomy of DHCD.

The practical implication for Devanagari recognition is to measure progress more honestly before claiming it. Without paired evaluation – McNemar or equivalent – a gain of a few hundredths of a percent is meaningless. The field needs fresher, harder benchmarks: DHCD can no longer discriminate between strong models. The next necessary step is high-resolution scanning or a completely independent pool of writers.

References↩︎