1 Introduction↩︎

The Armenian press spans over two centuries, from Azdarar (Madras, 1794) through a 19th-century golden age in Constantinople, Tiflis, Moscow, and the early diaspora; the 1915 Genocide redirected publishing toward Middle Eastern, European, and American diasporas, while the Soviet era (1920–1991) and post-1991 independence each produced their own periodicals. The resulting corpus combines Classical, Western, and Eastern Armenian (with frequent multilingual inserts), evolving orthographies, and printing technologies ranging from letterpress and lithography to offset and clandestine mimeographs—yielding highly heterogeneous scan qualities. Our work targets OCR and structuring of thousands of these pages, mostly in Western Armenian¹, contributing to a broader effort to build a Western Armenian reference corpus and to enable downstream resources for an endangered language.

A major obstacle is accurate reading order reconstruction: these newspapers use multi-column designs with irregular article placement, overlapping titles, and non-linear flows, and OCR models adapted to the script still produce disordered outputs. We address this by introducing a newly annotated dataset of Armenian newspapers, proposing a specialized OCR model for historical Armenian, and comparing strategies—layout heuristics, YOLO-based detectors, and large language models—under realistic OCR constraints, both within and across pages.

Our LLM-based methods are framed as an annotation accelerator—generating candidate orders that humans verify—rather than a production pipeline, since cloud-API LLMs scale poorly in cost and latency.

2 Related Works↩︎

Reading order detection is a key component of document structure analysis, impacting information retrieval, OCR post-processing, and large-scaxle dataset preparation for LLMs. Traditional methods rely on heuristics (top-to-bottom, left-to-right) or classical models based on Bayesian inference and logic programming [1]; both struggle with multi-column and irregular layouts. Deep learning brought layout-aware architectures: LayoutLM [2] integrates visual and positional cues with text, and LayoutReader [3] predicts reading order directly with a sequence-to-sequence paradigm. End-to-end document models such as Kosmos-2.5 [4], GOT [5], Nougat [6], and ECLAIR [7] combine OCR, region detection, and reading order in a unified encoder-decoder (ECLAIR uses a ViT-H (RADIO) encoder with an mBART decoder [8] and was benchmarked against LayoutLMv3, Kosmos-2.5, and GOT). These systems excel on structured English corpora but perform poorly on degraded documents and minority scripts. Two-stage pipelines remain common: Quiros et al. [9] separate layout detection from sequence modeling, and recent work on Chinese historical documents [10] adopts a similar hierarchical strategy.

Sequence-to-sequence models such as LayoutReader and LayoutLM-based approaches require token-level training data in the target script; their fine-tuning for Western Armenian is precluded by the absence of large annotated corpora—precisely the under-resourced setting this work targets.

3 Dataset↩︎

Digital archives of the Armenian press—mostly scans rather than plain text—are accessible through four main institutions². The scanned data used in the present study are drawn from one or more of these archives (for further details, see Git repository).

The dataset consists of 66 scanned pages (1913–2009) with high variability in layout, scan quality and typography, provided in PageXML and YOLO bounding-box formats. All annotations were created using the Calfa Vision annotation tool [11]. While the dataset primarily reflects Armenian newspaper production from the French diaspora throughout the 20^th century, the diversity of layouts, typefaces, and digitization conditions remains representative of broader Armenian newspaper design, excluding periodicals.

Three levels of annotation have been defined: (i) semantic text regions to describe document content, (ii) semantic areas used for reading order experiments, and (iii) structural separators. A total of 11 classes are annotated. The classes and their frequencies across the dataset are summarized in Table 1 and Table 2.

The dataset includes pages from 35 distinct newspaper issues (1913–2009), primarily in Western Armenian (MWA)—the under-resourced variant we target—with a few examples in Eastern Armenian (MEA) and some bilingual MWA / French issues. Scan quality ranges from good to poor (blur, damage, curvature, uneven exposure), and layouts span compact book-like formats to wide newspaper pages with two to six columns. For each issue, we selected two pages: typically the front or last page plus one interior page containing content that continues from the front page (Tasks 1 and 2, see Fig. 1).

Dataset splits.

For all reading order experiments, we use a stratified 80/10/10 train/val/test split, stratified by newspaper title to prevent leakage across issues of the same publication. An additional 20 out-of-domain images (unseen newspaper titles, sourced from the same digital libraries) form an independent test set for Tables 3 and 4; Tables 5 and 6 use the in-domain test split.

Synthetic data.

We augment the corpus to 500 images via image-to-image translation [12], [13] from semantic masks derived from PageXML annotations (see Table 2); the masks guide the generation of realistic newspaper-like layouts without textual content. To mimic typical scan degradations we apply randomized transformations from the Albumentations library (ElasticTransform, RandomBrightnessContrast, Flip+Flop, RandomFog, GaussianBlur, ZoomBlur). Each high-resolution image (typically \(>3000\times5000\) px) is split into four quadrants prior to generation and recombined afterwards to preserve coherence.

Table 2: Examples of original images, text region annotations, semantic blocks, and structural separators. Source: BULAC and ARAM.

	3 cols. (1949)	4 cols. (2001)	5 cols. (1937)
Image
Text regions
Semantic blocks
Separators

4 Tasks and experiments↩︎

We use the dataset for two reading order tasks. Task 1 addresses reading order within a single page; Task 2 identifies the global reading sequence across multiple pages, a common newspaper scenario where an article begins on the front page and continues on later ones. Fig. 1 illustrates both.

4.1 Preliminary task about OCR↩︎

Using LLMs requires transcriptions, and Armenian OCR is already difficult on its own: open models Tesseract and EasyOCR perform poorly on historical fonts and low-resolution scans, and their internal language models cover Western Armenian sparsely. A generic Armenian OCR/HTR model has been proposed by Calfa [14], [15], but it targets handwritten archives and ancient manuscripts. We therefore train our own Tesseract-based model on the dataset of this paper (CRNN, \(<\)​300k parameters; see Fig. 2; average inference 1.3s per page, \(\sim\)​0.1s per line), tailored to rare typefaces and variable scan quality.

Absolute CER reductions range from 8 to 35 percentage points (Table 3), with the largest gains on noisy and damaged historical fonts; most residual errors localize on non-Armenian words and curved lines.

4.2 Preliminary task about layout analysis↩︎

YOLO-family models perform strongly on complex and historical layouts [16]. We compare YOLOv11x (default hyperparameters, dropout 0.1, image size 1,560) and DocLayout-YOLO [17], a layout-aware model pretrained on the synthetic DocSynth dataset. Table 4 shows that DocLayout-YOLO + SynthData consistently achieves the best results, confirming the benefit of combining a layout-specific architecture with synthetic pretraining; DocLayout-YOLO outperforms YOLOv11x on mAP@50 and recall by a small margin (1–2 pp), while YOLOv11x remains competitive when fine-tuned.

Synthetic data improves both models, especially for harder classes (Marginalia, MetaData); gains are more pronounced for YOLOv11x, which benefits more from layout exposure than the already-pretrained DocLayout-YOLO. Marginalia, MetaData, Signature, and Advertisement remain poorly detected due to their rarity and variability—acceptable here as they are peripheral to reading order. The classes that matter most for our task, Title and Paragraph, are reliably detected across all models.

4.3 Reading order strategies↩︎

4.3.0.1 Global Topological Sort (GTS).

Inspired by Breuel [1], we adapt a line-segment topological method to paragraph-level bounding boxes: layout elements are segmented from geometric criteria (baseline alignment, spacing, font size) and the reading order is recovered as a topological sort over the resulting partial order, \(\mathcal{O} = \text{TopSort}(\{b_i\},\{b_i \prec b_j\})\), robust to multi-column layouts and floating elements.

4.3.0.2 Whitespace deduction + LTS (W ded. + LTS).

For boxes \(\mathcal{B}=\{b_i\}\) predicted by YOLO, we compute horizontal and vertical projections \[\mathcal{H}(y) = \sum_{i} \mathbf{1}_{[y_{1,i}, y_{2,i}]}(y), \quad \mathcal{V}(x) = \sum_{i} \mathbf{1}_{[x_{1,i}, x_{2,i}]}(x),\] and deduce whitespace separators as long zero-valued spans after smoothing (Alg. 3, with band-height threshold \(T\) set to the median paragraph height). The procedure assigns each box a paragraph_id and a hierarchical row_id; reading order is then produced by a Local Topological Sort (LTS) within each group, ordering by row_id and then by horizontal position. Adapted from whitespace-based segmentation [18], [19] but applied to detector outputs, this is robust to prediction noise.

4.3.0.3 Whitespace detection + LTS (W det. + LTS).

As an alternative to deducing separators from box projections, we train a dedicated YOLO model to detect horizontal and vertical separators directly (more reliable when boxes do not align cleanly with paragraph structure), then run the same grouping and LTS pipeline.

4.3.0.4 MLP.

Adapting [9], we train a Multi-Layer Perceptron on YOLO-detected paragraph boxes with ground-truth annotations to predict reading order from spatial features—a learned alternative to topological sorting.

4.3.0.5 SD + LTS.

The two-pass YOLO above (semantic blocks then paragraphs) is also evaluated without an LLM: paragraphs are grouped by semantic block and ordered within each block by a Local Topological Sort, then blocks are concatenated top-to-bottom. This isolates the contribution of semantic grouping from that of the LLM.

Inspired by hierarchical detectors for complex Chinese reading order [10], we also use a two-pass YOLO that first detects semantic blocks (Table 2) and then individual paragraphs—the basis of our SD + LLM pipeline below.

We benchmark ECLAIR [7] alongside our proposed SD + Generative LLM pipeline (Fig. 5). The LLM (Mistral-8B-Instruct-2410) is queried under a role-play system prompt with two-shot in-context learning (Fig. 4): one example requires reordering, the other is already in correct order, so the model learns when not to act. The pipeline runs in three steps: (i) two-pass YOLO yields SDs and their paragraphs; (ii) paragraphs within each SD are ordered by a Local Topological Sort (LTS, no LLM call); (iii) the global order across SDs is decided by the LLM through pairwise comparisons restricted to the first and last paragraphs of each pair of SDs—reducing LLM calls from \(O(n^2)\) to \(O(k^2)\) with \(k \ll n\). All runs use temperature \(0\) (\(\sim\)​8 s/page on one GPU). We additionally evaluate Gemini 3.0 Flash as a drop-in replacement for Mistral-8B to test the sensitivity of the pipeline to the underlying LLM.

5 Results and discussions↩︎

5.1 Metrics↩︎

Following Quirós & Vidal [20], we use Kendall’s tau distance \(\tau\) (number of pairwise inversions) and Spearman’s footrule \(F\) (sum of absolute positional differences), defined as \[\tau = \frac{2K}{n(n-1)}, \qquad F = \frac{\sum_{i=1}^{n} |\pi(i) - \sigma(i)|}{\left\lfloor n^2/2 \right\rfloor},\] where \(K\) is the number of inverted pairs, \(n\) the number of elements, and \(\pi(i), \sigma(i)\) the predicted and ground-truth ranks of item \(i\). Both are normalized to \([0,1]\) with \(0 =\) perfect agreement.

Metric scope and independent articles.

Both metrics assume a single fully-ordered ground truth, but newspaper pages routinely carry several independent articles whose relative order is arbitrary: reading column 2 before column 1 is equally valid. Such inter-article permutations are penalized identically to genuine intra-article errors. Scores should therefore be read as an upper bound on true ordering errors; the limitation is inherent to single-sequence evaluation and affects all methods equally.

Abbreviations (Tables 5 and ¿tbl:tab:llm95comparison?). GTS: Global Topological Sort. W ded.   +   LTS: Whitespace deduction + Local Topological Sort. W det.   +   LTS: YOLO separator detection + LTS. MLP: Multi-Layer Perceptron on bounding-box features. SD + LTS: Semantic block Detection + LTS. ECLAIR: end-to-end OCR/layout model (ViT-H + mBART) [7]. Gen. LLM: Mistral-8B-Instruct-2410, prompt-only. SD + Gen. LLM: SD + Generative LLM (proposed). SD + Gemini: SD + Gemini 3.0 Flash (ablation).

5.2 Reading Order Evaluation Results↩︎

Table 5 reports \(\tau\) and \(F\) (both in \([0,1]\), lower is better) under two settings: Exp. 1 randomly shuffles ground-truth bounding boxes; Exp. 2 uses predicted boxes from detection and recognition. LLM variants within the SD + Generative LLM pipeline are compared separately in Table ¿tbl:tab:llm95comparison?.

SD + Generative LLM performs best in both experiments, achieving the lowest scores (0.07/0.20 in Exp. 1 and 0.10/0.18 in Exp. 2)—a 76% reduction in \(\tau\) over the strongest geometric baseline (GTS Exp. 2: \(\tau=0.42\)), the figure underlying the abstract claim. ECLAIR and SD + LTS also perform well (0.14/0.38 and 0.17/0.35 for SD + LTS), confirming that structured document representations—semantic or learned—are beneficial.

Note on metric behaviour.

While \(\tau\) degrades from Exp. 1 to Exp. 2 for all methods (as expected with added detection noise), \(F\) often improves slightly. This is a normalization artefact: missed detections in Exp. 2 reduce \(n\), deflating the \(\lfloor n^2/2 \rfloor\) denominator independently of ordering quality. We therefore rely primarily on \(\tau\) for cross-experiment comparison.

5.3 LLM Comparison↩︎

We swap the LLM in the pipeline across four models varying in size and deployment mode to assess the pipeline’s sensitivity to the underlying LLM (Table ¿tbl:tab:llm95comparison?, Task 1 Exp. 1).

This comparison addresses the bootstrapping use case: small local models (Ministral-3B, Mistral-8B) offer the lowest cost for offline annotation, while cloud-API models (Gemini 3.0 Flash, Mistral-Large) trade cost for accuracy. Ministral-3B is clearly weaker than Mistral-8B; Mistral-Large performs on par with Mistral-8B, suggesting that scaling alone within the Mistral family is insufficient. Gemini outperforms all Mistral models, reaching the lowest scores; we cannot attribute the gap to a single cause (architecture, scale, or language coverage all plausibly contribute).

W det. + LTS outperforms W ded. + LTS, confirming that explicit whitespace detection helps with nested column layouts; both still struggle on irregular structures. GTS underperforms layout-sensitive models, especially on column splits (0.42/0.50 in Exp. 2). MLP scores near-randomly (\(\tau \approx 0.70\)): it lacks sequential inductive bias and is trained on fewer than 70 pages, insufficient for learning reliable spatial layouts.

Generative LLM alone: a systematic failure.

The standalone Generative LLM scores \(\tau=0.75\) / \(F=0.82\) in Exp. 1—well above the random baseline (\(\tau\approx 0.5\)): the model is not confused but systematically inverting the reading order. We hypothesize two compounding factors: (1) Western Armenian is sparsely covered by Mistral’s pretraining; (2) without spatial cues, the model reorders text thematically rather than spatially, conflicting with column structure. Substituting Gemini 3.0 Flash for Mistral-8B within the same SD + LLM pipeline lowers \(\tau\) from \(0.07\) to \(0.03\) (Table ¿tbl:tab:llm95comparison?); we cannot disentangle architecture, scale, and language coverage from a single substitution, but the gap is consistent with stronger underlying capabilities on this task.

For Task 2 (Table 6), given the large gap in Task 1, we evaluate only SD + Generative LLM since other methods are unlikely to handle inter-page dependencies. It remains effective: \(\tau=0.27\)/\(F=0.42\) (Exp. 1) and \(\tau=0.26\)/\(F=0.45\) (Exp. 2). The higher scores than in Task 1 (\(\tau=0.07\)) reflect the difficulty of long-range, cross-page dependencies. Notably \(F\) worsens but \(\tau\) marginally improves between experiments—the opposite of Task 1—indicating that cross-page errors stem more from structural ambiguity than from local displacement. Empirically, \(\tau<0.1\) corresponds to minor line-level swaps, whereas \(\tau>0.3\) signals disruption across columns or content blocks; SD + Generative LLM in Task 1 falls in the former regime.

OCR error cascade.

The \(\tau\) degradation from Exp. 1 to Exp. 2 reflects both detection noise and OCR errors. SD + Generative LLM shows the smallest relative degradation (\(0.07 \to 0.10\), +43%) versus W det. + LTS (\(+50\%\)) and GTS (\(+20\%\)): the LLM can leverage partial contextual cues even when individual transcriptions are noisy.

MLP anomaly.

For MLP, \(\tau > F\) in both experiments (\(0.70 > 0.68\); \(0.73 > 0.66\)). Although \(F \geq \tau\) holds asymptotically, the normalized denominators \(n(n-1)/2\) vs \(\lfloor n^2/2 \rfloor\) can invert this for small \(n\) (\(n\leq5\)), suggesting some test pages contain very few detected blocks.

6 Conclusion↩︎

We benchmarked reading order strategies for historical Armenian newspapers and introduced a dedicated annotated dataset. Combining semantic zone detection with a generative LLM yields the most accurate and robust predictions in both single- and multi-page settings; we frame this approach as a data bootstrapping tool—generating candidate orders that humans verify—rather than a production system. Our LLM comparison (Ministral-3B, Mistral-8B, Mistral-Large, Gemini 3.0 Flash) informs the quality/cost trade-off for this role.

Limitations and future work.

On simple, unambiguous layouts, classical geometric methods (GTS, W det. + LTS) suffice and the LLM cost is unjustified; the proposed pipeline shines when the layout itself is ambiguous. Three directions follow. (i) Reduce the quadratic LLM-call count via smarter pairwise selection (transitive pruning, tournament sort, geometry-based filtering). (ii) Improve Armenian—especially Western Armenian—language coverage in the underlying LLMs, a plausible contributor to the Gemini gap and a long-term need for the field; this calls for larger corpora and dedicated language models. (iii) Use the bootstrapped annotations to train layout-aware models (LayoutLM, LayoutReader) for Armenian, removing the LLM from inference. Extending the approach to other endangered scripts and to more complex multi-article layouts are concrete next steps. Finally, we did not evaluate a full vision-language approach in which a VLM directly ingests the page image and returns the ordered text—an avenue that could in principle bypass both the OCR and the explicit SD detection stages, and worth exploring as VLM coverage of low-resource scripts improves. In any case, the bootstrapped annotations produced by our pipeline provide the supervision needed to fine-tune compact local models (e.g., Qwen3-VL-3B/8B) toward an offline, deployment-ready alternative.

Acknowledgements, disclosure, data.

This research received support from ANR DALiH (ANR-21-CE38-0006); Claude AI was used to help condense the paper and proofread the English (any remaining mistakes are our own). The authors declare no competing interests. OCR model: https://github.com/calfa-co/hye-tesseract; dataset and code: https://github.com/CVidalG/HYEpress-RO.

References↩︎