1 Introduction↩︎

Punctuation restoration represents an essential task in natural language processing, particularly for languages with limited computational resources. The absence of punctuation in raw text—whether from automatic speech recognition, informal digital communication, or historical documents—severely impacts the performance of downstream NLP tasks including machine translation, text summarization, and sentiment analysis.

None

Despite the growing maturity of Persian NLP, punctuation restoration has received limited attention compared to other languages. The critical importance of punctuation in Persian is evidenced by dramatic semantic changes that occur with minimal punctuation modifications, as shown in Figure 1. Existing Persian studies have been constrained by small-scale datasets, domain-specific applications, or lack of publicly available models, highlighting the critical need for comprehensive approaches that can handle the full complexity of Persian text across diverse domains and writing styles.

This work addresses these challenges through a comprehensive approach to Persian punctuation restoration using fine-tuned BERT models and large-scale dataset curation. We present a dataset curation methodology that systematically aggregates multiple Persian text sources, resulting in a high-quality corpus for training robust punctuation restoration models. Our main contributions are:

• We present PersianPunc, a large-scale Persian punctuation restoration dataset containing 17 million filtered and deduplicated samples spanning diverse domains, sourced from six complementary corpora covering both formal and informal Persian text.

• We provide a systematic dataset curation framework including detailed preprocessing, quality filtering, and train/validation/test splits, with comprehensive analysis of punctuation distribution patterns in Persian.

• We achieve strong performance on Persian punctuation restoration with a fine-tuned ParsBERT model, demonstrating competitive results compared to large language models while requiring significantly lower computational resources and avoiding over-correction issues.

2 Related Work↩︎

Our research is situated at the intersection of punctuation restoration, Transformer-based NLP, and Persian text processing. This section reviews the evolution of methodologies for this task, starting from general approaches and progressively narrowing the focus to the specific challenges and prior work in the Persian language.

2.1 Punctuation Restoration as a Sequence Modeling Task↩︎

Historically, punctuation restoration was tackled with statistical methods, including n-gram language models [1], [2] and models incorporating prosodic features from speech to predict boundaries [3], [4]. These early systems laid the groundwork but were often limited by the scope of their handcrafted features and statistical models.

The advent of deep learning marked a significant shift. Recurrent Neural Networks (RNNs), particularly models using Bidirectional Long Short-Term Memory (BiLSTM) units, became the standard, framing the problem as a sequence labeling task where each token is classified with a punctuation mark (or none) [5]. This paradigm was often enhanced with Convolutional Neural Networks (CNNs) to capture local character-level features [6], [7], leading to substantial performance gains over classical methods. Our work follows this successful sequence labeling formulation, leveraging a more powerful neural architecture.

2.2 Transformer-Based Approaches for Punctuation Restoration↩︎

The introduction of the Transformer architecture [8] and pre-trained language models like BERT [9] revolutionized NLP. For punctuation restoration, fine-tuning BERT-based models quickly became the state-of-the-art approach in high-resource languages, demonstrating superior performance in capturing long-range dependencies crucial for understanding sentence structure and punctuation placement [10]–[12]. These models are typically lightweight and efficient, making them suitable for real-time applications like ASR post-processing.

More recently, Large Language Models (LLMs) have demonstrated impressive capabilities in a zero-shot or few-shot capacity for various text generation and correction tasks [13]. However, their application to focused tasks like punctuation restoration comes with potential drawbacks, including high computational inference costs and a tendency for over-correction, where they may alter the source text beyond simply adding punctuation. A key part of our contribution is to rigorously evaluate these trade-offs against a fine-tuned, specialized model.

2.3 State of Persian Text Processing and Punctuation↩︎

Early work on Persian punctuation restoration was pioneering but limited in scale. [14] introduced the first known corpus for this task, achieving an F1-score of 69.0% with a Conditional Random Field (CRF) model. While foundational, this work highlighted the need for larger and more diverse datasets and more powerful models.

More recently, [15] proposed ViraPart, a multi-task text refinement framework for Persian that handles punctuation restoration, Zero-Width Non-Joiner (ZWNJ) recognition, and Ezafe construction. Using ParsBERT [16], they achieved a strong F1-score of 92.13% for punctuation on the Bijankhan corpus [17]. Their work demonstrated the effectiveness of Transformer-based models for Persian text refinement. However, ViraPart focuses on multiple text refinement tasks simultaneously, and was evaluated on a smaller, single-domain corpus.

Despite these advances, critical gaps remain. First, there is a lack of a large-scale, publicly available dataset specifically curated for punctuation restoration across diverse domains. Most existing efforts rely on smaller or general-purpose corpora like Bijankhan. Second, the capabilities and limitations of modern LLMs for Persian punctuation restoration have not been systematically studied, particularly regarding over-correction behavior.

3 Methodology↩︎

3.1 Dataset Construction↩︎

3.1.1 Data Sources and Collection Strategy↩︎

We construct a comprehensive Persian punctuation restoration dataset by systematically aggregating high-quality corpora spanning diverse domains and registers. Our multi-source approach addresses the linguistic diversity challenges of Persian NLP through careful curation. We selected source datasets through manual inspection, verifying that at least 100 random samples from each contained proper punctuation usage.

Our dataset combines sources across two primary categories:

Formal Academic Text: Bijankhan-Peykare Corpus [18], Persian Medical QA [19], and Persian Wikipedia [20] provide standardized punctuation patterns in formal contexts, covering literary, medical, and encyclopedic domains.

Contemporary Informal Text: Persian Telegram Channels [21], Farsi Stories [22], and Blog Dataset V2 [23] capture modern conversational patterns and varied punctuation usage, representing social media, narrative fiction, and personal blogging styles.

3.1.2 Preprocessing and Quality Control↩︎

3.1.2.1 Normalization Pipeline

All texts undergo systematic preprocessing to ensure consistency:

1. Punctuation standardization: English punctuation marks (comma, semicolon, question mark) are converted to their Persian equivalents (Persian comma ،, Persian semicolon ؛, Persian question mark ؟).

2. Character filtering: Non-Persian characters are removed while preserving common Persian script variants and Arabic letters used in Persian text.

3. Whitespace normalization: Multiple spaces are collapsed, and leading/trailing whitespace is removed.

3.1.2.2 Sentence Segmentation and Filtering

We first segment each source corpus into sentence-level units using end-of-sentence punctuation marks (period, exclamation mark, question mark). Each candidate sentence then undergoes multi-stage filtering:

• Structural requirements: Minimum length of 10 characters; at least two target punctuation marks from the set {., ،, ؟, !, ;, :}; proper sentence termination with period, exclamation mark, or question mark.

• Content filtering: Removal of sentences containing URLs, email addresses, social media handles (@ mentions), emojis, excessive special symbols (more than 20% non-alphabetic characters), or substantial mixed-language content (more than 30% non-Persian text).

• Linguistic quality: Pattern-based detection and removal of repetitive punctuation (e.g., “.…”, “!!!!”), enumerative sequences (numbered lists, bullet points), and fragmented text (sentences with more than 50% single-character tokens).

3.1.2.3 Rationale for Filtering Criteria

The requirement for at least two punctuation marks ensures that samples present meaningful punctuation restoration challenges beyond simple sentence termination. While this filtering criterion does exclude simple sentences (which are underrepresented in formal Persian writing), it ensures the dataset focuses on the core challenge of internal sentence punctuation, which is critical for ASR applications where sentence boundaries are often detected separately. We acknowledge this as a dataset characteristic rather than a limitation, as it creates a focused benchmark for comma, colon, and question mark insertion—the most challenging and impactful aspects of Persian punctuation restoration. Future work could address simple sentence coverage through stratified sampling or separate evaluation sets.

3.1.3 Deduplication and Dataset Splitting↩︎

To ensure dataset quality and prevent data leakage, we perform exact deduplication across all source corpora. Due to the large dataset size (initial pool of over 20 million samples), we implement an efficient SHA-256 hash-based deduplication strategy with whitespace normalization. Each sentence is normalized (lowercased, whitespace-collapsed) before hashing to detect duplicates that differ only in formatting.

After deduplication, our final dataset contains 17,102,014 unique samples. For model training and evaluation, we randomly sample a 1M subset stratified by source corpus. This subset is split into training (989,000 samples), validation (10,000 samples), and test (1,000 samples) sets. The sampling strategy maintains the source distribution proportions to ensure representativeness across domains.

3.2 Dataset Statistics and Punctuation Analysis↩︎

We conducted a comprehensive punctuation analysis on the complete dataset of 17,102,014 samples to understand the characteristics of Persian punctuation usage in our corpus.

3.2.1 Punctuation Distribution↩︎

Table 1 presents the distribution of punctuation marks across the entire dataset. All samples contain at least one punctuation mark (by design), with an average of 2.51 punctuation marks per sentence.

The distribution reflects typical Persian text characteristics, where commas are heavily used for clause separation and complex sentence structures.

3.3 Punctuation Restoration Model and Training Setup↩︎

We formulate punctuation restoration as a token-level sequence labeling problem. Given an input sequence of tokens without punctuation, the model predicts a punctuation label for each token position. We define five classes: EMPTY (no punctuation), COMMA (،), QUESTION (؟), PERIOD (.), and COLON (:). Note that we focus on the four most common and semantically important punctuation marks, excluding exclamation marks and semicolons which are less frequent and often interchangeable with periods and commas in Persian.

3.3.0.1 Model Architecture

Our model architecture consists of a pre-trained ParsBERT encoder [16] followed by a linear classification layer with dropout regularization. ParsBERT is a monolingual Persian BERT model pre-trained on a large Persian corpus, making it well-suited for Persian NLP tasks.

Given an input sentence with punctuation removed, we:

1. Tokenize using ParsBERT’s WordPiece tokenizer

2. Pass tokens through ParsBERT to obtain contextualized embeddings

3. Apply dropout (p=0.1) for regularization

4. Project embeddings to 5-dimensional class logits via a linear layer

5. Assign the predicted punctuation class to each token position

For subword tokens generated by WordPiece tokenization, we assign punctuation labels only to the first subword of each word, ignoring continuation subwords during both training and evaluation. This aligns the token-level predictions with word-level punctuation placement.

3.3.0.2 Training Configuration

We train on the 1M sample subset described in Section 3.1.3. The model is optimized using AdamW with a learning rate of \(2 \times 10^{-5}\), weight decay of 0.01, and trained for 3 epochs. We use a batch size of 85 with gradient accumulation over 8 steps (effective batch size of 680). The loss function is cross-entropy computed over all token positions.

3.3.0.3 Evaluation Metrics

We employ standard sequence labeling metrics:

• Per-class metrics: Precision, recall, and F1-score for each punctuation class (COMMA, PERIOD, QUESTION, COLON)

• Macro-averaged F1: Arithmetic mean of per-class F1-scores, giving equal weight to each punctuation type regardless of frequency

• Micro-averaged F1: F1-score computed from the sum of per-class true positives, false positives, and false negatives, effectively weighting classes by their frequency

• Full Sentence Match (FSM) Rate: Percentage of test sentences where the predicted punctuation sequence exactly matches the gold standard. This metric is particularly important for evaluating LLMs, as it captures whether the model made any edits beyond punctuation insertion (over-correction).

Throughout this paper, unless otherwise specified, “F1-score” refers to the macro-averaged F1-score, which provides an overall measure of punctuation restoration accuracy giving equal weight to each punctuation type regardless of frequency.

4 Results and Analysis↩︎

4.1 Overall Performance↩︎

Our fine-tuned ParsBERT model achieves a macro-averaged F1-score of 91.33% and a micro-averaged F1-score of 97.28%. The macro-averaged score is lower due to the class imbalance (periods and commas dominate the dataset), but performance remains strong across all punctuation types as shown in Table 6 in the appendix.

4.2 Comparison with Prior Work↩︎

It is important to note that the CRF and ViraPart results shown in Table 2 are evaluated on different datasets (the Hosseini et al. corpus and Bijankhan corpus, respectively), and therefore cannot be directly compared to our results. We include these numbers for reference to situate our work within the Persian punctuation restoration literature, but we make no claims of superiority over these methods without evaluation on the same test set.

The substantial improvement over the CRF baseline (69.00% vs 91.33%) likely reflects both the advancement in modeling approaches (Transformer-based vs. CRF) and potential differences in dataset difficulty. The ViraPart score (92.13%) is more competitive, though direct comparison remains inappropriate due to the different evaluation sets.

4.3 Comparison with Large Language Models↩︎

We evaluated two variants of GPT-4o on our test set using the zero-shot prompt shown in Appendix 2. The prompt explicitly instructs the model to only add punctuation without modifying the source text.

Our ParsBERT model achieves 91.33% macro F1, outperforming both GPT-4o (85.96%) and GPT-4o-mini (79.54%) on the same test set. More importantly, the FSM Rate reveals a critical limitation of LLM-based approaches: GPT-4o achieves only 50.10% exact matches, while our model achieves 61.80%.

Analysis of the mismatches reveals that GPT-4o exhibits over-correction in approximately 5% of samples, making undesired edits such as:

• Removing words deemed unnecessary

• Replacing informal words with formal equivalents

• Correcting perceived spelling or grammatical errors

Notably, we observed no cases of word additions, only deletions and substitutions. This over-correction behavior is particularly problematic for ASR post-processing pipelines, where the source text (transcribed speech) should be preserved verbatim with only punctuation added.

Additionally, GPT-4o requires substantially higher computational resources for inference compared to our lightweight ParsBERT model, making it less suitable for real-time applications or deployment in resource-constrained environments.

4.4 Analysis of Model Performance↩︎

Table 6 (Appendix) provides detailed per-class performance metrics. The model performs exceptionally well on periods (F1: 98.71%), which is expected given their high frequency and relatively consistent usage patterns. Performance on other punctuation types remains strong: colons (90.45%), question marks (88.89%), and commas (80.03%).

The lower performance on commas reflects their more nuanced usage in Persian, where comma placement can be somewhat flexible and context-dependent, leading to greater ambiguity in the gold standard annotations themselves.

5 Conclusion and Future Work↩︎

This work presents PersianPunc, a large-scale dataset of 17 million samples for Persian punctuation restoration, constructed through systematic aggregation and quality filtering of diverse Persian text sources. We demonstrate that a fine-tuned ParsBERT model achieves strong performance (91.33% macro F1) while avoiding the over-correction issues and computational overhead of large language models.

Our primary contribution is the dataset itself, which addresses a critical gap in Persian NLP resources. The curation methodology, including detailed preprocessing pipelines, quality filtering criteria, and comprehensive punctuation analysis, provides a framework applicable to other low-resource languages.

Future work should explore several directions. The development of domain-specific models for Persian literature, news, and social media text could address the variation in punctuation usage across different domains. Additionally, incorporating prosodic information from Persian speech could improve punctuation restoration for speech-to-text applications. Furthermore, extending the model to jointly handle punctuation restoration and Zero-Width Non-Joiner (ZWNJ) insertion would address the broader text normalization challenges specific to Persian writing systems.

6 Limitations↩︎

This work has several limitations that should be acknowledged. First, the dataset creation process relies on existing Persian texts, which may contain punctuation errors or inconsistencies that could propagate to the trained model. Second, the model’s performance is optimized for contemporary Persian writing styles and may not generalize well to historical or highly specialized Persian texts. Third, our evaluation is limited to 1,000 test sentences due to resource constraints, including expensive API costs for commercial LLM evaluation and lack of GPU access for extensive experimentation. More extensive evaluation with larger test sets, multiple training runs, and statistical significance testing would strengthen our findings but was not feasible given these constraints.

7 Punctuation Analysis Details↩︎

In this appendix, we provide comprehensive analysis of punctuation patterns and model performance.

7.1 Punctuation Co-occurrences↩︎

Analysis of punctuation co-occurrence reveals common patterns in Persian writing. Table 3 shows the most frequent punctuation pairs appearing together in the same sentence.

The combination of period and Persian comma appears in nearly 80% of sentences, indicating that most sentences contain multiple clauses separated by commas before the final period.

7.2 Sentence-Level Punctuation Coverage↩︎

Table 4 shows the percentage of sentences containing each punctuation mark (counted once per sentence regardless of frequency).

7.3 Distribution of Punctuation Counts per Sentence↩︎

Table 5 presents the distribution of the number of punctuation marks per sentence. The majority of sentences (68.37%) contain exactly 2 punctuation marks, which is a direct consequence of our filtering criterion requiring at least 2 marks per sentence combined with the natural distribution in source texts.

7.4 Punctuation-Specific Performance↩︎

Table 6 presents a detailed analysis of per-class performance. The macro-averaged F1-score of 91.33% demonstrates strong overall performance across all punctuation classes.

8 LLM Evaluation Prompt↩︎

We used the prompt shown in Figure 2 to evaluate GPT-4o and GPT-4o-mini. The temperature was set to 0, and maximum tokens were set to 2048 to accommodate longer outputs. Prompts were issued in English, as we found LLMs demonstrate better instruction-following in English compared to Persian.

References↩︎