CrimeNER Demo: Named-Entity Recognition
in the Crime Domain

Miguel Lopez-Duran, Julian Fierrez, Aythami Morales, Daniel DeAlcala, Gonzalo Mancera, Javier Irigoyen, Ruben Tolosana, Oscar Delgado, Francisco Jurado, Alvaro Ortigosa

July 16, 2026

Abstract

We present CrimeNER Demo, an AI-powered platform that enables us to extract general crime-related information from documents and classify them into entity types with two levels of granularity. We provide pretrained NER models on the CrimeNER database, and we give the possibility to users to provide their own annotated data to train models for their own specific cases. This demonstrator aims to promote crime-related NER research and provides a practical tool to automatically extract crime information for researchers and law enforcement agencies. The demonstrator includes: i) Pretrained NER models on the crime domain; ii) Possibility to finetune the models on specific data annotated by the user; and iii) An automatic pipeline to extract and annotate crime entities from documents. The demo platform, a tutorial to run the demo, and a video demonstration are publicly available on GitHub.1

1 Introduction↩︎

Law enforcement agencies are increasingly required to extract and process information from crime-related documents. However, the number of documents that need to be processed is increasing rapidly, so manually extracting these data is not feasible. In addition, manually annotating sufficient data to train reliable models from scratch is really time consuming and inefficient [1].

Automatic extraction of criminal information from document repositories (including web/html locations) may be viewed as a Named Entity Recognition (NER) task [2]. In forensics, the kind of entity law enforcement agencies are interested in can change from case to case. However, when working in criminal cases, information like who committed or is being accused of a crime, or the agents and agencies involved in a case, are general and typically useful kind of information. NER has a lot of work done in several fields. Initially, it was focused on news and general documents, but it grew rapidly to other areas such as biomedicine [3]. In relation to the crime domain, there are works that focus on the extraction of legal entities [4] and on cyber threat Intelligence [5]. However, these works revolve around legal texts or specific types of crime, and are not suitable for general crime extraction for day-to-day criminal cases. Coarse entities are annotated with different colors, while fine entities are annotated as metadata inside the document.

Despite all this work, there is still a huge gap in the NER literature in the crime domain. As an initial step to fill this gap, we present a demonstrator of CrimeNER, a platform to extract crime-related entities from input documents. CrimeNER Demo leverages pretrained NER models in the CrimeNER database (CrimeNER-db) [6]. We label each detected entity into a predefined set of entity types with two levels of granularity, namely coarse and fine-grained entity types, as done in previous works [7]. The coarse entity types defined cover basic information about crimes, e. g., the criminal who committed the offense. Fine-grained entity types are defined to better contextualize each of the coarse entity types extracted from the document, e.g. the typology of the criminal activity mentioned or the number of criminals that committed the crime.

We acknowledge that different law enforcement agencies work with different types of crime, and in languages different from English. For this reason, we also let the users upload their own annotated data to the CrimeNER Demo following our schema and finetune the selected models on their data. In this way, only a small number of annotated samples is enough to extract meaningful crime information, avoiding the need for a large annotated database in each target domain [8], [9]

This work presents a demonstrator of the CrimeNER platform and serves researchers and practitioners in the line of Forensic Document Analysis using NER techniques.

2 CrimeNER: Database↩︎

The CrimeNER database (CrimeNER-db) is a dataset comprising more than 1.5K annotated documents from real-world scenarios, such as terrorist reports or real press notes from the US for the extraction of criminal entities.

The primary goal of CrimeNER-db is to provide researchers with a fine-grained general crime dataset. In order to do that, based on previous work on fine-grained NER databases [7], we define a two-level hierarchy for entity types, coarse and fine-grained, and classify each token as part of a coarse entity with its corresponding fine-grained entity type, or as a non-entity. The coarse and their corresponding fine entity types are as follows:

  • Crime: Illegal activities performed by an individual or group of people, including terrorist attacks. As fine-grained crime entity types, we consider: Terrorism, Fraud, Illegal Traffic, Theft, Drug-related, Homicide, Sexual crimes, Hate crimes, and Others.

  • Actor: Individuals or organizations that commit or are accused of committed a crime. The fine actor entity types are: Criminal Person, Criminal Organization, Terrorist Person, and Terrorist Organization.

  • Agent & Agency: Individuals and organizations acting against criminal activities or involved in criminal cases. We also consider government officials and bodies as this type of entity. We consider three fine entity subtypes: Law Enforcement, Government, and Legal. 2

  • Logistic: Specific details of the crimes mentioned in the documents that may be useful to agents. The fine logistic entity types are Location, GPE, Date, Weapons & Explosives, and Money.

3 CrimeNER: System↩︎

Table 1: Average Strict and Flexible F1 Scores on coarse and fine entity types after training on CrimeNER-db. Strict evaluation requires the same entity span and type to be correct. Flexible evaluation requires same entity type, and an overlap between the predicted span and the ground truth.
Model Coarse Fine
2-3 (lr)4-5 Strict Flexible Strict Flexible
XLM-RoBERTa-Base [10] \(\mathbf{0.650}\) \(0.900\) \(\mathbf{0.650}\) \(0.879\)
DeBERTa-V3-Base [11] \(0.649\) \(\mathbf{0.902}\) \(0.627\) \(\mathbf{0.892}\)
RoBERTa-Base [12] \(0.620\) \(0.899\) \(0.631\) \(0.882\)
AlBERT-Base-V2 [13] \(0.607\) \(0.881\) \(0.401\) \(0.888\)
DistilBERT-Base-Cased [14] \(0.519\) \(0.890\) \(0.643\) \(0.886\)
BERT-Base-Cased [15] \(0.514\) \(0.839\) \(0.644\) \(0.889\)
Figure 1: Example of different annotations for a sample document using the CrimeNER Demo with different model configurations. From left to right, the annotations for coarse entities were made using XLM-RoBERTa-Base [10], DeBERTa-V3-Base [11] and DistilBERT-Base-Cased [14] and the fine entity annotations were made using DeBERTa-V3-Base, XLM-RoBERTa-Base and AlBERT-Base-V2 [13]. Coarse entities are colored in the document and fine entities are annotated as metadata inside the document.
Figure 2: Overview of the CrimeNER Demo. Documents are processed with the selected model (fine-tuned if specific data is uploaded). Crime-related entities are extracted with two levels of granularity, processed, and shown to the user.

The CrimeNER Demo platform extracts crime related entities from input documents and texts with two levels of granularity. The system comprises 4 processing modules: 1) Document Preprocessing, 2) Specific Fine-tuning (if selected), 3) Coarse and Fine Entity Extraction, and 4) Postprocessing.

3.1 CrimeNER Modules↩︎

3.1.1 Document Preprocessing↩︎

Given an input document or batch of documents, the user is asked to decide whether to process the document using the pretrained NER models or to upload annotated data and finetune them on the data and then process the document. The models were trained in CrimeNER-db by performing a train/val / test split and evaluated on the test split. The results of this training with the selected NER models are shown in Table 1.

3.1.2 Specific Fine-tuning↩︎

Given the wide variety of criminal cases and languages law enforcement agencies may be interested in, we provide the option to further finetune the released criminal NER models on specific target data. This makes our platform even more useful in real-world deployment as it can be adapted to most criminal cases. The only drawback is that the target data need to be formatted as we specify in our demonstrator with the same types of entity. We believe that the types defined on CrimeNER-db are informative enough for most cases. In future work, we will consider new types of entity.

3.1.3 Coarse and Fine Entity Extraction↩︎

Given the processed documents and the pretrained or finetuned NER models, coarse entities are extracted. After that, on the basis of the extracted coarse entities, we extract the fine entities from them. All this process let us classify each entity into 4 coarse entity types and a total of 21 fine entity types, which makes the extracted information highly detailed and specific for each case.

3.1.4 Document Postprocessing↩︎

After the entity extraction, the entity spans are further processed and injected into the input documents. Input documents and texts are returned to the user with the detected crime entity spans highlighted with the detected coarse- and fine-grained entity types detected. A JSON file with the annotation of each document with its extracted entities may also be provided if requested by the user.

4 CrimeNER: Demonstrator↩︎

This study presents the CrimeNER Demo platform, a platform for extracting general criminal entities from documents. The platform was evaluated using CrimeNER-db, a database consisting of 1.5K real-world documents from the U.S. Department of Justice and other terrorist and crime reports.

The main objective of this work is to promote our platform, where we make CrimeNER Demo available to researchers and practitioners. The CrimeNER Demo allows users to pass input documents or texts and extract the crime-related entities with two levels of granularity within those documents. After processing the documents, users will receive the input documents with the extracted crime entities annotated and highlighted, as shown in Figure 1. This platform opens up new opportunities in document analysis and crime detection for researchers and law enforcement agencies.

We provide several pretrained models to extract the crime-related entities. The selection of different models for coarse and fine entities results in different annotations, as shown in Figure 1. Qualitative analysis shows that the difference between model annotations is notable, especially the differences in span length for the same entity. There is some misalignment between the annotation and the PDF text, as the positions in the PDF metadata are not exactly aligned with the visual text position in the document.

Figure 2 shows a screenshot of the CrimeNER demonstrator and its main features that we discussed earlier. Initially, users are required to upload one or more documents and select the NER models trained to process the documents. Among these models, there can be the finetuned models on specific data uploaded by the user, for which we provide a tutorial on the CrimeNER demo repository. After model selection, coarse and fine entity types spans are extracted from the documents as we described earlier. Finally, the extracted entities are annotated and highlighted in the documents, and a JSON file with the entity annotations is generated if the user requires it. The CrimeNER platform also stores previous document analysis as part of a history so that users can download documents and annotations already processed again, if necessary.

In future work, we will explore multimodal architectures [16], including the combination of NLP models like those used here and visual models that process text images [17], [18]. Detecting AI-generated information [19], fakes [20], other types of manipulation [21], and other general risks [22] when analyzing document repositories are also key topics on our agenda.

4.0.1 ↩︎

PID2024-160053OB-I00 MICIU/FEDER, Cátedra ENIA (NextGenerationEU PRTR TSI-100927-2023-2), and R&D Agreement DGGC / UAM/FUAM for Biometrics & Applied AI. Lopez and Robledo are supported by FPI-UAM-2025, DeAlcala is supported by FPU21/05785, Mancera is supported by FPI-PRE2022-104499, Irigoyen is supported by FPI-PREP2024-003107.

References↩︎

[1]
A. Peña et al., “Continuous document layout analysis: Human-in-the-loop AI curation, database & evaluation in the domain of public affairs,” Inf. Fusion, vol. 108, 2024.
[2]
G. Mancera, A. Morales, J. Fierrez, et al., “PBa-LLM: Privacy-and bias-aware NLP using Named-Entity Recognition (NER),” in ICDAR workshops, 2025.
[3]
B. Song et al., “Deep learning methods for biomedical named entity recognition: A survey and qualitative comparison,” Brief. Bioinform., vol. 22, no. 6, p. bbab282, 2021.
[4]
T. W. T. Au, V. Lampos, and I. Cox, “E-NER: An annotated named entity recognition corpus of legal text,” in Proc. Of the NLP workshop, 2022, pp. 246–255.
[5]
X. Wang, S. He, Z. Xiong, X. Wei, et al., “APTNER: A specific dataset for NER missions in cyber threat intelligence field,” in IEEE CSCWD, 2022, pp. 1233–1238.
[6]
M. Lopez-Duran, J. Fierrez, et al., “Named-entity recognition in the crime domain (CrimeNER): Case study and dataset,” arXiv:2603.02150, 2026.
[7]
N. Ding, G. Xu, Y. Chen, X. Wang, X. Han, et al., “Few-NERD: A few-shot named entity recognition dataset,” in Proc. ACL, 2021, pp. 3198–3213.
[8]
G. Mancera, D. DeAlcala, A. Morales, J. Fierrez, et al., “Auditing training data in domain-adapted LLMs: LoRA-MINT,” in IEEE COMPSAC, 2026.
[9]
M. Lopez-Duran, E. Marrero, et al., “Comparative study of domain-adapted VLMs for general document visual question answering,” in ICDAR workshops, 2026.
[10]
A. Conneau et al., “Unsupervised cross-lingual representation,” in ACL, 2020.
[11]
P. He et al., “DeBERTa: BERT with disentangled attention,” in ICLR, 2021.
[12]
Y. Liu et al., “RoBERTa: A robustly optimized BERT pretraining approach,” CoRR, vol. abs/1907.11692, 2019, [Online]. Available: https://arxiv.org/abs/1907.11692.
[13]
Z. Lan, M. Chen, S. Goodman, K. Gimpel, et al., “ALBERT: A lite BERT for self-supervised learning of language representations,” CoRR, vol. abs/1909.11942, 2019, [Online]. Available: https://arxiv.org/abs/1909.11942.
[14]
V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter,” ArXiv, vol. abs/1910.01108, 2019.
[15]
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” CoRR, vol. abs/1810.04805, 2018, [Online]. Available: https://arxiv.org/abs/1810.04805.
[16]
A. Peña et al., “Human-centric multimodal machine learning: Recent advances and testbed on AI-based recruitment,” SN Computer Science, vol. 4, no. 5, p. 434, 2023.
[17]
M. Lopez-Duran, J. Fierrez, et al., “Benchmarking graph neural networks for document layout analysis in public affairs,” in ICDAR workshops, 2025.
[18]
D. DeAlcala, G. Mancera, J. Fierrez, et al., “Is my vision-language data in your AI? Membership Inference Test (MINT) Demo 2,” in IEEE COMPSAC, 2026.
[19]
D. DeAlcala, C.-Y. Ko, A. Morales, J. Fierrez, et al., “DocAI: A framework for generating and identifying AI-edited documents,” in arXiv preprint, 2026.
[20]
J. Muñoz et al., “Privacy-aware detection of fake identity documents: Methodology, benchmark, and improved algorithms (FakeIDet2),” Inf. Fusion, vol. 128, p. 103969, 2026.
[21]
P. Korshunov et al., “DeepID challenge of detecting synthetic manipulations in ID documents,” in IEEE intl. Conf. On computer vision workshops, 2025.
[22]
J. Irigoyen et al., “Overview of risk assessment and management for intelligent systems under the AI Act and beyond,” in IEEE ICCST, 2026.

  1. https://github.com/BiometricsAI/CrimeNER.git↩︎

  2. In the original CrimeNER-db work, the Agent and Agency coarse entities are different types, but we decided to merge them into one, as we found it more informative.↩︎

Subjects

Artificial Intelligence (cs.AI)

Updated on Academus

2026/07/17 18:37 UTC