1 Introduction↩︎

Rare diseases are defined as conditions affecting 1 in 2,000 patients in the European Union [1], [2]. Due to the low prevalence of these diseases, their symptomatological similarity to common diseases, and clinical unfamiliarity, patients undergo a diagnostic odyssey, taking several years to receive the final diagnosis [1]–[3]. Machine Learning on Electronic Health Records (EHRs) can accelerate the diagnosis of patients with rare diseases and therefore help shorten the path to diagnosis. Recent examples demonstrate that applying Machine Learning (ML) algorithms alone to EHRs can identify rare diseases, thereby providing clinicians with the information necessary to perform targeted interventions [4]–[6]. However, rare diseases generally pose a challenge to ML approaches: We are normally presented with highly imbalanced datasets, resulting in increased false positives due to Bayes’ theorem. Tailored and specialized ML approaches have to be developed and tested in controlled environments to demonstrate their capabilities for the application in those cases. However, due to privacy and ethical concerns [7], access to extensive real-world data is limited and consequently poses a significant roadblock for research on the limitations and suitability of novel ML methods for EHR data of patients with rare diseases. Synthetic data helps overcome these barriers, allowing researchers to conduct rapid benchmarking and evolution of the capabilities of their ML models in a controlled environment. For example, a research group can test the ability of a classifier or outlier detection method for the early detection of rare diseases and train and evaluate it on synthetic patient data under realistic circumstances before undergoing the diligent access procedure to finally access real-world data.

We present SYNRARE, a GUI that leverages the Synthea framework [8], enabling researchers to quickly modify existing modules of common diseases to generate synthetic EHRs, including examples of slightly modified patients with a simulated rare disease. SYNRARE is readily available at https://gitlab.sdu.dk/screen4care/synrare, thereby enabling researchers to quickly set up a testbed for technical ML evaluation. It is important to note that SYNRARE is intended for the technical evaluation of ML procedures, where researcher seek to evaluate the technical capabilities of their algorithm to differentiate a minority class from a majority class with realistically appearing EHR data. Furthermore, we encourage users, especially clinical experts, to contribute to the project to capture accurate disease progression. While SYNRARE allows for the inclusion of literature-based knowledge, the data is not intended to help understand the rare disease in question, but merely to create cohorts that have a definable degree of dissimilarity from the majority class. Synthea is an open-source, rule-based synthetic data generator for patients that simulates disease trajectories for each patient from birth to the present. The framework consists of selectable disease-specific modules designed and developed by medical experts. We choose Synthea because it generates fully synthetic patient data without relying on real EHR data, unlike data-driven models such as Syntegra [9], MedGAN [10], and CTGan [11]. Synthetic data generated by rule-based data generators is preferable to data-driven models, as it allows the user to gain full control over disease trajectories and demographic parameters, thereby avoiding any biases introduced by underlying real-world data distributions. Therefore, Synthea is well-suited to systematically evaluate the performance limits of various ML models for EHR-based disease classification. The concrete contributions are as follows:

1. Generation of patients with common diseases and a subgroup of simulated RD patients with a definable degree of dissimilarity for benchmarking purposes.

2. No-code interface that reduces the barrier to entry for synthetic data generation.

3. BMI-driven comorbidity modeling based on empirical evidence.

4. Legacy module migration support.

2 Synthea Overview↩︎

To fully understand SYNRARE, it is necessary to explain Synthea, as SYNRARE depends on its data generation capabilities. Synthea generates a medical history of synthetic patients on predefined modules. These modules typically represent a disease and generate several events, including symptom onset, encounters, medications, and other clinical events. These events are also known as states. This process simulates the patient’s progression from the disease’s initial manifestation to diagnosis, eventual recovery, or death. For example, a module representing Cystic Fibrosis is a rare genetic disorder resulting in abnormally thick and sticky mucus clogging up the lungs and pancreas [12]. The majority of affected patients are diagnosed early in their age through newborn screening, while there are a few receiving a diagnosis later in their age. A sweat test is a reliable procedure for measuring the chloride level, where greater than or equal to 60 mmol/L indicates Cystic Fibrosis [12], [13]. A state where the patient receives a diagnosis can therefore be defined as a probability exponential distribution for all patients, followed by another state capturing the chloride level in a uniform distribution between 60 - 80 mmol/L. A synthetic dataset is generated based on the selected module for use in research studies. This example is simplified and serves solely to illustrate the process of synthetic patient generation. It is possible to manually edit modules to generate a rarer variant of a disease. However, this process demands a thorough understanding of Synthea’s framework, domain knowledge, and the state being modeled. Therefore, SYNRARE was developed to accelerate the generation of rarer disease variants and, therefore, enable them to establish a controlled test bed for research purposes.

3 Features, Implementation and Results↩︎

One of the main features of Synthea is to view and modify an existing module. However, a key missing component is applying a global modification to the module, meaning the current user must manually modify each state to accommodate their needs.

3.1 Installation↩︎

SYNRARE requires Python 3.8+ and Java JDK 11 to run both the software and Synthea. Additional instructions on installing and running the software are available on GitLab. After installing the software, the user can generate synthetic data based on the modules provided by Synthea.

3.2 Module Selection↩︎

SYNRARE offers three modes: Single Module, Multiple Modules, and All Modules. Single Module allows the user to select, view, and modify states of the selected module. The viewing and modification of the states are described later in this section. Multiple Modules enable simultaneous patient generation by selecting a subset of available modules, thereby supporting the creation of customized disease combinations. All Modules include all available modules and offer a quick way to generate data of patients representing the general population in a hospital. The user can load their custom module into SYNRARE to either generate synthetic data or view and modify it.

3.3 Module Viewing↩︎

SYNRARE extends Synthea’s functionality by enabling users to view and modify the underlying distribution of each state in Single Module mode, as well as the probabilities associated with these events and their respective distribution types. To be precise, these states follow a Uniform, Gaussian, or Exponential distribution or can be assigned an exact value. The parameters for the Uniform distribution are set to span the range from the lower to the upper bound. The Gaussian distribution requires both the mean and the variance. Finally, the Exponential distribution only requires the mean. A detailed overview of the module is shown in Figure 1A and 1B, allowing the user to check and modify some states.

Many legacy modules often contain missing fields or lack support for specific distributions. SYNRARE supports legacy module migration, allowing modules to be upgraded to the latest version and, therefore, enabling users to modify them accordingly.

3.4 Module Modification↩︎

The user can apply the Random Modification to adjust the magnitude of the variance, the range shift, and the probability of the modification occurring, thereby globally changing each state as depicted in Figure 1B and 1C. The modification depends on the distribution. As the variance increases, the Uniform and Gaussian distributions become taller or wider, respectively. In contrast, the Exponential distribution remains unaffected by the change. A range shift moves the distribution, whereas a Uniform distribution shifts both its lower and upper bounds simultaneously to the left or right. Additionally, the means of the Gaussian and Exponential distributions are similarly shifted, moving left or right as a result of this modification. To illustrate the magnitude of the shift, if the user sets the range shift to 2, a state with a uniform distribution in the range of 25 to 50 could, in extreme cases, be shifted to the range of 50 to 75. Therefore, the user should typically set the slide adjustment from 1 to 1.5 to apply minor modifications.

Another component of the module modification feature is the ability to adjust body mass index (BMI) to simulate known health effects across the categories: Normal, Overweight, Obesity Class I, Class II, and Class III. SYNRARE automatically updates the associated health effects. An individual with obesity shows increased risks of cardiovascular events, particularly in cases of severe obesity. Epidemiological meta-analyses indicate that overweight adults (BMI \(25–29.9\)) have roughly 20–50% higher odds of coronary heart disease and stroke, whereas obesity (BMI \(\geq\) 30) roughly doubles the risk of these outcomes [14]. Furthermore, Table 1 presents clinical evidence that the patient generally experiences more severe symptoms with increasing BMI. For instance, the systolic blood pressure of an obesity III patient is modeled as being, on average, 25 mmHg higher than that of a normal-weight patient to simulate the high prevalence of hypertension in that group. After applying any of the changes, the user receives a summary of the affected states and their updated values, which can be approved or rejected. A new module reflecting the modifications is created; however, if the user is dissatisfied with the current modifications, they may remove it.

3.5 Example case of generating a rare variant disease↩︎

To exemplify the usability of SYNRARE, consider a use case involving bronchitis and a rare variant of lung disease for a classification task. The user can modify the distribution for most states in the bronchitis module to create a close variant of the disease and thus determine the degree of similarity between bronchitis and the variant. We selected the Random modification mode and set the variance, range shift, and probability to modify all states to 1.1, 1.2, and 100%, respectively. Thereafter, the user can apply different ML models to assess their performance on binary classification tasks. Figure 1D depicts a PCA plot of bronchitis and its close variant. Both diseases are similar, making simple, interpretable models, such as Logistic Regression, difficult to apply to binary classification tasks. One could consider instead whether applying complex black-box models yields stronger performance. The user can create different scenarios to make the rare variant increasingly distinguishable from bronchitis to simulate classification tasks. The use case is not limited to binary classification tasks; it can be extended to general supervised and unsupervised tasks, such as outlier detection and clustering.

4 Discussion and Conclusion↩︎

SYNRARE is an ideal solution for researchers seeking to develop and benchmark ML models for accelerating rare disease diagnosis. An intuitive user interface enables domain experts to generate synthetic data and customize and tailor their modules for their test scenarios without compromising patient privacy, making SYNRARE an essential tool for supporting the rapid development and testing of ML approaches tailored for RD. SYNRARE enables users to generate synthetic data for their specific scenarios and subsequently rapidly benchmark ML algorithms used to assess their limitations. This software is particularly suitable for users with domain knowledge to model and generate comparable disease variants. To make the best use of SYNRARE, the user should ideally have domain knowledge to increase the realism of their disease scenario. This includes, but is not limited to, a strong understanding of disease progression, familiarity with clinical guidelines, and knowledge of clinical phenotypes that characterize the disease. Users should recognize the inherent limitations of Synthea, as synthetic data may not fully represent the complexity and variability present within real-world EHRs. In practice, EHRs, particularly those of rare disease patients, often contain missing data and confounding information [15], [16]. In contrast, Synthea generates "idealized" disease trajectories that include relevant symptoms and clinical interventions, but it nevertheless allows the systematic delineation of the capabilities of ML tools by creating datasets of varying difficulty.

5 Acknowledgements↩︎

We thank our former master’s student, Andreas Rosenstjerne, for implementing and testing SYNRARE.

6 Conflicts of interest↩︎

Nothing declared.

7 Funding↩︎

This work was supported by the Innovative Medicines Initiative 2 Joint Undertaking (JU) under grant agreement No. 101034427.

References↩︎