Mental Health Disorder Detection Beyond Social Media:
A Systematic Review of Available Datasets
July 03, 2026
The prevalence of mental health disorders is a global concern. In the USA, for example, one in every four adults experiences a diagnosable mental health disorder each year1. Furthermore, research shows that the majority of individuals who die by suicide have an identifiable mental health condition such as depression [1] or substance use disorder [2].
The limited access to mental health care services has become an urgent societal issue [3], [4]. This motivated research on applying NLP and ML models to social media data for early identification of mental disorders and supporting individuals at risk. Social media platforms such as Twitter ([5]; [6]), Reddit ([7]; [8]) and Facebook ([9]; [10]) offer a rich, unobtrusive stream of user-generated content that captures real-life expressions with reduced reporting bias, aiding in the detection and understanding of mental disorders [11]–[13].
Using social media data for mental health screening raises ethical and privacy concerns regarding user consent, construct validity, and the potential for algorithmic misuse [14]–[16]. Collecting data from social media can lead to unintended biases as the data may only reflect the experiences of individuals who are willing to openly discuss their mental health online, which primarily includes those who are active on social media platforms [17]. Finally, demographics vary across platforms and they may not be representative of the general population. For instance, X users are primarily male, while TikTok and Instagram users are mostly female; likewise, many platforms tend to be used by teens and young adults [18], [19]. Pragmatically, many social media sites have limited access to APIs for research.
The widespread use of social media data in this domain is partly due to the significant shortage of non-social media free-text based datasets related to mental health disorders derived from clinical and other reliable sources. Clinical data, such as electronic medical records (EMRs) or electronic health records (EHRs) with discharge summaries or clinical notes, psychiatric interview transcripts, and responses to standardized open-ended questionnaires, offer rich, detailed insights into patients’ mental health. Unlike social media data, these sources contain carefully documented information from healthcare professionals, including diagnostic details, symptom descriptions, and treatment histories, often supported by validated clinical scales.
Despite their great potential, non-social media free-text based datasets remain underexplored primarily due to privacy concerns, data access challenges, and annotation complexities. This scarcity presents a barrier to advancing robust, generalizable NLP models that can be effectively integrated into clinical practice. We aim to address this gap by systematically reviewing non-social media free-text based datasets for mental health research. We explore their diversity in terms of source, structure, clinical annotation, types, and research adoption to guide future efforts in dataset development and application. Previous related surveys have primarily focused on datasets from social media platforms [20]–[25]. To the best of our knowledge, this is the first systematic review2 [26] of free-text based mental health data sources beyond social media.
We address the following research questions:
What non-social media free-text based datasets are available for mental health research, and how do they vary by source, structure, and population?
How are mental health conditions defined and labeled in these datasets, and what are the clinical implications of these labeling methods?
What factors contribute to the popularity and adoption of non-social media free-text based mental health datasets in research?
This systematic review adopts the PRISMA3 reporting guidelines to comprehensively map and systematically analyze the landscape of the free-text based datasets in mental health research beyond social media, building on prior frameworks (e.g. [27]–[29]). PRISMA is a standardized guideline aimed at ensuring clear and thorough reporting of systematic reviews and meta-analyses. It features a 27-item checklist that helps authors include all essential components of their review, from initial identification to final conclusions. A key part of PRISMA is its flow diagram (Figure 1), which visually maps the process of selecting studies, making the review process more transparent and easy to follow [30].
To follow the checklists of PRISMA, we began with a systematic literature search that continued through June 2025. We used Publish or Perish4 to query multiple academic databases, including Google Scholar, and PubMed. The searches were carried out separately for key mental health related terms: ‘depression’, ‘anxiety’, ‘suicidal ideation’, ‘suicide’, ‘mental disorder’, ‘mental health crisis’, and ‘mental health disorder’, combined with targeted keywords ‘identification’, ‘detection’, ‘prediction’, and ‘analysis’ to capture each relevant disorder type of mental health and associated dataset corpus. We then manually screened the abstracts and included studies that explicitly referenced the use of any datasets or the application of any computational models or techniques after de-duplication, resulting in 543 papers within our initial scope. The rationale for selecting depression and anxiety as key terms is that they are among the most prevalent mental health disorders worldwide [31]. Additionally, given that suicide remains a leading cause of death globally among adults [32], we prioritize the inclusion of these mental health conditions in our study.
To ensure focus on non-social media sources, we excluded papers whose titles or abstracts included keywords related to social media platforms or mentions of datasets scraped from them, such as ‘social media’, ‘Reddit’, ‘Facebook’, or ‘Twitter’. This filtering was essential to isolate papers based on clinical notes, semi-structured interviews, and discharge summaries from other non-social media platforms, and we ended up with 259 total papers after this filtering. Subsequently, we conducted corpus novelty screening to remove studies based on pre-existing datasets, followed by modality screening to exclude papers that focused solely on audio or visual data, and data type screening to filter out datasets containing only numerical or categorical values, such as scale-based survey responses. Through this multi-phase screening, we identified 57 relevant papers, of which 40 proposed or analyzed newly collected free-text based mental health datasets.
In addition to keyword-based search, we followed a backtracking strategy; whenever a paper used an existing dataset, we traced it back to the original dataset publication, even if it did not contain our predefined keywords. We also examined all papers cited in the literature review or background sections of included studies and explored any dataset-related papers mentioned in relevant survey papers found from the initial search. As a result, our final collection may include important dataset papers whose titles or abstracts do not directly match the initial search terms, but were directly aligned with our objectives. This process yielded 5 additional papers, for a total of 45.
Several datasets were initially annotated for mental disorders, but their final task focused on emotion classification, including loneliness, fear, anger, hopelessness, and self-identification. Since emotions like loneliness and hopelessness can serve as indicators of depression [33] or suicidal ideation [34], we decided to include these studies in our survey.
In this section, we provide information on the 45 free-text based datasets included in this systematic review. Table 1 provides a comprehensive summary of the final set of datasets. The datasets were published between 2004 and 2025 in multiple languages, including English, Chinese, Polish, Korean, Japanese, Arabic, and some code-mixed. These datasets cover a broad spectrum of mental health conditions, such as depression, postnatal depression, anxiety, schizophrenia, suicidal ideation, Post-Traumatic Stress Disorder, and bipolar disorder, collected from platforms like clinics, colleges, mobile apps, and therapy sessions. Data types include interview transcripts, essays, discharge summaries, clinical records, forum posts, and suicide notes. While some datasets are publicly available, others are restricted or require agreements, with a few lacking availability details. We show their distribution below across disorders, languages, platforms, data types, demographics, and availability types.
| Dataset | Language | |||||||||
| Disorder | Platform | |||||||||
| Type | ||||||||||
| Procedure | ||||||||||
| Instrument | Label | Size | Availability | Citation | ||||||
| [35] | EN | SD | CLINIC | SD Notes | Manual | Krippendorff’s \(\alpha\) | 15 | 1004 | DUA | 236 |
| [36] | EN | DP, PTSD, OCD, BD | CLINIC | Clinical Notes | Manual | - | 4 | 816 | DUA | 45 |
| [37] | EN | SMI | CLINIC | EHR(DS) | Manual | Text Hunter | 50 | 37,211 | RSTR | 233 |
| [38] | EN | MDD | CLINIC | Interview | Manual | LIFE | 2 | 139 | RSTR | 313 |
| [39] | EN | SD | CLINIC | SD Notes | Manual | Ontology | 2 | 66 | RSTR | 322 |
| [40] | EN | MDD & DD | CLINIC | EMR | Manual | DSM-IV | 2 | 861 | RSTR | 88 |
| [41] | EN | DP | CLINIC | EHR (DS) | Manual | - | 3 | 1200 | UNK | 74 |
| [42] | EN | DP | CLINIC | EHR | Manual | ICD-9 & AD | 2 | 10,148 | UNK | 53 |
| [43] | EN | DP & AX | CLINIC | Interview | Manual | DSM-IV | - | 308 | UNK | 51 |
| [44] | EN | SZ | CLINIC | Essay | Manual | - | 2 | 56 | UNK | 71 |
| [45] | EN | SD | CLINIC | Clinical Notes | Manual | - | 3 | 210 | UNK | 233 |
| [46] | EN | |||||||||
| SD & Other | CLINIC | |||||||||
| Clinical Notes | Manual | - | 2 | 198 | UNK | 26 | ||||
| [47] | EN | MHD | FORUM | Post | Manual | |||||
| Cohen’s Kappa | 4 | 1227 | PUB | 142 | ||||||
| [48] | EN | PTSD | FORUM | Essay | Manual | DSM-IV & CAP Scale | 2 | 300 | UNK | 95 |
| [49] | EN | DP & AX | FORUM | Post | Self-disclosure | - | 2 | 16,975 | UNK | 55 |
| [50] | EN | DP, BD & SD | FORUM | Post | Self-disclosure | - | 2 | 267,964 | UNK | 303 |
| [51] | EN | SZ & BD | COLLEGE | Interview | Manual | DSM-V & DSM-IV | 3 | 644 | DUA | 11 |
| [52] | EN | DP | COLLEGE | Essay | Manual | BDI & IDD-L | 3 | 124 | UNK | 1543 |
| [53] | EN | AX | DEBATE | Political Speech | Manual | - | 2 | 4000 | PUB | 17 |
| [54] | EN | DP & AX | Telemedicine Platform | Message | Manual | |||||
| GAD-7 | ||||||||||
| GAD-7 | 10,718 | DUA | 43 | |||||||
| [55] | EN | PD | APP | Essay | Manual | EPDS | 2 | 1,091 | UNK | 14 |
| [56] | EN | DP & SD | Phone | SMS | Manual | Self-Disclosure | 2 | 94 | UNK | 127 |
| [57] | EN | DP & AX | Online Therapy Chat | Dialogue | Mixed | MALLET & LIWC | 2 | 882 | UNK | 78 |
| [58] | EN | DP & PTSD | Virtual Agent | Interview | Manual | PHQ-8 | 5 | 275 | DUA | 402 |
| [59] | EN | DP & SD | MIXED | SD Notes & Articles | Manual | - | 12 | 426 | UNK | 64 |
| [60] | EN | DP, AX & PTSD | MIXED | Interview | Manual | - | - | 621 | DUA | 759 |
| [61] | EN | DP & SDI | MIXED | SD Notes & Book | Manual | Cohen’s Kappa | 15 | 2393 | PUB | 45 |
| [62] | EN | DP & AX | mTurk | Interview | Manual | |||||
| GAD-7 | ||||||||||
| GAD-7 | 2674 | UNK | 29 | |||||||
| [63] | EN-ES | DP & SDI | Social Network & Forum | Post | Manual | Cohen’s Kappa | 4 | 102 | PUB | 24 |
| [64] | EN-ZH | |||||||||
| BD, AjD, DEM & Other | CLINIC | EHR(DS) | Manual | |||||||
| Sheehan Disability Scale | 9 | 4,836 | RSTR | 122 | ||||||
| [65] | ZH | MDD | CLINIC | Interview | Manual | HAMD & PHQ-9 | 2 | 78 | PUB | 48 |
| [66] | ZH | DP & AX | CLINIC | Interview | Manual | HAMD & HAMA | 3 | 1025 | PUB | 9 |
| [67] | ZH | DP & AX | CLINIC | EMR | Manual | DSM-V & ICD-10 | 4 | 1,160 | PUB | 1 |
| [68] | ZH | DP | CLINIC | Interview | Manual | MADRS | 2 | 113 | DUA | 5 |
| [69] | ZH | AX | CLINIC | EHR | Manual | ICD-9 & ICD-10 | 2 | 84,426 | RSTR | 0 |
| [70] | ZH | DP & SD | CLINIC | Interview | Manual | HAMD | 3 | 305 | UNK | 10 |
| [71] | ZH | DP | APP | Interview | Manual | SDS | 2 | 162 | PUB | 146 |
| [72] | ZH | AX | Phn. Recording | Essay | Manual | GAD-7 | 3 | 227 | UNK | 11 |
| [73] | KO | DP, AX & SD | CLINIC | Interview | Manual | |||||
| BAI & BSS | 2 | 166 | DUA | 22 | ||||||
| [74] | KO | SZ & FEP | CLINIC | Interview | Manual | DSM-V, PANSS | 3 | 133 | RSTR | 33 |
| [75] | PL | SZ | CLINIC | Interview | Manual | ICD-10 | 2 | 94 | UNK | 20 |
| [76] | ES-CL | DP | CLINIC | Interview | Manual | DSM-V | 8 | 451 | UNK | 2 |
| [77] | TH | DP | FORUM | Post | Manual | Key-word Search | 2 | 944 | PUB | 18 |
| [78] | JA | DP | FORUM | Post | Self-disclosure | - | 2 | 108 | UNK | 29 |
| [79] | AR | DP | FORUM | Post | ||||||
| Manual | ||||||||||
| QIDS-SR | 2 | 20,000 | UNK | 75 | ||||||
| [80] | EN | SD | Online DB | Song Lyrics | Manual | - | 2 | 810 | UNK | 37 |
| [81] | EL | SDI | - | POEM | Manual | - | 2 | 90 | UNK | 12 |
| [82] | CF | DP, AX & PA | Tablets | - | - | - | - | - | UNK | 50 |
Temporal Distribution Figure 2 illustrates the number of free-text based datasets proposed and analyzed in published studies each year between 2004 and 2025. The introduction of such datasets remained minimal and steady up to 2012. In 2014 and 2017, there was a noticeable rise in the number of proposed datasets, indicating intensified research efforts in this domain during that period. The proposal of such datasets peaked in 2022, which can be attributed to greater attention to mental health concerns after the COVID-19 pandemic [25]. The lower count in 2025 is probably a result of the systematic search being conducted until June 2025. Despite this overall growth, the process of collecting such datasets often involves complex procedures, including ethics reviews, annotator training, and repeated permission requests, which may explain the limited number of datasets over the years.
Figure 3 illustrates the distribution of the datasets by language group. English (EN) accounts for the majority, comprising 62.2% of the datasets, highlighting its predominant role in this research area. Chinese (ZH) represents 17.8% of the total. The ‘Other’ group includes a variety of less-represented languages, such as Korean (KO), Polish (PL), Greek (EL), Japanese (JA), Thai (TH), and Arabic (AR); collectively representing another 13.3%. Additionally, 6.4% of the datasets fall under the “Code-mixed” category, which includes multilingual combinations like EN-ES (English-Spanish), EN-ZH (English-Chinese), and ES-CL (Spanish-Chilean). While there is some linguistic variety, English datasets dominate in mental health research, indicating a need for more inclusive and multilingual dataset development.
We consider the distribution of datasets for different mental health disorders and language diversity. In Figure 4, we present a heatmap with the distribution of mental health datasets across different disorders and languages. Since some datasets cover more than one disorder, the total count of disorders is higher than the number of datasets. To make things clearer, we group similar disorders; depression related conditions, grouped under ‘DP’ (including Depression (DP), Major Depressive Disorder (MDD), Minor Depression (MinDP), and Postnatal Depression (PD)) are the most studied, appearing in 33 datasets. Anxiety (AX) and suicidal ideation/suicide(SD) follow with 12 and 11 datasets, respectively. Other less common disorders, such as Adjustment Disorder (AjD), Obsessive–compulsive disorder (OCD), Dysthymic Disorder (DD), and Dementia (DEM) are grouped under ‘Others’; each accounts for 8 datasets. Schizophrenia (SZ) and First-episode Psychosis (FEP), grouped together as ‘SZ’, are analyzed in 5 datasets, while Post-Traumatic Stress Disorder (PTSD), and Bipolar Disorder (BD) are the least represented ones.
English datasets dominate the landscape, especially for depression (DP) and suicidal ideation (SD). Code-mixed and non-English data remain underrepresented across most disorders. This highlights a concentration on depression and a linguistic imbalance, emphasizing the need for more diverse mental health datasets.
Figure 5 shows the distribution of data types across four platform types: CLINIC, FORUM, MIXED (from multiple sources), and OTHER (including apps, virtual agents, online therapy chat, telemedicine platform and suicide notes). For simplicity, we group similar data types, like Electronic Health Records (EHR), Electronic Medical Records (EMR), clinical records (CR), and Discharge Summaries (DS) are grouped under ‘EHR’; essays and questionnaires fall under ‘questionnaires’ category; while types like interviews and posts are kept distinct.
As shown in Figure 5, clinical datasets mainly stem from interviews (48%) and EHRs (39%), reflecting structured clinical data sources. Forum datasets are heavily composed of user-generated posts (86%). Mixed platform datasets are more balanced, with one dataset each from interviews, posts, questionnaires, and other forms (each 25%). In the “Other” category, data forms are diverse: interviews and unconventional sources (like phone recordings, SMS, app data) both account for above 80%, and questionnaires make up the remaining.
Figure 6 displays the distribution of citation counts (as per Google Scholar5) for mental health datasets categorized by their availability: datasets that are publicly available (PUB), datasets that require a Data Use Agreement (DUA), datasets labeled as Restricted (RSTR), and datasets with unspecified availability (UNK). Citation counts are normalized by year to account for differences in publication age. Datasets categorized as DUA and RSTR generally have higher citation counts compared to those that are publicly accessible or have unclear availability. Notably, DUA datasets show a wider range and a higher median citation rate, while RSTR datasets demonstrate a more consistent citation pattern. In contrast, PUB and UNK datasets have lower medians and are more tightly clustered around fewer citations, though some outliers exist.
The citation trends can also be partly explained by language distribution (Figure 7).
Many English datasets are in the UNK category, while Chinese datasets are more publicly available (PUB). However, English-language PUB datasets rarely include depression-related data, and none originate from clinical settings. In contrast, English-language DUA datasets predominantly focus on depression and are collected in clinical contexts – this contributes to their higher credibility and research utility. Then again, most academic research is conducted in English; the higher citation rates of DUA and RSTR datasets reflect language dominance. Meanwhile, the lower citation rates for PUB datasets may be influenced by their association with less widely used languages like Chinese.
Figure 8 shows how frequently different demographic attributes appear in the datasets shown in this survey.
Demographic attributes like gender and age dominate, appearing in over 24 datasets, while others like education, race, and ethnicity are moderately included. Fewer datasets report attributes like income, relationship status, or health indicators (e.g., height, weight, blood pressure). Some authors [38], [78] use demographic information to select negative samples that match the same profile type as the positive ones, which is rarely done in social media-based data, resulting in poor performance on the minority class [83], [84]. Some datasets focus on specific populations such as students [52], [56], [71], veterans [45], [60], seafarers [72], new mothers [55], or politicians [53]. Some have used demographic features in classification tasks, often reporting improved model performance [42], [62], [85].
Most datasets in our review are manually annotated by annotators or expert clinicians, following widely used tools and techniques. This section describes these datasets and the tools employed. Some datasets, such as those by [49], [50], [78], and [79], are based on self-reported user content, typically collected from forums where individuals voluntarily share their thoughts under relevant discussion threads.
Clinical diagnostic tools are assessments carried out by qualified healthcare professionals using standardized frameworks. They typically involve face-to-face evaluations, structured interviews, and expert judgment to ensure reliable and consistent mental health diagnoses. This systematic review highlights multiple studies that utilized clinical diagnostic tools for labeling and assessment.
The DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition) [86] and its updated version, the DSM-V [87], offer standardized criteria for diagnosing a wide range of mental health conditions. Diagnoses based on DSM are often recorded during clinical visits and stored in EHRs, making them a key source of labeled data. Several works in this survey rely on DSM-IV for annotation, including [40], [48], [43] and [64], while studies such as [67], [79], and [76] utilize the updated DSM-V guidelines. Notably, [51] incorporated both frameworks to account for changes in diagnostic criteria across versions.
The ICD (International Classification of Diseases), particularly ICD-9 and ICD-10 [88], is used for coding clinical diagnoses, including mental and behavioral disorders. While the DSM focuses on mental health, the ICD covers all diseases. For instance, the code F32.9 in ICD-10 represents an unspecified single episode of major depressive disorder. In this survey, ICD-9 is used by [42], while [67] and [75] utilize ICD-10 for data annotation, and [69] uses both editions to label anxiety.
The LIFE (Longitudinal Interval Follow-Up Evaluation) is a structured method used to monitor the long-term progression of psychiatric conditions, typically every six months, particularly within longitudinal studies [89]. [90] underlines that LIFE is used in both clinical practice and research to assess the long-term impact of psychiatric disorders. In this survey, only [38] uses LIFE to classify the interviews.
The CAPS (Clinician-Administered PTSD Scale) is a structured interview for diagnosing PTSD. It assesses 20 core PTSD symptoms, onset, duration, impairment, and dissociative features related to a specific traumatic event and is considered the gold standard for PTSD evaluation [91]. Clinicians in [48] use both the DSM-IV PTSD module and the CAPS scale to annotate the dataset with binary labels for PTSD presence or absence.
The HAMD (Hamilton Depression Rating Scale) [92] is a clinician-administered scale used to assess depression severity through patient interviews, including 17 core items that rate depression from mild to severe. Similarly, HAMA (Hamilton Anxiety Rating Scale) [93] evaluates anxiety symptoms. In this survey, [65], [70], [73] use HAMD to annotate the dataset for MDD or suicidal ideation, while [66] use both HAMD and HAMA for depression and anxiety labeling.
The MADRS (Montgomery-Åsberg Depression Rating Scale) [94] is a clinician-administered tool for assessing depression severity, emphasizing core symptoms. It is often used in clinical trials and is sensitive to changes in symptoms. [68] uses MADRS to identify depression from transcribed interviews.
Screening questionnaires and self-report scales are tools where individuals assess their own mental health by answering standardized questions. These are often based on validated clinical criteria for quickly screening for symptoms. Several studies included in this review employ screening questionnaires and self-report scales for mental health disorder identification.
Some studies in this review, [62], [65], [79], [54] and [73] use the Patient Health Questionnaire (PHQ-9) [95] for dataset annotations and defining gold standard labels for depression. [58] uses the older version of PHQ-9, PHQ-8, which consists of the same questions, excluding the suicidal ideation item [96]. [52] employs the Inventory to Diagnose Depression–Lifetime (IDD-L) [97] to analyze college students’ essays, identifying depression through their reflections on past experiences. The Beck Depression Inventory (BDI-II) [98], a self-reported tool used to assess the intensity of depressive symptoms, [52] employs both BDI-II and IDD-L for dataset annotation. In addition, the Beck Anxiety Inventory (BAI) [99] and the Beck Scale for Suicide Ideation (BSS) [100] are used to measure anxiety and suicidal thoughts, respectively, with [73] applying both to label transcribed interviews. In a study by [55], the Edinburgh Postnatal Depression Scale (EPDS) [101] is used to identify depression in essays collected from pregnant women through an app. [64] combines Sheehan Disability Scale [102] with DSM-IV guidelines to identify major depressive disorder (MDD), schizophrenia (SZ), bipolar disorder (BD), and other psychiatric conditions from discharge summaries, extracted from patients’ EHRs; whereas [71] uses Zung Self-Rating Depression Scale (SDS) [103] to label depression from transcribed interviews collected through an app. [62], [54] and [72] use the GAD-7 (Generalized Anxiety Disorder-7) [104] to classify anxiety from their respective datasets. [79] uses the Quick Inventory of Depressive Symptomatology- Self-Report (QIDS-SR) [105] tool to label depression in posts from a psychological forum in Arabic.
Several datasets in this review do not explicitly mention the annotation tools used; however, they do indicate that trained annotators or clinicians conducted the labeling [36]. In some instances, studies report inter-rater agreement measures like Cohen’s Kappa (e.g. [47], [61], [63]), Fleiss’s Kappa [47], or Krippendorff’s \(\alpha\) [35] to justify the labeling decisions, and occasionally a third annotator was used to resolve conflicts. A few datasets generate gold-standard labels using NLP-based methods such as Text Hunter [37], custom ontologies [39], or frameworks like MALLET and LIWC [57]. [77] uses keyword search (i.e., “depression",”anxiety") to label the datasets.
Few miscellaneous datasets contain song lyrics [80], poems [81] from artists who have committed suicide, even ancient texts scraped from Babylonian tablets [82]; the authors do not mention using any labeling techniques.
Different labeling strategies in mental health datasets carry distinct clinical implications. Clinical diagnostic tools are widely considered the most reliable or gold standard [106], offering standardized and consistent evaluations that support accuracy across clinicians6. However, these tools have limitations, including time constraints (e.g., typical sessions lasting 15–20 minutes), limited accessibility (shortage of clinicians), and missed opportunities for monitoring between appointments (often referred to as “clinical whitespace") [107].
To address these gaps, self-report tools have become increasingly valuable. They capture patients’ first-hand perspectives and are ideal for digital use [106], especially when clinicians are unavailable, such as during clinical whitespace periods [108] or while patients await intake appointments [109]. While self-reports offer efficiency and scalability, they should not replace in-person evaluations, as they are susceptible to social desirability effects, recall bias [110], and trust issues. Instead, they should serve as supplementary tools [106]. In fact, research supports combining clinician ratings and self-reports for a more comprehensive understanding of patient conditions [111].
When clinical labels are unavailable, manual annotation, together with annotator agreement techniques, is often used to scale data labeling. While useful, these methods can suffer from reduced accuracy if not guided by trained professionals [112]. Overall, each labeling method presents trade-offs between scalability, clinical rigor, and data quality.
The studies reviewed in this systematic review have been used to develop mental health classification and prediction systems using a broad spectrum of approaches [20], [21], [25]. While a thorough review of computational approaches is outside the scope of this survey, in this section, we provide the reader with a brief summary of the computational models applied to the task. Exploring computational models allows us to better understand prevailing methodological trends, evaluate benchmarks, and identify potential limitations or biases inherent in different approaches. This insight is vital for ensuring replicability and shaping future research directions.
Early works use manual or qualitative techniques while later studies applied traditional machine learning (ML) models like support vector machine (SVM) [35]–[37], [39], [44], [48], [49], [56], Logistic Regression [35], [42], [51], [57], [62], [70], [72], [78], Decision Trees [41], [48], [71], Adaboost [36], [39], [66], [72], XGboost [68], [75], [76], and Naive Bayes [48], [59], [61], [78]. Traditional ML classifiers are often supported by feature engineering techniques such as Linguistic Inquiry and Word Count (LIWC) [113] analysis [52], [56], [57], [70], Term Frequency-Inverse Document Frequency (tf-idf) [40], [56], [69], or keyword extraction. Deep learning models, particularly Convolutional Neural Networks (CNN) [36], [49], [61], Long Short-Term Memory (LSTM) [36], [55], [68], [77], and Gated Recurrent Unit (GRU) [37], [114] are also used to capture nuanced patterns in unstructured text.
Some datasets have been used to develop interpretable models or hybrid systems that combine rule-based methods with ML (e.g., Conditional Random Field (CRF) [35], [64], TextHunter + ConText [37]). More recent work adopts transformer-based models like Clinical-BigBird [69], MentalBERT [51], MentalRoBERTa [51], and large language models (LLMs) like Qwen2-72B [67], highlighting a shift from fine-tuning pre-trained language models to using instruction-tuned LLMs for domain-specific tasks with little or no fine-tuning.
This section summarizes the main findings of this review along with suggestions for future directions. Unsurprisingly, most available datasets are in English. Chinese follows in prevalence, while there are very few datasets in other languages, such as Korean, Arabic, or Polish. There are no datasets in low-resource languages. This gap may be partly due to the complexity of creating these datasets, particularly in clinical contexts. Collecting data often requires patient consent, approval from ethics boards, and substantial manual effort to annotate datasets.
Another important finding is that depression is the most studied mental health disorder within these datasets. Studies either focus exclusively on depression or include related conditions such as anxiety and bipolar disorder. Other conditions, such as PTSD, schizophrenia, and eating disorders, are not well represented. This highlights the need to develop datasets that encompass a broader range of mental health issues. On the other hand, compared to the number of social media datasets [115], [116], clinical datasets are scarce due to their sensitive nature. Releasing more public or even agreement-governed clinical datasets could expand research opportunities and improve reproducibility.
Moreover, the annotation procedures and labeling techniques vary across the datasets. Some use clinical diagnostic tools such as DSM and ICD, while others rely on self-reported questionnaires like PHQ and BDI. However, most papers do not explain why a specific tool was chosen, indicating a lack of reporting standards [16]. Additionally, some datasets are labeled manually or through keyword searches, and a few do not clarify their labeling methods at all. This lack of transparency can make it difficult to trust or replicate the research findings.
Advancements in NLP offer promising solutions to the challenges posed by inconsistent and opaque labeling practices in mental health datasets. With instruction-tuned language models and structured prompting techniques, NLP can support (semi-)automation and standardization of labeling based on formal diagnostic criteria such as DSM or ICD. Approaches like chain-of-thought (CoT) prompting [117] can emulate clinical reasoning and even improve labeling accuracy [118], while chain-of-empathy (CoE) [119] frameworks are well-suited for understanding emotionally nuanced texts, such as therapy transcripts or suicide notes. These strategies can enhance both the consistency and interpretability of labels, enabling the creation of scalable and clinically relevant datasets, even when direct clinician involvement is limited.
In this paper, we presented the first systematic review of mental health free-text datasets beyond social media. We analyze these datasets with respect to different dimensions such as their distribution in terms of languages and mental health disorders, the data types included in them, and their availability.
We revisit the research questions (RQ) posed in the introduction (Section 1) and present the main findings of our review below:
What non-social media free-text based datasets are available for mental health research, and how do they vary by source, structure, and population?
RQ1 Findings: Most datasets are in English and primarily focus on depression, while other languages and mental health conditions remain underrepresented. More Chinese datasets are publicly available than English ones. In terms of data collection, some actively balance samples by selecting equal numbers from the control group matching the same profile type. The nature of text-based data also varies widely, ranging from phone recordings and messages to essays, interview transcripts, and suicide notes, mostly collected in clinical settings.
How are mental health conditions defined and labeled in these datasets, and what are the clinical implications of these labeling methods?
RQ2 Findings: Clinical diagnostic tools, screening questionnaires, self-report scales, and manual contextual labeling have been used to annotate mental health disorders. These labeling techniques can complement one another and collectively serve as gold standards.
What factors contribute to the popularity and adoption of non-social media free-text based mental health datasets in research?
RQ3 Findings: Along with dataset availability, factors such as the data collection setting (clinical vs. informal), language, data type (interviews, EHRs, essays), mental health disorders covered, and the credibility and reliability of the labeling and annotation methods all contribute to the popularity and adoption of non-social media free-text based mental health datasets in research.
There is significant room for improvement in the development and use of these types of datasets. Expanding dataset creation to include more languages and geographic regions can help address current imbalances. Additionally, increasing the representation of a broader range of mental health disorders, not just depression, would make research findings more comprehensive. Establishing clear and consistent labeling methods is essential for ensuring reproducibility and building trust in research outcomes, and advances in NLP, particularly in explainable AI, can help bridge these gaps. Lastly, improving access to high-quality datasets, while maintaining ethical and privacy standards, can facilitate collaborative research.
In this systematic review, we used the PRISMA methodology and conducted a comprehensive literature search using the Publish or Perish software. This approach ensured thorough coverage of datasets on mental health disorders across both the NLP and clinical domains. Although we used mental health-related terms for our keyword searches, it is possible that studies may use alternative terms or less common phrases to describe mental health disorders, which may not have been included in our search strategy. As a result, some relevant works might have been unintentionally overlooked. Additionally, the search queries were in English, which may have excluded relevant non-English publications. Finally, Publish or Perish does not index certain databases, such as CINAHL, potentially limiting coverage of some clinically oriented studies.
This systematic review draws from previously published studies to guide future research in identifying mental health disorders beyond social media. While we present the available datasets used for this purpose in our review, we did not make any attempts to build prediction models using this data. We acknowledge that detecting early signs of mental health disorders requires adherence to ethical protocols. Moreover, misuse of these sensitive data or of the models trained on the data can lead to stigmatization or harm to individuals with mental health disorders [17].
We would like to thank the anonymous reviewers for their constructive feedback and thoughtful suggestions, which helped improve the clarity and quality of this work. We are also grateful to our collaborators for their valuable contributions and insightful discussions throughout the project. Finally, we acknowledge the creators and maintainers of the datasets studied in our review for developing and curating these resources, thus enabling and advancing research in this domain.