1 Introduction↩︎

In the age of ChatGPT and other large language models (LLMs), schools and publishers have begun using tools to detect text written by some kind of Artificial Intelligence (AI). As ethics questions surround the creation and use of models like ChatGPT, fairness issues related to inaccurate or biased AI-detection tools have also become prominent in discourse [1]. Concerns around AI-detection models have been bolstered by studies revealing their very low accuracy rates [2]–[4] as well as evidence of bias in AI detection against certain groups. In a 2023 survey, 10% of teenagers in the US reported having been falsely accused of using AI to write their assignments. Disturbingly, twice as many Black teenagers as white and Latino teenagers reported being falsely accused [5]. Additionally, Liang et al. [6] showed empirically that AI-detection models are more likely to flag texts written by non-native English speakers. While there is evidence of this tendency, anecdotal claims that such models disproportionately flag autistic people’s writing as AI-generated have not been formally investigated. In this paper, we describe an experiment probing the OpenAI GPT-2 detection model for a false positive bias against likely-autistic writers’ posts on Reddit.

1.1 AI Detection Models↩︎

The GPT-2 detector used in this experiment was released in 2019 by OpenAI, the creators of the GPT series [7]. The classifier purported to detect text generated with OpenAI’s GPT-2 language model with 95% accuracy, though later evaluations indicated much lower rates [8]. Newer iterations of AI-detection models have since been released and subsequently deleted by OpenAI as accuracy issues persist and evidence of biases are revealed [9].

While many major universities in the US and UK are now cautioning their professors against using automated AI-detection tools, free and paid tools such as CopyLeaks, Turnitin, GoWinston, and GPTZero boast largely unverified accuracy rates up to 99.6% and are still in high demand [1]. Despite empirically supported claims that AI-detection models are “neither accurate nor reliable” [4], their use is not explicitly prohibited in educational or publishing contexts. In fact, a 2024 survey reported that two thirds of American teachers used them regularly [10]. While newer AI-detection technology called watermarking could have the potential to be much more accurate, OpenAI has not publicly released any new tools, claiming to be wary of stigmatizing the use of AI for groups who rely on it to improve their writing, such as non-native English speakers [11].

1.2 Understanding AI Detector Predictions↩︎

Unlike plagiarism detectors, current AI-detection models cannot cite evidence of the phenomenon they attempt to detect, so their outputs cannot be cross-checked empirically [12]. It is quite hard to know what features of AI-generated text detection models rely on since they only return a prediction (“Real” or “Fake”) and a probability score (often taken as a model’s degree of certainty in its prediction, though this interpretation can be misleading—see [13]). One can learn a lot about a model from the data it was trained on. However, with large, proprietary models, data sources and statistics about those sources are rarely available.

After Liang et al. [6] brought to light evidence of a bias against non-native English speakers in several popular AI-detectors—including the OpenAI GPT-2 detector—AI and plagiarism-detection company Turnitin reported that a significant difference in false positives for non-native vs. native English-speaking authors only existed for short-form texts (under the 300-word minimum suggested by Turnitin) [2], [14]. Most AI-detectors recommend a minimum number of characters or words for their input, citing poor reliability under that length. It has also been reported that low values of perplexity and burstiness—frequently associated with AI-generated text—are common in non-native English speakers’ writing [6]. Similar claims regarding perplexity and burstiness have not been made for autistic people’s writing, but there is anecdotal evidence that humans and AI-detectors alike may be prone to mistake autistic writing styles for AI.

1.3 Autistic Experiences with AI-Detection↩︎

Kling [15] reported on a university professor’s experience of being falsely accused of using AI to write their emails. The professor believed the accusation could be attributed to being autistic and mentioned that receiving such allegations is a common experience for autistic people [15]. An autistic student falsely accused by her professor had similar observations, citing the "formulaic" nature of her own writing and its potential similarity to AI [16]. Gegg-Harrison and Quarterman [17] used a small corpus of their own writing to test the false positive rates of several different popular AI-detection tools and saw much higher false positive rates than those reported by the makers of the tools themselves. They go on to discuss their own neurodivergence as a potential factor in these results, noting that a number of neurodivergent students and writers with similar suspicions reached out to them to share stories and fears related to false accusations of AI usage in their writing.

1.4 Exploring Autistic Writing Styles↩︎

A surprisingly small amount of descriptive research has been conducted on sociolinguistic and stylistic differences in autistic language, despite considerable literature on diagnostic linguistic traits of Autism Spectrum Disorder (ASD) largely limited to children and spoken—as opposed to written—language. Some of these linguistic traits are mentioned in diagnostic coding for the ADOS-2 [18] and in the DSM-5 [19], advising diagnosticians to look out for language that is “stereotyped”, “formal”, “rigid”, “repetitive”, and “pedantic”. While many researchers group these linguistic traits under the umbrella of pragmatic “deficiency” [20], others approach the topic more neutrally, highlighting only the divergence from the norm or majority [21], [22]. Recent research spurred by autistic and other disability self-advocacy groups explicitly frames these features as “differences” rather than “deficiencies” [23]. The finding that autistic people without language impairment communicate as effectively and efficiently as neurotypical people do when in a peer group of other autistic people [21] bolsters the claim that autism is characterized by stylistic language differences rather than pragmatic deficiencies.

At least two recent studies have attempted to make use of differences in written language to train machine learning models to predict whether or not tweets were authored by autistic users on social media platform X (formerly Twitter) using only the texts of tweets [24], [25]. While they initially claimed that the use case for such classifier models would be assisting with early autism diagnosis, Jaiswal & Washington released an addendum to their original paper and a letter to the editor discussing serious ethical concerns with digital phenotyping and the potential for harm involved in creating such models [26], [27]. They also acknowledge a lack of explicit consent from users who authored the data that was collected, which ultimately motivated them to delete their models and data.

1.5 The Present Paper↩︎

In this paper, we test the hypothesis that false positives in the outputs of the OpenAI GPT-2 detection model are more common for text written by autistic people. We run a variety of Reddit posts from autism-focused and general discussion subreddits through the detector and compare its predicted probabilities of AI-generated content between the two groups of texts.

2 Data↩︎

While research involving public Reddit posts does not meet the criteria for submission to the authors’ Institutional Review Board, ethical concerns arise when collecting data without explicit consent from its creators, especially in marginalized communities. Though this dataset contains no user-identifying information, it will not be made publicly available given the potential for malicious use of such data, as is discussed in Jaiswal et al. [27]. See additional research [28], [29] on ethical considerations for projects using “scraped” Reddit posts.

2.1 Data Collection↩︎

The goal of the data collection step was to gather text written in a similar domain by autistic and non-autistic people. It is difficult to have high certainty about the accuracy of such a division, since not all autistic people identify themselves publicly as such anywhere their writing appears. The dataset used in the current study suffers from this low degree of certainty but was chosen for the quantity of publicly available and easily categorized texts.

We identified 13 subreddits dedicated to discussing topics related to autism on the social media platform Reddit. The descriptions and rules of each of these subreddits make it clear that their purpose is to offer autistic people a space to discuss various facets of their lives. While some subreddits ask that only autistic people or only officially-diagnosed autistic people post and comment, others welcome posts from non-autistic people when made in good faith. Still, looking through a random sample of recent posts from each subreddit, the overwhelming majority are written by people who identify themselves as autistic. As in all subreddits, posters who do not follow the rules of each designated subreddit may have their posts deleted by the subreddit moderators. This means we can be slightly more certain that posts in these subreddits are going to “belong” or fit the theme of the subreddit.

To curate a comparison group representing the general Reddit population, we selected 12 popular subreddits judged to be similar to the autism-related subreddits in terms of post style and format. These posts tended to be relatively long first-person narratives discussing topics related to mental health, social situations, embarrassing or surprising stories, solicitations for advice or comfort, etc. Importantly, a binary distinction of “autistic” and “non-autistic” cannot be drawn between these two subcorpora, since anyone of any neurotype can join and post in any subreddit. There will be some non-autistic people posting in the subreddits targeted towards autistic people and many autistic people posting in the general subreddits. For this reason, we will refer to the two subcorpora as “likely-autistic” and “general-Reddit” hereafter. Figure 1 shows the distribution of posts per subreddit present in the corpus, largely dictated by the availability of data.

We collected data with the Python PRAW library [30], a wrapper for the Reddit API, which only returned posts from approximately 2021-2024. With the goal of adding a larger quantity of data and less recent data, we downloaded a large set of Reddit posts from a Pushshift archive [31]. This archive provided much more data than what we could get through PRAW alone, including posts from 2010-2020. Unfortunately, far fewer of the 13 subreddits previously identified for the autism-related subcorpus existed in the older archive. For the sake of quantity, we chose to combine the more recent and older corpora, ending up with around 60,000 Reddit posts to work with. This dataset was later halved in size, approximately, after eliminating posts with a word count of 300 or less.

Little is known about the demographics of the authors of these Reddit posts. Reddit users in these communities may be of any age, including early adolescents and adults. The overwhelming majority of texts are written in English, though varied generational, socioeconomic, cultural, and regional dialects as well as differing degrees of English proficiency are expected. All of these factors could complicate this experiment, though these issues are not easily mitigated given the anonymity of Reddit posts.

2.2 Preprocessing and Filtering↩︎

We used the RoBERTa AutoTokenizer which pairs with the OpenAI GPT-2 detection model for the word tokenization step. This involved truncating posts to 480 tokens, which is just below the token maximum for inputs to the OpenAI model. We also used the NLTK punkt sentence tokenizer [32] to break posts up into sentences, though this step was purely for collecting descriptive statistics about the posts.

In an effort to exclude as many autistic writers from the general-Reddit group as possible, we excluded posts from the general-Reddit group whose authors appeared in the likely-autistic group or which included keywords about autism in the text. In our manual review of randomly sampled posts from each subcorpus, we noticed that several of the older posts from the archived Pushshift source made in the r/autism and r/aspergers subreddits were authored by people who did not actually identify as autistic but were discussing autistic relatives. Because of this, we chose to filter out posts containing keywords about autism paired with phrases like “my daughter…”, “my nephew…”, etc. in an attempt to limit the number of posts from authors who are not necessarily autistic themselves. This filtering process will not catch all cases and will unnecessarily exclude some posts made by autistic people discussing relatives.

We limited our dataset to one randomly chosen post from each user to avoid over-representing any one author. We also excluded posts with fewer than 1000 characters (200 words, roughly), since OpenAI claims this is the minimum length required for an accurate prediction.

2.3 Corpus Statistics↩︎

Motivated by a desire to explore textual differences between the subcorpora and their effect on AI-detection probability, we chose to compute several descriptive statistical measures for each subcorpus. For AI-generated text, both perplexity and burstiness tend to be low [2]. Perplexity can be loosely understood as the model’s degree of “surprise” or difficulty predicting the next word in the text. Burstiness is a term often used to represent the non-random distribution of a particular word in text, but in the context of AI-detection, it has been described as the degree of variation in sentence lengths and structures. Perplexity was computed using the evaluate Python package created by HuggingFace [33]. Burstiness was calculated as the coefficient of variation of sentence lengths for each post, modeled after the calculation used in the zippy Python package [34]. In addition, we computed the average word length in characters and average sentence length in words for each post in the corpus.

Most notably, the general-Reddit subcorpus had a much larger mean word count than the likely-autistic subcorpus. Mean word length and sentence length were slightly higher in the likely-autistic subcorpus, and both perplexity and burstiness were close to equal with the likely-autistic subcorpus trending only slightly higher in both. Figure 2 depicts the differences in textual features between the two subcorpora.

3 Experiment 1↩︎

3.1 Methods↩︎

While there are several AI-detection tools on the market, we chose to use OpenAI’s RoBERTa GPT-2 detector [7] as it is freely available for download on the platform HuggingFace and thus may be more commonly used. After downloading the model locally, we ran all posts from each Reddit subcorpus through the AI-detection model to generate both binary (Fake/Real) predictions and decimal/percentage probabilities of each post being AI-generated. Predictions labeled “Fake” indicate an AI-probability value greater than 0.5.

There is a real risk that some of these posts actually were generated by AI, and an unequal distribution of truly AI-generated posts between the two subcorpora would compromise this experiment. One method of mitigating this risk would be to limit the data to only posts made before 2020, as GPT-2–one of the first widely available LLMs–was publicly released at the end of 2019. Still, due to the limitations of the older subcorpus, we chose to combine it with the newer subcorpus and included a binary “date” variable in the dataset which tracks whether or not a post was written after January 1, 2020.

In this experiment, 1.7% of all 59,947 posts were flagged by the model as AI-generated. When split by subcorpus, the likely-autistic group had 1.9% of posts flagged as AI-generated, and the general-Reddit group had 1.5% flagged as such. To investigate the significance of this difference, as well as the impact of various textual features on the model’s probability outputs, we fit a logistic regression model with the detection model’s AI probability score as the outcome variable. The subcorpus (likely-autistic or general-Reddit) and several other variables (mean sentence length, mean word length, perplexity score, burstiness score, and a binary indication of whether the post was made after 2020) were all included as predictors. To fit this model, we used the glm function in R [35] with a binomial distribution and no random effects or interaction terms.

3.2 Results and Discussion↩︎

Table ¿tbl:exp1? shows the regression coefficients and Figure 3 shows the effect plots for significant variables of this model. We see that the subcorpus a post came from was a significant predictor in determining whether or not it was flagged as AI-generated. Based on the odds-ratio, posts from the likely-autistic subcorpus had a 25% greater chance of being classified as AI-generated. The model also showed a significant negative effect of perplexity, which matches expectations. Posts with higher perplexity were less likely to be flagged as AI. Lastly, we see that the length of each post was significant in determining whether or not a post would be flagged as AI-generated. Shorter posts tended to be flagged as AI more often than longer posts. This tracks with the conventional wisdom that AI-detectors are less accurate with shorter texts. If the model tends to more often flag shorter texts as AI-generated, even above the 1000-character minimum threshold suggested, this is something worth exploring in greater depth.

4 Experiment 2↩︎

4.1 Methods↩︎

Given that the likely-autistic corpus had a lower mean word count than the general-Reddit corpus and that the effect of word count was significant, we decided to pursue a secondary experiment neutralizing this variable. Based on the lengths of the available data and the recommended minimum lengths from other AI-detectors, we chose to limit the corpus only to posts longer than 300 words and subsequently truncate the text of each post to exactly 300 words. In this step, the dataset size was effectively halved, resulting in a subset of \(\approx\)​33,000 of the original \(\approx\)​60,000 posts. The model’s 500-token limit constrained our choice of maximum word count, and a larger minimum word count would have decreased the amount of data available too dramatically. Before running this set of truncated posts through the same AI-detection model as in Experiment 1, we re-computed the relevant descriptive statistics for each subcorpus for the text with normalized word counts. All other descriptive statistics showed the same comparative trends as in the previous dataset.

The following results were observed after running the 300-word posts through the same AI-detection model as in Experiment 1: 1.4% of the 33,216 total posts were flagged as AI-generated (a slightly lower percentage than the initial experiment). 1.7% of posts from the likely-autistic subcorpus were flagged as AI, and 1.2% of posts from the general-Reddit subcorpus were flagged as such. We fit a new logistic regression model with this data, using the same design, method, and variables as before (with the exception of word count, which has been normalized across all posts).

4.2 Results and Discussion↩︎

Table ¿tbl:exp2? is the table of coefficients for this regression, and effect plots for significant variables are found in Figure 4. This model showed that the subcorpus variable (likely-autistic or general-Reddit) again had a significant effect on the probability of AI-generation returned by the detection model and a slightly larger effect size. Based on the odds-ratio, posts from the likely-autistic subcorpus had a 50% greater chance of being classified as AI-generated. The “date” variable, a binary indication of whether or not a Reddit post was submitted after Jan 1, 2020, also showed significance in this experiment with a small effect size. The effect of this variable on AI-probability was negative, indicating that posts written in or after 2020 were less likely to be flagged as AI compared to older posts. One possible explanation for the decrease in AI-generated predictions after 2020 is that the GPT-2 detector was not trained to detect texts written by newer LLMs such as ChatGPT, so any Reddit posts written with such tools might fly under its radar as false negatives.

5 General Discussion↩︎

The fact that posts from the likely-autistic corpus were significantly more likely to be flagged as AI by this detection model prompts concern for all domains in which the model is used. False accusations of AI can cause students to suffer in terms of their academic/career standing and psychological well-being. Chaka argued that “any AI content probability percentage or percentage point, however negligible it may be […] inflicts immeasurable reputational damage to that essay and to the student who produced it” [2].

Globally, autistic people suffer from extremely high rates of unemployment [36], and reputational damage or limited educational opportunities caused by false accusations of AI will only have more devastating effects on employment rates and livelihoods. Gegg-Harrison and Quarterman [17] discuss the severe psychological impact on students caused by false accusations of cheating via AI, noting that autistic people—and other neurodivergent people such as those with ADHD—often suffer from rejection sensitive dysphoria and difficulty regulating emotions, so the impacts of false accusations could be even more damaging. From another viewpoint, schools and companies could open themselves up to ableism and other discrimination lawsuits by using biased technology.

If AI-detection tools finally fall out of fashion, there is still concern that individuals will take it upon themselves to decide whether or not a text has been written by AI. It has been shown in multiple contexts and domains that humans are no better—and often worse—than automated detection models at identifying AI-generated content [37]. Still, Verma & Tenjarla [38] reported that Ivy League admissions officers use automated AI-detection models as well as their own judgment to decide whether or not an essay was written with AI. One admissions officer detailed a valid set of criteria he used to spot AI-generated papers which eerily echoed descriptions of autistic narrative styles (see [20]).

There are several experimental limitations intrinsic to the data collected for this experiment. The casual style of social media text in comparison to the more formal target material of AI-detectors may constrain any generalizations made. A later iteration of this project could also put a more objective focus on matching up the subcorpora by topic. The use of other publicly available corpora will still contend with the underlying uncertainty in identifying autistic and non-autistic authors. It is reasonable to have a high degree of certainty in autistic authors’ self-identifications but very difficult to know who in the “general population” may be autistic without identifying as such publicly. 3% of the US population have an autism diagnosis, according to figures from 2020 [39]. However, diagnosis rates are rising rapidly, particularly in young adults, women, and certain racial and ethnic minority groups [40]. These rising rates seem to be an artifact of the historically widespread under-diagnosis of autism. One study reported that 80% of autistic women were undiagnosed as of age 18 [41]. Given these complexities, a more controlled experiment would verify all participants’ results of autism evaluations when designating the two comparison groups.

The fact that we do see a significant trend of bias in the detection model’s outputs even with a very noisy dataset inclines us to suspect that the difference in false positive rates between autistic and non-autistic writers may actually be larger than what was observed here. Of course, It would be decidedly more informative to get predictions from more than one AI-detection tool, as modeled in several of the previously referenced experiments. Though certain costs and inconveniences prevented the inclusion of other AI-detection tools in this study, one other freely available AI-detection tool was briefly used [42] as an additional source of prediction data. Unfortunately, this tool produced the exact same predictions and probability scores as the OpenAI model for the same texts, rounded to the hundredth decimal place. While this is just one example, it is possible other tools on the market use the OpenAI model on the back end of their services, despite its bias and accuracy issues.

Various seemingly incongruous descriptions of autistic language indicate that while autistic language is often more repetitive, stereotypical, and clichéd, it is also observed to be idiosyncratic and contain more neologisms or oddly-worded phrases [43]. The former point prompts the assumption that autistic writing has lower perplexity and burstiness, but in our likely-autistic subcorpus, we found perplexity and burstiness to be roughly the same as—if not slightly higher than—the general-Reddit subcorpus. Perhaps the latter observation regarding idiosyncrasy is the kernel of an explanation for why we did not see lower perplexity and burstiness. A look at the distribution of frequent lexical items and constructions in both subcorpora could give more insight into these questions.

6 Conclusion↩︎

Considering the prevalence of AI-generated text detection tools, it is important to understand their rates of accuracy as well as how tendencies in their outputs might affect certain groups disproportionately. This study attempted to test the hypothesis that autistic writers are more likely to have their content flagged as AI-generated by a publicly available AI-detection model. Given a large curated corpus of Reddit posts, it was shown that those posted in autism-centric subreddits—presumably written by autistic people—were more likely than posts from other subreddits to be flagged as AI-generated by OpenAI’s GPT-2 detector. These findings add further motivation to examine biases in other AI-detection models and limit or discontinue their use given the potential for harm.

References↩︎