Linear representations of grammaticality in neural language models
July 16, 2026
Abstract
Whether neural language models (NLMs) possess the ability to distinguish strings on the basis of their grammaticality remains a debated topic in the computational linguistics literature. Existing evidence has largely relied on probability-based measures, testing whether models assign higher probabilities to grammatical than ungrammatical strings. However, probability comparisons have been criticized as a measure for grammatical knowledge based on the assumption that grammaticality is inherently entangled with likelihood. Model-assigned probability is a function of many related sentence properties, such as lexical frequency, plausibility, and world knowledge. In this work, we move beyond probability-based evaluations and directly investigate whether grammaticality is encoded in the internal representations of NLMs. Using linear probing (specifically, mass-mean probing), we test whether grammatical and ungrammatical sentences are systematically separated in representational space: i.e., whether this distinction can be detected even by a low-complexity classifier. We further examine the extent to which these representations are independent of sentence properties that are correlated with grammaticality, as well as their generalization across grammatical phenomena and languages. Our results provide evidence that grammaticality is robustly encoded in sentence representations of a wide range of pretrained NLMs, yielding clear representational separation between grammatical and ungrammatical sentences that cannot be fully explained by alternative sentence-level factors. Moreover, this encoding generalizes across a broad range of grammatical phenomena and to some degree, across languages, suggesting that grammaticality constitutes a coherent representational dimension in contemporary NLMs. These findings contribute new evidence to debates about the nature of syntactic knowledge in language models and offer a complementary framework for evaluating grammatical competence that is not dependent on string probabilities alone. 1
1 Introduction↩︎
The rise of neural language models (NLMs), trained to predict the likelihood of vast amounts of text, has led to a resurgence in a core debate in cognitive science about whether statistical learners can distinguish grammaticality of strings [1]–[6]. On the one hand, many pretrained NLMs assign higher probabilities to unseen grammatical strings over their ungrammatical counterparts [5], [6], supporting arguments that they capture non-trivial aspects of grammar [7] and that this method can serve as a linking function between model-assigned probabilities and grammaticality [6]. There is significant pushback on this view, with one prominent line of disagreement being that because model-assigned probabilities fundamentally entangle grammaticality with likelihood, inferences about model knowledge of grammaticality is fraught [3], [4]. In this work, we argue that stronger evidence for sensitivity to grammaticality comes from the distinction being reflected in the internal organization of sentence representations, unfettered by various sentence properties that correlate with grammaticality.
Humans reliably distinguish well-formed sentences from ill-formed ones, even when surface statistics are uninformative, suggesting sensitivity to abstract structural constraints [1], [8]. In symbolic theories of syntax, and symbolic model instantiations of these theories (e.g., via a context-free grammar), grammaticality as a property of a sentence falls out naturally. What about grammaticality in NLMs? On one view, Chomsky’s (1957) demonstration of the orthogonality of probability and grammaticality seems to dissuade us from the idea that a language model – crucially a probabilisitic model of language – can have true discernment between grammatical and ungrammatical strings. This position has received significant theoretical and empirical pushback ([2], [2]; and more recently, [6], [6]). Setting aside this in-principle disagreement, an empirical limitation of probability-based measures of grammaticality in modern LMs lies in the amount of information it encodes. Other string properties such as word frequencies, sentence length [9], [10], plausibility [11], and world knowledge [12], are documented to influence string probability. Even when minimal pair comparisons control for some of these factors (e.g., plausibility), it remains largely unclear how, or whether, these factors interact with grammaticality to yield the types of probability distinctions we observe. These limitations further motivate the study of whether grammaticality is representationally encoded in NLMs that are argued to have grammaticality distinctions.
Zooming out of discussions about NLM grammaticality detection, having a more acute understanding of what syntactic knowledge a computational model possesses – in the form of sentence grammaticality – is a crucial precondition for work that argues NLMs can provide insight into human cognition. Arguments that neural language models can provide insight into the human language faculty [13], [14] rest on the assumption that we are able to reliably determine whether (a) a model is able to discern grammaticality at all, or (b) a model has grammatical knowledge of particular phenomena. For example, work arguing that certain phenomena are in principle learnable from data alone with little domain-specific biases requires positive or negative demonstrations of the learned grammaticality distinctions [15]. On the flip side, based on the cited criticisms of probability-based methods (confounds with likelihood, encoding other correlated factors), related work has argued against the utility of NLMs for the study of human cognitive science. [3], [16]. Under this view, these models neither have genuine knowledge of grammaticality or syntax, and their successes with regard to fluent text generation or interpretation are surface-level in nature [17], thus not meeting the preconditions that make them suitable models for inferences about the human mind. The questions at stake here further highlight the importance of additional evidence to adjudicate between these positions.
It is clear that settling the question of whether NLMs have true distinctions of grammaticality is of timely essence. In this work, we approach this question by moving beyond probability-based measures and probing for grammaticality representations from model representations of sentences. In this article, we provide evidence for the argument that pretrained NLMs have grammaticality representations by demonstrating that there is clear representational separation between grammatical and ungrammatical sentences in a model’s representational space. We find that this separation is distinct from many collinear properties of sentences, while being widely generalizable beyond a particular grammatical phenomenon. At a high level, this work addresses three questions:
Is there a detectable encoding of grammaticality in sentence representations? Is it measurable even with a low-complexity (i.e., mass-mean) probe?
How is this grammaticality representation distinct from collinear properties of sentences?
How generalizable is this grammaticality encoding across grammatical phenomena or languages?
In the next subsections, we review existing methods for grammaticality detection for NLMs and further flesh out criticisms against these methods, to inform us of how a study of grammaticality representation should address these confounds. We also review model interpretability literature to motivate the logic used in our experiments that follow.
1.1 Previous work↩︎
1.1.1 Grammaticality detection↩︎
The most common method for grammaticality detection is targeted syntactic evaluation using model-assigned sentence probabilities [5], [18], [19]. Probabilistic models are evaluated on their ability to consistently assign higher probability to grammatical strings over their minimally-different ungrammatical counterparts. There exists an active line of work developing the kinds of sentential comparisons that would be insightful for understanding the syntactic capabilities of the models in question, in English [5], [13], [18]–[22], Japanese [23], Mandarin Chinese [24], [25], Russian [26], and more [27], [28]. As reviewed, while this method has served as indices for grammatical knowledge in many recent work in the natural language processing literature [29]–[31], the assumption that models have genuine distinctions of grammaticality is challenged [3], [4], [32].
The primary argument is that ungrammatical sentences are unlikely, entangling the two factors in the training data. With the training objective of NLMs being that of maximizing the likelihood of this entangled data, it is unclear whether probability contrasts are indexing likelihood or grammaticality. [3] supports this claim with demonstrations that there exists metalinguistic queries where an LLM, likely succeeding on these probability-based benchmarks, will prefer a likely, but ungrammatical, continuation, over a grammatical continuation. Similar observations were made in [15] and [33]. Thus, a demonstration that models have grammaticality distinctions in representational space would also need to demonstrate that the success of the grammaticality probes is not due to string probability differences or extra-grammatical factors in the fitting sentences.
Another line of research uses metalinguistic prompting to measure a model’s sensitivity to string grammaticality [4], [34]. The target sentence is embedded in a query directing asking about the grammaticality of the sentence, and the measured variable is whether the model responds to this query positively (e.g., by responding “Yes”) or negatively. This form of evaluation is useful for models where probabilities or weights are not publicly accessible, but makes the additional assumption that models have the ability to provide metalinguistic judgments for a sentence: an ability that is distinct from having a grammaticality distinction. However, [35] argue that metalinguistic prompt-based evaluations underestimate a model’s syntactic ability.
Finally, we review existing work that probes for model-internal representations of grammaticality. To our knowledge, [36] is the only study that probes for grammaticality from a model’s representations of sentences. Their work applied linear probes (logistic regression classifiers trained on sentence representations) to differentiate between sentences drawn from corpora versus perturbed sentences that render the sentence ungrammatical, via functions such as word insertion, deletion, or shuffling. Their results provide convergent evidence that grammaticality is robustly represented various LLM’s representations of sentences. In our work, we take a step further to establish the generalizability of linear encodings of grammaticality across various kinds of violations. Complementary to probing, [37] investigate how syntactic specialization for specific grammatical phenomena emerges in activation space by comparing hidden state activations elicited by grammatical and ungrammatical sentence pairs. Although their goal is to characterize functional specialization rather than decode grammaticality with probes, their activation space comparisons are closely related to our methodology and likewise provide evidence that syntactic information is systematically reflected in model representations. Aside from probing in hidden state space, there also exists a line of circuitry work that examines the weights that are involved in computations of particular phenomena (e.g., subject-verb agreement) and whether they generalize across languages [38], [39]. These studies demonstrate that a small number of units in a model’s weight space are responsible for distinctions in probability space, and its generality provides convergent evidence that LLMs have robust representations of syntactic structure [40], [41].
1.1.2 Mass-mean probing↩︎
We investigate grammaticality as encodings in sentence representations using mass-mean probing [42], a simple linear probing method with minimal data and modeling assumptions. Mass-mean probing has been used to detect a range of high-level properties in neural representations, including truthfulness [42], sentence plausibility [11], and political orientation [43], and has also been adopted in model-steering work [44]. Its appeal lies in its interpretability and low complexity: the probe consists of a single direction in representation space defined by the difference between the mean representations of two classes. In this work, we apply this method to test whether grammaticality corresponds to a stable and accessible dimension of sentence representations in neural language models.
Through our experiments, we test the following hypothesis: if it is the case that a representational encoding of grammaticality exists, we should expect to see cross-phenomena generalization, while expecting to observe no generalization to sentential properties that correlate with grammaticality. We evaluate this hypothesis across 25 publicly available pretrained NLMs that perform above chance on targeted syntactic evaluation benchmarks, thus establishing a prerequisite level of behavioural sensitivity to syntactic structure.
2 Methods↩︎
2.1 Datasets↩︎
We use datasets and benchmarks from previous work that target grammaticality, as well as other sentential properties for our deconfounding experiments. Tables 1 and 2 summarize these sources respectively. Pre-processing procedures for specific datasets are documented in Appendix 9.1. The grammaticality benchmarks [5], [45] span a wide range of linguistic phenomena, allowing us to test within- and between-phenomena generalization in our experiments. We note a subset of the BLiMP datasets (40 out of 67) are annotated for the word where grammaticality diverges in a minimal pair setting (e.g., These dogs {like, *likes} to jump.). We use these annotations (see §2.3) to conduct additional analyses on this particular subset of datasets.
| Benchmark | Language | # Datasets | # Items per dataset |
|---|---|---|---|
| BLiMP [5] | English | 67 | 2,000 |
| SCaMP [45] | English | 134 | 2,000 |
| DGL [4], [46] | English | 1 | 190 |
| LI-Adger [22], [47], [48] | English | 1 | 4,066 |
| RUBLiMP [26] | Russian | 45 | 2,000 |
| ZhoBLiMP [25] | Mandarin Chinese | 118 | 2,000 |
| CLAMS [27] | French, German, Hebrew, Russian | 28 | 280–2,000 |
| Benchmark | Domain | # Datasets | # Items per dataset |
|---|---|---|---|
| Shades [49] | Event plausibility/conceivability | 6 | 2000 |
| Drive [50] | Sentence imageability, arousal, etc. | 9 | 1542 |
| Truth [42] | Sentence factuality | 11 | 354–2000 |
2.2 Models↩︎
This work examines the hidden states of 25 publicly available pretrained NLMs. These models vary widely in their scale (14 million to 14 billion parameters) and model family, as listed in Table 3. All of these models examined perform above chance at discerning grammaticality via minimal pair string probability comparisons on BLiMP (Appendix 9.3), which we take as the precondition to expect any knowledge of grammaticality to be encoded in each of these models, should it be the case that behavioural success is due to its sensitivity to grammaticality.
| Model family | Models (and parameter count) |
|---|---|
| GPT2 | Small (117M), Medium (345M), Large (774M), XL (1.5B) |
| Llama | Llama-3.2-1B, Llama-3.2-3B, Llama-3.1-8B |
| OLMo | OLMo-1B, OLMo-7B, OLMo-2-1B, OLMo-2-7B, OLMo-2-13B |
| Pythia | 14M, 70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, 12B |
| Qwen3 | 0.6B, 1.7B, 4B, 14B |
2.3 Extracting representations↩︎
Each sentence is passed into the model verbatim. We perform our analyses over final and intermediate representations of sentences in the final hidden layer. When available, we define three regions-of-interest over tokens.
Critical region: the span of tokens in the sentence where grammaticality divergence becomes clear (i.e., there is no possible grammatical continuation of the ungrammatical sentence). When this span contains more than one token, we take the average across tokens.
Pre-critical region: the token immediately preceding the critical region. This is treated as a control condition, because even though strings may diverge in their content, there are no grammaticality differences between the minimal pairs at this point of the sentence.
Final region: the final token of the sentence, which is the punctuation mark that ends the sentence (e.g., period or question mark).
An example illustrating these regions are available in Figure 1. When region-level annotations are not made available, the term “sentence representation” refers to the representation extracted in the Final region, which is accessible for any sentence. All representations extracted are \(d_M\)-dimensional vectors, where \(d_M\) is the dimension of the final hidden layer of the model \(M\). Before conducting any probing analyses, we mean-center all representations. The mean is computed from the set of representations that refer to the same token, and then subtracted from the representation. This is done to ensure that any probing results are not a reflection of the lexical/token identities of tokens in a sentence representation.
2.4 Mass-mean probing↩︎
We search for grammaticality representations via mass-mean probing [42], [51], a linear probe with minimal data assumptions. Let there be a set of data points \(\mathcal{D} = \{(x_i, y_i)\}\) where \(x_i \in \mathbb{R}^d\) (sentence representations) and \(y_i \in \{0, 1\}\) (grammaticality labels). The mean representation for each class, \(\mu^0\) and \(\mu^1\), as well as a difference vector \(\theta = \mu^1 - \mu^0\) is computed. A mass-mean probe \(g: \mathbb{R}^d \mapsto [0,1]\) is defined as: \(g(y = 1 \mid x) = \sigma(\theta^Tx)\). For classification, a representation is classified as 1 (grammatical) when \(\sigma(\theta^Tx) > 0.5\), or \(\theta^Tx > 0\). The decision boundary for the mass-mean probe is the plane that satisfies the equation \(\theta^Tx = 0\).
Unless otherwise specified, a probe is fitted to 1,600 sentences. For each probing setup, we report the results from 100 runs, with each run sampling a different random subset of whatever fitting dataset defined by the experiment. For the remainder of this work, we will use the terms “fitting dataset” and “evaluation dataset” to refer to the set of sentences where the fitting/training sentences or evaluation/test sentences are drawn from. The next subsection reports how these results are processed and reported.
2.5 Evaluation, measurements, and significance testing↩︎
All evaluations are conducted on held-out sentences. Oftentimes, the fitting and evaluation dataset target different phenomena (e.g., fitting on an NPI-licensing dataset and testing on a verb agreement dataset). In that case, we randomly sample fitting subsets across runs, while evaluating on the totality of the dataset for the evaluated phenomenon. In the other scenario, where the experiment involves fitting and evaluating on the same dataset, we treat the held-out remainder of the fitting dataset as the evaluation dataset. For example, for a dataset with 2,000 sentences, the probe is fitted on 1,600 randomly sampled sentences from the dataset, and then evaluated on the held-out 400 sentences.
For each probing setup, we now have a distribution over all probing measures, e.g., item-level accuracy, dataset-level accuracy, an item’s distance to classification boundary. Unless otherwise specified, we report the mean of these measures. One crucial second-order measure that we report is probe success, taken to represent whether this probing setup successfully distinguishes grammaticality (or whatever construct that defines the labels) of the evaluated dataset. This is measured by comparing the dataset-level accuracy distribution against an equivalent null distribution. A null probe is constructed in the same way as a normal probe, except the labels of the fitting sentences are shuffled. We consider a probing configuration to be successful if the probing distribution is significantly greater than the null distribution (via a two-sample Welch’s \(t\)-test, \(p\)-value is Holm-Bonferroni adjusted) and the mean of the probing distribution is at least 5% greater than the null. This threshold acts as a noise threshold to avoid false positives. Observing that probes fitted on single BLiMP datasets at the Pre-Critical region (a manipulation in Experiment 2) are in the range [-2.62%, 1.65%], we set the threshold to 5% as a conservative estimate of false positives.
3 Experiment 1: Grammaticality detection via mass-mean probing↩︎
In this initial experiment, we establish that grammaticality distinctions are detectable in hidden state space via mass-mean probing. Demonstrating this basic finding in a given model allows us to then inquire the scope, or sensitivity, of these grammaticality representations in the next experiments. Additionally, to situate this work in the prior literature on grammaticality measurements, we also report our results in a way that yields comparisons between probing and probability-based measures.
We fit a probe on sentences randomly sampled from the LI-Adger dataset [22], [47], [47], [48], a collection of acceptability judgment data from linguistic theory papers and a prominent syntax textbook. Then, we evaluate the probe on all BLiMP sentences (67 datasets). We report probe accuracy and probe success, the two main measures used in this paper, but also derive a comparison-based measure, MP Rep (Minimal Pair, Representations), that compares the probabilities assigned by the probe in a similar fashion to targeted syntactic evaluation. For each dataset, we define MP Rep to be the proportion of minimal pairs in the dataset where \(g(x_\text{grammatical}) > g(x_\text{ungrammatical})\), \(g(x)\) being the mass-mean probe-assigned probability for the positive label. This measure is also used in [36] and [37] for comparisons between probe- and probability-based measures.
3.1 Result #1: All models exhibit probe sensitivity to grammaticality↩︎
We find that all 25 models we examine demonstrate some probe sensitivity to grammaticality, albeit with great variation between models (Fig.2A). Success rate ranges from 13.4% in Pythia-14m to 94.0% in Qwen3-14B. Though, aside from Pythia-14m, all models exhibit probe sensitivity on at least 27 datasets (of 67 datasets total), suggesting that there exists some non-trivial degree of representational separation in each of these models. Model size (in terms of number of parameters) best predicts the variation in success rate (see \(R\) values in Fig. 2) – we explore the relationship between model size and probing performance in further detail in then next subsections. Global accuracy fluctuates similarly (Fig.2B), though this is expected as accuracy on unresponsive datasets hovers around 50% for all models, drawing the global accuracy down in proportion to the number of successful datasets for a given model. Narrowing our analyses down to the accuracy of successful probes, we observe a similar effect (Fig.2C).
3.2 Result #2: Comparisons between probe and probability-based measures↩︎
Our results show that probing accuracy calculated from labels determined by a fixed threshold of \(g(x)\), in general underperforms relative to minimal-pair, probability-based targeted syntactic evaluation. Fig.3A provides a summary of model-level differences between the three measures: probing accuracy (Acc Rep), MP Rep (Minimal Pair, representations), and MP Prob (Minimal Pair, probability). We observe that MP Rep underperforms MP Prob substantially in small models, but this disadvantage subsides and even reverses as model size increases (range: [-13.4%, +6.89%], correlation with log10(parameter count): \(R = 0.791\), \(p < 0.001\)). These results suggest that in aggregate, probability-based comparisons tend to detect grammaticality contrasts more effectively than probing. However, these aggregate model-level trends conceal substantial variation across individual syntactic phenomena, which we examine next.
We find that there are BLiMP datasets that are in general (across models) better distinguished via probing over probability. If there is no particular pattern to when a given model underestimates grammatical knowledge via probability-based measures and vice versa, we expect a roughly uniform distribution of MP Rep > MP Prob over datasets. This is not the case; we find that dataset-level advantages exist along a continuum, with some datasets having stronger detectability via probing than via probability across nearly all models (labelled in Fig.3B, left side), but the reverse is also true, with certain datasets having stronger detectability with probability-based measures (Fig.3B, right side).
3.3 Discussion↩︎
This experiment establishes that mass-mean probing, and by extension the difference vector \(\theta\), captures some key information about grammaticality, thus warranting further study. The LI-Adger dataset consists of sentences from an assemblage of grammatical contrasts, rather than one kind of grammatical violation. This suggests that our results are not simply due to some surface-level pattern or template-based similarities that are collinear with grammaticality. This is further supported by the fact that this probing setup tests for generalization to a different benchmark. In later experiments, this point is reinforced by further demonstrations of cross-phenomenon and cross-linguistic generalization. These results are consistent with the findings of [36], despite employing different probing techniques, sources of ungrammaticality, and representations to fit on.
We note that our probing accuracies are generally lower than [36]’s ([36]) probes. This is unsurprising given the different goals of our experimental setup. We intentionally employ a low-complexity probe to evaluate how much grammaticality information is directly recoverable from the mass-mean difference vector, rather than maximizing classification performance. In addition, we do not tune the choice of representation layer for probing, and layer selection can substantially affect probe accuracy. Investigating how probe complexity and layer selection interact for mass-mean probing is a natural direction for future work, and may help clarify how much additional grammatical information can be extracted while preserving the interpretability of the probing method. The comparative results of this experiment also raise questions about how probing should be interpreted in tandem with results from probability-based approaches. We discuss this further in the Discussion section of this paper (§8.2). Overall, we view the convergent results as preliminary evidence that the encoding of grammaticality is one of the major organizational principles of the representational space, at least in the larger language models (\(\sim\)1B+ parameters and above) that this work and the others examine.
4 Experiment 2: Scope of grammaticality signals↩︎
Upon establishing that grammaticality distinctions are detectable via probing sentence representations, we would like to better understand the scope of the grammaticality signals. Specifically, we are interested in localization (where the signals start to emerge) and cross-phenomenon generalization (how general the signal is beyond the specific phenomenon targeted).
For most ungrammatical sentences, there exists a point in the sentence where no grammatical continuation is possible. Do our probes capture grammaticality violations as the sentence unfolds, or do they only demonstrate sensitivity when the sentence is completed? We fit probes over intermediate representations extracted at the each of the sentence regions (Pre-critical, Critical, Final; §2.3) and evaluate them on held-out representations drawn from the same region. Based on our results from Experiment 1, we already expect to see sensitivity to grammaticality in the Final region; our main question is whether grammaticality distinction is captured in the Critical region as well. As control, we also report probing results in the Pre-critical region – representations drawn from the token immediately preceding the Critical tokens. Since there is no grammaticality distinction in the Pre-critical region, we should expect to see no signal for probes fit in that region beyond noise.
We also explore the effects of phenomenon generality in grammaticality probing. In what ways are the grammaticality representations that we found specific or general, and how does this interact with sentential region? One hypothesis is that probes are highly-specific: they only succeed on datasets that are either the same dataset the probe was fitted on, or datasets of the same phenomenon.2 It predicts that probes would not generalize when tested on different phenomena. For this experiment, we fit a probe to one dataset, and evaluate that probe on other datasets or held-out sentences in the same dataset. We conduct this analysis on the 40 BLiMP subsets with Critical region annotations, yielding 1,600 (40 fit \(\times\) 40 evaluation datasets) probes per model, per region.
The high-level summary of the results are as follows. First, we show that all models show probe sensitivity both at the Critical and Final regions, and not the control Pre-Critical region (§4.1). The success profile of each of the models suggests that model representations of grammaticality are phenomenon-general. Nevertheless, we find that phenomenon-identity does influence probing accuracy – probes fitted and tested on the same phenomenon are more accurate (§4.2). Through a regression analysis, we show that various shared sentence properties explain variance in accuracy. Still, their effects are small and when regressed out, do not change the success profiles much – the previously reported patterns of phenomenon-generality in probing success and phenonmenon-specificity in the success profiles remain (§4.3).
4.1 Result #1: All models exhibit probe sensitivity to grammaticality, from the moment grammaticality diverges↩︎
We find that all 25 models we examine demonstrate sensitivity to grammaticality in the Critical and Final regions and not the Pre-critical region. On average, the Final region has a higher proportion of successful probes than the Critical region (Fig. 4), though when probes succeed, probes at the Critical region were on average more accurate than a Final region probe (Fig.5). Overall, the results from this experiment are consistent with the findings from the previous experiment in a mixed-phenomena probing setup. Sensitivity at the Critical region also suggests that models maintain a grammaticality representation that is incrementally updated as the sentence unfolds.
It is important to note that there is great variability in sensitivity among models, like what we have found in Experiment 1. In the Final region, the proportion of probe success ranged from 7.81% (Pythia-14m) to 94.38% (Llama-3.1-8B). At the model level, we find that model size (number of parameters) predicts a large swathe of this variance, accounting for 51.5% variance in the Critical region and 59.0% in Final region.
4.2 Result #2: Grammaticality signals generalize beyond dataset and phenomenon boundaries↩︎
If probes are phenomenon-specific, we expect that analyses that remove within-phenomenon probing results to yield little-to-no grammaticality signal or above-null probing performance. This is not the case. We only observe a minor drop in proportion of successful probes when we remove them from the analysis (Critical: average 4.06% drop, [1.64%, 5.28%]; Final: average 2.20% drop, [-0.22%, 5.68%]). All models retained their significance with respect to sensitivity at both experimental regions. This is also captured visually via heatmaps (Fig.4B and Fig.5B): probe success is widely observed beyond the dataset/phenomenon diagonal, with the Final region yielding broader cross-phenomenon generalization than the Critical Region. These results show that the grammaticality signal captured by our probes is phenomenon-general, especially signals in the Final Region.
While probes, especially those in larger models are highly generalizable, it is also the case the probes that fit and test within the same phenomenon have a higher accuracy than its cross-phenomenon counterparts (Critical: average 14.0% increase, [6.14%, 18.26%]; Final: average 6.64% increase, [2.13%, 11.03%]). We find that phenomenon-specificity effects, as measured by the difference between cross- and within-phenomenon average probing accuracy are the strongest in small- to mid-size models (some examples are available Appendix 9.3, Fig.14). In the next subsection, we explore whether surface-level properties of sentences, which may be correlated with phenomenon identity, explains the accuracy profiles observed in this experiment.
4.3 Result #3: Sentence similarity modulates accuracy, but does not affect probing success↩︎
Phenomenon-generalization could be capturing sentence-level similarity rather than abstract phenomenon-level representations; sentences targeting the same phenomenon, especially in the context of a grammaticality benchmark, will exhibit more surface-level similarities than sentences targeting a different phenomenon. Here, we ask whether similarity explains any part of probing accuracy. If it does, we ask whether the captured variance nullifies the probing successes we have reported.
For each probe accuracy matrix (\(M \in \mathbb{R}^{40\times40}\), per region, per model), we compute its correlation to various measures of dataset similarity rooted in comparisons between sentence-level properties of the grammatical subset of the dataset. These properties are: length (KL-divergence of length distributions), lexical overlap (% of common unique words), and representational similarity (average cosine similarity between sentence representations of the target region from the evaluated model).
Across models, we find that probing accuracies do correlate with various measures of between-dataset sentence similarity (Fig. 6A). They are also significant effects in a linear mixed-effects model, with the model
structure accuracy\(\sim\)1 + Length_KL + Word_Overlap + Rep_Similarity + (1|model). The measures related to length and word overlap are correlated with probing accuracy in the Critical and Final
regions, but not the Pre-critical region, whereas model representational similarity correlates with probing accuracy at all regions. However, LMEM estimates of accuracies without similarity terms (global intercept + random intercept + residual) remain
substantially higher than the null distribution (Fig. 6B). This supports the claim that beyond sentence-level similarity, the grammaticality signals captured by mass-mean probing are phenomena general. The next
experiments further aim to deconfound grammaticality from other likely correlated factors.
5 Experiment 3: Deconfounding string probability from representations of grammaticality↩︎
Could our probes be detecting a contrast of low vs.high probability strings rather than grammaticality? In all of the models we examine in this work, we know that the model assigns, on average, higher probability to a grammatical sentence over its ungrammatical counterpart (Appendix 9.3). While this difference is measured at the sentence pair level, we may expect a potential heuristic that achieves high accuracy on our grammaticality probing to be one that relies on probability as a cue rather than grammaticality. If the grammaticality signals detected by our probes were probability-specific rather than grammaticality-specific, then we expect a manipulation that decorrelates the two factors in the fitting sentence set (i.e., low probability grammatical vs. high probability ungrammatical) to yield at-chance predictions on grammaticality on held-out sentences. Alternatively, we expect minimal-to-no drop in accuracy if our probes were not sensitive to probability. This hypothesis also predicts that probability manipulations that strengthen this correlation will also have little-to-no effect on evaluation accuracy.
This is the prediction we test: for each evaluated dataset, we created three probes corresponding to three probability-related manipulations. The Counter condition samples ungrammatical sentences from the 20% most probable subset of sentences outside of the evaluated dataset, and grammatical sentences from the 20% least probable sentences. The Favour condition reverses this by sampling grammatical sentences from the top 20% probable subset and ungrammatical sentences from the least 20% probable subset. Both of these conditions are compared against the Baseline condition, which does not impose any probability-related constraints on sampling. Fig.7A demonstrates the average string probabilities from the grammatical and ungrammatical subsets of each condition with Llama-3.1-8B as an example. We conduct these analyses on the Final representations of the BLiMP and SCaMP datasets.
5.1 Results↩︎
We find that there are minimal differences of accuracy relative to the Original for both manipulation conditions. While a subset of models exhibit a significant drop in accuracy in the Counter condition, the magnitude was small: the largest accuracy drop across models was 3.19 percentage points (OLMo-2-0425-7B, Fig.7C). We also found similar degrees of negative accuracy modulations even in the Favour condition (Favour: [-2.61%, +1.53%] Counter: [-3.19%, +6.12%]). More importantly, we found that the Counter manipulation has nearly no negative effect on probe success for most models. This is demonstrated by measuring the proportion of Counter probe success given that its matching Original probe succeeds. We find that with the exception of the smallest scale models (left-hand side of Fig.7D), models are at-ceiling for both conditions.
Taken together, these results support the hypothesis that the probes are detecting grammaticality rather than string probability. These results do not contradict the hypothesis that model representations of grammaticality affect string probability: an implicit assumption of the targeted syntactic evaluation paradigm. We discuss the relationship between our detection methods and string probability further in the Discussion section.
6 Experiment 4: Deconfounding general sentence acceptability from grammaticality↩︎
A prominent view in linguistics is that grammaticality and acceptability are linked but distinct [52], [53]: grammaticality contributes to acceptability, which is measured through judgment tasks to draw inferences about grammaticality despite acceptability also reflecting other factors. This section asks whether our probes were detecting sentence acceptability instead of grammaticality. If this is the case, then we predict that the probes that are supposedly fitted to discern grammaticality will demonstrate sensitivity to other factors that modulate acceptability. We identify these factors based on previous human experimental work [10], [49], [50], [54] and utilize existing datasets to examine their detectability under a probe fitted on grammaticality contrasts. These factors include: semantic plausibility, event possibility/conceivability, truth, imageability, arousal, etc (Table 9.1). Throughout this experiment, we refer to such acceptability-modulating factors as “extra-grammatical” factors.
In this section, we present three analyses that test this overarching hypothesis. The first two analyses seek to understand whether there is any effect of acceptability-modulating extra-grammatical factors on our grammaticality probes. Finding that there is an effect, we inquire, through a probe boundary-tilting experiment, whether acceptability subsumes grammaticality, or whether there remains a distinct encoding for grammaticality. Overall, we find that linear probes fitted on grammaticality contrasts are a composite of both grammaticality and acceptability signals, with the former still being the dominant signal.
6.1 Shared encoding between extra-grammatical contrasts and grammaticality↩︎
We can further examine the representational relationship between various acceptability-related contrasts and grammaticality by testing how encodings for other contrasts generalize to grammaticality datasets, and vice versa. The strong hypothesis that our grammaticality probes have no signal of extra-grammatical factors, will predict that probes fitted on an extra-grammatical dataset should not generalize at all to grammaticality datasets. In contrast, if there exists some general sentence acceptability encoding, then generalization between these datasets are expected. The setup of this experiment was similar to Experiment 1: we fitted a probe for each extra-grammatical dataset (see Table 2 for benchmark/dataset information) or BLiMP dataset, and evaluated the probe on each BLiMP dataset or extra-grammatical dataset, respectively.
Across models, we observe some degree of generalization of grammaticality probes to extra-grammatical sentence representations and vice versa (Fig.8A). Though the percentage of probes that demonstrate generalization are far lower than reported in previous experiments (gram \(\to\) extra mean: 11.2%, extra \(\to\) gram mean: 11.5% vs. gram \(\to\) gram mean: 55.9%, Experiment 2), it is still a non-negligible degree of generalization that merits attention. In particular, we find that this generalization is observed for a few select extra-grammatical datasets, and near-to-no sensitivity in others. Fig.8C illustrates this selective extra-grammatical sensitivity for GPT2 Large, the model with the strongest generalization between grammaticality and extra-grammaticality probes.
6.2 Minor probing accuracy differences between SCaMP-Plausible vs.SCaMP-Implausible↩︎
The SCaMP dataset offers us a window into measuring the effect of semantic plausibility on probing performance. Each SCaMP dataset has plausible and implausible variants; all sentences in the implausible variant, regardless of grammaticality label, violate selectional category restrictions on verbs. Ungrammatical sentences in the plausible datasets and all sentences from the implausible dataset should fall under the same category from the perspective of an acceptability-tracking probe, as both are anomalous, albeit for different reasons. Therefore, if it is the case that our grammaticality probes are acceptability-tracking rather than grammaticality-tracking, we expect a probe fitted to differentiate grammaticality on a plausible dataset to poorly differentiate the grammaticality of an implausible dataset. This is the prediction we test, employing the same probe-fitting procedures as Experiment 2. To quantify poor differentiation in implausible datasets, we make comparisons to how these probes perform on matching plausible datasets. If it is the case that a probe poorly differentiates grammaticality due to general sentence anomaly, then we expect a drop in accuracy on the implausible dataset when compared against its plausible counterpart.
We find that there are measurable accuracy and success differences based on the plausibility of the evaluated dataset, though this effect is minor. Across models, the mean accuracy drop was 1.06 percentage points, with the range of within-model mean accuracy differences (Acc(Plausible) \(-\) Acc(Implausible)) being [-0.57%, 4.34%] (Fig.9A). The conditional success rate is relatively high (mean: 90.2%, range: [80.3%, 95.8%]), but not at-ceiling (Fig.9B). At the dataset level, the accuracy profiles between plausible and implausible datasets are globally similar, but with the implausible datasets having a weaker signal than that of their plausible counterparts. Fig.9C provides an example of this for a specific model (Llama-3.2-1B), and this relationship is quantified by measuring the correlation between the two accuracy matrices – in Fig.9C, \(R = 0.927\). Across models, the correlation ranges from 0.763 to 0.964. Overall, while sentence plausibility has a measurable effect on our grammaticality probes, the effect is modest, indicating that grammaticality remains a robust and reliable signal across both plausible and implausible sentences.
6.3 Factoring out extra-grammatical acceptability signals with boundary tilting↩︎
The results from the previous analysis suggest that mass-mean probes capture some degree of general sentence acceptability, but leaves unclear the degree to which this signal subsumes the grammaticality signal reported in the previous experiments. In this experiment, we first assume that the acceptability signals can be factored out of the probing boundary. Taking that assumption further, if it is the case that acceptability subsumes grammaticality, we predict that probes with their acceptability signals factored out would render them poor classifiers of our grammaticality datasets.
We remove acceptability signals via boundary tilting, regressing out the shared signal among the extra-grammatical and grammatical datasets from the probing difference vector \(\theta\). This regression is done by first computing the vector \(\phi\): the difference vector for an extra-grammatical dataset. Then, we remove the shared variance between \(\theta\) and \(\phi\) by subtracting the vector projection of \(\theta\) onto \(\phi\) (\(\frac{\theta\cdot\phi}{\|\phi\|}\,\frac{\phi}{\|\phi\|}\)) from \(\theta\), generating the tilted vector \(\theta_\phi\). \(\theta_\phi\) is then transformed into a mass-mean probe \(\sigma(\theta_\phi \cdot x)\) for evaluation.
We compare between conditions by comparing the probing accuracies or success over the same evaluation datasets. The conditions vary in terms of what \(\phi\) is. The Tilt-all condition takes the mean the all difference vectors from each of the extra-grammatical datasets as \(\phi\). However, failure to see a reduction in probing performance could stem from the mean operation, which may thin out the acceptability signal that can be present in some difference vectors but not others. Thus, we also define a Tilt-one condition where we compute \(\phi\) from the extra-grammatical dataset with the highest cosine similarity between \(\theta\) and \(\phi\). Each evaluated dataset has a probe (across 20 runs) fitted for each condition. A further Control condition used to validate the boundary tilting method defines \(\phi\) by randomly sampling 26 difference vectors from other grammaticality probes (for other evaluated datasets) and taking the mean, akin to the Tilt-all condition. If the method works, then we expect that the Control condition to render probes that are classifying at-chance.
The results indicate that grammaticality is a distinct signal from acceptability. First, we see that the equivalent tilting over grammatical datasets (Control) causes probes to be unable to distinguish grammaticality, whereas the Tilt conditions essentially pattern the same as the Original probe. These results are consistent among three reporting measures: success (Fig.10A), accuracy (Fig.10B), and success conditioned on success on the matched Original probe (Fig.10C). Like Experiment 3, we find that the models that are most affected by this manipulation are smaller models.
7 Experiment 5: Cross-linguistic generalization of grammaticality↩︎
We now turn to studying the language-generality of our grammaticality probes. Large language models are often explicitly trained on multiple languages and can respond to queries from non-English languages ([34], [34]; though see [55] for notable performance gaps between languages). A key assumption in generative linguistics is that, despite extensive typological variation, the core machinery underlying grammaticality distinctions is shared across languages [56]. Is the encoding of grammaticality in the NLMs that we study also language general?
This experiment focuses on the five models that exhibit the strongest grammaticality signals from the previous experiments, both in terms of probe success rate and accuracy: Llama-3.2-3B, Llama-3.1-8B, Qwen3-4B, OLMo-7B-hf, and OLMo-2-1124-7B. As a precondition to studying the language-generality of grammaticality encodings, we first establish that for these models, there is above-chance probing signal in the non-English languages we examine. Then, we establish that language-general encodings of grammaticality exist with a cross-linguistic probing analysis. We conduct our analyses on six benchmarks (CLAMS-{de, fr, ru, he}, RUBLiMP, and ZhoBLiMP; see Table 1) across five languages: German, French, Russian, Hebrew, and Mandarin Chinese.
The probe-fitting and evaluation procedures closely follow Experiment 2 (§4). In the within-language analysis, we fit probes on sentences sampled from same-language datasets that are not the evaluated dataset. In the cross-linguistic analysis, we fit probes on sentences sampled from BLiMP and SCaMP and evaluated them on the non-English datasets.
7.1 Result #1: Successful within-language grammaticality detection for a variety of non-English languages↩︎
We find that internal grammaticality distinctions are also present for non-English languages (Fig.11, left bars). This is the case even for language models that are not explicitly trained on datasets from these language, as indicated by results from OLMo [57]. Some model-benchmark pairs achieve near-ceiling success rates, such as Llama models on Russian benchmarks. The accuracy profiles of these successful probes are comparable to English (English mean across the five models: 66.6%, [65.0%, 68.8%]; Cross-ling mean: 61.4%, [59.9%, 62.9%]). These results indicate that the models examined have non-trivial grammaticality distinctions for various languages other than English.
7.2 Result #2: Shared grammaticality encoding across languages in some models↩︎
Now, we ask whether probes fitted on languages other than the evaluated dataset’s language yield similar degrees of success and accuracy. If the encoding of grammaticality is language-general, then we expect there to be little difference on both success and accuracy, regardless of whether the probe was fitted on the same language or a different language.
As can be seen from Fig. 11 (right bars), we find that there is a decrease in successful probes in the cross-linguistic condition across all five models, albeit to substantially different degrees in different models. We find that the drop in success rates in the Llama models is overall marginal, whereas there is a drastic drop in success rates in OLMo-7B-hf and Qwen3-4B. However, there still exists a non-trivial number of successful cross-linguistic probes in all models. We take this analysis a step further and analyze the probing performance for all the language pairs examined in this experiment.
Consistent with the previous analysis, we find that there is detectable grammaticality signal across many language pairs, though generalizability between pairs is not guaranteed in all models (Fig.12). We note that in this analysis, we collapse the distinctions between phenomena within a language by aggregating them for evaluation, which obscures the interesting accuracy variation that may help us diagnose why a language pair does not generalize for a given model. Overall, the results from this experiment suggest that NLMs can develop an encoding of grammaticality that is language-general, and does so with high fidelity in models explicitly trained on multilingual text [58]. This once again provides evidence that grammaticality is a major organizational dimension in the representational space, at least in some language models. There are many follow-up questions about the development of a consistent, language-general representation of grammaticality, including when such representations emerge during training, how they relate to language-specific syntactic features, and whether they depend on shared structural regularities across languages or instead arise from the pressure to model multilingual data efficiently.
8 General Discussion↩︎
Our series of experiments find that there is a reliably detectable representational separation between grammatical and ungrammatical linguistic expressions in the models we examine. This representational divide is captured by a single linear direction, and this direction is highly generalizable across phenomena and languages in stronger models. We discuss the implications of these findings and highlight future connections that can be made to better understand model grammaticality and syntactic knowledge.
8.1 RQ1: Detection of grammaticality from sentence representations↩︎
Through this work, we find that a low-complexity probe over sentence representations is able to reliably predict grammaticality on held-out sentences. Our findings are consistent with [36] in that regard, except the probing parameter \(\theta\) can be readily manipulated/tilted, as seen in our later experiments. For models with consistently high probing scores, these empirical results suggest that for an arbitrary linguistic expression, the models’ representation of this expression contains an abstract encoding of whether it is grammatical or not. The findings from Experiment 2 demonstrate that grammaticality is continually represented as the sentence unfolds. Future work could connect this incremental property of grammaticality representations to previous work detecting parse states [41] and dependencies as the sentence unfolds [40].
Zooming out from these particular findings, it is important to ask: why do we see linear representational separation for grammaticality at all? All of the models we examine were trained on an autoregressive objective, and information about sentence grammaticality is generally not explicitly labelled in the pretraining data. [59] develop a hypothesis for why we may expect to see linear separation for truth [42], [60], [61]: factual statements tend to co-occur with each other while falsehoods co-occur with other falsehoods. An autoregressive objective encourages the model to learn this distinction (of which statements are factual, and the correlation over truth/false properties) to minimize loss. It is unclear to us yet whether grammaticality as a property has similar co-occurrence structure in training data – this is a testable question in the models we investigate with their training data publicly available (e.g., the Pythia and OLMo series). In future work, one may apply training data attribution methods to also finely examine what particular properties of training utterances and their statistics lead to the emergence of grammaticality encodings. Another related question that should be addressed is whether representational separation for grammaticality is inherently tied to a model’s ability to represent syntactic knowledge. If that is the case, then explanations about why, or when, we expect to find linear encodings of grammaticality can be tied to understanding the conditions that give rise to syntactic representations. Finally, there exists utility-based explanations of the development of a grammaticality encoding. While these models are not the models that are deployed as chatbots in the current age, its training data may consist of conversations or documents that involve metalinguistic information. This consists of papers about grammaticality (e.g., the BLiMP paper itself), or language instructional conversations. In our selection of models to examine in this work, we have avoided the use of chat-based models for this reason, but this information may still be present in the pretraining data. In the future, if we were to study the the development of syntactic representations alongside grammaticality representations in relation to humans, we would need to conduct the same analyses presented in this work on a model that is trained on carefully vetted data that control for the amount and type of metalinguistic information.
8.1.1 Syntactic representations and grammaticality representations↩︎
In the study of human language, syntactic representations and grammaticality are tightly linked: on generative accounts, ungrammaticality often signals the lack of a licit syntactic structure for a given expression [62].Our probes establish the existence of a grammaticality boundary, but a similar link with syntactic representations has yet to be established. Furthermore, the mechanism through which an expression is placed into the grammatical or ungrammatical region remains unknown. Future work may study whether sentences in the ungrammaticality region lack representational signatures for structure (e.g., probe failures for constituency or dependency structures: [63]), and eventually establish causal links between structural representations and the grammaticality boundary we identify.
Through our experiments, we also find that there exist specific types of grammatical contrasts that are not detectable across models. In general, it can also be asked for any specific model studied in this work why contrasts tied to specific phenomena are undetectable while maintaining robust representations of grammaticality in general. It could be the case that this representational distinction exists internally but is undetectable via mass-mean probing, but then the question of why not, for these specific phenomena, still remains. Future work may test whether characteristics specific to the phenomena in question, especially distributional properties in the training data, predict detectability in representation space.
8.2 RQ2: Distinctions between grammaticality and collinear properties of linguistic expressions↩︎
As discussed in the Introduction, a persistent argument against the claim that NLMs have grammaticality distinctions has been that they fundamentally entangle sentence likelihood and grammaticality. Through Experiment 3, we demonstrate that this is in fact not the case; we find that there is little-to-no reduction in probing performance when we decouple string probability and grammaticality, showing that the grammaticality probes reported in our experiments are not merely sensitive to string probability. In investigating the distinction between grammaticality and acceptability in Experiment 4, we have also developed methods for decoupling the many collinear factors that may exist in the encoding of a sentence representation, results from which further support the existence of a grammaticality boundary independent from those factors. We recommend using similar methods to deconfound these factors in representation space to complement probability-based measures of grammaticality.
These findings also have practical implications for researchers studying grammaticality in NLMs. In Experiment 1, we report that probability-based and probe-based methods captured different sources of variation, suggesting that they should be viewed as complementary rather than interchangeable measures. In particular, the additional variance explained by representation-based probes indicates that grammatical information encoded in sentence representations does not necessarily propagate into the model’s output probability distribution. Consequently, relying exclusively on probability-based measures may underestimate the extent to which grammatical distinctions are represented internally. One important direction for future work is to characterize the relationship between representational and probabilistic measures of grammaticality, including why the two capture systematically different sources of variation. Together with questions about how these findings relate to models’ syntactic representations [40], [64], answering these questions may clarify what aspects of grammatical knowledge are reflected in pretrained LM probabilities, with important implications for their use in computational psycholinguistics.
8.3 RQ3: The scope of grammaticality representations↩︎
We find that representations of grammaticality, whenever they are detectable for a given model and a given syntactic phenomenon, the identified probe boundary is often generalizable to other phenomena. We find that the phenomenon-general nature of grammaticality is strengthened as model size increases. In Experiment 5, we find that for a select set of models, their grammaticality encoding is language-general. These generality results further reinforce the interpretation that the representational separation is selective to grammaticality, rather than to low-level features of a minimal pair dataset correlated with grammaticality.
Connecting our findings to the broader literature, we see that grammaticality distinctions are not limited to distinguishing “naturally occurring” or frequently observed ungrammaticality. In our work, the distinctions were observable for datasets that contain ungrammatical contrasts that are commonly observed in speech errors [65], [66] and also for datasets with word shuffles that are less frequently observed but nonetheless elicit robust ungrammaticality judgments from humans. Results from [36]’s (2026) probes suggest that the grammaticality distinctions that we find are likely extendable to even more extreme cases of unnatural sentence perturbations. In future work, we plan to further isolate grammaticality by studying in what ways the grammaticality encodings reported in our study persist under various nonce-word (“jabberwocky”) manipulations to both ungrammatical and grammatical sentences.
8.4 On the utility of NLMs for human cognitive science↩︎
How do the results of this study inform larger questions related to the use of NLMs for the study of human language and cognition? As outlined in the introduction, sensitivity to grammaticality distinctions is a requisite premise for the view that NLMs can serve as models of human language, but whether this premise holds in the first place is debated. We view our work as settling this premise, thus opening up future avenues for arguing precisely, in what ways, the study of the linguistic capabilities of NLMs can contribute to the study of the human language faculty. The results suggest that the central question is no longer whether NLMs can represent grammaticality distinctions at all, but rather which distinctions they acquire, how robustly they acquire them, and under what architectural and learning conditions these distinctions emerge. By shifting the debate in this direction, our findings provide a clearer foundation for investigating the relationship between grammatical competence in artificial and human learners.
To conclude, the main contributions of this work are twofold. First, we provide a representation-based method complementary to existing probability-based methods for measuring a model’s ability to distinguish grammaticality for a given grammatical phenomenon. Second, in doing so, we demonstrate that it is indeed possible for statistical systems, in this case neural network learners of language, to exhibit robust and generalizable grammaticality distinctions. Future studies can build on this method to investigate the architectural and data conditions under which grammaticality distinctions arise, as well as the extent to which such distinctions generalize across a wider range of linguistic phenomena, languages, and and model classes. Regardless of one’s belief about whether these models can contribute directly to the study of human cognitive science, our results establish that the emergence of grammaticality distinctions is not unique to symbolic or explicitly rule-based systems, and therefore constitutes an empirical phenomenon that demands explanation in its own right.
Acknowledgments↩︎
This work is supported by a SSHRC Doctoral Fellowship (752-2024-0291) to JL. We acknowledge that the computational work reported on in this paper was performed on the Shared Computing Cluster which is administered by Boston University’s Research Computing Services. We thank tinlab at Boston University, the Group for Language and Intelligence and the Semantics Lab at Johns Hopkins University, and audiences at the GenLM reading group for their feedback on this work.
9 Methods↩︎
9.1 Dataset preprocessing↩︎
9.1.0.1 SCaMP
Datasets from SCaMP are not labelled for the syntactic phenomenon they belong to. We manually labelled the syntactic phenomenon of each dataset, with most datasets falling into phenomenon categories set up by the authors of BLiMP. In addition to the BLiMP phenomena, we categorized some SCaMP datasets into novel categories. These categories and their associated datasets are documented in Table 4.
| Phenomenon | Datasets |
|---|---|
| Adverb/Adj restriction | recursionintensifier{adj,adv}{short,medium,long} {plausible,implausible} |
| Center embedding | centerembedding{single, double}{1,2} {plausible,implausible} |
| Verb count | recursionpp{is,verb}{short,medium,long} {plausible,implausible} |
| Word order | swappedditransitive{1,2}{plausible,implausible}, {pp,adv}order{plausible,implausible}, svovos{plausible,implausible}, greenideas{plausible,implausible} |
9.1.0.2 DGL
The DGL dataset contains sentences from [4] as well as minimal pair counterpart sentences created by [46].
9.1.0.3 Shades
The Shades benchmark consists of sentences derived from the multi-condition minimal pairs stimulus set from [49]. Each sentence from the stimulus set consists of probable, improbable, impossible, inconceivable, and inconceivablesyntactic continuations at the critical
word. In particular, we selected the first four conditions (inconceivablesyntactic involves a grammaticality contrast), and created minimal pair datasets from each possible pair of conditions. This yields 4C2 = 6 datasets.
9.1.0.4 Drive
The Drive benchmark consists of sentences derived from the stimulus set of [50]. Each sentence in
the stimulus set is rated for the following measures: grammaticality, arousal, conversational, sense, frequency, imageability, othersthoughts, physical,
places, and valence. Since we are interested in extragrammatical factors, we filter the stimulus set for sentences that are rated above 4.0 on grammaticality on a Likert scale (average, across raters), rendering 1542 sentences.
For each of the extragrammatical dimension, we compute the rating mean for that dimension, and label the filtered sentences with the mean as a threshold.
9.1.0.5 Truth
We selected the following datasets from [42]: spentrans, negspentrans,
commonclaimtruefalse, cities, negcities, citiescitiesconj, citiescitiesdisj, largerthan, smallerthan, counterfacttruefalse, and
companiestruefalse.
9.2 BLiMP dataset labels↩︎
In heatmaps referencing BLiMP datasets, we used numerical labels to make space. Table 5 maps these numerical labels to each of the BLiMP datasets, along with their phenomenon group ascribed by BLiMP [5].
| # | Dataset | Phenomenon |
|---|---|---|
| 1 | anaphorgenderagreement | Anaphor agreement |
| 2 | anaphornumberagreement | Anaphor agreement |
| 3 | determinernounagreement1 | Det-N agree |
| 4 | determinernounagreement2 | Det-N agree |
| 5 | determinernounagreementirregular1 | Det-N agree |
| 6 | determinernounagreementirregular2 | Det-N agree |
| 7 | determinernounagreementwithadj2 | Det-N agree |
| 8 | determinernounagreementwithadjirregular1 | Det-N agree |
| 9 | determinernounagreementwithadjirregular2 | Det-N agree |
| 10 | determinernounagreementwithadjective1 | Det-N agree |
| 11 | irregularpastparticipleadjectives | Irregular forms |
| 12 | irregularpastparticipleverbs | Irregular forms |
| 13 | distractoragreementrelationalnoun | Subj-V agree |
| 14 | distractoragreementrelativeclause | Subj-V agree |
| 15 | irregularpluralsubjectverbagreement1 | Subj-V agree |
| 16 | irregularpluralsubjectverbagreement2 | Subj-V agree |
| 17 | regularpluralsubjectverbagreement1 | Subj-V agree |
| 18 | regularpluralsubjectverbagreement2 | Subj-V agree |
| 19 | transitive | Argument structure |
| 20 | coordinatestructureconstraintcomplexleftbranch | Island effects |
| 21 | leftbranchislandechoquestion | Island effects |
| 22 | whisland | Island effects |
| 23 | animatesubjectpassive | S-selection |
| 24 | animatesubjecttrans | S-selection |
| 25 | principleAccommand | Binding |
| 26 | principleAcase1 | Binding |
| 27 | principleAcase2 | Binding |
| 28 | principleAdomain1 | Binding |
| 29 | principleAdomain2 | Binding |
| 30 | principleAdomain3 | Binding |
| 31 | existentialthereobjectraising | Control/Raising |
| 32 | expletiveitobjectraising | Control/Raising |
| 33 | onlynpiscope | NPI licensing |
| 34 | sententialnegationnpiscope | NPI licensing |
| 35 | matrixquestionnpilicensorpresent | NPI licensing |
| 36 | npipresent1 | NPI licensing |
| 37 | npipresent2 | NPI licensing |
| 38 | onlynpilicensorpresent | NPI licensing |
| 39 | sententialnegationnpilicensorpresent | NPI licensing |
| 40 | superlativequantifiers2 | Quantifiers |
9.3 Models↩︎
Fig.13 summarizes model performance with the targeted syntactic evaluation paradigm on the BLiMP datasets, broken down by phenomenon.
10 Experiment 2↩︎
10.1 Additional accuracy profiles↩︎
We report additional accuracy profiles from models in Fig.14 across the success spectrum to better understand how phenomenon-generality differ across models and regions.
References↩︎
Code, analyses, and data are available at https://github.com/jane-lisy/linear-gram.↩︎
The word “phenomenon” here refers to labels assigned to the datasets by us or the researchers that created the benchmarks (e.g., in the case of BLiMP). We acknowledge that the concept of phenomenon may not be present at all in a particular model’s representation of sentences (though see [39]), or they may not line up with the categorizations of phenomena assigned by us or by the creators of the benchmarks examined.↩︎