1 Introduction↩︎

The advent of large-scale generative models such as GPT-4 [1], DALL-E [2] and Stable Diffusion [3] has revolutionized the field of artificial intelligence. These models, trained on vast web-scale datasets, exhibit remarkable capabilities in generating text, images, and other media [4], [5]. However, as these models become more widely used, an increasing amount of generated data populates the web. This raises a critical question: what are the consequences of training generative models on datasets containing their own outputs?

Recent studies have investigated this question, revealing that training generative models on their own outputs can cause the performance of such models to progressively degrade with each model-fitting iteration, eventually rendering newer models useless. This phenomenon was consequently labeled model collapse [6]–[13] (see App. Sec 5 for summarization and discussion of prior work). Model collapse warns that democratizing access to generative models runs the risk of polluting the very data necessary to train future iterations of generative models.

To theoretically understand model collapse, [13] recently studied an analytically tractable setup involving a sequence of linear models, each trained on the predictions of the previous linear model, and demonstrated that the test error increases linearly with the number of iterations \(n\). Critically, this analysis assumed that each linear model undergoes training exclusively on data from the single preceding iteration. This assumption that data does not accumulate was similarly made by other recent mathematical and empirical investigations of model collapse [8], [9]. However, this assumption might not match the real world, where both human-generated and machine-generated data are continually added to existing data, and hence, training solely on the previous model’s generated data seems much less likely than training on combined and accumulating real and synthetic data.

In this work, we study what effect accumulating data has instead of replacing data. We begin with the analytically-tractable setup of [13] and prove that if data accumulates, the test error no longer grows linearly with the number of model-fitting iterations, but instead has a finite and relatively small upper bound independent of the number of model-fitting iterations. This finding suggests that data accumulation could serve as a robust solution for mitigating model collapse. We then test this conjecture on sequences of deep generative models trained on real data: causal transformers on text (Sec. 3.1), diffusion models on molecules (Sec. 3.2) and variational autoencoders on images (Sec. 3.3). After confirming that replacing data at every iteration indeed causes error to increase with the number of iterations, we test our conjecture that accumulating data prevents such model collapse. We empirically find that for all models and for all data modalities, accumulating synthetic data with real data prevents model collapse. Altogether, our work emphasizes the importance of considering data accumulation in the analysis of model collapse and suggests that accumulating data can be a robust approach for avoiding model collapse when training future generations of generative models on web-scale data.

2 Accumulating Data Prevents Model Collapse in Linear Models↩︎

We consider a linear regression setup, which, despite its simplicity, already gives useful insights into the cause and mitigation of model collapse. Our process is nearly identical to the setting introduced by [13], with the simple but critical modification that we assume data accumulate across model-fitting iterations. After gleaning insights from linear regression trained on toy data, we then carry our insights to deep generative models trained on real data (Sec. 3).

2.1 Notation and Preliminaries↩︎

2.1.0.1 Original Data Distribution.

Define the distribution \(P_{\Sigma,w,\sigma^2}\) on \(\mathbb{R}^d \times \mathbb{R}\) given by \((x, y) \sim P_{\Sigma,w,\sigma^2} \quad \text{iff} \quad\): \[\begin{align} \text{(Input)} \quad & x \sim \mathcal{N}(0, \Sigma), \\ \text{(Noise)} \quad & \epsilon \sim \mathcal{N}(0, \sigma^2), \text{ independent of } x, \\ \text{(Label)} \quad & y = x \cdot w^* + \epsilon. \end{align}\] The positive integer \(d\) is the input-dimension, the matrix \(\Sigma \in \mathbb{R}^{d \times d}\) is the true covariance structure of the input \(x\), the vector \(w^*\) is the true linear relationship used to generate the original data and the scalar \(\sigma\) is the level of label noise. We start at iteration \(n=1\) with \(T\) initial independent data points \((x_i, y_i)\) each following \(P_{\Sigma, w^*, \sigma^2}\), that is, \(y_i = x_i\cdot w^* + \epsilon_i\) for each \(i=1,2,\cdots, T\). We form the design matrix \(X\in\mathbb{R}^{T\times d}\) with \(x_1^\top,\cdots, x_T^\top\) as rows. We also form the vectors \(Y\) and \(E\) with \(i\)-th coordinate \(y_i\) and \(\epsilon_i\) respectively. In whatever follows, we will assume that \(X\) has full column rank, i.e., \(T\geq d\), \(X^\top X\) is invertible and the model is underparameterized.

2.1.0.2 Synthetic Data Generation Process.

We generate synthetic data from the following sequence of distributions

\[\begin{align} P_{\Sigma,w^*,\sigma^2} \to P_{\Sigma,\hat{w}_1,\sigma^2} \to \ldots \to P_{\Sigma,\hat{w}_n,\sigma^2}, \end{align}\] where \(n \in \mathbb{N}\) is the number of iterations. The scheme is outlined as follows.

• For \(n=1\):

  • Accumulating Covariates/Features: \(\tilde{X}_1 \mathop{\mathrm{\stackrel{\text{def}}{\;=\;}}}X\)

  • Accumulating Targets: \(\tilde{Y}_1 \mathop{\mathrm{\stackrel{\text{def}}{\;=\;}}}\hat{Y}_1 \mathop{\mathrm{\stackrel{\text{def}}{\;=\;}}}Xw^* + E_1\), where \(E_1 \mathop{\mathrm{\stackrel{\text{def}}{\;=\;}}}E \sim \mathcal{N}(0, \sigma^2 I_T)\)

  • Fit linear model: \(\hat{w}_1 = \tilde{X}_1^{\dagger} \tilde{Y}_1\)

  • Make predictions for the next iteration: \(\hat{Y}_2 \mathop{\mathrm{\stackrel{\text{def}}{\;=\;}}}X\hat{w}_1 + E_2\), where \(E_2 \sim \mathcal{N}(0, \sigma^2 I_T)\)

• For \(n \geq 2\):

  • Accumulating Covariates/Features: \(\tilde{X}_n^\top = [\tilde{X}_{n-1}^\top; X^\top] \in \mathbb{R}^{nT \times d}\)

  • Accumulating Targets: \(\tilde{Y}_n^\top = [\tilde{Y}_{n-1}^\top; \hat{Y}_n^\top] \in \mathbb{R}^{nT \times 1}\)

  • Fit linear model: \(\hat{w}_n \mathop{\mathrm{\stackrel{\text{def}}{\;=\;}}}\tilde{X}_n^{\dagger} \tilde{Y}_n\)

  • Make predictions for the next iteration: \(\hat{Y}_{n+1} \mathop{\mathrm{\stackrel{\text{def}}{\;=\;}}}X \hat{w}_n + E_{n+1}\), where \(E_{n+1} \sim \mathcal{N}(0, \sigma^2 I_T)\)

Here, for a matrix \(A\) with full column rank, \(A^\dagger=A(A^\top A)^{-1}\) is the Moore-Penrose pseudo-inverse of \(A\). The noise terms \(E_1, E_2, \ldots, E_n\) are independent of each other and of the covariates/features. Since \(X\) has full column rank, so does \(\tilde{X}_n\) for every \(n\geq 1\).

2.1.0.3 Test Error.

We are interested in the dynamics of the test error \(E_{\text{test}}(\hat{w}_n)\) of this sequence of linear model \(\hat{w}_1, \hat{w}_2, ...\). Note that evaluation of the model is done on the true distribution \(P_{\Sigma,w^*,\sigma^2}\), even though the model is trained on the accumulated synthetic data. For any linear estimator \(\hat{w}\) computed from the training data, we measure test error in the standard way:

\[E_{test}(w) \mathop{\mathrm{\stackrel{\text{def}}{\;=\;}}}\mathbb{E}\left[(x_{test}^T w - y_{test})^2 \right] - \sigma^2 = \mathbb{E}[\|w - w^*\|_{\Sigma}^2]\] where the expectation is taken over the training data and \((x_{test}, y_{test})\sim P_{\Sigma,w^*,\sigma^2}\) independent of the training data.

2.1.0.4 A Note on Extensions to Ridge Regression and Kernel Methods.

Although we present our results in the context of ordinary linear regression in \(\mathbb{R}^d\), our analysis can be readily extended to ridge regression and the kernel setting [14]–[17]. We leave such contributions to future work.

2.2 Precise Test Error Characterization Under Accumulating Data↩︎

Our goal is to establish an analytic formula for the test error of the \(n\)th model in the data accumulation setting. We begin by characterizing the relationship between the fitted linear parameters \(\hat{w}_n\) and the true parameters \(w^*\). We remind the reader that we assume that \(X\) has full column rank, i.e., \(X^\top X\) is invertible. Proofs are deferred to App. 7.

How does test error with accumulating data compare against test error with replacing data? Under otherwise identical assumptions, [13] showed in the data-replacing setting that the test error is given by⁵: \[E_{\text{test}}^{\text{Replace}}(\hat{w}_n) = \frac{\sigma^2 d}{T - d -1} \times \color{red}n \color{black} \label{dohmatob2024thm4p1}\tag{1}\]

When data are replaced, the test error grows linearly with the number of iterations \(n\) (Fig 2 top), with the rate of growth determined by a noise-to-signal ratio: the amount of noise per dimension \(\sigma^2\) times by the number of dimensions \(d\), adjusted by the (per-iteration) sample size \(T\). In contrast, when data accumulate, Theorem 2 shows the test error is upper bounded regardless of the number of iterations \(n\): \[E_{\text{test}}^{\text{Accum}}(\hat{w}_n) \leq \frac{\sigma^2 d}{T-d-1} \times \color{red}\frac{\pi^2}{6}\color{black}\]

This striking difference can be intuitively explained by the way data is handled across iterations. In the data replacing setting, because previous data were discarded, the model fits to the noise that each iteration of generated data introduces, with no way to recognize and ignore the noise. But in the accumulating setting, because iteration \(i\) contributes \(1/i\) to the total amount of data, the errors from the \(i\)th iteration are suppressed proportional to \(1/i^2\) (due to squared error), preventing the test error from growing indefinitely. This suggests that accumulating generated data with real data can be a robust solution to model collapse.

2.3 Numerical Confirmation of Analytical Results↩︎

To confirm the analytical results, we numerically simulate the setup. The numerics almost perfectly matched the analytics (Fig. 2): when data are replaced, the test error grows linearly with the number of iterations \(n\), with the prefactor set by the noise-to-signal ratio \(\sigma^2 d / (T - d -1)\) (Fig. 2 Top), but when data accumulate, the test error rapidly flatlines with the prefactor similarly set (Fig. 2 Bottom). For log test error and higher model-fitting iterations, see App. Fig. 9.

3 Accumulating Data Prevents Model Collapse in Deep Generative Models↩︎

The linear regression setup is useful for studying replacing data versus accumulating data because it displays the phenomenon of interest and is analytically tractable, but its applicability to deep generative models trained on real data is unclear. We thus test whether the same results qualitatively hold for in causal transformers, diffusion models, and variational autoencoders, respectively, trained on real text, molecular, and image data. We find that for all models and all datasets, replacing data causes model collapse, whereas accumulating data prevents model collapse.

3.1 Transformer-Based Language Models Pretrained on Text Data↩︎

3.1.0.1 Experiments.

We first turn to causal transformers [19] trained on text data. We pretrain 9M parameter GPT-2 [20] and 12M, 42M and 125M parameter Llama2 [21] language models for a single epoch on TinyStories [18], a 470M token GPT-3.5/4-generated dataset of short stories at a kindergarten reading level. For each model-fitting iteration \(n \geq 2\), we sample a new dataset of the same size as TinyStories from the previous iteration’s language model and then either replace or concatenate the previous dataset with the newly generated dataset. In each model-fitting iteration, we then pretrain a newly initialized model on the replaced or concatenated dataset from the previous iteration. We experiment with sampling the new datasets using temperatures \(0.3\) or \(1.0\). We chose this combination of architectures, scales, dataset, and sampling because the setup necessitates pretraining multiple iterations of language models – a computationally costly endeavor – but we also wish to study realistic conditions where generative models are high-performing and generative diverse outputs because small language models (below 10M parameters) pretrained on TinyStories were shown to be able to generate coherent-albeit-simple English sentences, this choice of architectures, scales, dataset and temperature hopefully strikes a good balance between being representative, being diverse and being computationally feasible.

3.1.0.2 Results.

We found that for all architectures, parameter counts, and sampling temperatures, as the number of model-fitting iterations increased, replacing data led to an increase in test cross entropy (Fig. 3 top). We also found that for all architectures, parameter counts and sampling temperatures, as the number of model-fitting iterations increased, accumulating data led so equal-or-lower test cross entropy (Fig. 3 bottom). Lower temperature (0.3) led to a faster increase in test error than higher temperature (1.0) (App. Fig. 10), but the trend was consistent for both temperatures. Table 1 shows samples of generated texts for GPT2 (9M) and Llama2 (125M) models at model-fitting iterations 3-5 when both accumulating and replacing data, as well as iterations 8-10 (replacing only).

3.1.0.3 Ablations.

We ablate for several additional potential confounds, in addition to generation temperature. First, when accumulating data, subsequent model iterations are trained on larger datasets than when replacing data. To control for this, we also perform experiments in which data is replaced, but the size of the (fully synthetic) dataset is grown to match the training set size in the accumulation regime. We find that model performance still degrades (albeit at a lower rate). This is shown in App. 8, Table 2, right-most column. Second, a possible concern could be that degrading performance when replacing data could be due to low model performance in iteration 1 (and thus the quality of the first synthetic dataset). We control for this by varying the amount of training performed in iteration 1 only, and find that this has no significant impact. Lastly, we find that our results are also consistent across varying dataset sizes and training epochs. These ablations are discussed in App. 6.

3.2 Diffusion Models on Molecular Data↩︎

Experiments We next turn to diffusion models trained on molecular data. We used GeoDiff [22], a geometric diffusion model for molecular conformation generation, trained on the GEOM-Drugs [23] dataset. We down-sample the training split of GEOM-Drugs to \(40,000\) molecular conformations, which we use as our initial training set, and perform \(50\) diffusion steps for each prediction.

Results Over \(8\) model-fitting iterations, we find a similar increase in test loss when replacing data matching our language model experiments, and similarly a relatively constant performance when accumulating data (Fig. 5). Unlike with language models, we found that while performance when replacing data worsens significantly in the first model-fitting iteration trained on synthetic data, it does not degrade further substantially in subsequent iterations.

3.3 Variational Autoencoders on Image Data↩︎

3.3.0.1 Experiments.

We lastly evaluate the training dynamics of variational autoencoders (VAEs) [24], [25] trained on image data. We choose CelebA [26], a widely-used dataset of 200k human faces split between train and test sets, as a balance between being a realistic dataset with many samples, color images and resolution, and computational feasibility of training multiple iterations of models on accumulating data. We discuss more details of our experiments in Appendix 9.

3.3.0.2 Results.

We find that replacing data at each iteration indeed causes model collapse: the test error rises swiftly with each additional iteration, and each iteration yields lower quality and less diverse generated faces until all model generations represent a single mode as shown in left panel of Figure 7. In contrast, accumulating data at each iteration significantly slows model collapse: the test error increases significantly slower with each additional iteration. While the diversity of generations does go down as compared in the middle and right panel of Fig. 7, it still represents major axes of variation in the dataset, such as gender, but no longer seems to generate other details, along more minor axis of the data manifold, such as glasses and accessories. We discuss further analysis of VAE reconstructions in Appendix 9.

Interestingly, unlike language modeling, the test error of accumulating data does increase with the number of iterations (albeit much more slowly than with replacing data). We also note that [7] found slightly contradictory evidence, specifically that a different architecture on a much smaller dataset exhibits fast deterioration of performance even with accumulating data. Understanding under what conditions and why these discrepancies exists is an interesting direction we leave for future research.

4 Discussion↩︎

This work explored the phenomenon of model collapse, an important concern as AI-generated content permeates the internet and finds its way into future training datasets. Prior work has shown that training on model outputs can lead to degraded performance [6], [8]–[10], [13], implying that future model training faces a difficult challenge of ensuring strict training dataset hygiene. Our findings extend these prior works to show that if data accumulates and models train on a mixture of “real” and synthetic data, model collapse no longer occurs.

We show this both theoretically for linear regression, as well as in experiments on causal transformers for language modeling, diffusion models for molecule generation, and variational auto-encoders on image data.

Taken together, these results strongly suggests that the “curse of recursion" we face is not as dire as had been feared. Nevertheless, many question worth investigating remain. For instance, in future work we would like to explore different data generation and accumulation regimes, such as additional”real” data being introduced in each model-fitting iteration and different schedules of how much synthetic data is generated at each iteration.

5 Summarization and Discussion of Related Work↩︎

A growing body of recent work has investigated the phenomenon of iteratively training models on data generated by previous models, e.g., [6]–[13] and (in a slightly different context) [27]. [6] and [10] conducted experiments replacing real training data with generated data at each iteration, assuming that the dataset size remains fixed over time. They found that this iterative retraining procedure can lead to model degradation if the proportion of synthetic data becomes too high. Similarly, [8] ran experiments with Gaussian mixture models, VAEs, and language models in which the total number of samples per iteration was held constant, and the samples always originated with the previous model rather than aggregating over time. Building on this work, [9] considered three treatments of data: fully replacing real data with synthetic data, augmenting a fixed real dataset with additional synthetic data, and mixing new real data with synthetic data at each iteration. In almost all of their experiments, they drew a fixed size dataset from the most recent model at each iteration, without accumulating data. [11] also assumed that dataset size and mixing proportions are constant over time in their theoretical stability analysis and empirical validation. [13] also theoretically analyzed a simplified setting where a sequence of linear regression models are fit to previous predictions, characterizing why collapse occurs if the data are replaced at each iteration.

The two papers we found that considered accumulating data are [7] and [9]. [9] did so in one-half of one experiment: StyleGAN2 trained on FliqrFaces 128\(\times\)​128 (App. Fig. 8). The authors concluded that accumulating data does not prevent model collapse, but merely slows it down. However, we believe that a closer examination of their results (App. Fig. 8) reveals that accumulating data causes the test error to plateau to a relatively low error with increasing numbers of model-fitting iterations. This result would support our conclusion that accumulating data prevents model collapse and does not merely delay it. The results from [7] are harder to evaluate; the paper provides scarce methods, no quantitative losses or metrics, and model collapse only seems to occur when the amount of synthetic data added per model-fitting iteration is 2\(\times\) the total amount of accumulated data. We think understanding what conditions and why these discrepancies exist is an interesting question.

6 Additional Linear Regression Analytical and Numerical Results↩︎

7 Proofs of Mathematical Results↩︎

We point out a lemma useful to prove Theorem 2.

Assuming Lemma 1 and Theorem 1, we present the proof of Theorem 2.

Finally, we prove Theorem 1.

8 Additional Details and Ablations on Language Model Experiments↩︎

Implementation Details↩︎

Model training was implemented using Huggingface Transformers [28]. Dataset generation was implemented using vllm [29].

Ablations↩︎

In addition to the experiments shown in the main paper, we conducted several ablation studies.

8.0.0.1 Controlling for dataset size.

One possible concern is that when accumulating data, the train dataset size will grow at each model-fitting iteration, meaning subsequent models will be trained on more aggregate data than their counterparts in the replacement regime. To control for this, we run experiments controlling for this. In this “replace-multiple” regime, we create a fully synthetic dataset at the end of each model-fitting iteration, but grow the size of this dataset to match that of the accumulated data in the accumulation regime. Table 2 rightmost column shows that in this regime, evaluation loss still increases over model-fitting iterations.

8.0.0.2 Generation temperature.

Most of our language model experiments were run with sampling temperature \(1.0\) during generation of new datasets. To ensure that this choice is not critical, we also run one experiment with temperature \(0.3\), and see that this shows similar results (with even larger increases in validation loss in the replacement regime than temperature \(1.0\)), as shown in Table 2, row 2, and Figure 10.

8.0.0.3 Dataset size and training epochs.

We similarly vary the size of the initial (and subsequent) training datasets and number of training epochs, and see that this has no qualitative effect on the results (Table 2, rows 3 & 4 show training on 1/5th of the TinyStories dataset for 1 & 3 epochs, respectively).

8.0.0.4 Model quality after first model-fitting iteration.

Finally, we control specifically for model (and thus synthetic dataset) quality after the first iteration, to rule out an undue influence of a “bad” first synthetic dataset on subsequent training. Figure 11 shows performance in subsequent iterations for different amounts of training in the first iteration, showing no qualitative differences.

9 Additional Details on VAE Experiments↩︎

9.0.0.1 Experiment Details

As pre-processing, we crop and down-sample the images to 64x64 pixels. We use a standard convolutional architecture for the VAE model consisting of 5 convolutional layers with 32, 64, 128, 256, and 512 channels, respectively, and a similar convolutional decoder structure. The latent space is 128-dimensional isotropic Gaussian, represented by 2 MLP layers. Each data iteration consists of 100 training epochs, after which we generate 163K new training images by sampling latents from the Gaussian prior and the passing them through the generator model.

9.0.0.2 Analysis of Reconstructions

Figure 12 shows reconstructions after replacing (left) and accumulating (center) data, compared to baseline (right). Analyzing the reconstruction of test set images also reveals interesting findings - the model trained only on data from the prior iteration has indeed collapsed and cannot represent any other classes besides the single mode it generates. Interestingly, the model trained on aggregated data still maintains it’s capabilities and generates accurate reconstructions, including smaller details such as glasses and hats. We hypothesize that this model maintains it’s generative capabilities, but these details become a more minor sub-manifold in the latent space, which is realigned with the newly-generated data, hence why they appear less often in the generated images, which use samples from the prior.

References↩︎