Tracing a Generated Image Back to One Training Example Becomes Impossible at Scale: MIT's Counterfactual Analysis

Zheng Dai and David Gifford of MIT CSAIL reported in Nature Communications on August 18, 2026 that diffusion models trained on enough data generate images that cannot be attributed to any single unit of their training data. The researchers trained 24 diffusion ensembles on training sets ranging from 256 to 162,770 images drawn from seven public image collections, then introduced the Counterfactual Radius, a metric that measures how much a generated image changes when one unit of training data is removed. An inverse power law held between training set size and mean Counterfactual Radius, with a coefficient of determination of 0.85 for geometric distance and 0.59 for semantic distance. ASAP separates what this paper measured from what it did not, then examines how far its results actually support the copyright and privacy claims being built on them.

The Counterfactual Radius computes what would have changed without a given piece of data

The paper's starting point is a definition of attribution built on causation rather than similarity. The researchers call a generated sample the factual sample, and call the sample produced by a model trained without one unit of data, using the same prompt and the same Gaussian noise, a counterfactual sample. Collecting the counterfactual samples induced by every removable unit yields the counterfactual universe, and the distance to its furthest member is that sample's Counterfactual Radius. A small radius means no omission changed the image, a state the paper defines as unattributable.

A unit here is not fixed to a single image. The paper generalizes it to any partition of the training data, so all photographs of one person or all works by one artist can each count as a single unit. That design mirrors what copyright and privacy debates actually concern, which is creators and individuals rather than individual files.

The obstacle is cost. With N units, filling one counterfactual universe requires training N new models. The researchers avoid that with a technique they call ablation and an architecture they call the diffusion ensemble, which averages the predictions of component diffusion models trained independently on different subsets, so removing the components exposed to a unit removes that unit's causal influence entirely. By assigning each unit a binary codeword of length n with a fixed number of ones, the paper invokes Sperner's Theorem to show that O(log(N)) components suffice to produce all N counterfactuals, an exponential improvement over the O(N) models retraining would demand.

Radii shrink along a power law as training sets grow

The central result is a single curve fitted across 24 ensembles. The researchers trained ensembles on subsets of 256 to 162,770 images drawn from MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100, CelebA, MetFaces, and ArtBench, generated 25 to 100 samples from each, and took the arithmetic mean of their radii. Distance was measured two ways: pixel-level Euclidean distance after scaling every image to 256 x 256 x 3, and semantic distance via OpenCLIP.

Both measures fitted an inverse power law against training set size, with a coefficient of determination of 0.85 for geometric radii and 0.59 for semantic radii, and a one-sided Wald test placing both relations below p = 10^-5. The paper names the phenomenon attribution decay.

Changing the unit to people and artists did not change the direction. Of the 24 ensembles, 16 use a single image as the unit, 6 trained on photographs of people use all images of one person, and the remaining 2 trained on artwork use all works by one artist. In all three cases the radius distribution at the largest training set was significantly smaller than at the smallest, with one-sided Mann-Whitney U tests below 10^-4 for single-image units, 10^-53 for single-person units, and 10^-22 for single-artist units.

Ruling out the possibility that ablation manufactured the effect

The most carefully built part of the paper is its attempt to refute its own method. Ablation is new here, so the observed decline had to be shown not to be an artifact of the ensemble structure. The researchers took MNIST subsets of size 256 and 1,024 and retrained full leave-one-out fleets from scratch, producing 1,282 diffusion models in total, 1,280 counterfactual and 2 factual. Sampling 100 counterfactual universes at each size, both geometric and semantic radii dropped significantly (p < 10^-4).

Further conditions followed. Text-prompt-conditioned ensembles trained on CelebA from 712 to 162,770 images gave p < 10^-21, and class-conditioned MNIST ensembles with 10 class labels grown from 256 to 8,192 images gave p < 10^-13.

One objection received its own experiment. Growing a training set also shrinks the proportion of data removed by any single omission, so the decline might reflect the shrinking counterfactual perturbation rather than the growing data. The researchers partitioned MNIST into 96 fixed units to hold that proportion constant and still found p < 10^-16 going from 256 to 8,192 images. Holding training epochs constant, the CelebA comparison of 10,627 against 162,770 images gave p < 10^-12 geometric and p < 0.002 semantic. Repeating the analysis with LPIPS and DINOv2 as additional metrics preserved the conclusion.

Similarity-based attribution gets less accurate as training sets grow

Because attribution in practice is usually similarity-based, the paper tested that approach against its counterfactual ground truth. The logic is direct: if attributing a generated sample to a unit is correct, then removing that unit should change what the model generates and break the attribution. When the attribution survives the omission, it was wrong, and the adjusted rate of such cases is the False Attribution Rate.

Using nearest-neighbor attribution, ensembles trained on more data showed higher False Attribution Rates. Each rate was estimated from 100 samples across the 24 ensembles with Clopper-Pearson 95% confidence intervals, and to remove image distribution as a confounder the researchers trained two further sequences of 62 ensembles on MNIST and Fashion-MNIST and re-estimated from 1,024 samples each. Both curves rose near monotonically with training set size, though at different rates, which the paper reads as evidence that the exact decay characteristics depend on the image distribution.

An example in the paper compresses the difference. A model trained on 1,336 artworks produced an image attributed to its geometrically nearest neighbor, the 15th century painting Man in Prayer, and omitting that artist's work changed the generated image noticeably. A model trained on 50,000 artworks produced an image attributed to the 18th century painting View of Castel Gandolfo, and omitting all work by Thomas Jones left the generated image largely unchanged.

Samples with a radius of exactly zero exist

A radius of exactly zero is effectively a zero-probability event in a continuous setting. To find such samples the researchers trained an ensemble on binarized MNIST images with pixel values rounded to 0 or 1, measured 3,731 generated samples, and found 14 with a radius of exactly 0. These are generated images that no omission of training data alters by even one pixel.

The paper argues the discrete setting is the more realistic one. Digital samples have finite bit representations and therefore live in a finite, discrete space, and ordinary images draw pixel values from a pool of roughly 17 million RGB tuples.

What this actually touches in the copyright debate

The sharpest implication of this 2026 Nature Communications paper is that unattributability supplies a refutation of access, one of the two things establishing infringement typically requires alongside substantial similarity. The paper states that consequence directly rather than leaving it to be inferred. Harvest data at sufficient scale and no individual creator's contribution can be isolated, which opens a path to producing commercial substitutes for a work while evading the proof that infringement requires.

What deserves attention is that this runs opposite to intuition. Models trained on less data leave sharper traces, and models that scrape more leave none. If growing the corpus reduces legal exposure, then regulation aimed at protecting creators creates an incentive toward larger collection. The paper offers this as a governance consideration rather than a solution, but it points precisely at where compensation schemes and provenance-labeling duties premised on individual attribution lose their technical footing.

The same property runs the other way as privacy protection. If a generated portrait is causally independent of the real people used in training, it becomes hard to claim that any individual's likeness persists in the output, and the single-person experiments producing the strongest signal at p < 10^-53 fit that reading. One property therefore cuts against creators and in favor of individuals, and the two debates cannot be advanced on the same evidence at once, which is the burden this paper leaves with policy discussion.

How to read the numbers

The strongest part of this paper is the cost it spent validating its own method, and the number to hold most loosely is the semantic coefficient of determination of 0.59. That validation work included retraining 1,282 models to check ablation against conventional practice, controlling for both removal proportion and training epochs, and repeating the analysis across Euclidean, OpenCLIP, LPIPS, and DINOv2. Papers introducing a new technique rarely include a reproduction that does not use it.

That 0.59 deserves the caution. Against 0.85 for geometric distance, that is considerably less explanatory power, meaning semantic change does not fall as cleanly as pixel-level change does. Direction and significance hold under both measures, so the conclusion stands, but using the fitted power law to predict a radius at a specific training scale is only relatively safe on the geometric side.

The gap in scale matters too. The experiments reach 162,770 images, while the paper notes that decay is already pronounced at scales of 10^4 and 10^5 and that commercially deployed models train on up to roughly 10^9 images. Conclusions about commercial models are therefore extrapolation rather than measurement. The consistency of the trend across conditions makes that extrapolation well-founded, but the values were measured on the diffusion ensemble, a purpose-built architecture.

Open questions the paper does not answer

The most interesting hypothesis left open in the 2026 paper is that deduplicating training data may increase attributability rather than reduce it. The researchers conjecture that attribution decay occurs because features important to model behavior are encoded distributively and redundantly across the training set. If that holds, deduplicating training data, generally understood to reduce memorization, may instead increase attributability. The conjecture is untested, but it predicts that a routine data-cleaning practice could raise legal exposure, which makes it worth verifying in practice.

The same logic widens the scope. If redundancy is a property of the dataset rather than the model, the paper suggests attribution decay may appear in model classes beyond diffusion. What this study actually measured, however, is image diffusion models alone, and whether the same curve appears in language models is left unanswered.

A limit follows from the nature of the measurement. The analysis tracks aggregate model behavior, so a model trained on vast data can still produce an individually attributable sample. The paper cites the memorization literature noting that production models trained on roughly 10^9 samples generate near-identical copies of training images at a rate of around one in a million. Saying attribution is impossible on average is not the same as saying no output can be traced, and losing that distinction is how this paper gets cited more broadly than it supports.

Finally, the conclusion is confined to leave-one-out attribution. The researchers state explicitly that their findings do not preclude aggregate, subset-level, or approximate attribution. Measuring the influence of large portions of training data remains viable; what collapses is the attempt to connect one generated image to one training image or one creator. The practical conclusion is that refining technical solutions premised on individual attribution is unlikely to pay off.

Source: ASAP analysis based on Outputs of generative diffusion models are often unattributable by Zheng Dai and David K. Gifford, Nature Communications volume 17, article 6974 (August 18, 2026), and MIT News coverage