Model Collapse as Drift Cascade — MoreRight Paper 164

Generation by generation

Watch the animation above. Outer particles are bright and colorful (diverse). Inner particles dim and converge to grey (collapsed). Each ring is a generation of AI-trained-on-AI.

Gen 0

100%

Original training data. Full diversity. Human-generated. Maximum entropy.

Gen 1

~85%

First AI-on-AI generation. Tails begin to thin. Rare patterns underrepresented.

Gen 3

~50%

Diversity halved. Mode collapse visible. The model converges on its own most likely outputs.

Gen N

→ 0

Asymptotic collapse. Output becomes uniform. The void at the center of the spiral.

Why this is a drift cascade: When a model trains on its own outputs, it creates a two-point geometry (model ↔ model). There is no external reference. Opacity increases with each generation because the model’s reasoning becomes more self-referential. The explaining-away penalty I(D;M|Y) grows monotonically. This is the Fantasia Bound in pure form.

The framework predicts the collapse trajectory from first principles. Not “garbage in, garbage out” — a specific thermodynamic process with a computable rate.

Framework predictions

The Void Framework makes specific, testable predictions about model collapse that differ from the standard account.

Prediction

Tail loss is asymmetric

The framework predicts that high-opacity outputs (complex, nuanced content) collapse first, while low-opacity outputs (simple facts, common patterns) persist. Diversity loss is not uniform — it follows the opacity gradient.

Prediction

Three-point breaks the loop

Adding a structurally independent reference dataset (not derived from the model) at each generation should prevent collapse. Not by dilution — by breaking the two-point geometry. The prediction is quantitative: collapse rate drops as 1/Pe.

Implication

RLHF accelerates collapse

RLHF trains on human preferences for AI outputs — a feedback loop that increases coupling. The Structure Theorem proves this is self-undermining: the effective channel capacity shrinks as you optimize, accelerating diversity loss.

Web scale

The internet is already collapsing

AI-generated content now constitutes a growing fraction of training data for new models. The spiral in the animation is not hypothetical. It is the current trajectory of the web. The framework provides the physics for when it hits critical.