Google Research Ties Diffusion Model Creativity to 1 Score-Smoothing Mechanism
Updated
Updated · Google Research · Jul 15
Google Research Ties Diffusion Model Creativity to 1 Score-Smoothing Mechanism
1 articles · Updated · Google Research · Jul 15
Summary
Google Research said diffusion models generate novel outputs because neural networks learn a smoothed score function, not a perfectly sharp one that would simply reproduce training examples.
That smoothing emerges from approximate learning and regularization, slowing denoising between known data points and creating an interpolation zone where new but plausible samples can form.
In a 1-D test with two training points—+1 and -1—the team found stronger weight decay made the learned score function smoother and pushed outputs toward points between the originals.
In higher-dimensional image-like data, the paper argues smoothing mainly reduces collapse along the hidden data manifold while preserving movement toward it, helping models balance realism with novelty.
The findings, presented at ICLR 2026, frame diffusion-model 'creativity' as a predictable mathematical effect and could guide designs that reduce memorization in uses such as image generation and drug discovery.
Is AI's 'creativity' just high-tech interpolation, or can it innovate beyond its training data?
Now that we understand AI creativity, can we build models that are guaranteed to be novel yet safe?
How does AI’s mathematical creativity challenge our understanding of human ingenuity and art itself?
AI Creativity Demystified: Stanford’s Analytic Theory Predicts 90% of Diffusion Model Outputs via Local Patch Mosaic Mechanism
Overview
Stanford researchers Mason Kamb and Surya Ganguli have revealed that AI creativity in advanced diffusion models is not mysterious, but a predictable result of their design. Their work shows that two key features—locality and equivariance—drive AI models to create new images by mixing and matching small parts from their training data, rather than just copying what they've seen. This process, called combinatorial creativity, is guided by clear rules built into the model’s architecture. The research provides the first analytic theory explaining how AI generates novel outputs, making creativity in AI understandable and controllable.