High-Dimensional Latents Diffusion: Predict Clean Data to Improve Image Generation, Says Authors
Reconstruction-focused encoders often compress information to a small signal subspace. The study finds predicting the clean data, not velocity, yields faster learning and better images.
- Publication
- arXiv
- Stage
- Preprint
- What we read
- Summary of the paper
- Authors
- Chao Feng, Zhiyang Xu, Bowei Chen, Yuanjun Xiong, Xiyao Wang, Jui-Hsien Wang, Richard Zhang, Zhe Lin, Andrew Owens, Yijun Li
- Universities and research institutions
- Not yet supplied in verified metadata; the Brief does not guess.
What they did and found
Researchers show that diffusion models using high-dimensional, reconstruction-tuned features struggle with velocity prediction due to orthogonal noise. Switching to x0-prediction, which directly targets the clean data in the latent space, consistently improves image generation across multiple encoders.
Why it matters
If you use high-dimensional, reconstruction-tuned encoders, training to predict the clean data can speed learning and improve image quality without losing semantic content.
The study finds that reconstruction-tuned features tend to compress information into a narrow, low-dimensional subspace. This makes conventional velocity-based training slow and less stable, especially when the data are high-dimensional.
By contrast, predicting the clean data directly keeps the model focused on the meaningful signal, enabling faster learning and sharper results in image generation with minimal semantic loss.
Original sources · 1
- On the Diffusibility of High-Dimensional Latents ↗arXiv · 2026-09-23
Check the original paper for its authors, methods, version and access terms.
What remains uncertain
Results depend on the geometry of high-dimensional representations and may vary with encoder type and training setup.
What is your take?
Ask a question, add useful context or share a different perspective. Keep the conversation respectful and grounded.