Diffusion models also waste resources when the desired output is only a few tokens long. They have to do a lot more parallel work to whittle down to, say, five tokens that an autoregressive model does from beginning to end in just five steps.
文章客观地指出了扩散模型在短文本生成时的局限性，显示了平衡的观点。这值得深入了解扩散模型在不同任务长度下的效率表现，以及Google是否针对这一局限性进行了优化。
Diffusion models also waste resources when the desired output is only a few tokens long. They have to do a lot more parallel work to whittle down to, say, five tokens that an autoregressive model does from beginning to end in just five steps.
这是一个重要的技术限制说明，揭示了扩散模型在短文本生成中的效率问题。这个背景信息对于理解模型适用场景和局限性至关重要。
We argue that this gap stems from a fundamental failure of introspective consistency: AR models agree with what they generate, whereas DLMs often do not.
这是一个令人惊讶的深刻见解，揭示了扩散语言模型(DLMs)与自回归模型(AR)之间性能差距的根本原因。作者提出'内省一致性'概念，指出AR模型天生具有与自身生成内容一致的特性，而DLMs缺乏这种自我验证能力，这为理解DLMs的局限性提供了全新视角。
Mclachlan, S., Lucas, P., Kudakwashe Dube, Hitman, G. A., Osman, M., Kyrimi, E., Neil, M., & Fenton, N. E. (2020). The fundamental limitations of COVID-19 contact tracing methods and how to resolve them with a Bayesian network approach. https://doi.org/10.13140/RG.2.2.27042.66243