Z.ai further reduced the LLM’s hardware footprint using a method called linear attention. Usually, doubling the size of a prompt quadruples the amount of memory that a model’s attention mechanism consumes.
本周唯一一条正面证据：线性注意力确实把 KV 内存从平方降到线性。但它只作用于长上下文的 KV 缓存，权重本身照样要进 HBM。所以"架构创新会削弱 HBM 依赖"依然没被证实，最多算延缓。