A common 8-bit quantization saves half the space at almost no cost to quality. Going further down the road, models are smaller (and potentially - faster), but at the cost of quality
这里提供了关于模型量化的关键数据和最佳实践。8-bit(BF16到Q8)是性价比极高的“甜点”区间,能在节省一半内存的同时几乎不损失质量。而追求更激进的量化(如4-bit)则必须面对质量下降的权衡。初学者应以此为基准来选择适合自身硬件的模型版本。