The models are finally ready. Costs of inference are getting optimized with open models, and even on-device models.

Models are tricky. Budgets prevent defaulting everyone to state-of-the-art. The legion of other models each have a personality.

At a time when many companies are blowing through their AI budgets, those token cost savings have become a major selling point for the company.

Running every query through Opus 4.7 is the fastest path to negative gross margins. The best Rest of Oz companies route across tiers of models — frontier models for the hardest tasks, mid-tier for the bulk, smaller custom or fine-tuned models where they've earned the right to use them.

a lightweight surrogate trained on them can absorb a significant portion of future traffic at near-zero marginal inference cost

The extra tokens bought something measurable. +5pp on strict instruction-following. Small. Real. So: is that worth 1.3–1.45x more tokens per prompt?

Total cost: ~$29 ($20 in CPU VMs, $9 in API calls) over ~3 hours with 4 VMs.

Artificial Analysis has also positioned Gemini 3.1 Flash TTS within its 'most attractive quadrant' for its ideal blend of high-quality speech generation and low cost.

This means 2.7x more tokens from the same GB300 NVL72-based infrastructure and power footprint, reducing the cost to manufacture each token by more than 60%.