Every time you ask ChatGPT a question, your request triggers a data relay race. Information leaves memory, passes through a CPU for preprocessing, travels to a GPU for heavy computation, and then makes its way back and that entire journey repeats for every single word the AI generates.

The question it forces is not which model is best. It is who owns the inference layer your organization depends on, what happens when the economics of that layer stop being subsidized, and whether the thing in your pocket turns out to matter more than the thing in the datacenter.

We see continued gains from inference scaling on larger projects, suggesting they may be solvable given enough tokens.

Inter-node communication stalls: high batching is crucial to profitably serve millions of users, and in the context of SOTA reasoning models, many nodes are often required. Inference workloads then resemble more training.

Performing optimization in the latent space can more flexibly model underlying data distributions than mechanistic approaches in the original hypothesis space. However, extrapolative prediction in sparsely explored regions of the hypothesis space can be poor. In many scientific disciplines, hypothesis spaces can be vastly larger than what can be examined through experimentation. For instance, it is estimated that there are approximately 1060 molecules, whereas even the largest chemical libraries contain fewer than 1010 molecules12,159. Therefore, there is a pressing need for methods to efficiently search through and identify high-quality candidate solutions in these largely unexplored regions.