think about how many of those applications were built by people that you know didn't have the capabilities to just build this massive infrastructure they just wrote some code and deployed it to you and now you have it and now you have a superpower uh this is a a remarkable uh kind of Technology

Relays, string figures, pass-ing patterns back and forth, giving and receiving, patterning, holdingthe unasked-for pattern in one’s hands, response-ability; that is core towhat I mean by staying with the trouble in serious multispecies worlds.Becoming-with, not becoming, is the name of the game; becoming-withis how partners are, in Vinciane Despret’s terms, rendered capable.7 On-

Emergent abilities are not present in small models but can be observed in large models.

Houston, we have a Capability Overhang problem: Because language models have a large capability surface, these cases of emergent capabilities are an indicator that we have a ‘capabilities overhang’ – today’s models are far more capable than we think, and our techniques available for exploring the models are very juvenile. We only know about these cases of emergence because people built benchmark datasets and tested models on them. What about all the capabilities we don’t know about because we haven’t thought to test for them? There are rich questions here about the science of evaluating the capabilities (and safety issues) of contemporary models. 

As the metaphor suggests, though, the prospect of a capability overhang isn’t necessarily good news. As well as hidden and emerging capabilities, there are hidden and emerging threats. And these dangers, like our new skills, are almost too numerous to name.

There’s a concept in AI that I’m particularly fond of that I think helps explain what’s happening. It’s called “capability overhang” and refers to the hidden capacities of AI: skills and aptitudes latent within systems that researchers haven’t even begun to investigate yet. You might have heard before that AI models are “black boxes” — that they’re so huge and complex that we don’t fully understand how they operate or come to specific conclusions. This is broadly true and is what creates this overhang.

psychological and physical capability (i.e. the individual’s psychological and physical capacity to engage in the activity concerned, including the necessary knowledge and skills),

But at the level of the capability hierarchy where we wish to work, it seems useful to us to distinguish several different types of structuring--even though each type is fundamentally a structuring of the basic physical processes. Tentatively we have isolated five such types--although we are not sure how many we shall ultimately want to use in considering the problem of augmenting the human intellect, nor how we might divide and subdivide these different manifestations of physical-process structuring. We use the terms "mental structuring", "concept structuring", "symbol structuring", "process structuring," and "physical structuring."