Early on, Doug Engelbart recognized that what makes us capable, beyond the basic human abilities we were born, is a whole infrastructure of capabilities, with higher level capabilities depending on the execution of lower level capabilities. For example, the ability to solve a problem collaboratively depends on our ability to communicate with language, to use acceptable conventions and methodologies for working together, to identify opportunities, plan, implement, write, discuss, etc.

But this process took over two decades instead of a few hours in the case of AlphaZero because safety considerations put a limit on the extent to which each iteration of this loop could be scaled up compared to the previous one

Trump pledged as a presidential candidate to keep America out of “stupid endless wars.” But he also vowed to prevent the Islamic Republic from obtaining a nuclear weapon.

But neither Defense Secretary Pete Hegseth nor Gen. Dan Caine, the chairman of the Joint Chiefs of Staff, could immediately say whether Iran still retained the ability to make a nuclear weapon. Mr. Hegseth repeated President Trump’s assertion from the previous night that the nuclear sites had been “obliterated.” General Caine did not.

think about how many of those applications were built by people that you know didn't have the capabilities to just build this massive infrastructure they just wrote some code and deployed it to you and now you have it and now you have a superpower uh this is a a remarkable uh kind of Technology

Relays, string figures, pass-ing patterns back and forth, giving and receiving, patterning, holdingthe unasked-for pattern in one’s hands, response-ability; that is core towhat I mean by staying with the trouble in serious multispecies worlds.Becoming-with, not becoming, is the name of the game; becoming-withis how partners are, in Vinciane Despret’s terms, rendered capable.7 On-

Emergent abilities are not present in small models but can be observed in large models.

Houston, we have a Capability Overhang problem: Because language models have a large capability surface, these cases of emergent capabilities are an indicator that we have a ‘capabilities overhang’ – today’s models are far more capable than we think, and our techniques available for exploring the models are very juvenile. We only know about these cases of emergence because people built benchmark datasets and tested models on them. What about all the capabilities we don’t know about because we haven’t thought to test for them? There are rich questions here about the science of evaluating the capabilities (and safety issues) of contemporary models. 

As the metaphor suggests, though, the prospect of a capability overhang isn’t necessarily good news. As well as hidden and emerging capabilities, there are hidden and emerging threats. And these dangers, like our new skills, are almost too numerous to name.

There’s a concept in AI that I’m particularly fond of that I think helps explain what’s happening. It’s called “capability overhang” and refers to the hidden capacities of AI: skills and aptitudes latent within systems that researchers haven’t even begun to investigate yet. You might have heard before that AI models are “black boxes” — that they’re so huge and complex that we don’t fully understand how they operate or come to specific conclusions. This is broadly true and is what creates this overhang.

psychological and physical capability (i.e. the individual’s psychological and physical capacity to engage in the activity concerned, including the necessary knowledge and skills),

But at the level of the capability hierarchy where we wish to work, it seems useful to us to distinguish several different types of structuring--even though each type is fundamentally a structuring of the basic physical processes. Tentatively we have isolated five such types--although we are not sure how many we shall ultimately want to use in considering the problem of augmenting the human intellect, nor how we might divide and subdivide these different manifestations of physical-process structuring. We use the terms "mental structuring", "concept structuring", "symbol structuring", "process structuring," and "physical structuring."