We can expect that AI will lead to improvements in technologies that slow or prevent climate change, from atmospheric carbon-removal and clean energy technology to lab-grown meat that reduces our reliance on carbon-intensive factory farming.

I think of the issue as having two parts: international conflict, and the internal structure of nations. On the international side, it seems very important that democracies have the upper hand on the world stage when powerful AI is created. AI-powered authoritarianism seems too terrible to contemplate, so democracies need to be able to set the terms by which powerful AI is brought into the world, both to avoid being overpowered by authoritarians and to prevent human rights abuses within authoritarian countries.

Nevertheless, it is a thing of transcendent beauty. We have the opportunity to play some small role in making it real.

Both AI companies and developed world policymakers will need to do their part to ensure that the developing world is not left out; the moral imperative is too great.

Advanced computational neuroscience. As noted above, both the specific insights and the gestalt of modern AI can probably be applied fruitfully to questions in systems neuroscience, including perhaps uncovering the real causes and dynamics of complex diseases like psychosis or mood disorders

and resisting the temptation to rely on natural resource wealth); it’s plausible that “AI finance ministers and central bankers” could replicate or exceed this 10% accomplishmen

Given all this, many biologists have long been skeptical of the value of AI and “big data” more generally in biology

data is often lacking—not so much in quantity, but quality: there is always a dearth of clear, unambiguous data that isolates a biological effect of interest from the other 10,000 confounding things that are going on, or that intervenes causally in a given process, or that directly measures some effect (as opposed to inferring its consequences in some indirect or noisy way)

Thus, we should imagine a picture where intelligence is initially heavily bottlenecked by the other factors of production, but over time intelligence itself increasingly routes around the other factors, even if they never fully dissolve (and some things like physical laws are absolute)10

Physical laws. This is a starker version of the first point. There are certain physical laws that appear to be unbreakable

Need for data. Sometimes raw data is lacking and in its absence more intelligence does not help.

Speed of the outside world. Intelligent agents need to operate interactively in the world in order to accomplish things and also to learn

I believe that in the AI age, we should be talking about the marginal returns to intelligence77 The closest economics work that I’m aware of to tackling this question is work on “general purpose technologies” and “intangible investments” that serve as complements to general purpose technologies., and trying to figure out what the other factors are that are complementary to intelligence and that become limiting factors when intelligence is very high.

Second, and conversely, you might believe that technological progress is saturated or rate-limited by

In addition to just being a “smart thing you talk to”, it has all the “interfaces” available to a human working virtually, including text, audio, video, mouse and keyboard control, and internet access. It can engage in any actions, communications, or remote operations enabled by this interface, including taking actions on the internet, taking or giving directions to humans, ordering materials, directing experiments, watching videos, making videos, and so on

This means it can prove unsolved mathematical theorems, write extremely good novels, write difficult codebases from scratch, etc.

One thing writing this essay has made me realize is that it would be valuable to bring together a group of domain experts (in biology, economics, international relations, and other areas) to write a much better and more informed version of what I’ve produced here. It’s probably best to view my efforts here as a starting prompt for that gro

as if it’s their mission to single-handedly bring it about like a prophet leading their people to salvation.

I also think that as a matter of principle it’s bad for your soul to spend too much of your time “talking your book”.

The basic development of AI technology and many (not all) of its benefits seems inevitable (unless the risks derail everything) and is fundamentally driven by powerful market forces.

I don’t think that at all. In fact, one of my main reasons for focusing on risks is that they’re the only thing standing between us and what I see as a fundamentally positive future. I think that most people are underestimating just how radical the upside of AI could be, just as I think most people are underestimating how bad the risks could be.