The statement We Must Act Now, an appeal to economists, policymakers and technology leaders to prepare for the social effects of transformative AI, was released last week. It’s a short, three-paragraph statement, conveniently broken up into two premises and a conclusion.

AI may become radically more powerful over the next 10 years.

This could drive an unprecedented transformation of our economy, larger than the Industrial Revolution, but unfolding over a vastly shorter time frame. It could bring risks, including large-scale job displacement, as well as opportunities such as major gains in living standards.

Economists, policymakers and technology leaders must act now to understand the economics of transformative AI and to build the incentives, guardrails, and institutions needed to steer AI in a direction that complements humans and benefits society.

Pretty sound, no?

I’m pleased to see more public debate about transformative AI. Seeing for myself how AI outperforms mathematicians in many respects, I’ve been thinking about the effects of transformative AI on white collar workers more broadly¹.

One of my main worries right now is AI populism, popular backlash against AI companies. This seems much more worrying than, say, existential risk from AI systems. AI companies usually oversell the effects of AI – they are sellers, after all – and I’m wondering what will happen if people feel like they’ve been scammed a few years later.

Suppose AI develops as rapidly as the AI companies want, with AI capable of accelerating scientific progress and automating all remote work within 2-3 years. I think this could radically improve people’s lives – but not within 3 years. It might well take 10-20 years before the effects become more palpable, entering into public consciousness. Or maybe this won’t happen at all: a big chunk of the population, those growing up in an AI world, might not appreciate how slow things moved pre-AGI². So, even in this best case scenario, I expect things to feel much the same.

We’ve already seen plenty of examples of anti-AI sentiment. At commencement speeches, the audience boos when AI is mentioned; there have been protests against data center buildout; AI company CEOs have had their homes attacked. For many people, the term ‘AI’ appears to carry the same negative connotation as ‘big tech’.

Just like many mathematicians, I also have mixed feelings about AI. On the one hand, I’m pleased to see all the advances due to AI; on the other hand, I sometimes long for the days when all mathematics was ‘artisanal mathematics’. ‘Industrial mathematics’, the process of solving open problems by creating good agent harnesses, is very different from pen-and-paper work. Some mathematicians, like David Madore, have a much more negative view on AI.

While AI might well lead to some kind of utopia, much like the scenario in Machines of Loving Grace, it will probably feel very underwhelming. We better understand the social effects of transformative AI and, in particular, the causes of AI populism.