For most of human history, intelligence was expensive, and if you wanted more intelligence, what did you do? You get more people, such as experts, engineers, soldiers, and analysts. But human expertise is scarce and limited. Then something changed in 2022 when GenAI went mainstream with – at first, it was mostly a surprisingly capable chat assistant; then tools came along, and now we’re building agents. We have systems that can reason, write code, search the internet, read pictures, and increasingly act on our behalf, but unlike humans, they don’t need to sleep and can keep running as long as there is compute and token budget.
All of this has led to discussions about how intelligent AI will become. But what if that’s the wrong question and the interesting question is:
What happens when intelligence becomes cheap?
Let’s think about what happened when other things became cheap. For example, when computing became cheap, we put it everywhere. When storage became cheap, we started collecting enormous amounts of it – some call it Big Data. When we make something abundant, it not only gets cheaper, but it also changes what becomes possible. And I think AI is about to do something similar to intelligence.
So imagine a company where intelligence is no longer constrained by headcount, where an engineer doesn’t just have one assistant but hundreds of agents researching, coding, testing, monitoring, and basically working around the clock. The limitation is no longer
How many intelligent people can we hire?
But
How many intelligent processes can we safely operate?
And when intelligence becomes cheap, everyone can access it, such as companies, students, defenders, criminals, attackers, and bad actors alike. We know now that AI will make *good* people more capable, but also bad actors. We’ve spent decades building security around a world where expertise was relatively scarce, and
what happens when such expertise can be deployed on demand?
There is another, more subtle dimension to this. Give an agent a goal, say “fix this problem”, and it may pursue the goal in ways its creator never intended (and we have seen these today: AI gym hack and hijacked german website)
Humans naturally apply assumptions about boundaries, authority, consequences, concepts of right and wrong, but an autonomous system may not. This makes a question like to what extent should AI systems go to achieve their given task now extremely important. This isn’t about a SkyNet scenario, but something much more mundane: we’re building increasingly capable systems where a small misunderstanding could have unintended large consequences.
Guess what? This pattern isn’t new. The internet gave us global communication, social media and co, and so thus global-scale misinformation. Smartphones put computing and cameras in billions of pockets, and changed privacy and social behavior as well. Warfare followed a similar trajectory – the rise of drones has changed the economics of warfare. Previously, achieving significant air capability required expensive aircraft, super-trained personnel, and massive infrastructure. Now relatively cheap, scalable autonomous and semi-autonomous systems are challenging that. As these capabilities became cheaper, more actors gained access to them, and now the battlefield has changed. AI feels like another step in this same pattern – except this time, the thing we’re making abundant isn’t computing.
It’s Intelligence.
Should we slow down AI then? Of course not, and I’m actually quite optimistic about AI and the journey here. The systems we’re going to build over the next decade will probably make today’s models look primitive. But as intelligence gets cheaper and capability increases, so does the need to get better at answering questions like:
- Who is allowed to do what?
- What can an agent access?
- Can we stop it?
- How do we defend against actors using such capabilities at a massive scale?
- What happens when thousands, perhaps millions of these agents interact with each other?
At some point, we may not be able to understand every action individually, and we will need to just trust the system as a whole. Maybe that’s where the next technology race begins –
not more Artificial intelligence (that is inevitable), something else, maybe Artificial Trust?
i.e the infrastructure around intelligence – the identity, permissions, security, observability, and governance layer around autonomous systems. Not because we want to put AI in a cage- on the contrary, I will say- but because we’re about to let it out of an increasing number of cages.
A hundred years from now, perhaps we’ll look back at this period and say that the biggest thing AI did wasn’t replace programmers, lawyers, or teachers but something much more fundamental –
It changed the economics of intelligence.
And when intelligence is no longer scarce, capability becomes more abundant too. That creates enormous opportunities. Which inevitably leads to more ways to fail, more ways to be abused, to be misused, so basically a whole new class of problems.
So maybe the question isn’t really
How intelligent can we make AI?
But
What kind of world do we build when intelligence is no longer scarce?
I don’t know the answer, but I suspect it will be one of the defining questions of the next decade.
