The discussion about AI revolves around a couple of big companies - from frontier labs and hyperscalers to neo-clouds and chip producers. Bloomberg covered the circular nature of reciprocal investments among the big players (see graphic below).

This focus on a few big players is not only evident in media reporting and general debate, it also impacts how organizations think about their AI adoption. That conversation usually revolves around how many licenses to get from which provider and how to balance different models and hyperscalers against each other. Under the megatrend Digital Sovereignty, alternatives to established actors are also feeling lucky but follow the same idea of AI as a service or as infrastructure. The thinking goes that you rent the AI capability as the investments in performant on-premise solutions are out of reach for most organizations and not deemed sound investments given the rapidly evolving nature of the AI market.

We've been here before

Doesn't that sound familiar? That's right, we've been here before, at the height of the mainframe era. Before the personal computing revolution, you would get access to the power of machines not by buying the machine but by buying time using a machine. Only large institutions such as university research labs or major banks could afford mainframes; they then worked out how to sell or allocate time on them to customers and staff.

But technological advancements meant that personal computing became an alternative. The cost of memory and processing power decreased, miniaturization did its thing and soon enough, people who were either barred from using a mainframe or deemed it too costly had a viable alternative for some of their use-cases.

If we look at AI we see a similar dynamic: smaller models that require less compute become ever more powerful, catching up with frontier models, often while being released as open-weight models themselves. Combine this with the ongoing progress in consumer hardware and you get a trend that allows local AI to close the gap to frontier LLMs. The latest example of the often-overlooked quest to make locally run AI a reality comes in the form of the project "Bonsai", shrinking powerful models to fit on modern smartphones.

Solving the problems of the future

Just as with the PC revolution, similar drivers are at work: more control, less reliance on a supplier, better access and ideally lower costs in the long run than sticking with mainframe, i.e. the frontier lab licence. True, performance is worse, as anyone trying local AI for demanding use-cases on a standard office laptop will tell you. But that is only a snapshot, ignoring the deeper trends like growth in small-model performance and performance increases in consumer hardware.
It also assumes that the most demanding AI tasks are the ones that people care about or that performance is everything for customers, when that might not be fully accurate. For example, I ran an older open-source LLM on a reasonably modern laptop to summarize documents. Yes, I wait a bit longer than I would with my Claude licence. But it is still much faster than reading everything myself.

AI also brings a driver the PC revolution lacked: governance. The use of AI in the "mainframe" way comes with many thorny governance issues, from cost to data protection and digital sovereignty. Running AI locally bypasses many of those questions, making it interesting for use-cases involving sensitive data - which, in industries from healthcare to finance and the public sector, is probably where AI can deliver the most value.

In addition to performance, there is currently another blocker - reminiscent of the PC revolution: usability. There is still a bit of effort and tinkering required to get local AI up and running. Various software projects try to close that gap, facilitating model selection with assistants or even integrating complete use cases like RAG or transcription of meeting audio files. But the field of local AI is only beginning.

While the "big boys" are set to dominate the debate around AI, the trend for local AI should not be overlooked. This raises the question: who will be able to produce the Apple II equivalent of local AI, i.e. something that makes local AI usable for the mass market?

Subscribe to this newsletter and follow me on social media for more publications and thoughts:
LinkedIn
Mastodon