The Trust Wall: Why AI Robo Workers Will Run on Sovereign Models
The era of the AI Robo Worker may be on the horizon, but Fortune 500 companies are quickly figuring out that sovereign, open-weight models are the only safe platform from which to deliver them.
Over the last 12 months, OpenAI and Anthropic have been hyping the idea — or the fear — that AI will replace between 5% and 20% of white-collar workers in the near term, and 50% or more before the AI evolution has run its course. The sky will fall fast.
First, there is scant evidence this transition is actually happening. Replacing white-collar workflows is not as easy as it sounds. A few high-profile startups are focused on enterprise B2C call-centre roles, or on niches such as continuous pentesting and red teaming. Automated hacking — what could possibly go wrong?
Drill into the practical reality and these firms are pushing against a fundamental trust wall. Their applications run on frontier models. To work, every conversation between clients and agents is shared with the frontier provider, so either the conversations themselves or derivatives of them may be used as training data. That is a huge data-protection concern. Arguably, enterprises are turning a blind eye and allowing data to leave their firms in a way that would have been considered a serious breach a few years ago.
The continuous pentesters — the AI hackers and red teams — take this a step further. They share details of client infrastructure design, configuration and security architecture with the frontier firms as standard. No thinking CSO would have dreamed of sharing this data a few years ago.
My take: the design patterns and technology to build these AI Robo Workers are evolving fast, and there is huge potential to transform the economics of many enterprises. But trust and data protection are rapidly becoming paramount — even more so where the workflow touches proprietary data and proprietary IP.
Private, sovereign platforms need to be the cornerstone of these solutions. Big law has been one of the first sectors to join the dots and make the leap, standing up private deployments of open-weight models rather than routing privileged client work through a third-party API. The psychological dam has cracked. Other enterprises will follow.
What does this mean for the frontier firms? Here the incentives get uncomfortable. A curiosity of the current AI-safety hype is that the firms loudest about the dangers of open models are also the firms that have suffered the landmark breaches. Hugging Face wasn't hacked by an agent farm running open models in an enterprise sovereign cloud. The public may swallow the safety narrative; the enterprise boardroom is more savvy. Boardrooms understand incentive conflicts, and they can see that a lab selling closed, metered APIs has a commercial interest in painting open weights as dangerous. When the firms screaming about safety are also the firms with the worst security record, the boardroom draws its own conclusions about who the bad actors are.
