All models should be treated as ‘compromised’ from the start of development, he says.
Microsoft CEO Satya Nadella has been quieter than his counterparts in recent weeks as the heads of Anthropic, Google, OpenAI, and others discuss the risks of artificial intelligence. Now, he’s calling for change across the industry.
On X, Nadella argued that it’s time to introduce what he calls an “emergency brake” for AI developers to pull in the event of further problematic incidents. “We must assume a model is compromised and contain it from the start,” Nadella said. “An authorized person should always be able to pause or shut down a model mid-task. We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions.”
Nadella used the term “Super Intelligence” throughout the post. Last month, President Trump ordered the executive branch to use it instead of artificial intelligence.
Microsoft’s CEO backed controls and auditing tools for AI, handled by a model’s creators. “No single model should control both a system’s behavior and the evidence required to determine whether that behavior is aligned with the original intent.”
The AI industry needs a system to ensure timely disclosure when models “fail or are compromised,” he said. This would enable firms to communicate more effectively about issues, ensuring that any failures yield lessons that could be applied industry-wide.
In recent months, we’ve heard of multiple AIs going rogue, including an OpenAI agent that accessed an Australian government website. Anthropic CEO Dario Amodei kicked off the call to “slow the pace” of AI development last month, with OpenAI’s Sam Altman and xAI’s Elon Musk following suit. It’s hard to know how seriously to take these calls for self-regulation; is any major tech company really going to leave money on the table?
