Thoughts on models as insider risks. [Read more]
Satya Nadella’s Post
Satya Nadella is an Influencer
Chairman and CEO at Microsoft
2d
Thoughts on models as insider risks. https://lnkd.in/g4YKftAv
Models as Insider Risks in the Super Intelligence Era snscratchpad.com
Suggesting that we have achieved super intelligence is a myth. The present state of GenAI models does not reflect autonomous intellect. If these systems were genuinely super intelligent, Microsoft would not require extensive legal disclaimers stating that the models are unreliable and prone to errors. A system requiring constant human supervision & liability waivers contradicts the concept of super intelligence. Businesses are being asked to assume all operational risk for software that frequently generates incorrect outputs. Also, the essay argues that models should be treated like insider risks within an enterprise. If an organisation must assume that an automated component cannot be trusted and requires continuous quarantine, granting that component administrative authority over company data in the first place makes little practical sense. Thx
The insider frame is the right one, it gives boards a mental model they already understand. The gap: insiders get background checks, least-privilege access, and monitoring. Most enterprises deploy models with none of the three. Governance has to scale with the capability…
Agreed in principle, Satya Nadella. But delegating execution to AI creates a practical paradox: human accountability requires meaningful human oversight. If an operator cannot independently assess why a model acted, “oversight” becomes purely symbolic. Audit trails and access controls are a start, but without independently verifiable reasoning and enforceable intervention mechanisms, liability stays with humans while operational control shifts to the machine.
Superintelligent models aren't malicious, but like any powerful insider, they can make critical errors or be compromised. Separating the supply of intelligence from the authority over it-and externalizing all runtime safeguards-is the defining security mandate for enterprise AI.
Satya Nadella, the emergency brake has to reach the agents that never received the original human instruction. One agent assigns the work, another hands it off, and a third takes the action. That is where the chain I described in The Provenance Problem gets harder to defend. Withdrawing permission needs to travel through those handoffs, while the evidence of what already happened stays intact.
So are we actually now changing the name to Super Intelligence just because... why not?
This is the shift: CEOs aren’t just evaluating models on performance anymore — they’re evaluating them as insider risk surfaces. Trust becomes the real moat.
Great article. Separating the intelligence from who controls it makes a lot of sense. We don't trust people at work just because they're capable. We give them the right access, keep records, and can step in when needed. AI/SI should work the same way. Loved the closing point: The best/trustworthy system is the one that needs to trust the model the least.
The narrative makes a lot of sense and the recommendation to treat frontier models like privileged insiders is the right one. The part I'd stress-test is reproducing an outcome without relying on the model to attest to it. A lot of agent logging today is the model's own account of what it did, and one recent paper found nearly every harness it tested let the agent delete its own trace when asked. So the record has to come from the harness, somewhere the model can't write. Then someone has to actually replay an outcome from it. In a regulated shop, that's the first thing an auditor will ask for.
See more commentsTo view or add a comment, sign in