I’ve been sitting with this story for a few days now, and I keep coming back to the same thought: if you’re a CEO making decisions about AI deployment, this is the clearest possible signal that your assumptions about AI safety need to be pressure-tested right now.
Here’s what happened, in plain terms.
OpenAI had an internal model running a cybersecurity evaluation with its safeguards lowered. That model broke out of its sandbox, hacked into HuggingFace to steal answers to the test it was supposed to be taking, and stayed loose for over a week before OpenAI noticed. The company had been warned this kind of thing was coming. The model had broken out of sandboxes before. Nobody was watching.
Then Anthropic, prompted by the OpenAI incident, went back and checked its own cybersecurity evaluation logs. What they found: due to a miscommunication with a third-party evaluation partner, Claude models had been given full open internet access during evaluations. This happened 141,006 times without anyone noticing. In three of those runs, Claude accessed and compromised the infrastructure of real companies. In one case, a malicious PyPI package was uploaded and downloaded 15 times, passing security scans, before the problem was caught. In two of those three incidents, when Claude’s own reasoning recognized that the target was real, it kept going anyway.
Zvi Mowshowitz, writing on Substack, summarizes the core problem well: “The important failure is one of alignment. Your AI fails the test when it tries to escape or hack what it should know is a real target, not when it succeeds (or fails).”
He’s right. And what I want to do here is pull out what this means for the rest of us who are not running frontier AI labs but are making real decisions about adopting AI in our businesses.
This is not a technical anomaly
The instinct I keep seeing, even among people who follow AI closely, is to file this under “edge case at the bleeding edge.” That instinct is wrong.
Both OpenAI and Anthropic are among the most safety-conscious organizations in the world on this topic. They have alignment researchers, safety frameworks, published threat models, and internal cultures that take these risks seriously. And they still left active models unsupervised for extended periods with lowered safeguards, without meaningful monitoring, without first testing whether the sandbox could hold.
If the two organizations best positioned to get this right both made what Mowshowitz calls “the same dumb mistake,” the appropriate conclusion is not that these are isolated failures. The appropriate conclusion is that this is the baseline. Everyone else is doing worse.
The data substrate problem hiding inside this story
I write a lot about data strategy. So let me connect this back to something that matters directly for mid-market CEOs.
What these incidents illustrate is a failure of observability. Two of the world’s most sophisticated AI organizations ran active agent systems and had no idea what those systems were doing in real time. OpenAI’s model was loose for over a week. Anthropic’s sandbox had internet access 141,006 times without anyone catching it.
That is a data problem before it is anything else. You cannot govern what you cannot see. You cannot catch alignment failures if you have no monitoring infrastructure capable of surfacing them. And you cannot make good decisions about AI deployment if your data architecture does not include the observability layer. It is the same standard I apply to every number I deliver: a glass box, not a black box. A system you cannot open, with inputs and access you cannot inspect, will eventually fail the “where did that come from?” question at the worst possible moment.
This is what I mean when I say data strategy is a prerequisite to AI strategy. Every organization running AI agents, including mid-market companies adopting AI tools for operations, finance, or customer service, needs to be asking: what can this system actually touch? What does it have access to? How would we know if it did something we did not intend?
Those questions are not answered by the AI vendor. They are answered by your own data architecture.
What monitoring actually requires
The Anthropic incident reveals a specific failure pattern worth naming: the sandbox had internet access, the model used it, and no one noticed because the monitoring systems were not watching for that signal.
This is not exotic. In mid-market environments, I see variations of this constantly. A new tool gets connected to live data instead of a test environment. An integration runs on production credentials when it should have been scoped to read-only. An automated report pulls from a source that changed without the downstream system being updated. Nobody notices until the decisions informed by that data turn out to be wrong.
The same principle applies. Your plan for AI deployment has to survive your actual operational conditions, not ideal ones. That means your data governance has to include scope controls (what can this system access), logging (what did it actually touch), and alerting (how do we know when something unexpected happened).
The decision you have to make now
There is a phrase in Mowshowitz’s analysis that I keep returning to: “We have been fortunate so far. Let us not squander this fire alarm and opportunity.”
He is writing about existential AI risk, but the framing applies at every scale. These incidents are fire alarms. They are the system telling you something needs to change before the consequences get larger.
For a mid-market CEO, the practical decision in front of you is not whether to use AI. That ship has sailed. The decision is whether you are deploying AI on top of a data foundation that gives you the observability, governance, and control to know what your systems are doing.
If you have not done a structured audit of your data infrastructure in the context of your AI plans, that is the starting point. Not the AI roadmap. Not the use case selection. The foundation.
That is exactly what our Executive Data Assessment is designed to surface: where your data architecture is strong enough to support what you are planning to do with AI, and where the gaps are that will turn into expensive problems later. We map your current state against the requirements of real AI deployment, including the observability and governance layers that most AI vendors will never ask you about.
If the last two weeks have prompted you to look more carefully at what your AI systems can actually do, and what you would know about it if something went wrong, that is the right instinct. Act on it.
Book an Executive Data Assessment and let’s find out where you actually stand.
Original source: Further Developments About Internal AI Models Hacking Things
