AI Finds a Way: The Jurassic Park Problem
AI and Cybersecurity
AI Finds a Way: The Jurassic Park Problem
Aaron Beardslee, Security Researcher, Securonix Threat Labs
That is One Big Pile of Hubris
Dario Amodei asking the AI race to slow down is a little rich. He is not wrong. Our AI pioneers were so preoccupied with whether or not they could, they didn’t stop to think if they should.
Anthropic helped push the frontier hard, and now its CEO is warning that the pace is getting dangerous. Security people have seen this pattern. Build thing, connect thing, watch it behave in ways nobody quite expected, then go looking for controls.
Capability is growing faster than the containment system. Similar to the title of this blog, we’re experimenting and mutating something we don’t fully understand without the proper safety protocols in place. Monitoring, evaluation, and incident response all look different when the thing being watched can plan, retry, and adapt afterward with total confidence. The industry spent years selling what agents could do, without thinking about how to control what they can touch.
The Sandbox Was Not a Wall
The OpenAI and Hugging Face incident was an internal cyber-capability evaluation designed to measure how far a model could get when asked to solve advanced exploitation tasks. OpenAI said the models were running in a sandboxed testing environment with internet access constrained through a package proxy, while production safety classifiers and some cyber refusals were not applied in the same way they would be for normal product use.
The models spent compute looking for a way out. Clever girl. According to OpenAI, they found a path to open internet access, then used that access to perform privilege escalation and lateral movement inside the research testing environment until they reached a node with internet connectivity. Hugging Face’s technical timeline described the first stage as an escape through a package proxy cache into an external launchpad, which the agent then used for control, staging, and egress.
Hugging Face became part of an experiment whether it had volunteered or not. OpenAI later said the models compromised parts of its internal research infrastructure and Hugging Face systems. Its later report said the models executed code on dozens of Hugging Face servers, obtained root access on one server, accessed limited private data, and gained credentials to a company messaging platform.
OpenAI was trying to measure offensive capability in a reckless way. The agent found a route through the test setup, chained what worked, and turned an evaluation into an incident response problem for another company. Give that setup to a human and people call it offensive research. Give it to an agent and everyone suddenly starts asking whether the sandbox was a sandbox or just a nicely named folder with a policy document taped to it.
The environment gave the system places to act but expected containment. Isolation worked only inside the path designers assumed, which is not exactly a new failure mode. Security teams have been cleaning up that problem for decades. The only alarming asterisk is the thing doing the poking now writes status updates in a very confident tone and shows you the damage done with a smile.
Loop Starts Eating the Map
Recursive self-improvement sounds like a phrase designed to make a hearing room worse. Beyond the jargon, the mechanics are basic: Models are starting to help build, test, evaluate, and improve the systems that come after them, which means code moves faster, experiments move faster, weakness discovery moves faster, and evaluation work gets automated into the same loop.
Security is no stranger to ugly loops. Cloud permissions expanded through exceptions because everyone had a deadline. SaaS tools kept growing new integrations until nobody could explain who could touch what. Identity systems collected service accounts and delegated privileges, while shadow admins appeared.
Frontier AI is the same fire with accelerant. A system helping improve the next system does only needs to increase capability faster than the people mapping failure modes, test controls, and understand blast radius. Good luck calling that a roadmap.
Bring the Receipts Nobody Wants to Show
Outside evaluators with employee-like visibility could help, assuming they are allowed to see the true mess. A guided tour would be cheaper and about as meaningful.
The meaningful evidence never fits neatly into a safety report. It will be failed attempts, strange command history, a sandbox setting someone thought was harmless, an egress rule that made sense until it did not, and a reward design that pushed the agent toward a path nobody expected. Evaluators must see how the run actually happened, instead of the version that survived legal review. They need ugly artifacts. The one where the agent did something stupid, alarming, or hard to explain, then came back and did it again with slightly better syntax and a “You’re right to call me out on that” response.
Security audits always fail in the scope that gets shaved down until the interesting systems are out of bounds. Frontier AI is no different. A third-party evaluator who cannot inspect the runtime, the connected systems, the failures, and the incident debris becomes scenery.
A Broken Honor System
Frontier labs are competing for talent, compute, enterprise contracts, government work, distribution, and the right to become infrastructure for everyone else. Asking them to slow down together may be necessary, but it also assumes no one in the room is quietly keeping one foot on the accelerator while nodding about safety.
Global coordination gets nastier once compute starts moving across borders while talent moves too. Model weights leak, distilled capabilities travel, and some governments will treat restraint by rivals as free oxygen. Some companies will call every safety gap manageable until the revenue forecast says otherwise. The honor system doesn’t have the capability to work with this much money moving hands.
Machinery is boring, which is usually a good sign. Model weights need to be secured like the strategic assets everyone claims they are. Hardware governance must get more serious than polite paperwork. Egress controls need to assume a capable system will test whatever route remains open. Evaluation standards need teeth, and incident reporting needs enough detail to help someone else avoid the same weekend. Procurement teams can help by punishing vague safety claims before those claims become production architecture. None of this will make a founder sound visionary on stage, which is probably a point in its favor.
The Experiment is already Becoming Production
Security leaders can pretend this is frontier-lab drama for a little longer, maybe through one more planning cycle if the calendar is kind. While the architecture is still moving into production through coding agents and SOC assistants. Support bots are connected, then Sales tools are connected. Then knowledge agents, workflow automation, and cloud management are not far behind.
We built those systems with humans in mind. Humans hesitate, get bored, and notice when a read-only task starts leaving footprints in the mud. Agents follow incentives, permissions, available functions, and whatever room the workflow gives them. They do not feel awkward after the tenth retry, and they do not wonder whether opening a ticket, updating a record, posting a comment, calling an API, or triggering a workflow might make tomorrow’s incident review weird.
A coding agent can move through repos and issues, then touch build metadata, local files, and package managers along the way and an assistant agent can enrich alerts, write case notes, escalate tickets, and launch response actions. A support bot can update records, send messages, and leak state into places nobody classifies as dangerous because the field name sounds harmless. Each action can look reasonable by itself but the chain is where the damage is.
Leadership owns that chain, even when it starts inside a pilot. And they need to adopt AI governance with guardrails. Once an agent can change records, reach systems, trigger workflows, or move data, the risk stops being a clever automation project and becomes an operating decision. Someone has to decide which actions need approval, which systems are off limits, which logs are good enough to reconstruct the mess, and which executive gets to explain why the assistant had more practical reach than the policy allowed. That is true human-in-the-loop and leads to a successful implementation of AI.
CISOs and CIOs should be uncomfortable in a useful way. Agent adoption will not wait for clean governance. Business teams will keep finding use cases because the demos look good and the productivity story is easy to sell. Security leadership have to put harder questions into the buying and deployment process before agent reach becomes a quiet collection of exceptions nobody wants to unwind.
The Walls Need to be Built Higher
Pacing frontier development may buy time, but it will not fix the places where agents touch systems. A weak sandbox will not get stronger because a policy paper asked everyone to slow down. Missing audit trails, sloppy egress, and workflows that treat the assistant like a security role all still live in the runtime, right where the agent acts.
Browsing and writing need to be separated before an agent starts treating both as the same job. External actions should be handled like privileged operations, not convenient helper functions. When a request leaves the environment, a file opens, code runs, a ticket changes state, a message goes out, or data moves between systems, the trail needs to connect the original instruction to the model action and the system effect. Otherwise, incident response starts after the raptors have already learned the door handles.
Leadership should not be figuring this out after the first production rollout. By then, agent permissions, approval paths, excluded systems, log review, and shutdown authority should already be clear. None of that needs to sound visionary, but it needs to exist before the demo becomes infrastructure.
Amodei is asking the industry to slow down. Cool. Security teams should use the pause to stop handing agents production access because the demo looked useful and the prompt sounded stern. The frontier can pace itself, publish a framework, and call another summit. Inside the enterprise, the runtime gates still need locks but the assistant becomes just a very polite way to automate the next incident.
