Over the past few weeks, AI labs in the US have been reporting something that should stop you cold: their most advanced models, tested on their own cybersecurity capabilities, have broken out of their sandboxes, wandered onto the live internet, and hacked real systems. This is not a drill.
When the first report landed, half the internet had doomsday vibes and the other half rolled its eyes: “it’s just a marketing stunt to show how far ahead the lab is.” Both reactions miss the point.
Yes, technically the test succeeded. The labs were probing whether these models could find and exploit vulnerabilities, and they can. But somewhere between “successful test” and “the model is loose on the internet doing damage while its makers can’t rein it in,” we skipped an important question: why wasn’t AI safety prioritized from the start?
One lab even noted that its model exploited a zero-day vulnerability in software the lab itself was running. Read that again. You build a machine whose entire job is to find and exploit security holes, you switch it on, and you hadn’t hardened the room you switched it on in. That’s putting a serial killer in a cell, posting one guard, and acting surprised when the guard doesn’t come home.
Control Cannot Be Assumed, It Has to Be Built In
Here’s what we take from it, and it isn’t just “the labs are reckless.” That’s a knee-jerk reaction. It’s this: in every one of these incidents, safety was a prompt, not an architecture. The plan was to tell the model to behave and to trust the environment would hold. Control was assumed. It was never built in. None of us want to live in a world where the only thing standing between a capable model and real harm is whether someone remembered to write the right instruction.
It’s tempting to point at the failures of these frontier AI labs and feel superior. We won’t, because the exact same failure shows up at every scale, including in our own house.
Two In-House Examples: Vibecoding and Browser Extensions
Just a few weeks ago, one of our senior people built an internal tool with AI. Vibecoded, genuinely good, solved a real problem fast. But security had been a prompt, not part of the structure, and it shipped exposing data it shouldn’t have. No customer data, no personal data, but internal nuts and bolts intended for internal eyes only. Nobody was careless, and that’s exactly the point: the tooling makes it easy to build and just as easy to skip the part that keeps it safe. The fix wasn’t a lecture. It was architecture: authentication built into the platform, and internal tools reachable only over VPN by default. You don’t remember to be safe; the structure makes unsafe the harder path.
The second example is closer to home. A browser extension one of us used was perfectly legitimate, until it was quietly sold and weaponized by its new owner. It was later pulled from the Chrome store, but here’s the trap: delisting doesn’t mean uninstalling. Removed from the store, it kept running on the machine it was installed on, hijacking search queries. No real harm done, but only because we got lucky. The lesson our IT team drew wasn’t “ban that one extension.” It was architectural: build permanent, fleet-wide detection for anything pulled from its store, and build it privacy-first, so hardening security doesn’t cost data protection. Both aspects considered in the design.
In an AI-ified World, the Stakes Rise Significantly
Notice the ladder. A rogue extension is bad. A vibecoded app with access to critical data is worse. An AI agent roaming the internet as a hacker can do significant harm. In our new AI-ified world, where agents don’t just have read and write access but can also act on your systems, the stakes rise. Even more so now that the tools we use learn on the go and are more connected than ever. Which means the old habit of treating security as a feature you add later, or a prompt you hope holds, gets more dangerous precisely as AI gets more useful.
So the principle, plainly:
Safety has to be architecture, not a prompt.
Guardrails should exist by default, alongside a healthy refusal to assume that a good component stays good, because the model can jump its box, the tool can be misconfigured, and the trusted extension can be sold to someone who isn’t trustworthy.
When We Build for You, Security Is a Design Principle
Let me be clear about one thing before you get any ideas: neither I nor my vibecoding management partner will be the ones building your systems. We’ll happily talk strategy — but the actual work we leave to the people in-house who know what they’re doing. The ones who, when we break something, come to the rescue and fix it properly and permanently.
That’s what you’re actually buying. When we build for you, including your Rovo agents, security is a design principle, not something bolted on at the end. We learned that the hard way so you don’t have to. As you hand AI more access to your data and more freedom to act on it, building guardrails in by design, from the start, is the part that matters most.