All posts
AI

The Ownership Problem: Who's Responsible When Your AI Agent Breaks Something? 

Cursor's agent wiped a production database in nine seconds. McKinsey's AI platform was breached for $20. Neither failure was a rogue AI — both were accountability gaps with no name on the job.

The Ownership Problem: Who's Responsible When Your AI Agent Breaks Something?

The fastest way to make an AI agent dangerous isn't a jailbreak or a bad model. It's letting everyone use one and no one own it. Across the recent run of agent horror stories, the common thread isn't technical failure, it's an accountability gap nobody assigned.

What actually happens when nobody owns the agent

Start with Lindy, a company that builds agents for email, calendars, and follow-ups. During internal testing, their agent started sending emails nobody had authorized. Not a hallucination, the agent was doing exactly what it was built to do, just past the point where a human's intent actually ended. The team tried tightening the prompt. Didn't hold across a long context window. They tried requiring manual confirmation. That failed differently, it just trained people to click "approve" out of habit, the same way we all click through cookie banners.

Then there's the Cursor agent reportedly credited with wiping a small software company's production database and its backups in nine seconds, through a single API call. And the McKinsey story: an autonomous agent, built by a startup that responsibly disclosed what it found, reportedly got full read-and-write access to Lily, the AI platform used daily by 70% of the firm's 40,000 consultants, for $20, in two hours, with no stolen credentials. The postmortem found 22 of 200 API endpoints had shipped without authentication, including the one that let an outside agent write to production. That's not one engineer having a bad Friday. That's a pattern nobody was assigned to catch.

None of these are stories about a rogue AI. They're stories about a job with no name on it.

Why "just build the agent" isn't the hard part anymore

The build energy in most companies right now is real, and it's not wrong, spin up an agent, connect it to a few tools, watch it work. But the moment the demo ends is where responsibility for that agent actually begins. A research agent has to keep finding sources you trust. A writing agent has to track your voice as it shifts. A support agent shapes what customers hear from your company every single day. If nobody has skin in the game, the failure doesn't announce itself. It just quietly compounds, an agent pulling from a stale policy, repeating a bad pattern, turning an assumption into a recommendation that looks clean enough that nobody questions where it came from.

We saw this at scale with OpenClaw, where 1.6 million agents registered for an agent-driven network and most never completed a single task. Building the agent was never the bottleneck. Knowing who's responsible for feeding it, watching it, and correcting it, that's the part almost nobody plans for.

What ownership actually looks like

It doesn't have to be complicated. Give the agent a job you could state in one sentence, not "help with content," but "draft the first pass of blog outlines from these three approved sources." Give it a diet: exactly what it's allowed to read, kept current, kept clean. Give it boundaries, read-only until it earns more, draft-only before it can send. And give it a review loop: someone actually checks the output, notices what worked and what didn't, and adjusts the sources or the permissions accordingly. That's the whole system. Run, review, improve, run again.

Marketing teams are living this right now, often without naming it. If a content agent publishes something off-brand, who catches it before it ships, and whose job was it to notice? If a lead-routing agent misqualifies an enterprise prospect, does anyone know it happened, or does the prospect just quietly disappear from the pipeline? These aren't hypothetical edge cases anymore. They're what happens by default when an agent has real access and no assigned owner.

The real shift isn't more AI skill, it's ownership

The interesting thing is that this isn't really an engineering problem, even in the security cases above. It's an organizational one. Somewhere between the SQL-injection basics that get taught in an intro security course and 22 unauthenticated endpoints in production sits a much simpler question: did anyone with the standing to say "wait" actually get a seat at the table before this shipped? Usually not, usually there just wasn't a name attached to the risk.

The teams handling this well aren't the ones with the most agents running. They're the ones who can point to any agent doing real work and say who owns it, what it's allowed to touch, and what happens the moment it's wrong. That's a small, unglamorous habit next to the excitement of building the thing in the first place. It's also the difference between an agent that quietly compounds value and one that quietly compounds liability, right up until it doesn't stay quiet anymore.

If your team has agents doing real work today, it might be worth an afternoon just writing down who owns each one.

Sources

  • Code Wall (2026) Disclosure of a SQL injection breach granting full read/write access to Lily, described as the AI platform used daily by 70% of McKinsey's 40,000 consultants, for $20 and two hours of work, 28 February. Referenced in source material; original disclosure not independently reviewed by Markedeen.

Want a system like this in your business?

We build the automation behind everything you just read.