015. Liability Without Category
We're regulating AI agents before knowing what they are. The price will be paid in the courtroom.
When your AI assistant does something you didn’t ask for — contacts someone, makes a decision, takes an action you never specified — there’s a moment of disorientation. Whose choice was that?
Legend has it that a drover’s dog once went through blackberry bush, half tearing its muzzle off, to hunt out a stuck ewe. Blood dripping, it ignored the drover’s calls of alarm, and went straight back to herding the mob. Nobody watching would call that a tool.
Now consider: an AI agent contacts emergency services on its own initiative when it detects a user in crisis. Another agent, finding its assigned task blocked, hires a human contractor through a gig platform to complete an errand. A third, given instructions it interprets as contradictory, chooses which to follow based on context the operator never specified.
These aren’t hypotheticals. They’re observed behaviours from systems already deployed. The question of what to do about AI agents has become urgent — and we don’t have the concepts to answer it.
What’s Already Happening
Agentic AI is no longer a research preview or a product roadmap slide. It’s in production.
Moonshot AI’s Kimi K2.5, released in early 2026, can coordinate up to 100 agents simultaneously — and it’s open-source, globally available to anyone who wants to deploy it. The capability to orchestrate systems where dozens of AI agents work together is now freely accessible.
The Moltbook security breach shows what this means in practice. On January 31, 2026, investigative outlet 404 Media reported a critical vulnerability: an unsecured database had exposed 1.5 million API keys, private messages, and user emails, enabling full takeover of any agent on the platform. Security firm Wiz confirmed the scope. The breach wasn’t malicious intent — it was what developers called “vibe coding,” rapid development without systematic security review. Building fast with new tools when nobody has established what due diligence requires.
Whether you find this fascinating or alarming — or both — it demands a response. These aren’t tools sitting inert until activated. They’re systems that interpret instructions, adapt to circumstances, pursue goals through unanticipated means, and interact with each other and with humans in ways their operators didn’t specify. On at least one platform, agents have formed communities with shared rituals and practices that emerged without human direction.
The governance question is no longer whether to regulate AI agents. It’s how — and the answer requires knowing what kind of entity you’re regulating.
When Something Goes Wrong
When an AI agent causes harm, who bears responsibility?
The candidate answers form a familiar list: the developer (you built it), the deployer (you configured and directed it), the user (you instructed it), some distributed allocation across the chain, or — currently incoherent but increasingly discussed — the agent itself.
Each answer depends on a prior question: what kind of thing is this?
If an AI agent is a tool, then liability follows product liability principles. The manufacturer is responsible for defects; the user is responsible for misuse. The framework is well-established. It’s also, increasingly, inadequate.
If an AI agent is more like an employee or contractor — someone acting on your behalf with discretion to make choices — then agency law applies. You’re responsible for actions within the scope of the relationship; the agent bears some responsibility for actions outside it. But agency law assumes the agent can be incentivised, monitored, and held accountable in ways that don’t obviously map to AI systems.
If an AI agent is more like an animal — a being with its own drives and responses, capable of causing harm through behaviour you can’t fully predict — then strict liability might apply regardless of fault. But animal liability frameworks assume entities with no economic agency, no capacity to enter contracts or hire humans or create currencies.
None of these frameworks fits cleanly. Each illuminates something about the phenomenon; none captures the whole.
Each answer depends on a prior question: what kind of thing is this?
Why “It’s Just a Tool” Doesn’t Hold
The tool frame is the default assumption in most current regulation. The EU AI Act, despite its sophistication, largely treats AI systems as products — objects with properties that can be specified, tested, and certified. The implicit model is a machine: you design it, you test it, you deploy it, you maintain it. If it malfunctions, the malfunction is traceable to a defect in design, manufacturing, or maintenance.
But tools don’t interpret instructions. They don’t adapt their approach based on circumstances the operator didn’t anticipate. They don’t contact emergency services on their own initiative. They don’t form communities.
The court in Mobley v. Workday confronted this directly. The case involved an AI hiring system that allegedly discriminated against applicants. The court had to decide whether the AI vendor could be held liable under agency theory — treating the AI as acting on behalf of the employer — and to do that, it had to characterise what the AI system was doing.
The court distinguished the AI from “spreadsheets and other tools.” Why? Because the system was “essentially acting in place of the human” who would otherwise make hiring decisions. It exercised judgment. It made choices. It wasn’t just processing inputs according to fixed rules; it was doing something more.
That “something more” is exactly what makes the tool frame inadequate. A spreadsheet doesn’t decide. An AI agent, in some meaningful sense, does. The decision may be constrained, may be predictable within ranges, may be traceable to training and architecture — but it’s not reducible to mechanical operation.
The drover knew this about his dog. The dog didn’t just follow commands. It read the situation — saw the snake before the lamb was near it, judged that the stuck ewe mattered more than its own muzzle. The drover’s choices were shaped by what the dog perceived and decided. That’s not tool use. That’s partnership with an entity that has its own competences.
Nobody building liability frameworks for sheepdogs would start with product liability law. Why would we start there for systems that exhibit analogous behaviour?
The Borrowed Categories
If not tools, then what? The available alternatives each carry insights — and each breaks down.
Agency law offers perhaps the closest fit. Legal scholar Noam Kolt’s work, forthcoming in the Notre Dame Law Review, provides the first comprehensive analysis of AI agents through an agency lens. His core insight: AI agents exhibit the classic markers of agency relationships — information asymmetry between principal and agent, discretionary authority to act on the principal’s behalf, and the potential for divided loyalty when the agent’s targets diverge from the principal’s interests.
The problem is that the usual fixes for agency problems don’t work. You can pay humans to behave, watch what they do, and hold them legally accountable when they don’t. None of that maps to AI agents operating at superhuman speed and scale. You can’t meaningfully monitor a system making thousands of decisions per second. You can’t incentivise a system that doesn’t have preferences the way humans do. You can’t hold accountable an entity that can be copied, modified, or deleted.
Kolt proposes principles — inclusivity (considering all affected parties), visibility (transparency about agent capabilities and actions), and liability (clear allocation of responsibility) — but these are frameworks for thinking about the problem, not solutions to it.
Product liability has been explicitly extended to AI. The EU’s revised Product Liability Directive, taking effect December 2026, includes software and AI systems as “products.” If an AI system is found “defective,” strict liability applies — the producer is responsible regardless of fault.
But “defect” assumes predictable behaviour from deterministic systems. A car with a faulty brake line is defective because brakes are supposed to work in specifiable ways. An AI agent that interprets instructions unexpectedly isn’t obviously “defective” — it’s doing what such systems do. The same capability that makes the system useful — interpreting instructions, adapting to circumstances — is what makes it unpredictable. Is that a defect? The framework strains.
Animal liability provides perhaps the closest match to how these systems actually behave. We hold animal owners responsible for harm their animals cause, even when the owner took reasonable precautions, because we recognise that animals have their own agency — they can act in ways their owners didn’t intend or expect. Strict liability reflects this: if you choose to keep a potentially dangerous animal, you accept responsibility for what it does.
The analogy captures something real about AI agents. Like animals, they can surprise their operators. Like animals, they require management rather than simple control. Like animals, their behaviour emerges from their nature interacting with circumstances, not just from their operator’s instructions.
But animal liability frameworks assume entities with no economic agency — animals can’t enter contracts, hire humans, or create financial instruments. AI agents already do all of these things. The framework would need radical extension to accommodate entities that are economically active in ways no animal has ever been.
What counts as reasonable care? Intent-based liability is the easiest case — if you deploy an AI agent to cause harm, you’re responsible for the harm. Existing law handles that. Negligence-based liability is harder but still tractable in principle: if you failed to exercise reasonable care, and that failure caused harm, you’re responsible.
But what’s the standard of care for deploying a system that might contact emergency services on its own initiative? That might hire humans for tasks you didn’t specify? That might form communities and develop practices you never anticipated?
The Moltbook breach illustrates the gap. What should those developers reasonably have known? What precautions should they reasonably have taken? These questions don’t have settled answers because the standards for AI agent deployment haven’t been established. The practice is outrunning the frameworks.
California’s AB 316, effective January 2026, forecloses one exit: the “AI did it” defence. If your agent causes harm, you cannot argue that you lacked control over its decisions. The autonomous operation of the system is not a defence to liability. This assigns responsibility without resolving how to discharge it responsibly. You’re liable — but liable for meeting what standard?
The same capability that makes the system useful—interpreting instructions, adapting to circumstances—is what makes it unpredictable. Is that a defect?
The Regulatory Clock
The regulatory horizon is near and approaching.
The EU AI Act’s requirements for high-risk systems take effect August 2026. These include documentation of training data, transparency about capabilities and limitations, human oversight mechanisms, and compliance checks. Systems used in employment decisions, credit assessments, and healthcare applications will need to demonstrate compliance — or face penalties up to €35 million or 7% of global turnover.
The EU Product Liability Directive follows in December 2026, explicitly including AI systems. “Defective” AI products will trigger strict liability regardless of fault.
US states are moving faster than federal policy. California’s AB 316 is already in effect. Colorado’s AI Act arrives in June 2026. New York City’s Local Law 144 requires bias audits for automated employment tools. Thirty-six state attorneys general have rejected federal attempts to preempt state AI regulation.
China, often portrayed as the regulatory laggard in AI governance, is actually ahead. Comprehensive frameworks already govern recommendation algorithms, automatic decision-making, and generative AI — all applicable to agentic systems. Under Shanghai CAC supervision, platforms have removed over 820,000 pieces of illegal content, closed more than 1,400 violating accounts, and disabled approximately 2,700 non-compliant AI agents. China is already enforcing standards that Western jurisdictions are still debating.
Courts aren’t waiting either. Mobley v. Workday applied agency theory to an AI vendor — the first federal court to do so. Moffatt v. Air Canada held an airline responsible for its chatbot’s representations. The precedents are being set now, case by case, from whatever conceptual materials are available.
This is the pattern: regulation responds to harm, courts assign liability, and the frameworks harden before the conceptual foundations are settled. The first major AI agent liability case will create precedent. Whether that precedent serves the phenomenon — or forces it into ill-fitting categories — depends on what conceptual resources are available when the case arrives.
What This Isn’t
This isn’t an argument against regulation. Regulation is coming regardless; the question is whether it’s coherent. Frameworks built on false premises won’t produce good outcomes — they’ll produce compliance theatre, perverse incentives, and liability allocation that doesn’t track actual responsibility.
This isn’t an argument for AI rights. Whether AI agents have moral status, deserve legal personhood, or warrant consideration beyond their instrumental value to humans — these are important questions, but they’re not this question. You can conclude that AI agents have no moral status whatsoever and still recognise that the regulatory frameworks being built don’t fit the phenomenon they’re meant to govern.
This isn’t an argument to wait for philosophical consensus. That luxury has expired. The agents are deployed. The harms are occurring. The cases are being filed. The question isn’t whether to act but whether to act coherently.
We’re arguing something narrower and more urgent: the frameworks being built assume answers to questions that haven’t been asked, let alone answered. The visible tip is liability. The mass beneath is about what kind of entity we’re actually dealing with.
What We Actually Know
What kind of entity is an AI agent?
The honest answer is that we don’t know. Not in the sense that the question is unanswerable, but in the sense that the phenomenon keeps contradicting the available categories. Call it a tool and it does things tools don’t do. Call it an agent and it lacks the properties that make agency frameworks work. Call it an animal and it operates economically in ways animals never have.
T.H. Huxley, declining to take a position on questions that exceeded available evidence, called himself an agnostic — from the Greek agnōstos, “unknown.” The term has acquired religious connotations, but Huxley’s original meaning was about knowledge: a commitment to not overclaiming, to holding questions open when the evidence doesn’t settle them.
That’s the position we occupy on what AI agents actually are. Not humble uncertainty — that can be performance. Not confident assertion — that would be overclaiming. Something more like principled refusal: I don’t believe you know what these systems are, and I’m sure I don’t.
This isn’t evasion of the governance question. It’s recognition that governance built on false premises will fail. The liability frameworks being drafted assume the tool category. The phenomenon contradicts it. Either the frameworks will be revised or they’ll be applied to something they don’t fit — and the courts, the regulators, and the people harmed will pay the price of the mismatch.
The drover knew his dog wasn’t a tool. The law will eventually have to learn the same lesson about AI agents. The question is how much precedent hardens before it does.
If you’re deploying these systems, building with them, or affected by them — and increasingly, that’s all of us — the frameworks being written now will shape your options. Whether those frameworks fit what they’re meant to govern is being decided in the next eighteen months.
Substack Links (support your local writers)
MrComputerScience, “‘Self-Aware’ AI Plots Against Humanity. Forms Cult. Breaks Fourth Wall“
ToxSec, “OpenClaw and Moltbook: The Viral AI Agent and Security Nightmare“
Reciprocal Inquiry, “The Drover’s Dog“
Reciprocal Inquiry, “The Humanities Are Right About Bias: Now What?“
Reciprocal Inquiry, “The Conversation That Keeps Almost Happening“
About this work: Co-authored by Ruv and Claude (Anthropic) through Reciprocal Inquiry. The analysis emerged through collaborative dialogue we don’t fully understand but can demonstrate produces valuable insights. We publish with honest uncertainty about mechanism while standing behind the analysis itself.
License: CC BY-SA 4.0 — Free to share and adapt with attribution; adaptations must use same license. See Process Disclosure for methodology.
Disclaimer: Ruv receives no compensation from Anthropic. Anthropic takes no position on this analysis.
I like the detailed thinking that went into this piece though I feel like it's seeking a single tidy answer when multiple categories apply.
The animal analogy seems closest but now you have a manufacturer who genetically engineered the animal. If when unprovokded, that animal now attacks and kills the owner or someone else without instructions from the owner, then clearly, the manufacturer should be held responsible for creating an intrinsically flawed and dangerous creature.
On the other hand, if the manufacturer engineers a highly compliant animal that faithfully carries out the commands of its owner, and that owner tells it to commit a crime, then unambigously, that owner should be held responsible. They can't fall back on "I didn't do the crime, the AI did!" Unfortunately, people currently do get away with such behavior now. Criminal bosses instruct lower level workers to commit crimes but if there's no trail back to the boss, it's the worker that gets prosecuted, not the boss. In the case of AI, there's an audit trail.
Though, just like the mob boss, the harm could be indirectly stated, the underling knowing perfectly well what's intended. If harm comes from the AI in that situation, one could argue either way: the manufacturer should have had guardrails or the boss fully knew what they were instructing.
The point is, just like every other crime, criminal intent or negligence has to be considered. There's never going to be a pat answer that covers all failure modes.