Who Do You Sue When the AI Is Wrong?

Somebody asks me this at nearly every talk now, and it’s usually a CFO, and it’s usually about 4 minutes after I’ve shown a model pulling a real number out of a live Essbase cube in front of the room: who do you sue when the AI is wrong?

Short answer: nobody, and I say that as a man who has read the contracts, which is not something I recommend as a hobby (I do it anyway, on planes, and Dawn has learned to book the seat across the aisle).

Let me back that up with what the courts have actually said so far, because there have been a couple of real cases and they point the same direction. In February 2024 a tribunal in British Columbia (small claims, basically) ruled on Moffatt v. Air Canada. Jake Moffatt’s grandmother had died, he asked the airline’s chatbot about bereavement fares, and the chatbot told him he could book now and apply for the discount within 90 days. That was wrong. The real policy said the opposite, and the chatbot had helpfully linked to it in the same answer, which I find funnier every time I think about it. I’ve read the decision 3 times now, mostly to enjoy the sentence where the tribunal summarized Air Canada’s position as being that the chatbot was “a separate legal entity that is responsible for its own actions,” which is the best thing any defendant has argued in my lifetime, and I have sat through a lot of vendor escalation calls. The tribunal didn’t agree, and Air Canada paid about $812 Canadian (about $600 US). Nobody sued the chatbot, nobody sued whoever built the chatbot, and the airline that put it on its website paid, because it was the airline’s website, and the decision is about 10 pages if you’d like to read it yourself.

The other case is the one every lawyer already knows, and if you’re a lawyer you’ve now heard about it at 4 straight CLEs. In 2023 two attorneys in New York filed a brief in Mata v. Avianca that cited 6 court decisions that did not exist, because ChatGPT had made them up, complete with quotes and docket numbers and (this is the part that gets me) internal citations to other fake cases. Judge Castel fined the lawyers $5,000. He did not fine OpenAI, and nobody even tried to make him, because it was the lawyers’ names on the signature block. These aren’t the precedents you’re looking for, and unlike Obi-Wan the model was fully sincere about it.

So that’s 2 rulings and the same answer both times: the vendor wasn’t in the room, and whoever handed the output to somebody else paid for it. Both amounts were embarrassing rather than ruinous, and I don’t think that part holds.

Finance is where this lands hardest, and I’ll tell you why, since I’ve spent 25 years in it. If you’re a public company CFO, you already sign your name, personally, to the financial statements under Section 302 of Sarbanes-Oxley (SOX, to everybody who has to live with it), and Section 906 adds prison time (up to 20 years of it) to the willful version. That signature has no exception for numbers that came out of a model: there’s no box on the certification that says “an AI did this part,” and your auditor, who has never once accepted “the software said so” as a control, isn’t going to start now just because the software got more articulate (and it has gotten very articulate). So when a CFO asks me who they sue, the honest answer is that the signature is now carrying more than it used to, because there are more ways than there used to be for a well-formatted wrong number to reach your desk at 6PM on the last day of the close, and I have personally watched that exact number arrive (buy me a coffee sometime and I’ll tell you which company, and the number, and how long it took to find, though the number is the boring part of that story).

What about the vendor? Read the agreement, all of it (the exhibits too), cause the part you need is never in the first paragraph. Every major model vendor’s terms say the output is provided as is, with no warranty of accuracy, and the indemnities that do exist are about copyright. Microsoft’s Customer Copyright Commitment from September 2023 and OpenAI’s Copyright Shield from that November both say, roughly, that if somebody sues you because the model reproduced their book, the vendor will defend you. Neither says a word about the model telling you the wrong depreciation schedule. That’s a real gap and I don’t think it’s an accident. Nobody wants to warrant a number that came out of a system that’s guessing, and having sold software for 25 years I don’t entirely blame them. They’ll warrant uptime at 99.9% and hand you service credits when they miss it, and they will not warrant accuracy at any number at all. (If you have a signed enterprise AI agreement with an accuracy warranty in it, I would genuinely like to see it, and I’ll buy the coffee this time.)

My bold claim, with a date on it so you can come back and mock me: by the end of 2028, the standard enterprise AI contract for finance will carry an accuracy warranty with a cap, the way every SLA carries uptime credits today, and it’ll be finance that forces it, because finance is the only department that already signs its name to numbers under penalty of law and therefore the only one that has to care. The caveat is that I’ve been writing about enterprise software for 25 years and the timelines in my books aged about as well as the screenshots did (the screenshots were all Essbase 11.1.2, which looked great in 2011 and which I’d have to install in a virtual machine to see again), so I’m sure about the direction and a lot less sure about the year.

In the meantime, what I actually do about it is plumbing. I spent the last year coding MCP servers so that a model asked for a number goes and reads it out of Essbase or Oracle EPM instead of remembering something that sounds like it. Essbase and Planning are shipping, Financial Close and Consolidations is in beta, and Lillian Buziak’s team at Caprus does the part where it has to actually work for a customer. I want to be honest that this does nothing for the liability question. All it does is move the wrong number from something the model invented to something a human typed into a cube in 2019, and it turns out that’s an enormous improvement, because when a human’s number is wrong there’s a name and a date sitting right on the cell, and you can go ask that person what they were thinking in 2019, which I have done, and they usually remember. (Edward, you wrote 15 instruction manuals for these exact products and spent 25 years billing people to install them. You are not the plaintiff in this story. At best you’re an expert witness for the other side, and at worst you wrote the catalog.)

Which brings me to the website this week, in case you got here from the front door and you’re wondering why it’s a lawsuit. Coyote vs. Acme opened yesterday, a customer suing a vendor whose premium products failed him on a schedule, and I looked at that and saw my whole career, so the site is ROSKE v. ACME for the week, I’m the plaintiff, and the court finds against me on every count, because I bought all of it voluntarily and I’d read the instructions, having written several of them. As far as I can tell that’s where AI liability actually sits in 2026, which is that there’s nobody on the paperwork but you (and I checked, twice, because I wanted there to be somebody else).

If you want something to do about it on Monday, the things that have actually held up for the finance teams I’ve watched are unglamorous: ask the vendor’s counsel, in writing, who carries the liability when an output reaches a filing (it takes 10 minutes and the answer is you, but now you have it in writing), keep a human signature on anything a regulator will eventually read, point the model at the system of record instead of at its memory, write the rollback on one page before you turn anything on, and a couple of other things that don’t fit in a sentence that is already this long. Nobody sells any of that, so nobody’s going to show it to you in a demo.

If you’ve got a contract in front of you and can’t tell what it covers, send me the paragraph (Edward@Roske.AI) and I’ll tell you what I think it means, for free, with the disclaimer that I’m a data scientist and not a lawyer. Or come argue about it in person: the Puerto Rico AI Community (I live here) meets every other month and it’s free, and the Caribbean AI Summit is October 9-10 at the Convention Center in San Juan. Come to one of them! I’ll be the one in the fedora, which narrows it down to one.

Asking good questions, Edward