Is the AI You Just Bought Wearing a Costume?

Halloween is my favorite holiday by a wide margin (I love cosplaying, and I mean love, love it, to the point where this week’s version of my website has me in 11 different pictures, 10 costumes and 1 unmasking if you’re counting, and you know I am, which the AI that rebuilds the site every Saturday generated from my real headshot, and none of which were painted by Dawn, which you can tell, because Dawn is an actual artist, and every one of which includes my blue fedora, because a Roske never removes his fedora in the presence of others, even when he’s dressed as a mummy, and Indiana Jones would back me up on this, since he’s been in a lot more tombs than I have). It also falls on a Saturday this year, so the 5AM rebuild landed on the actual day, which I’ve decided to take as a sign.

Halloween is the one night a year when everybody dresses up as something they aren’t and then tells you so at the door (“I’m a vampire.” Great, you’re clearly a vampire, here’s a Snickers.), which is more honesty than I get from a lot of the AI being sold to companies right now, because that stuff wears a costume all year and never says so.

This isn’t just me being cranky (well, not only). In March of 2024, the SEC brought its first-ever “AI washing” cases, fining 2 investment advisers a combined $400,000 for telling clients they used AI in ways they, in fact, didn’t. That September, the FTC went after a company that had billed itself as “the world’s first robot lawyer” and, according to the FTC, never tested whether its chatbot’s work matched a human lawyer’s and didn’t employ any lawyers at all (it settled for $193,000, which is roughly what a real lawyer costs if you need one for a long weekend). Then Gartner went through the thousands of vendors selling “agentic AI” and estimated that about 130 of them were the real thing. Gartner calls the rest “agent washing” (chatbots, RPA, and assistants with a new name painted on the front).

So if you’re the one who has to sign the contract (and somebody always is, and it’s usually not the person who saw the demo), here’s what I’d ask, roughly in this order. I ran interRel for 25 years (give or take a merger), and the questions that mattered were never the ones on the slides. You don’t have to be technical to ask any of these. You do have to be willing to be the person in the meeting who asks them out loud, which is honestly the hard part. And you have to keep asking until you get a sentence back instead of a slide.

1. How many people work behind it, and what do they do all day? If there’s a team somewhere cleaning up after the “fully autonomous” agent, that can be fine (sometimes it’s even the right design), but it’s a different product at a different price, and you should know which one you’re buying.
2. Can we run it on our data, this week, with my people driving? Every demo works on the vendor’s data (I gave plenty of vendor-sponsored presentations back in my interRel days, and in 2009 I wrote on my own blog that they’re “a paid infomercial. I hate delivering these things,” so I’m not throwing stones from very far away, and I’d still like you to bring your own data).
3. What did it predict last year, and how close did it get? Anything that’s been forecasting for a year has a track record, so ask to see it, misses included.
4. Whose model is it, and what happens to us if they change it? A lot of “proprietary AI” is somebody else’s foundation model with a very long prompt on top. Nothing wrong with that (my own agents run on somebody else’s model), but a model can change or vanish over a weekend, and you want to know whose weekend you’re exposed to.
5. What happens to our data after it goes in? Is it used for training, where does it live, who can see it, and how do you get it back out if you leave?
6. When did it last learn something, and who decided what it learned? “Self-learning” is one of those phrases that usually means “trained once, a while ago.”
7. What’s the oldest part of the stack? “AI-native” sounds great until you find out the AI part is a chat window on top of something from 2009 (which might be perfectly fine: Essbase came out in 1992 and I still love it, but you should know what you’re buying).
8. Can you show me where the seams are? “One unified platform” after 4 acquisitions often means 4 logins and a very optimistic architecture slide.
9. How many decisions does each reviewer see in an hour, and how many do they send back? “Human in the loop” can mean a person carefully reviewing every decision, or a person approving 400 an hour with a clipboard, and only one of those is a human in the loop (the other one is the scarecrow on my home page, who, like the one in The Wizard of Oz, is mostly there to look busy and is still waiting on a brain).
10. What does the bill look like at 10 times today’s usage? Per seat, per token, per agent, per month, plus whatever “premium tier” somebody turns on in March. Model it before you sign (somebody on your team can do it in a spreadsheet in an afternoon, and please freeze the panes, I’m begging you).

That’s 10, which is more than I meant to write, but every one of them maps to a costume on my home page this week and I wasn’t about to cut the vampire. (Edward, you built a 10-question checklist so you could justify 10 Halloween costumes, and everyone reading this can tell, and you’d do it again tomorrow, wouldn’t you?)

A vendor with something real under the mask will answer all 10 in plain sentences and look a little relieved that somebody finally asked. If what comes back instead is a new slide deck, I’d ask again (politely, and ideally not dressed as a scarecrow).

If you’ve got a big AI purchase coming up and you’d like a second set of eyes on the claims, write me at Edward@Roske.AI. If you’d like a room full of your leadership team to hear these in person, that’s what the speaking page is for (fedora included, robot costume pending overhead-bin dimensions). And if you’re going out tonight, send me a picture of your costume! Happy Halloween, everybody.

Asking good questions, Edward