"AI Makes Mistakes"

It does, but people also make mistakes, and we’ve learned how to create reliable organizations and systems in spite of it.

I learned this one personally. Last year I was helping my wife wrap up her Dad’s estate. Final tax return. I’d been leaning hard into AI tools for my own work, so naturally I figured I’d put them to work here too. I asked ChatGPT for help with the return, and it confidently told me I could save a significant amount of money by filing a separate “rights or things” return under Subsection 70(2) for a RRIF.

Great. Done. Filed.

The CRA disagreed.

Turns out there are published CRA bulletins that spell this out clearly. Subsection 70(2) does not apply to a RRIF. The information existed. The AI just didn’t have it.

But here’s the part that really got me. When I went back to ChatGPT with the CRA’s response, it didn’t back down. It offered to help me draft a letter explaining to the CRA why they were wrong. Confident. Articulate. Completely wrong.

So I did something simple. I took the entire transcript and dropped it into a different model. Claude. Asked it to fact-check the whole exchange. Within moments it found the error, cited the relevant guidance, and explained exactly where the reasoning broke.

One AI auditing another AI. And it worked.

I took three things away from that experience, and they’ve shaped how I think about every AI engagement since.

Context is everything. The AI didn’t fail because it’s stupid. It failed because I didn’t give it what it needed. If I’d been using a system with access to CRA technical bulletins, through retrieval or even just a well-constructed prompt, it probably gets this right. When AI gets something wrong, the first question isn’t “is the AI broken?” It’s “did we set it up to succeed?”

That’s a question every CEO already knows how to ask about their people. Same question. Different employee.

Check the work. I used one AI to audit another. That’s not a hack. That’s a pattern. The same way you’d get a second opinion on a legal contract or have someone review a financial model, you can build verification into an AI workflow. My mistake wasn’t using AI for taxes. It was treating one answer from one tool as gospel.

Human judgement is still the thing. I don’t tell the companies I work with to keep humans in the loop because AI can’t be trusted. I tell them to keep humans in the loop because trust has to be earned. You start with tight oversight. You watch where the system is reliable. You gradually extend its reach as it proves itself. That’s not an AI management strategy. That’s just management. That’s how you’d onboard a new hire.

Here’s my take for any CEO who’s holding back on AI because “it makes mistakes.”

You already run an organization full of people who make mistakes. You’ve spent your career building processes and culture around managing that reality. AI isn’t different. It’s new. The question isn’t whether it’s perfect. Nothing in your business is. The question is whether you’re set up to get the most out of it while catching the errors that matter.

That’s not a technology problem. It’s a leadership problem.

And you already know how to lead.