A staff member came to me excited last month. Claude had built her something brilliant — a practice management dashboard that would transform how we track client deadlines. I looked at it and saw the gap immediately. The data that would make it meaningful lives in three systems that don't talk to each other the way the dashboard needs. No pipeline. No architecture. A beautiful output with no viable input.
She was disappointed. I was frustrated. And the training she'd completed last quarter didn't prepare her for any of this.
That gap — between what she envisioned and what she could build — shows up in every CAS practice I talk to. Teams aren't failing because AI doesn't work. They're hitting the same four walls, and nothing in their training teaches them to get past them.
The four walls your team keeps hitting
These aren't theoretical categories. They're patterns you've already seen in your own practice.
The better Google. Staff treat AI as a smarter search engine. "How do I fix a bank rec problem in Xero?" instead of "Here are the three transactions that won't match, here's what I've ruled out, here's the client's history on this account — what am I missing?" Same question they'd type into Google, slightly better answer. That's not AI adoption. That's a search upgrade.
The platform wall. Staff try to use AI on problems inside Xero or QBO — categorize these transactions, reconcile this account. AI can't reach into those platforms from the outside. The staff member concludes AI doesn't work. The approach is wrong — and they don't know enough about what's possible to find the path around it.
The output fantasy. The dashboard story. Staff see AI produce something impressive and want to deploy it tomorrow. They haven't thought about where the data lives, what it takes to make a tool work firm-wide rather than just on their screen. The vision outruns the infrastructure by months.
The deployment gap. Someone builds a workflow that works beautifully — for one client, on one machine. You try to scale it. Different clients, different structures, edge cases multiplying. What works for one person is an experiment. What works for everyone is engineering.
What every failure has in common
Every one of these is a gap between vision and architecture. The staff member can see what they want AI to do but can't decompose the problem into buildable steps. Can't identify where data lives or how to move it. Can't anticipate what breaks at scale.
That's a skills problem — and the skills that solve it aren't the ones most training programs teach.
Artifacts over certificates
Every accountant understands this intuitively. You've hired someone with a Xero Advisor certification who couldn't produce a clean set of financial statements. The certificate proved they completed a module. It didn't prove they could handle a messy chart of accounts, unreconciled bank feeds, and a business owner who mingles personal and business expenses. The certificate tells you they attended. The work product tells you they can deliver.
AI training has the same problem. The skills your practice actually needs — understanding where AI can and can't reach in your tech stack, designing input pipelines, building workflows that handle edge cases, deploying at scale — don't come from video courses. They come from building.
I built an AI skill for something unrelated to accounting — searching for flights — and learned more about encoding professional knowledge in two hours than months of theory. An ugly Excel diagnostic got rebuilt as a polished interactive scoring tool in 30 seconds — because someone knew what "better" looked like. A purpose-built agent now produces Caseware-format working trial balances from raw data, replacing a paid subscription entirely. Those are artifacts. An encoded skill. A rebuilt deliverable. A shortcut that eliminated a tool. Each one is proof someone can think through inputs, architecture, edge cases, and deployment.
Why the stakes just got higher
Here's where the landscape sharpens the argument. This week, two companies — one in coding, one in customer service — independently proved that training AI on real domain interaction data outperforms the most expensive general-purpose models. The coding tool ships improvements every five hours from actual user sessions. The customer service model resolves two million issues a week with 65 percent fewer errors.
Accounting is next. Over a billion dollars in combined funding is building vertical AI for this space. When those models arrive, the practices that benefit will be the ones generating structured interaction data — the workflows, correction logs, and encoded decision rules. The artifacts. If your team has been building, their work feeds the next generation of tools. If they've been collecting certificates, their work is invisible to that pipeline.
And the capability ceiling keeps rising. Anthropic confirmed this week that its next model represents a "step change" in reasoning. The four walls don't get lower as models get smarter. They get higher — because more capable output still requires someone who can evaluate it, architect around it, and deploy it.
The question isn't whether your team is learning AI
They probably are — most firms claim to use it. The question is whether they're producing artifacts or collecting certificates. Whether they're developing the skills that break through the four walls — or developing comfort with tools they'll never fully use.
Which one are you investing in?
There's additional content for newsletter subscribers on this page covering a comprehensive AI Training Audit — a team diagnostic that assesses your firm against the four failure modes and surfaces exactly where the gaps are. The assessment works best as a team exercise, running as a guided conversation where your team discusses the questions together. The partner sees the training investment. The manager sees what scaled. The bookkeeper knows which walls the team is hitting. Together, the conversation itself surfaces more than the scores do.