There’s no shortage of hype about AI, especially in the accounting world. From automation to chat assistants, we’re being told that bots can do just about everything, research, analysis, writing, even legal and tax documentation.
But every now and then, reality steps in with a good reminder: AI still gets things very, very wrong.
This week, someone shared a painfully accurate story. A potential client sent them a full “Transfer Pricing Memo” and “Services Legal Agreement” that they wanted reviewed. Problem was, the client had generated it entirely using ChatGPT.
The post quickly caught fire, not just because it was funny, but because every accountant in the group had seen something similar.
Here’s what happened, and what the group had to say about it.
AI can’t replace technical expertise (yet)
The client who submitted the memo didn’t say they wrote it with ChatGPT. But it didn’t take long to figure it out.
The language was overly polished, filled with misplaced references, and cited the wrong PCG entirely. The industry examples were irrelevant to the actual business. It was clear the model had pulled together something that sounded right, but wasn’t even close.
As the poster said:“It was clearly prepared by ChatGPT. They were a company in waste management and chatty had PCG references for offshore drilling. The whole thing was completely wrong.”
They were asked to quote on reviewing it, but replied:“I won’t be ‘reviewing’ it. I’d need to completely rewrite it.”
This is a good reminder that while AI can mimic tone, structure, and formatting, it does not understand the underlying rules, regulations, or context behind professional work.
It cannot, and should not, be trusted for anything involving real compliance risk. Not in transfer pricing. Not in legal agreements. Not in client-facing deliverables.
Clients still don’t get how AI works
This wasn’t the only story in the thread. Others chimed in with similar experiences.
One member shared they were asked to review a client’s Google Gemini research, with no mention of a fee. The assumption? That AI had done 95 percent of the work, and a quick skim from a professional would fill in the blanks.
But this misses the entire point.
AI doesn’t “know” facts. It doesn’t access real-time legislation. It doesn’t “read” the ATO website. It generates answers based on prediction, guessing the next most likely word or phrase. That’s useful for basic questions and first drafts. But dangerous when precision matters.
As someone else wrote:“Only mad dogs and fools use it for anything other than searching.”
And even then, searching might be generous.
The issue isn’t that AI is being used. The issue is how it’s being treated, like a qualified advisor, not a language model. Clients don’t see the gaps. But we do. And those gaps can cost them if we don’t catch them early.
AI works when you ask better questions
One of the more thoughtful insights came from someone who pointed out that AI isn’t necessarily wrong by default. It’s often wrong because people don’t know how to prompt it properly.
“I have found the most common reason it is wrong is because you ask it the wrong question.”
This is spot on. Most users type in vague, open-ended queries and expect detailed, accurate answers. But AI’s output is only as good as the input.
Ask it to draft a memo without giving it the full scenario, industry, structure, tone, and jurisdiction, and of course it will get things wrong.
The takeaway here isn’t that AI is useless. It’s that non-professionals should not rely on it to replace actual advice. The tool is not the expert. You are.
Research? No. Summarising? Yes.
One of the best distinctions raised in the thread was this: AI is bad at research, but great at summarising.
Give it a slab of technical writing or legal commentary, and it can boil it down into something human-readable. Use it to write email drafts, proposal outlines, or summarise legislation, once you’ve verified it yourself. Just don’t expect it to get the source data right.
As one person wrote:“Yeah it’s not good at research, but it’s great at summarising material.”
That’s the real utility for accountants and bookkeepers right now. Using tools like ChatGPT to save time on admin, prep, and communication. Not replacing high-stakes thinking. Not generating content clients will rely on for legal or tax outcomes.
There’s a reason platforms like ATO SmartDocs or TP toolkits exist. They are built with precision and verification in mind. ChatGPT is built to predict language. That’s a massive difference.
The group’s verdict: useful, but risky
The consensus in the group was clear: AI is here, it’s helpful, and most of us are using it. But we’re using it wisely. Clients? Not so much.
The group had a laugh, of course. There was a suggestion to reply to any wrong ChatGPT answer with:“Computer says no. No other explanation.”
Another added:“I love that it gets it wrong sometimes. Means humans are still the superior race.”
Humour aside, there’s a real issue behind these stories. Clients are starting to assume AI-generated content is accurate. They’re not asking you to write things anymore. They’re asking you to “review” it. But what they’ve created isn’t close to being right, and now they want you to fix it on the cheap.
That’s not just a workflow problem. It’s a positioning one.
If we’re not careful, clients will assume our role is to polish what ChatGPT writes. That’s not the job. Our job is to apply judgment, experience, context, and accuracy. That’s not review. That’s creation.
What this means for firms
Here’s the practical takeaway for your firm:
• Start educating clients. Add a line in proposals or emails: “Please note we do not review or validate AI-generated documentation. Where required, we will create compliant documents from scratch.”
• Price for rework. If someone hands you an AI draft, treat it as if they handed you an outdated Word doc from 2012. It might help with structure, but you’ll still need to do the actual work.
• Use AI internally, not externally. AI is a great tool for internal productivity. But once it becomes client-facing, you need to own what’s produced, and that means limiting its role.
• Train your team. Make sure juniors know where AI fits and where it doesn’t. The worst outcome is an internal team member trusting AI too much and introducing risk you didn’t see coming.
Final word
AI is moving fast. But so are the risks.
It’s a useful tool. It can help you draft things faster, structure ideas, summarise dense legislation, and automate the boring parts. But it cannot replace deep expertise. It cannot verify law. And it definitely cannot be trusted with client-facing compliance documents.
The good news? Most accountants already get this. The bad news? Your clients probably don’t.
If you’re seeing more “Can you review what I wrote in ChatGPT” requests, be cautious. And be clear. AI might help them sound smart. But it can’t save them from being wrong.
Just smile and say:“Computer says no.”
