A real example from our own work at innmotion. During research for a client brief, Claude cited the specific clause number of an Australian consumer protection law. The clause number was wrong. The law existed. The clause summary was approximately right. But the specific number was invented. If we'd cited it in a client document without checking, that's an embarrassing error sitting on a partner's desk.
That's the failure pattern that matters for your business. AI says things with confidence that are completely wrong, and the confident wrong answer reads exactly like the confident right one. There is no visual tell.
This is not a one-time bug. It's a structural feature of how these models work. Understanding why, and knowing how to catch it before something gets sent to a client or posted on your website, is as important as learning how to prompt well.
---
Hallucinations: When AI Makes Things Up
"Hallucination" is the industry term for when an AI model generates information that is plausible-sounding but incorrect. The model isn't lying. It doesn't have intentions. But it is filling in gaps in its knowledge with pattern-matched plausibility, and sometimes the plausible thing isn't the true thing.
This shows up in predictable places for business owners.
Statistics and research. Ask ChatGPT or Claude for "the latest statistics on tradie workforce shortages in Australia" and it will give you numbers. Some will be real. Some will be confected from surrounding context. They will all sound confident. Never use an AI-sourced statistic without checking the primary source.
URLs and citations. AI models frequently invent URLs. They will give you the name of a real organisation (say, the ATO, or the Fair Work Commission) and invent the URL for the specific page you need. Click it. It's often a 404.
Names of real companies and people. Models can get the name of a real company right and still generate a fake employee name, a fake address, or a fake contact number. For any specific factual claim about a real business, check it.
Australian-specific context. This is where the hallucination risk is highest for Australian operators. The models are trained predominantly on English-language text with a significant US bias. They know what an ABN is. They will sometimes confuse how it works with how a US EIN works. They know superannuation exists. They may give you the wrong rate, the wrong threshold, or a rule that applied three years ago.
Never use AI-generated advice on Australian tax law, employment law, or licensing requirements without checking with the ATO website, Fair Work, or a registered professional.
---
Training Cutoffs: The Information Time Bomb
Every AI model was trained on data up to a certain date. After that date, it knows nothing.
Every model is different, and the cutoff dates shift with each new version release. The right habit is to check the model's own documentation when it matters, or just ask: "What's your training cutoff?" The model will usually tell you. Gemini has web search grounding available, which sidesteps the cutoff problem for many current-event questions (more on that in Module 6).
What this means practically: if you ask any of the models about a regulation that changed recently, it may not know it changed. If you ask it about a competitor who launched in the last year, it may not know they exist. If you ask it about current material prices or building code changes, you're asking someone who read the news a long time ago.
The models are often not well-calibrated about this. They will answer questions about recent events with the same confidence they answer questions about history. The uncertainty doesn't show.
The fix is simple: for anything time-sensitive (pricing, regulations, legislation, competitor landscape), verify with a current source. Use the AI for drafting and structure. Use Google for facts that change.
---
Confident Tone on a Wrong Answer
This is the hardest problem to catch, because there's no signal.
When you Google something and get a bad result, the page usually looks sketchy. The design is terrible, the writing is keyword-stuffed, and your intuition fires.
When an AI model gives you a wrong answer, it reads exactly like a right answer. Same confident tone. Same grammatical precision. Same structured reasoning. There's no visual tell.
The discipline here is straightforward: any time you're planning to act on specific factual information that came from an AI, ask yourself where you'd find the original source. If you can't think of one, or if the information would be hard to verify, don't treat it as fact.
The rule: treat AI outputs the same way you'd treat information from a smart colleague who does a lot of reading but isn't always right. Useful starting point. Not primary source.
---
Australian-Specific Context: Where to Be Extra Careful
A few specific areas where the models are less reliable for Australian operators:
Payroll and employment. Minimum wages, award rates, superannuation percentages, and long service leave entitlements change. The Fair Work website is the source of truth, not an AI.
GST and BAS. The broad strokes are right. The specific thresholds and edge cases are where errors creep in. ATO.gov.au only.
Licences and registration. Building, electrical, plumbing, real estate, the specific licensing body and renewal requirements vary by state. AI will often give you the right category of answer (yes, electricians need a licence in Victoria) but wrong specifics (the specific board, the specific renewal period).
Local council and state planning rules. Essentially useless. Too fragmented, too frequently updated, and too sparsely represented in training data. Always go directly to your council or state planning portal.
---
How to Catch AI Errors in Practice
Develop three habits:
1. The "show me where" habit. When AI gives you a specific fact you plan to use, ask it: "Where would I verify this?" If it can point you to a real source, follow the link. If it can't or invents a URL, treat the fact as unverified.
2. The second-pass habit. After getting any factual output, ask: "What are you uncertain about in this response?" A well-designed AI model (Claude in particular is good at this) will flag where its confidence is lower. That flag tells you where to verify first.
3. The Australia-specific habit. Any time AI gives you a number, rate, threshold, or legal requirement relevant to your Australian business, mentally flag it as "needs ATO/Fair Work/state authority check." Default assumption: AI knows the concept, but may have the Australian-specific detail wrong.
---
A Real Hallucination You Might Believe
Here's a type of hallucination that catches business owners specifically. This was drafted with Claude to illustrate the point (yes, we used AI to write the AI course, and we're naming it):
If you ask ChatGPT something like: "What's the average hourly rate for a licensed electrician in Melbourne right now?", it will give you a number. Confidently. It might say something like "$90 to $120 per hour." That might be in the right ballpark. Or it might be based on data from several years ago, or national data averaged across much cheaper markets, or a blended rate that includes junior apprentices.
You might see it, feel like it confirms what you already thought, and quote it to a client.
The fix here isn't to never use AI for this kind of question. The fix is to treat the answer as a starting hypothesis, not a confirmed fact, and then spend three minutes on Google looking at current job listings, industry association pages, or Service Seeking rate guides to triangulate.
---