The Lie
Last week I asked my AI to research local networking groups in Portland for a project I’m working on, when it came back with six organizations, complete with meeting times, locations, and contact information. At first, I thought “This is great.” Then I did a little research and found out that three of them didn’t exist.
Not that they “used to exist” or they “moved locations.” They were completely fabricated. Made up. My AI Digital Assistant (DA) generated plausible-sounding organization names, assigned them real-looking addresses, and presented them with the same confidence as the three that were actually real.
This is a problem for those who “just use AI for research” without any validation. AI doesn’t know when it’s wrong. It doesn’t have a doubt meter. It delivers fiction with the same tone and formatting as fact. If you’re a business leader making decisions based on AI output, this is something that you need to at least take into consideration.
Why AI Makes Things Up
AI models don’t “know” things the way humans know things. They predict the next most likely word in a sequence based on patterns in their data. When they have solid data to draw from, the predictions are accurate, but when they don’t, they predict anyway. For anyone using AI to drive real-world operations, this gap between prediction and truth is where the risk lives. If you treat a probabilistic prediction as a hard fact, you aren't automating your business functions, you're gambling.
The commonly used term is “hallucination”, but I’m not generally a fan of that word because it sounds like a glitch. It’s not a glitch in the technology though. It’s the default behavior of the technology. The AI is always generating. It doesn’t have a mode where it says, “I don’t have enough information to answer this.” It just answers anyway.
Think of it like asking a very confident intern to research something. If they know the answer, they give you the right one. If they don’t, they might just give you a plausible-sounding wrong one instead of saying “I couldn’t find it.” Same energy. Same risk. For a business leader, that means bad data in your proposals, wrong numbers in your forecasts, and fake sources in your research, all delivered with zero hesitation.
The Three Places AI Lies Most
In my experience running AI systems for business use, these are the areas where hallucination is most dangerous:
1. Facts and data. Numbers, dates, statistics, company names, product specifications. AI will cite statistics that don’t exist, reference studies that were never published, and quote prices that are wildly wrong. If the output contains a specific number, verify it.
2. Local information. AI training data is global and often outdated. Ask it about local businesses, events, regulations, or market conditions and often you’ll get a mix of accurate and invented results. Portland’s business landscape changes faster than any training dataset. This is why I’ve given my AI local tools to do current research to move it beyond the baseline dataset.
3. Legal and compliance. AI will confidently tell you the wrong filing deadline, cite regulations that don’t apply to your state, or describe compliance requirements that are partially correct. In regulated work, “partially correct” is the same as wrong.
