When AI Gets It Wrong: Hallucinations and Fact-Checking

Start with an example

In 2023, two lawyers in New York used an AI chatbot to help research a court filing. The AI supplied several past court cases that seemed to support their argument, complete with case names, dates, and quotes. The lawyers submitted the filing.

There was one problem: those cases didn’t exist. The AI had invented them. The judge noticed, the lawyers were fined, and the story made national news.

The lawyers weren’t careless people. They trusted output that looked professional. This lesson is about making sure that never happens to you.

What a hallucination is

A hallucination is information an AI produces that sounds true but isn’t. It might be a wrong number, a fake quote, a person’s title that’s slightly off, or an entire source that was never written.

As you learned in Lesson 2, language models predict likely-sounding text. When the model doesn’t have a real answer, it doesn’t stay quiet. It predicts what an answer would probably look like. The result can be fluent, specific, and completely made up.

Three facts make hallucinations dangerous:

1. They sound confident. The AI uses the same calm, certain tone for true and false statements.
2. They’re specific. A fake statistic like “68% of shoppers” feels more believable than a vague claim.
3. They mix with true information. A paragraph might be 90% accurate, which makes the 10% easy to miss.

The five places mistakes hide

You don’t need to distrust every word. Most hallucinations show up in predictable places. Train your eye to slow down when you see these:

Where	What it looks like	Example
Numbers	Statistics, percentages, prices, dates	“Small businesses using AI save an average of 14.5 hours a week.”
Names	People, titles, companies, places	Gives the wrong person as the mayor of your town
Quotes	Words put in a real person’s mouth	A “famous quote” that the person never said
Sources	Citations, links, books, court cases	A real-sounding article that doesn’t exist
Recent events	Anything after the model’s training cutoff	Describes a law or product as current when it has changed

When you review AI writing, highlight every claim in these five categories. Those are the ones you check.

Don’t ask the AI to grade itself

It’s tempting to ask, “Are you sure that’s right?” The problem is that the second answer comes from the same prediction process as the first. The AI might confidently confirm a wrong fact, or it might apologize and “correct” something that was actually right.

Real fact-checking happens outside the AI. You need an independent, trustworthy source.

Good sources include:

• Official websites: government agencies (.gov), the organization itself, a company’s own site
• Original documents: the actual study, law, report, or press release
• Established news organizations with editors and corrections policies
• Your own records: your sales numbers, your policies, your calendar

Weak sources include random blogs with no author, social media posts, and, of course, another AI answer.

The SIFT routine

Use this four-step routine whenever AI output includes facts that matter:

1. S: Spot the claims. Highlight the numbers, names, quotes, sources, and recent events.
2. I: Investigate the source. Where would this fact come from? Go there directly. If the AI gave a link, open it and confirm it says what the AI claimed.
3. F: Find a second source. For important facts, confirm with a second, independent source. If two good sources agree, you’re on solid ground.
4. T: Then decide. Keep the claim, correct it, or cut it. If you can’t verify it, don’t publish it.

SIFT takes a few minutes. Fixing a public mistake can take weeks.

Prevent mistakes before they happen

You can also lower the risk with smarter prompts.

Give the AI your facts. Instead of asking it to know things, paste in the information and ask it to use it.

“Using only the program numbers below, write a two-paragraph update for our donors. Do not add any statistics that aren’t listed.”

Give it permission not to know. Models tend to fill gaps. Tell them not to.

“If any detail you need is missing, write [NOT PROVIDED] instead of guessing.”

Ask for uncertainty. Ask the AI to flag claims it’s unsure about.

“After the draft, list any facts I should double-check.”

These instructions don’t make AI perfect, but they cut down the number of problems you’ll find in review.

A worked example: fact-checking a draft

Keisha manages communications for a community center. She asks AI to draft a newsletter paragraph about youth sports, and it writes:

“Studies show that 85% of kids who play team sports earn higher grades. Our center has served over 3,000 young athletes since opening in 1998, and as former First Lady Michelle Obama said, ‘Sports teach children everything they need to know about life.’”

Keisha runs SIFT:

• Spot: She highlights “85%,” “3,000 young athletes,” “1998,” and the quote.
• Investigate: She checks the center’s records. They’ve served about 1,900 athletes, and the center opened in 2001. She searches for the quote and can’t find any record of Michelle Obama saying it.
• Find a second source: She looks for the 85% statistic and finds research suggesting a link between sports and grades, but no source with that exact number.
• Then decide: She fixes the numbers, removes the quote, and softens the research claim.

Her final version:

“Research suggests that kids who play team sports often do better in school. Since opening in 2001, our center has served about 1,900 young athletes.”

Shorter, less flashy, and true. Four of the five claims in the original draft needed fixing.

Think deeper: why AI makes things up

AI wasn’t trained to tell the truth. It was trained to predict text that sounds right. A sentence like “A 2019 study at the University of Michigan found…” sounds just as plausible whether or not the study exists. When the AI doesn’t know something, it still tends to produce a confident, specific-sounding answer, because confident writing was far more common in what it learned from than “I’m not sure.”

Hallucinations are more likely with:

• Rare topics the AI saw little about
• Exact details like page numbers, quotes, and statistics
• Leading questions: ask “What year did our town win state?” and it may invent a year instead of questioning whether it happened
• Recent events that happened after its training

The best fix is called retrieval: give the AI real documents first, and have it answer from those. When you paste your notes and say “use only this,” you’re doing a homemade version. It helps a lot, but the AI can still misread a source or cite a real one for a claim it doesn’t make. Retrieval improves the odds. Checking guarantees the result.

Questions to sit with:

• People also fill gaps in their memory with things that “sound right.” How is an AI hallucination similar to a person misremembering, and how is it different?
• Why might an AI that says “I don’t know” more often actually be more useful?

Practice: your own fact-check

1. Ask an AI tool to write a short paragraph about a local business, landmark, or public figure in your town. Ask it to include two statistics and one quote.
2. Spot every claim in the five risky categories.
3. Investigate each claim using official or original sources.
4. Find a second source for the most important claim.
5. Decide: keep, correct, or cut each claim.
6. Count how many claims held up. Most people are surprised by the result.

Next, you’ll widen the lens to three risks every AI user has to manage: accuracy, privacy, and bias.