Section 2.5 Spotting Hallucinations
A crucial aspect of AI literacy is learning how to fact-check and verify AI output—because simply put: AI makes things up! In computer science, this phenomenon is known as a hallucination, where a Large Language Model (LLM) generates information that sounds completely plausible and authoritative, yet is factually incorrect or entirely fabricated. As students increasingly rely on generative AI tools for research and coursework, hallucinations pose a growing threat to learning, critical thinking, and academic integrity. To navigate these tools safely and effectively, AI literacy must expand beyond simple prompt engineering to teach users how to spot, investigate, and respond to fabricated output before accepting it as truth.
Subsection 2.5.1 Why AI Fabricates Information
In particular, large language models (LLMs) such as ChatGPT or Google Gemini are built around fundamental generation mechanics:
-
Designed to predict patterns, not create a repository of truth.
-
Generate text based on what sounds right, not what is right.
-
Designed to "fill in the blanks" rather than say they don’t know.
These hallucinations arise because LLMs are trained to predict next-word sequences through statistical pattern prediction—selecting the most probable next word rather than retrieving verified facts from a database. Despite being technically inclined, many users still view AI as a search engine that looks up stored information. This misconception overlooks the core mechanism of LLMs and leads users to mistake fluent text for factual accuracy. Because these models lack built-in mechanisms to distinguish between learned patterns and plausible fabrications, they produce inaccurate information confidently and convincingly—so much so that AI hallucinations have even made their way into mainstream publications.
Subsection 2.5.2 Manifestations of Hallucinations
In practice, hallucinations manifest in several forms: fabricated citations with non-existent authors, false historical facts or dates, invented statistics, incorrect explanations of complex concepts, and flawed code presented with apparent confidence.
Students most easily detect hallucinations in coding and technical topics because the output is immediately verifiable—code can be executed and tested against expected outcomes. This highlights a critical verifiability gap: AI is safest when outputs can be objectively validated, yet students frequently rely on it for tasks where verification is difficult, such as evaluating conceptual understanding or argument quality. In these qualitative areas, plausible but incorrect reasoning easily goes unnoticed without checking external ground truths.
Subsection 2.5.3 Strategies for Fact-Checking AI Output
For this reason, you should always verify the facts and citations an AI provides. It is never a good idea to assume that AI is giving you accurate information. Outputs should always be checked using strategies such as:
-
Direct Source Requests: If AI makes a claim, ask it directly to cite the sources it is using for that claim.
-
Citation Verification: When AI provides sources or citations, verify that they actually exist.
-
Independent Cross-Checking: Make sure you can find the sources and citations outside of the AI tool you are using.
-
Source Evaluation: If sources and citations are real, check for bias and reliability as you would with any traditional source.
You have attempted of activities on this page.
