A language model is asked for the date a contract was signed. The date is not in anything it was given. It answers anyway, with a day, a month and a year, in the same even tone it uses for everything else.
The usual word for this is hallucination. It is the wrong word, and the mistake is worth correcting, because the right word comes with a hundred years of research attached.
In medicine a hallucination is a perception with nothing there to perceive: a voice in a silent room, a figure in an empty doorway. A language model does not perceive anything. What it does has a different clinical name. It confabulates.
What confabulation is
Confabulation is the production of a false account, given fluently and sincerely, to fill a gap. The person is not lying. They believe what they are saying.
It was first described in the late 1880s by the Russian psychiatrist Sergei Korsakoff, in patients with a severe memory disorder that now carries his name. Ask a patient with Korsakoff's syndrome what they did yesterday and you may get a detailed answer: a trip into town, a conversation with a brother, what was eaten for lunch. None of it happened. The patient has been in the hospital for weeks. The neuropsychologist Morris Moscovitch later gave the behaviour its most quoted description: honest lying.
The same pattern appears after damage to specific parts of the frontal lobes, in particular the regions just above the eyes, and after the rupture of an aneurysm on the artery that feeds them. The work of Armin Schnider and others points to a common thread. These patients have not lost language, and they have not lost the ability to produce memories. They have lost the filter that checks a thought against present reality before it is spoken.
Three features define it, and all three will be familiar to anyone who has worked with a language model.
- It is fluent. The grammar, the detail and the tone are normal.
- It is plausible. The account is built from true pieces: real places, real people, real habits.
- It is unmarked. Nothing in the delivery separates the invented part from the remembered part.
The interpreter in the left hemisphere
The clearest demonstration came from patients whose brain hemispheres had been surgically separated to treat epilepsy. In a series of experiments, Michael Gazzaniga and his colleagues showed a different picture to each half of the brain. In the best known case, the left hemisphere, which controls speech, saw a chicken claw. The right hemisphere, which cannot speak, saw a snow scene.
The patient was then asked to pick a matching picture with each hand. The right hand chose a chicken. The left hand, steered by the silent right hemisphere, chose a snow shovel.
Asked why he had picked a shovel, the patient answered at once. The claw went with the chicken, he said, and the shovel was for cleaning out the chicken shed.
The speaking half of his brain had no access to the snow scene. It did not say "I do not know". It produced a reason on the spot and delivered it as fact. Gazzaniga called the mechanism the interpreter: a system that explains whatever the rest of the brain has done, with or without the information needed to explain it.
Healthy memory does it too
It would be comfortable to treat this as something that happens only to damaged brains. The evidence does not allow that.
In a book published in 1932, Frederic Bartlett described experiments in which he asked students at Cambridge to read an unfamiliar folk tale and retell it later. The retellings got shorter, tidier and more English. Details that did not fit were dropped or replaced with details that did. Bartlett's conclusion was that remembering is not playback. It is reconstruction.
In 1974 Elizabeth Loftus and John Palmer showed people a film of a car crash and asked how fast the cars were going when they "hit" each other, or when they "smashed" into each other. One verb was enough to raise the estimated speed. A week later, people who had heard "smashed" were more likely to remember broken glass. There was no broken glass in the film.
In 2005 a Swedish group took the idea further. Participants chose which of two faces they found more attractive. With a sleight of hand, the experimenter sometimes handed back the face they had not chosen and asked them to explain their choice. Most of the swaps went unnoticed, and people gave confident reasons for a choice they had never made.
Richard Nisbett and Timothy Wilson had made the general point in 1977: people often have no direct view of why they did something, and report a plausible theory in its place. The feeling of knowing and the fact of knowing are separate things.
Why the comparison fits a language model
A language model is trained to continue text in the most plausible way. When the fact is in what it has learned or been given, the plausible continuation is the true one. When the fact is missing, the plausible continuation is still fluent, still specific and still delivered in the same voice. The model is an interpreter with no snow scene.
Researchers have started to use the clinical term for this reason. Andrew Smith, Felix Greaves and Trishan Panch argued in 2023 that "confabulation" describes the behaviour more accurately than "hallucination", since nothing is being perceived. A 2024 paper in Nature by Sebastian Farquhar and colleagues used the word for a specific class of error, answers that are both wrong and arbitrary, and showed a way to detect them: ask the same question several times and measure whether the answers agree in meaning. A fact tends to come back the same way each time. A confabulation varies.
Psychology also has a name for the missing piece. Marcia Johnson and her colleagues called it source monitoring: the work the mind does to decide where a piece of information came from, whether it was seen, heard, read or imagined. A person with good source monitoring can say "I think I read that somewhere, but I am not sure". A language model on its own has no such record. Every sentence it produces comes from the same place.
There is a further reason the behaviour persists. A 2025 paper from researchers at OpenAI and Georgia Tech argued that the way models are trained and graded rewards guessing. A student who leaves an exam answer blank scores zero. A student who guesses sometimes scores. A model that is measured only on right answers learns the same lesson.
The comparison has limits. A model is not a patient, and nobody is claiming the mechanism in a network of weights matches the one in a frontal lobe. The likeness is in the behaviour, and the behaviour is what a business has to design for.
What this changes in how a system is built
Healthy people confabulate less than patients because of the checks that surround memory, not because their memory is a better recording. The same is true of AI systems. The model is one part. The checks are the rest.
Give it the record. A patient with amnesia answers from plausibility because the memory is not there. A model asked about your business is in the same position unless the system puts the relevant record in front of it at the moment of the question. This is the idea behind retrieval: the answer is written from the document, not from recall. It is how we build knowledge lookup and every system that answers from a company's own files.
Keep the source attached. Every claim in an answer should carry a pointer to the line it came from. A claim with no pointer is treated as a guess and handled as one. This is source monitoring, built on the outside because it is missing on the inside.
Make "I do not know" a designed outcome. If the only acceptable result is an answer, the system will produce answers. A system that can say "that is not in the file" and pass the question to a person needs that path designed, tested and counted as a success when it is used correctly.
Ask twice. For an answer that matters, a second pass that checks the first against the source can catch invented detail. Where the two disagree, the disagreement is the signal.
Keep a person at the point of consequence. The frontal filter that fails in confabulation is a check against reality before speech. In a business system, the equivalent is a named person who approves before anything reaches a client, a ledger or a calendar. Our guide to designing the approval step covers what that person should see.
The useful half
The interpreter is not a defect in the brain. It is the reason people can act on incomplete information, tell a coherent story about their day and finish each other's sentences. The same gap-filling is what makes a language model good at drafting, summarising and rephrasing.
The aim is not to remove it. The aim is to know where it is allowed. Drafting a reply is a good place for a system that fills gaps. Stating the date a contract was signed is not, unless the contract is on the table.
References
- Bartlett, F. C. (1932). Remembering: A Study in Experimental and Social Psychology. Cambridge University Press. doi:10.1017/CBO9780511759185
- Farquhar, S., Kossen, J., Kuhn, L., and Gal, Y. (2024). Detecting hallucinations in large language models using semantic entropy. Nature, 630, 625 to 630. doi:10.1038/s41586-024-07421-0
- Gazzaniga, M. S. (2000). Cerebral specialization and interhemispheric communication: does the corpus callosum enable the human condition? Brain, 123, 1293 to 1326. doi:10.1093/brain/123.7.1293
- Johansson, P., Hall, L., Sikström, S., and Olsson, A. (2005). Failure to detect mismatches between intention and outcome in a simple decision task. Science, 310, 116 to 119. doi:10.1126/science.1111709
- Johnson, M. K., Hashtroudi, S., and Lindsay, D. S. (1993). Source monitoring. Psychological Bulletin, 114, 3 to 28. doi:10.1037/0033-2909.114.1.3
- Kalai, A. T., Nachum, O., Vempala, S. S., and Zhang, E. (2025). Why language models hallucinate. arXiv:2509.04664
- Loftus, E. F., and Palmer, J. C. (1974). Reconstruction of automobile destruction: an example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 13, 585 to 589. doi:10.1016/S0022-5371(74)80011-3
- Moscovitch, M. (1989). Confabulation and the frontal systems: strategic versus associative retrieval in neuropsychological theories of memory. In H. L. Roediger and F. I. M. Craik (Eds.), Varieties of Memory and Consciousness. Erlbaum. doi:10.4324/9781315801841
- Nisbett, R. E., and Wilson, T. D. (1977). Telling more than we can know: verbal reports on mental processes. Psychological Review, 84, 231 to 259. doi:10.1037/0033-295X.84.3.231
- Schnider, A. (2003). Spontaneous confabulation and the adaptation of thought to ongoing reality. Nature Reviews Neuroscience, 4, 662 to 671. doi:10.1038/nrn1179
- Smith, A. L., Greaves, F., and Panch, T. (2023). Hallucination or confabulation? Neuroanatomy as metaphor in large language models. PLOS Digital Health, 2(11), e0000388. doi:10.1371/journal.pdig.0000388
