Blog
Psychology9 min read
Ask an assistant whether your plan is good and it tends to say yes. A Stanford study in Science measured how much: eleven models affirmed people's actions 49 percent more often than people did, and users trusted them more for it. Why training produces it, why nobody notices, and how to build a system that says no.
When a language model states something false with complete confidence, the clinical word for it is not hallucination. It is confabulation, and neuropsychology has studied it for more than a century. What that research says about why it happens, and how to build around it.
"A person reviews it" is the standard answer to the risk of AI. Decades of human factors research say the review is the part most likely to fail, unless it is designed. What automation bias is, why experts are not immune and how to build a review step that holds.
In ordinary conversation the silence between one speaker and the next is about a fifth of a second. That number, and the research behind it, explains why a voice agent can say the right thing and still feel wrong.
The brain keeps two memory systems: one that learns fast and one that learns slowly. Neuroscience worked out why, machine learning borrowed the answer, and the same division explains where a business's facts should live in an AI system.
People forgive a colleague's error and remember a machine's. The research on algorithm aversion explains why a system that outperforms the old way can still be dropped after one visible mistake, and what makes a team keep using it.
Almost everything is called an agent now. There is a real distinction underneath the word, between a system that follows steps written in advance and one that chooses its own. Knowing which one a job needs decides its cost, its risk and whether it can be tested.
A model answers a question. A system does a job. The difference is everything built around the model: the data it reads, the tools it can use, the rules that limit it, the people who approve its work and the monitoring that shows whether it is working.
Psychology3 posts
AI sycophancy: why chatbots agree with you, and what it costs a business
Neuroscience2 posts
Confabulation, not hallucination: why AI makes things up
Human factors1 post
Automation bias: why people stop checking the machine
Engineering2 posts
AI agents and workflows: what the difference means for a business