In 1953 a surgeon in Connecticut removed a small part of a young man's brain to treat his epilepsy. The seizures eased. Something else stopped altogether.
The patient, known for decades as H.M., could hold a conversation, remember his childhood and solve puzzles. But he could no longer form new lasting memories. He would greet a doctor he had met many times as a stranger. He read the same magazine with fresh interest each day.
The tissue that had been removed, on both sides of the brain, included a large part of a structure called the hippocampus, along with tissue around it. The report by William Scoville and Brenda Milner in 1957 changed the study of memory. It showed that memory is not one thing. Whatever stores a new event is separate from whatever holds what a person already knows.
It took nearly forty more years to work out why the brain would be built that way. The answer turned out to matter for machines.
Two learners in one brain
In 1995 James McClelland, Bruce McNaughton and Randall O'Reilly published a theory with a plain name: complementary learning systems.
The neocortex, the folded outer layer of the brain, is a slow learner. It changes its connections a little at a time. Over many experiences it extracts what they have in common: what a dog is, how a restaurant works, what a sentence means. This slowness is a feature. It is what lets the cortex find structure and not be thrown by any single event.
The hippocampus is a fast learner. It can record a specific episode after a single exposure: where the car is parked today, what was said at this morning's meeting. It keeps episodes apart from each other so that today's parking spot does not blur into yesterday's.
Each system does what the other cannot. The question the theory answered was why one system could not do both.
The problem machines hit first
The answer came from artificial neural networks.
In 1989 Michael McCloskey and Neal Cohen trained a simple network on one set of arithmetic facts and then on a second set. The first set was nearly wiped out. Learning the new material had rewritten the connections the old material depended on. They called it catastrophic interference. It is now more often called catastrophic forgetting.
The cause is built into how such networks learn. Knowledge is not filed in one place. It is spread across many connections, and every piece of knowledge shares connections with others. Change them quickly for one purpose and everything else that used them shifts.
So a network faces a choice. Learn slowly, with old and new examples mixed together, and knowledge accumulates safely. Learn quickly from the newest thing alone, and it destroys what came before.
A brain cannot wait. An animal has to remember where the predator was after one encounter. Hence two systems: a fast one to capture the event now, and a slow one to absorb it later, safely.
Replay: how the fast system teaches the slow one
That leaves the matter of how an event gets from one system to the other.
The hippocampus contains cells that fire when an animal is in a particular place. John O'Keefe found them in rats in 1971. As a rat runs along a track, these place cells fire in sequence, one location after another.
In 1994 Matthew Wilson and Bruce McNaughton recorded from groups of these cells while rats ran a track, and then again while the rats slept. Cells that had fired together during the run fired together again in sleep. The hippocampus was playing the day back.
This is the mechanism the theory needed. By replaying stored episodes many times, interleaved with one another, the hippocampus gives the cortex what it requires: old and new experiences mixed together, a little at a time. The slow learner gets its gradual, shuffled training. Nothing is overwritten.
It is also one of the better explanations of why sleep matters for learning.
The same idea, borrowed
Machine learning arrived at the same solution, partly through its own work and partly by looking at the brain.
In 1992 Long-Ji Lin proposed that a learning agent should store its experiences and learn from them again later, a method he called experience replay. In 2015 a team at DeepMind used it at the centre of a system that learned to play dozens of Atari games from the screen pixels and the game score alone. Without replay, the network learned from each moment as it happened and was unstable, forever overwriting itself with the most recent few seconds. With a memory of past moments, sampled at random, it learned steadily.
The connection was explicit. Dharshan Kumaran, Demis Hassabis and James McClelland updated the complementary learning systems theory in 2016 with the new results from artificial agents in view. Other work drew on a different piece of neuroscience. In 2017 James Kirkpatrick and colleagues described a method that slows down learning on the connections most important to earlier tasks, modelled on the way the brain stabilises synapses that matter.
The exchange has run in both directions for decades, and it is one of the more productive conversations in science.
What this means for a business system
A large language model is a slow learner in exactly this sense. Its knowledge is spread across billions of connections, set during a long training run that ended on a fixed date. It is good at what the cortex is good at: general structure, language, the way things usually go.
It does not know what changed in your business this morning. Teaching it by further training runs into the old problem. Training is slow and costly, it risks disturbing what the model already does well, and a fact once trained in cannot be cleanly taken out again.
The brain's design suggests the alternative, and it is how production systems are built. Keep the changing, specific facts in a fast store: the documents, the records, the calendar. When a question arrives, fetch the relevant pieces and put them in front of the model. The model supplies language and general sense. The store supplies the facts.
Several practical things follow.
- Updates take seconds. Change the record and the next answer reflects it. Nothing is retrained.
- Answers can be traced. The system can show which document a statement came from, because it read that document a moment ago.
- Forgetting is possible. Delete the record and the system no longer knows it. For anyone holding client information, that matters.
- The slow loop still exists. Reviewing what the system got wrong each month, and improving its instructions and tests, is the equivalent of consolidation. It should be gradual and deliberate, as it is in the cortex.
This is the thinking behind knowledge lookup, and it is why one of the first things we ask for is a map of where a company's information lives. Our guide on asking for a data map explains what that involves.
Where the comparison stops
A document store is not a hippocampus. It does not replay anything in its sleep, and a language model is not a cortex. The comparison is about the division of labour, not the biology.
But the division is the lesson. A system that must both know a great deal and keep up with a changing world needs two kinds of memory, and it needs to keep them apart. Evolution reached that design a long time ago. It is a reasonable one to copy.
References
- Hassabis, D., Kumaran, D., Summerfield, C., and Botvinick, M. (2017). Neuroscience-inspired artificial intelligence. Neuron, 95(2), 245 to 258. doi:10.1016/j.neuron.2017.06.011
- Kirkpatrick, J., Pascanu, R., Rabinowitz, N., and others (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13), 3521 to 3526. doi:10.1073/pnas.1611835114
- Kumaran, D., Hassabis, D., and McClelland, J. L. (2016). What learning systems do intelligent agents need? Complementary learning systems theory updated. Trends in Cognitive Sciences, 20(7), 512 to 534. doi:10.1016/j.tics.2016.05.004
- Lin, L.-J. (1992). Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8, 293 to 321. doi:10.1007/BF00992699
- McClelland, J. L., McNaughton, B. L., and O'Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102(3), 419 to 457. doi:10.1037/0033-295X.102.3.419
- McCloskey, M., and Cohen, N. J. (1989). Catastrophic interference in connectionist networks: the sequential learning problem. Psychology of Learning and Motivation, 24, 109 to 165. doi:10.1016/S0079-7421(08)60536-8
- Mnih, V., Kavukcuoglu, K., Silver, D., and others (2015). Human-level control through deep reinforcement learning. Nature, 518, 529 to 533. doi:10.1038/nature14236
- O'Keefe, J., and Dostrovsky, J. (1971). The hippocampus as a spatial map: preliminary evidence from unit activity in the freely-moving rat. Brain Research, 34(1), 171 to 175. doi:10.1016/0006-8993(71)90358-1
- Scoville, W. B., and Milner, B. (1957). Loss of recent memory after bilateral hippocampal lesions. Journal of Neurology, Neurosurgery and Psychiatry, 20(1), 11 to 21. doi:10.1136/jnnp.20.1.11
- Wilson, M. A., and McNaughton, B. L. (1994). Reactivation of hippocampal ensemble memories during sleep. Science, 265, 676 to 679. doi:10.1126/science.8036517
