What you will learn
Separate storage, retrieval, and learning, then test a hybrid memory system for interference.
Before you begin: From neurons to brains; Spiking neural networks
START WITH A QUESTION
Picture this.
Remembering a phone number for ten seconds and remembering how to ride a bicycle are both called memory, yet they behave very differently. Artificial systems also need several kinds of memory, and choosing the right interface starts with asking what must be retained and for how long.
- Retrieval
- Selecting stored information using a query or cue.
- Attractor
- A state or set approached by nearby states under a particular dynamical rule.
- Interference
- One stored or incoming pattern disrupting the use of another.
Read for the idea first. Work through the numbers next. Try answering the questions before opening the explanations.
Temporary state, learned parameters and records
The recurrent state from Part 3 carries recent activity. Learned weights can retain a relationship acquired across many examples. An external store can keep explicit records for later retrieval. These are different mechanisms with different update rules. A neural model's current context is not the same thing as a permanent database, and a retrieved document does not automatically alter its learned parameters. Describe the operation you need: maintain a recent cue, recall a record, or learn a reusable association.
Compare a cue with stored patterns
A vector is an ordered list. A dot product multiplies matching positions and sums the results. In the toy cue [1, 0, 1], zero marks a missing component. Comparing it with [1, −1, 1] gives 1+0+1 = 2; comparing with [−1, 1, −1] gives −1+0−1 = −2. This is a similarity score, not a probability. With general vectors, magnitude affects the score too, so normalization may be needed before interpreting rankings.
Retrieval is not yet an attractor
Imagine a marble rolling into a valley. An attractor is a state or set toward which nearby states evolve under specified dynamics. Hopfield-style networks use recurrence to retrieve patterns through such dynamics under particular assumptions. Our example simply scores two stored vectors once; it has no iterative state update and therefore does not implement an attractor memory. This separation helps you ask a productive next question: what update rule and conditions would make a partial cue settle into a stored pattern?
Memory can confidently retrieve the wrong thing
A noisy cue may match several records. A once-correct record can become outdated. Two sources can disagree. Keep provenance and update time with explicit records, and allow the system to report ambiguity. For a robot, an old obstacle location may be worse than an unknown location. Test false recall and interference, not just easy successes. Similarity answers 'what resembles this cue?' Verification asks a separate question: 'is the retrieved information reliable for this use?'
SEE THE RELATIONSHIPS
From structure to behavior.
MAKE THE IDEA PRECISE
The mathematics.
| Symbol | Meaning |
|---|---|
| cue | Query vector, with zero representing a missing component in this toy |
| pattern_j | Stored candidate vector j |
| score_j | Unnormalized dot-product similarity |
| i | Vector component index |
Build the equation, step by step
- For a cue and candidate of equal length, multiply corresponding entries and sum the products. This is a dot-product similarity, not a calibrated probability.
- With signed binary patterns, matching signs contribute positively and opposing signs contribute negatively.
- Rank the candidate scores. For general real vectors, vector length affects the result; normalization or a different distance may be needed. Retrieval is only one component of a memory system.
WORK THROUGH THE NUMBERS
Worked example
A partial cue [1, 0, 1] scores 2 against [1, -1, 1] and -2 against [-1, 1, -1]. The result identifies a closer pattern in this tiny constructed set, not a general memory capacity.
A small Python experiment.
Python 3 · Standard library only · Synthetic educational example. Run this in your own notebook or a local file.
cue = [1, 0, 1]
patterns = [[1,-1,1], [-1,1,-1]]
scores = [sum(a*b for a,b in zip(cue,p)) for p in patterns]
print(scores)Read the code as a story
zip aligns corresponding cue and pattern components. Each product contributes evidence for or against a match. The example assumes equal lengths; ordinary zip would silently stop at the shorter list, so a reusable implementation should validate lengths. The output ranks these two synthetic candidates only.
Expected output
[2, -2]
Change one quantity, predict the result, and then run the code again. Explain any difference before adding complexity.
Test your understanding.
Try it: Corrupt one nonzero cue component. Does the selected pattern remain reliable?
Self-check: Does selecting the most similar stored pattern prove that the answer is true?
No. Similarity retrieves candidates; it does not validate their content, origin or current relevance. Keep provenance and uncertainty, and check a retrieved claim against appropriate evidence before using it as a fact.
Where this model stops: Similarity does not establish truth, and biological memory categories are not one-to-one software modules.
A research question: How should a hybrid system resolve conflicting memories with different provenance?
Record evidence in your mastery worksheet →PREDICT / TRY / EXPLAIN
Create an ambiguous memory
Use paper, a calculator or the Python example. Write your prediction before checking the result.
- Calculate the two scores for the original cue.
- Corrupt its first component to obtain [−1, 0, 1]. Recalculate both scores.
- Write a response rule for a tie. Compare reporting uncertainty with always choosing the first stored pattern.
Compare with a worked solution
The original scores are 2 and −2. The corrupted cue gives zero for both candidates. Always choosing the first would hide the loss of distinguishing evidence. A useful retrieval interface can return both candidates with a tie flag or request another observation. That policy is separate from the similarity calculation.
CLOSE YOUR NOTES
Can you explain it back?
These questions test the reasoning, not just a remembered phrase. Say your answer out loud or write a sentence first.
01Does a score of 2 mean a 100% probability of correctness?
Reveal the reasoning for question 1
No. A raw dot product is not a calibrated probability. Its scale depends on vector values and representation.
02What is missing from this example before it can demonstrate attractor retrieval?
Reveal the reasoning for question 2
A recurrent update rule, specified constraints and a test of how states evolve from partial or noisy cues. A one-pass ranking does not supply those dynamics.
TAKE THE IDEA WITH YOU
Three things to remember.
- Choose a memory mechanism based on the retention and update task.
- Dot products rank compatibility under a chosen representation.
- Handle ties, outdated records and provenance explicitly.
Memory describes what we carry forward from the past. A world model uses the current information to predict what could happen next.
Continue to Part 18 →THE NEXT LAYER
Chapter depth map.
This introduction provides a first model. The following chapters are outlines for deeper future editions; their full treatments have not yet been published.
Biological categories and mechanisms
17.1 Timescales and memory functions · Outline
- 17.1.1 Working, short-term and long-term memory
- 17.1.2 Episodic, semantic and associative memory
- 17.1.3 Behavioral categories versus neural mechanisms
- 17.1.4 Synaptic plasticity and consolidation
- 17.1.5 Species differences and limits of analogy
17.2 Dynamical memory · Outline
- 17.2.1 Recurrent attractors and persistent state
- 17.2.2 Noise, capacity and interference
- 17.2.3 Hippocampal-inspired models
- 17.2.4 Recall cues and pattern completion
- 17.2.5 Test mechanisms with perturbations
Artificial memory systems
17.3 Context, retrieval and storage · Outline
- 17.3.1 Transformer context and its boundaries
- 17.3.2 RAG and vector retrieval
- 17.3.3 External memory and agent state
- 17.3.4 Provenance, update policies and forgetting
- 17.3.5 Retrieval is not evidence that a claim is correct
17.4 Hybrid memory experiments · Outline
- 17.4.1 Fast state plus slower durable representations
- 17.4.2 Consolidation schedules as hypotheses
- 17.4.3 Retrieval accuracy and temporal consistency
- 17.4.4 Adversarial or conflicting memories
- 17.4.5 Compare with simpler task-specific state
Read with the original sources.
These readings support the concepts and deeper investigation. The worked numbers and illustrations above are teaching examples.
Primary paper / author manuscript
Vaswani et al. (2017): Attention Is All You Need ↗
Read after matrix multiplication and optimization. Reconstruct query-key-value shapes and distinguish sequence modeling from embodied control.
Author-hosted textbook
Neuronal Dynamics — Gerstner, Kistler, Naud & Paninski (2014) ↗
Use for neuron-model prerequisites and notation; this site provides an original curriculum, not a reproduction of the textbook.
Primary paper
Hopfield (1982) — neural networks and collective computation ↗
A source for attractor-network memory; the simplified pattern example here is original and does not reproduce its experiments.