PART 17 / Advanced

Memory: biological & artificial

Which kind of memory does a task need, and what mechanism supplies it?

Guided illustrated lesson

What you will learn

Separate storage, retrieval, and learning, then test a hybrid memory system for interference.

Before you begin: From neurons to brains; Spiking neural networks

START WITH A QUESTION

Picture this.

Remembering a phone number for ten seconds and remembering how to ride a bicycle are both called memory, yet they behave very differently. Artificial systems also need several kinds of memory, and choosing the right interface starts with asking what must be retained and for how long.

Retrieval
Selecting stored information using a query or cue.
Attractor
A state or set approached by nearby states under a particular dynamical rule.
Interference
One stored or incoming pattern disrupting the use of another.

Read for the idea first. Work through the numbers next. Try answering the questions before opening the explanations.

Temporary state, learned parameters and records

The recurrent state from Part 3 carries recent activity. Learned weights can retain a relationship acquired across many examples. An external store can keep explicit records for later retrieval. These are different mechanisms with different update rules. A neural model's current context is not the same thing as a permanent database, and a retrieved document does not automatically alter its learned parameters. Describe the operation you need: maintain a recent cue, recall a record, or learn a reusable association.

Compare a cue with stored patterns

A vector is an ordered list. A dot product multiplies matching positions and sums the results. In the toy cue [1, 0, 1], zero marks a missing component. Comparing it with [1, −1, 1] gives 1+0+1 = 2; comparing with [−1, 1, −1] gives −1+0−1 = −2. This is a similarity score, not a probability. With general vectors, magnitude affects the score too, so normalization may be needed before interpreting rankings.

Retrieval is not yet an attractor

Imagine a marble rolling into a valley. An attractor is a state or set toward which nearby states evolve under specified dynamics. Hopfield-style networks use recurrence to retrieve patterns through such dynamics under particular assumptions. Our example simply scores two stored vectors once; it has no iterative state update and therefore does not implement an attractor memory. This separation helps you ask a productive next question: what update rule and conditions would make a partial cue settle into a stored pattern?

Memory can confidently retrieve the wrong thing

A noisy cue may match several records. A once-correct record can become outdated. Two sources can disagree. Keep provenance and update time with explicit records, and allow the system to report ambiguity. For a robot, an old obstacle location may be worse than an unknown location. Test false recall and interference, not just easy successes. Similarity answers 'what resembles this cue?' Verification asks a separate question: 'is the retrieved information reliable for this use?'

SEE THE RELATIONSHIPS

From structure to behavior.

Network: a labelled conceptual diagram
Figure 17.1 · Network. Original educational diagram; synthetic, not a measured result.
The reasoning pathCue or observation then Retrieve / recurrent state then Check provenance then Use & update memory1. Cue or observation2. Retrieve / recurrent state3. Check provenance4. Use & update memory
A learning flow for this part. Follow the assumptions through to an observable result.

MAKE THE IDEA PRECISE

The mathematics.

score_j = sum_i cue_i pattern_(j,i)
Symbols and units
SymbolMeaning
cueQuery vector, with zero representing a missing component in this toy
pattern_jStored candidate vector j
score_jUnnormalized dot-product similarity
iVector component index

Build the equation, step by step

  1. For a cue and candidate of equal length, multiply corresponding entries and sum the products. This is a dot-product similarity, not a calibrated probability.
  2. With signed binary patterns, matching signs contribute positively and opposing signs contribute negatively.
  3. Rank the candidate scores. For general real vectors, vector length affects the result; normalization or a different distance may be needed. Retrieval is only one component of a memory system.

WORK THROUGH THE NUMBERS

Worked example

A partial cue [1, 0, 1] scores 2 against [1, -1, 1] and -2 against [-1, 1, -1]. The result identifies a closer pattern in this tiny constructed set, not a general memory capacity.

A small Python experiment.

Python 3 · Standard library only · Synthetic educational example. Run this in your own notebook or a local file.

cue = [1, 0, 1]
patterns = [[1,-1,1], [-1,1,-1]]
scores = [sum(a*b for a,b in zip(cue,p)) for p in patterns]
print(scores)

Read the code as a story

zip aligns corresponding cue and pattern components. Each product contributes evidence for or against a match. The example assumes equal lengths; ordinary zip would silently stop at the shorter list, so a reusable implementation should validate lengths. The output ranks these two synthetic candidates only.

Expected output

[2, -2]

Change one quantity, predict the result, and then run the code again. Explain any difference before adding complexity.

Test your understanding.

Try it: Corrupt one nonzero cue component. Does the selected pattern remain reliable?

Self-check: Does selecting the most similar stored pattern prove that the answer is true?

No. Similarity retrieves candidates; it does not validate their content, origin or current relevance. Keep provenance and uncertainty, and check a retrieved claim against appropriate evidence before using it as a fact.

Where this model stops: Similarity does not establish truth, and biological memory categories are not one-to-one software modules.

A research question: How should a hybrid system resolve conflicting memories with different provenance?

Record evidence in your mastery worksheet →

PREDICT / TRY / EXPLAIN

Create an ambiguous memory

Use paper, a calculator or the Python example. Write your prediction before checking the result.

  1. Calculate the two scores for the original cue.
  2. Corrupt its first component to obtain [−1, 0, 1]. Recalculate both scores.
  3. Write a response rule for a tie. Compare reporting uncertainty with always choosing the first stored pattern.
Compare with a worked solution

The original scores are 2 and −2. The corrupted cue gives zero for both candidates. Always choosing the first would hide the loss of distinguishing evidence. A useful retrieval interface can return both candidates with a tie flag or request another observation. That policy is separate from the similarity calculation.

CLOSE YOUR NOTES

Can you explain it back?

These questions test the reasoning, not just a remembered phrase. Say your answer out loud or write a sentence first.

01Does a score of 2 mean a 100% probability of correctness?

Reveal the reasoning for question 1

No. A raw dot product is not a calibrated probability. Its scale depends on vector values and representation.

02What is missing from this example before it can demonstrate attractor retrieval?

Reveal the reasoning for question 2

A recurrent update rule, specified constraints and a test of how states evolve from partial or noisy cues. A one-pass ranking does not supply those dynamics.

TAKE THE IDEA WITH YOU

Three things to remember.

  • Choose a memory mechanism based on the retention and update task.
  • Dot products rank compatibility under a chosen representation.
  • Handle ties, outdated records and provenance explicitly.

Memory describes what we carry forward from the past. A world model uses the current information to predict what could happen next.

Continue to Part 18 →

THE NEXT LAYER

Chapter depth map.

This introduction provides a first model. The following chapters are outlines for deeper future editions; their full treatments have not yet been published.

Biological categories and mechanisms

17.1 Timescales and memory functions · Outline
  1. 17.1.1 Working, short-term and long-term memory
  2. 17.1.2 Episodic, semantic and associative memory
  3. 17.1.3 Behavioral categories versus neural mechanisms
  4. 17.1.4 Synaptic plasticity and consolidation
  5. 17.1.5 Species differences and limits of analogy
17.2 Dynamical memory · Outline
  1. 17.2.1 Recurrent attractors and persistent state
  2. 17.2.2 Noise, capacity and interference
  3. 17.2.3 Hippocampal-inspired models
  4. 17.2.4 Recall cues and pattern completion
  5. 17.2.5 Test mechanisms with perturbations

Artificial memory systems

17.3 Context, retrieval and storage · Outline
  1. 17.3.1 Transformer context and its boundaries
  2. 17.3.2 RAG and vector retrieval
  3. 17.3.3 External memory and agent state
  4. 17.3.4 Provenance, update policies and forgetting
  5. 17.3.5 Retrieval is not evidence that a claim is correct
17.4 Hybrid memory experiments · Outline
  1. 17.4.1 Fast state plus slower durable representations
  2. 17.4.2 Consolidation schedules as hypotheses
  3. 17.4.3 Retrieval accuracy and temporal consistency
  4. 17.4.4 Adversarial or conflicting memories
  5. 17.4.5 Compare with simpler task-specific state

Read with the original sources.

These readings support the concepts and deeper investigation. The worked numbers and illustrations above are teaching examples.

Complete source register →

Where next?