PART 03 / Intermediate

From neurons to brains

How do interacting cells produce reliable sensorimotor behavior?

Guided illustrated lesson

What you will learn

Compare feedforward and recurrent circuits using explicit inputs, states, outputs, and perturbations.

Before you begin: Biological neuroscience

START WITH A QUESTION

Picture this.

Imagine deciding whether to cross a road. One glimpse may be ambiguous; a short sequence tells you whether a vehicle is approaching. A circuit can combine evidence across inputs and across time. These are two different operations, and separating them makes neural networks much easier to understand.

Bias
An added offset in an artificial unit's weighted sum.
Nonlinearity
A transformation that cannot be captured by one affine rule, such as a threshold.
Recurrence
Using previous internal activity in a later update.

Read for the idea first. Work through the numbers next. Try answering the questions before opening the explanations.

First, combine evidence

An artificial unit can multiply each input by a weight, add the results, and add a bias: z = w₁x₁ + w₂x₂ + b. A weight controls how strongly an input contributes; a bias shifts the baseline. With inputs 2 and 1, weights 0.5 and −1, and bias 0.2, z = 0.2. This is a weighted sum, like a scoring rule with positive and negative contributions. A perceptron adds a decision threshold: output one when z is at least zero, otherwise zero. It is a simple decision rule, not a tiny conscious agent.

Why put units into layers?

A vector is an ordered list of values, such as two sensor readings. A layer transforms one vector into another. A nonlinear activation changes the shape of that transformation; a threshold is one example. Without nonlinear activations, stacking affine layers still produces one affine transformation, so extra depth would not supply the same expressive benefit. Feedforward networks pass information through layers. Recurrent networks feed a previous state into a later calculation, giving the system a way to carry history.

A fading note to your future self

In x_next = a*x + b*u, x is the current note, u is fresh evidence, a tells us how much of the note survives, and b sets the input strength. With no new input and a = 0.8, the note fades by 20% each step. A negative a flips its sign while scaling it. This analogy helps predict the trace; the state could represent many quantities, not just a literal memory. The weights in this lesson are chosen by us. Part 7 will show what it means to learn a weight.

Contrast and feedback

Subtracting some neighboring activity can emphasize a local difference: a bright patch stands out against a dim surround. That is a useful toy motif, not a universal explanation of inhibition. Feedback can also amplify mistakes. In our scalar model, |a| below one makes an unforced state decay; |a| above one makes its magnitude grow. The bars mean absolute value. A large network has several interacting modes, so this one-number rule is a starting intuition rather than a general stability certificate.

SEE THE RELATIONSHIPS

From structure to behavior.

Network: a labelled conceptual diagram
Figure 3.1 · Network. Original educational diagram; synthetic, not a measured result.
The reasoning pathSensory input then Circuit state then Motor output then New sensory input1. Sensory input2. Circuit state3. Motor output4. New sensory input
A learning flow for this part. Follow the assumptions through to an observable result.
Inputs 2 and 1 contribute +1 and ?1; bias +0.2 gives a positive threshold decision.
Multiply, add, then decide. Trace each contribution before looking at the output. This is a synthetic perceptron calculation, not a biological cell diagram.

MAKE THE IDEA PRECISE

The mathematics.

x_(t+1) = a x_t + b u_t
Symbols and units
SymbolMeaning
x_tScalar circuit state at time step t
aRecurrent retention gain, dimensionless
bInput gain
u_tExternal input at time step t

Build the equation, step by step

  1. Separate the previous state contribution a x from the input contribution b u. This is a discrete linear approximation, not an anatomical identity.
  2. Set u to zero and apply the rule repeatedly: x after k steps equals a to the power k times the initial state.
  3. For this scalar model, magnitudes shrink when |a| < 1. A negative a alternates the sign. At |a| > 1, magnitudes grow unless a nonlinear limit intervenes.

WORK THROUGH THE NUMBERS

Worked example

With no new input and a = 0.8, an initial state of 1 becomes 0.8, 0.64 and 0.512. These are exact updates of this discrete model, not a fitted biological decay.

A small Python experiment.

Python 3 · Standard library only · Synthetic educational example. Run this in your own notebook or a local file.

x = 1.0
values = []
for _ in range(3):
    x = 0.8*x
    values.append(round(x, 3))
print(values)

Read the code as a story

x starts at one. Each loop replaces it with 80% of its current value, not 80% of the original value. append saves a snapshot for inspection. round only shortens the displayed values; it is not a biological mechanism. The three updates produce 0.8, 0.64 and 0.512.

Expected output

[0.8, 0.64, 0.512]

Change one quantity, predict the result, and then run the code again. Explain any difference before adding complexity.

Test your understanding.

Try it: Try a = -0.8 and a = 1.1. Explain alternating decay versus growth.

Self-check: Why does a stable recurrent circuit still need input?

Stability means disturbances do not grow without bound; it does not mean useful activity appears spontaneously. With |a| < 1 and no new input, this model forgets its initial state. Persistent memory needs an appropriate mechanism, not just the word recurrence.

Where this model stops: One scalar stability condition does not establish stability of a large nonlinear circuit.

A research question: Can a recurrent motif retain useful information while rejecting irrelevant perturbations?

Record evidence in your mastery worksheet →

PREDICT / TRY / EXPLAIN

Build an evidence accumulator

Use paper, a calculator or the Python example. Write your prediction before checking the result.

  1. Start with x = 0, a = 0.8 and b = 1. Apply inputs [1, 0, 0, 1] one at a time.
  2. Write each new state before proceeding. Compare with the same inputs when a = 0.
  3. Draw the two traces. Explain which trace could distinguish a recent earlier pulse from no earlier pulse.
Compare with a worked solution

With a = 0.8 the states are [1, 0.8, 0.64, 1.512]. With a = 0 they are [1, 0, 0, 1]. The last value in the first trace contains residue from the first pulse. That is temporal integration. It could help combine repeated weak signals, but could also retain a distracting signal.

CLOSE YOUR NOTES

Can you explain it back?

These questions test the reasoning, not just a remembered phrase. Say your answer out loud or write a sentence first.

01For z = 0.5×2 − 1×1 + 0.2, what does a threshold-at-zero perceptron output?

Reveal the reasoning for question 1

z = 0.2, so it outputs one under the stated at-least-zero rule. Always state what happens exactly at the threshold.

02Does recurrence imply a circuit can remember indefinitely?

Reveal the reasoning for question 2

No. Here, |a| < 1 makes old information fade without new input. Other recurrent mechanisms can preserve information, but recurrence by itself guarantees neither duration nor useful capacity.

TAKE THE IDEA WITH YOU

Three things to remember.

  • A weighted sum combines inputs; a nonlinear activation changes the mapping.
  • A recurrent state carries a trace of previous input.
  • Stable fading memory can be useful, but it also forgets.

We can now calculate a tiny circuit. Next we learn how a real wiring map becomes a graph—and why a map alone cannot tell us the circuit's full behavior.

Continue to Part 4 →

THE NEXT LAYER

Chapter depth map.

This introduction provides a first model. The following chapters are outlines for deeper future editions; their full treatments have not yet been published.

Circuit building blocks

3.1 Small circuits, useful computations · Outline
  1. 3.1.1 Neural circuits and microcircuits
  2. 3.1.2 Feedforward chains and convergence
  3. 3.1.3 Recurrent excitation and inhibition
  4. 3.1.4 Lateral inhibition and competition
  5. 3.1.5 Input-output curves and causal perturbations
3.2 Sensing and moving · Outline
  1. 3.2.1 Sensory transduction and sensory pathways
  2. 3.2.2 Motor neurons and motor systems
  3. 3.2.3 Reflex arcs and feedback loops
  4. 3.2.4 Central pattern generators and rhythmic behavior
  5. 3.2.5 Body and environment as part of the closed loop

Coordination over time

3.3 Choosing, attending and navigating · Outline
  1. 3.3.1 Decision circuits and evidence accumulation
  2. 3.3.2 Attention-like selection without anthropomorphism
  3. 3.3.3 Heading and navigation representations
  4. 3.3.4 Feedforward predictions versus feedback corrections
  5. 3.3.5 Distinguishing behavioral evidence from circuit speculation
3.4 Remembering and stabilizing · Outline
  1. 3.4.1 Persistent activity and memory circuits
  2. 3.4.2 Attractor dynamics and noise
  3. 3.4.3 Gain control and homeostatic regulation
  4. 3.4.4 Circuit lesions and model ablations
  5. 3.4.5 Emergence as a testable system property

Read with the original sources.

These readings support the concepts and deeper investigation. The worked numbers and illustrations above are teaching examples.

Complete source register →

Where next?