PART 09 / Advanced

Event-based vision

What changes when a sensor reports brightness changes instead of frames?

Guided illustrated lesson

What you will learn

Process a synthetic event stream and compare its information and cost with frame sampling.

Before you begin: Biological vision; Spiking neural networks

START WITH A QUESTION

Picture this.

A conventional camera is like taking a classroom photograph at regular intervals. An event camera is more like each seat raising a flag when something changes locally. The flags can arrive quickly, but a quiet seat does not tell you everything about the person sitting there.

Asynchronous
Events occur on their own timing rather than a single shared frame schedule.
Polarity
The direction of a reported brightness change under a stated convention.
Time bin
An interval used to group events for processing.

Read for the idea first. Work through the numbers next. Try answering the questions before opening the explanations.

Read an event as a small record

A common event is (x, y, t, p): horizontal position, vertical position, timestamp and polarity. Polarity distinguishes an increase from a decrease in brightness under the sensor's convention. Events arrive asynchronously, meaning pixels need not wait for a shared frame boundary. They are sensor reports, not biological spikes. To feed them into an SNN, define which input unit receives each position and polarity, and how timestamps map onto the simulator's time scale.

Why use a logarithm?

Compare brightness changing from 10 to 13 with brightness changing from 100 to 130. The absolute changes differ, but both increase by 30%. A logarithm turns a brightness ratio into an additive change: ΔL = ln(I_new/I_ref). ln is the natural logarithm, the inverse of exp. You do not need to calculate it mentally; first predict the sign. A ratio above one gives a positive change, below one gives a negative change, and one gives zero. Both intensities must be positive.

One threshold comparison is only the beginning

Our teaching rule emits +1 if the log change is at least C, −1 if it is at most −C, and zero otherwise. Real sensor behavior also involves reference updates, noise, refractory effects and potentially several events during a large change. The example below checks a single pair of intensities; it is not a full sensor simulator. Keep the distinction clear when interpreting the output. A single event says something changed enough at a pixel, not what object caused it.

Choose what to remember

A model can consume event times directly or accumulate them into time bins. A time surface stores how recently locations were active. Bins make conventional array processing convenient, but a long window delays the answer and loses timing detail. If the task needs static texture or absolute brightness, sparse changes may be insufficient by themselves. Compare the complete sensor-to-decision pipeline on the same task. High timestamp precision cannot rescue a program that waits in a long queue before reading its events.

SEE THE RELATIONSHIPS

From structure to behavior.

Events: a labelled conceptual diagram
Figure 9.1 · Events. Original educational diagram; synthetic, not a measured result.
The reasoning pathBrightness change then Timestamped events then Temporal state then Task decoder1. Brightness change2. Timestamped events3. Temporal state4. Task decoder
A learning flow for this part. Follow the assumptions through to an observable result.

MAKE THE IDEA PRECISE

The mathematics.

Delta L = ln(I_new / I_ref); event when |Delta L| >= C
Symbols and units
SymbolMeaning
I_new, I_refPositive intensity values on a common scale
LLog intensity
CIllustrative contrast threshold, dimensionless
pPolarity: sign of the threshold crossing

Build the equation, step by step

  1. Use log intensity so a multiplicative brightness ratio becomes an additive change: ln(I_new) − ln(I_ref) = ln(I_new/I_ref). Intensities must be positive.
  2. Compare the magnitude with the contrast threshold C. The sign determines event polarity when the threshold is crossed.
  3. A real event stream also depends on reference updates, timestamp resolution, refractory behavior and noise. The one-comparison example does not reconstruct all events that might occur during a large brightness jump.

WORK THROUGH THE NUMBERS

Worked example

An intensity rise from 100 to 130 gives ln(1.3) = 0.262. With a chosen threshold of 0.2, a simplified single-crossing rule emits a positive event. A real sensor may emit multiple events or exhibit noise and refractory effects.

A small Python experiment.

Python 3 · Standard library only · Synthetic educational example. Run this in your own notebook or a local file.

from math import log
change = log(130/100)
threshold = 0.2
polarity = 1 if change >= threshold else -1 if change <= -threshold else 0
print(f"change = {change:.3f}; polarity = {polarity}")

Read the code as a story

log(130/100) calculates the relative change in log intensity. The nested conditional checks positive crossing first, then negative crossing, then no event. Zero here is a code for no threshold crossing, not a dark pixel. Try log(0) only as a thought experiment: the model excludes nonpositive intensities, so real preprocessing needs a valid sensor-specific policy.

Expected output

change = 0.262; polarity = 1

Change one quantity, predict the result, and then run the code again. Explain any difference before adding complexity.

Test your understanding.

Try it: Try 100 to 80 and 100 to 110. Predict polarity and whether the threshold is crossed.

Self-check: Does an event camera report an absolute image at every timestamp?

No. Its events primarily describe brightness changes at particular pixels. Recovering a frame or inferring motion requires temporal state and assumptions; a quiet pixel is not proof of a dark pixel.

Where this model stops: This threshold example is not a calibrated event-camera simulator.

A research question: When does event filtering remove useful fast motion along with noise?

Record evidence in your mastery worksheet →

PREDICT / TRY / EXPLAIN

Build a tiny event table

Use paper, a calculator or the Python example. Write your prediction before checking the result.

  1. Use reference intensity 100 and new values 80, 110 and 130. Predict the sign and threshold decision for C = 0.2.
  2. Compute the three log ratios and compare with ±0.2.
  3. Now double both reference and new intensities. Check which decisions change and explain why.
Compare with a worked solution

The changes are approximately −0.223, 0.095 and 0.262, producing −1, 0 and +1. Doubling both intensities leaves every ratio unchanged, so this ideal rule gives the same decisions. This invariance belongs to the mathematical rule; a real sensor's noise and operating limits still matter.

CLOSE YOUR NOTES

Can you explain it back?

These questions test the reasoning, not just a remembered phrase. Say your answer out loud or write a sentence first.

01Does no event imply no light?

Reveal the reasoning for question 1

No. A bright pixel whose intensity stays near its reference can be quiet. Events describe changes, not a fresh absolute image at every instant.

02Why might a 100 ms event bin undermine a fast reflex?

Reveal the reasoning for question 2

A decision that waits for the bin to close can spend up to a bin interval waiting, in addition to processing and communication. Match the representation to the actual response deadline.

TAKE THE IDEA WITH YOU

Three things to remember.

  • An event is a timestamped local change report.
  • Log ratios express relative rather than absolute intensity change.
  • Representation windows trade timing detail against processing convenience.

Next we audit a public project that combines visual signals, neural activity and a command interface. We will separate what its components do from what its name might suggest.

Continue to Part 10 →

THE NEXT LAYER

Chapter depth map.

This introduction provides a first model. The following chapters are outlines for deeper future editions; their full treatments have not yet been published.

Sensor and representation

9.1 An asynchronous camera · Outline
  1. 9.1.1 DVS and the event tuple (x, y, t, p)
  2. 9.1.2 Log-intensity contrast thresholds
  3. 9.1.3 Polarity, timestamp precision and refractory effects
  4. 9.1.4 Latency, dynamic range and noise
  5. 9.1.5 Limits in static scenes and low contrast
9.2 From events to features · Outline
  1. 9.2.1 Event filtering and temporal windows
  2. 9.2.2 Time surfaces and voxel representations
  3. 9.2.3 Direct event-to-spike mapping versus frame conversion
  4. 9.2.4 Reconstruct synthetic motion with ground truth
  5. 9.2.5 Avoid treating an event as a biological action potential

Systems and comparisons

9.3 Event stream to action · Outline
  1. 9.3.1 Event camera to encoder to SNN
  2. 9.3.2 Optional circuit-inspired stage
  3. 9.3.3 Activity decoding into bounded commands
  4. 9.3.4 Clock synchronization and processing backlog
  5. 9.3.5 Simulation before sensor purchase
9.4 Is the event camera justified? · Outline
  1. 9.4.1 Matched camera-plus-CNN baseline
  2. 9.4.2 Whole-pipeline latency and energy
  3. 9.4.3 Flicker, vibration and hot-pixel stress tests
  4. 9.4.4 Dataset provenance and leakage
  5. 9.4.5 Report regimes where conventional frames win

Read with the original sources.

These readings support the concepts and deeper investigation. The worked numbers and illustrations above are teaching examples.

Complete source register →

Where next?