What you will learn
Produce a literature matrix, reproducible baseline, experiment report, and paper draft with honest limitations.
Before you begin: Foundations: matter, mathematics & computation; Connectomics: wiring as data
START WITH A QUESTION
Picture this.
Finishing a program answers 'can I make it run?' Research adds another question: 'what have I learned, and could someone else check it?' The final step is to make the evidence as understandable as the code.
- Reproducibility
- Making the inputs, methods and conditions available enough for another person to check a result.
- Experimental unit
- The independently varied or sampled unit relevant to the analysis.
- Effect size
- The magnitude of a difference, distinct from how uncertain its estimate is.
Read for the idea first. Work through the numbers next. Try answering the questions before opening the explanations.
Read a paper in three passes
First identify the question, the comparison and the main evidence. Do not try to understand every symbol immediately. Second, follow one central result back through its method, assumptions and data. Third, inspect limitations, supplementary details and alternative explanations. Keep a question list. An unfamiliar equation is often easier after you know what observation it is intended to explain. A review can locate the literature, but important claims should lead you back to their original experiments or derivations.
Write a claim-evidence ledger
Make columns for claim, supporting result, assumptions and what would weaken the claim. Distinguish observed results from your interpretation. If a paper reports success in one simulation, do not silently promote it to a hardware result. If your own experiment uses invented teaching data, label it accordingly. This habit keeps a notebook useful months later, when the excitement of a result has faded but the decisions still need to be understood.
Plan before tuning
Specify the hypothesis, baseline, primary metric, experimental unit and held-out conditions. Explain why runs are paired if you use paired differences. Keep development data for debugging and tuning; reserve evaluation data for the final comparison. If you inspect the evaluation result and then modify the method, disclose that reuse or obtain a fresh evaluation. Record failures and exceptions rather than quietly excluding difficult cases. A protocol can be modest and still be scientifically honest.
Tell a result-sized story
Start a report with the question and the actual outcome. Describe enough of the setup to rerun it, including versions, preprocessing, seeds, parameters and resource limits. Show individual outcomes alongside summaries. Explain what remains uncertain and what experiment would resolve it. A positive mean does not prove a mechanism; a negative result does not prove universal impossibility. The goal is a limited conclusion supported by inspectable evidence. Peer review and publication are later external outcomes, not automatic consequences of writing the report.
Return to the beginning
You began with a membrane that retains a little history. You can now connect that state to wiring, sensing, learning, action and experimental evaluation. Revisit your first sketch and explain each bridge in ordinary language. If one bridge still feels magical, that is a useful next study target. The chapter outlines and project ladder offer deeper directions, but the core habit stays the same: make a prediction, construct a test, inspect the mismatch and improve the explanation.
SEE THE RELATIONSHIPS
From structure to behavior.
MAKE THE IDEA PRECISE
The mathematics.
| Symbol | Meaning |
|---|---|
| i | Matched independent experimental run |
| score | Predefined task metric on the same evaluation scale |
| delta_i | Within-pair difference; pairing must be justified |
Build the equation, step by step
- Define the measured quantity and its desirable direction before comparing systems. Subtract baseline from candidate for each matched run.
- Keep the individual differences alongside the mean. Averages can hide a failure mode or a dependence on one favorable seed.
- Connect the numerical result to a limited claim, report uncertainty and resource budgets, and provide the protocol needed for someone else to test it. Negative or inconclusive results are still evidence.
WORK THROUGH THE NUMBERS
Worked example
Baseline results [0.70, 0.72, 0.68] and candidate results [0.74, 0.73, 0.72] yield differences [0.04, 0.01, 0.04], averaging 0.03. These are invented teaching numbers, not measured performance.
A small Python experiment.
Python 3 · Standard library only · Synthetic educational example. Run this in your own notebook or a local file.
from statistics import mean
baseline = [0.70, 0.72, 0.68]
candidate = [0.74, 0.73, 0.72]
delta = [b-a for a,b in zip(baseline,candidate)]
print([round(x,2) for x in delta])
print(f"mean difference = {mean(delta):.2f}")Read the code as a story
zip pairs the candidate and baseline scores by run. The subtraction direction makes positive values favor the candidate when larger scores are better. The printed mean keeps the individual differences visible. The lists must describe genuinely matched runs; pairing by list position does not itself justify the experimental design.
Expected output
[0.04, 0.01, 0.04] mean difference = 0.03
Change one quantity, predict the result, and then run the code again. Explain any difference before adding complexity.
Test your understanding.
Try it: Write a one-page protocol for one project, including a result that would count against your idea.
Self-check: What would count as a useful negative result?
A reproducible comparison showing that the proposed mechanism fails to improve the chosen outcome under clearly defined conditions. State what was tested, what the experiment could detect, and what remains unresolved. Do not turn failure in one setup into a universal impossibility claim.
Where this model stops: Neither three runs nor a positive mean alone establishes a reliable scientific improvement.
A research question: Can someone else reproduce your conclusion using only your report and public artifacts?
Record evidence in your mastery worksheet →PREDICT / TRY / EXPLAIN
Write your first research note
Use paper, a calculator or the Python example. Write your prediction before checking the result.
- Choose one completed book experiment. State its question, assumptions and an expected result before describing the output.
- Report the input, parameters, individual results and one failure or unresolved case. Identify which numbers are synthetic.
- Ask whether a reader could repeat the calculation without asking you for a hidden file. End with one narrow next experiment.
Compare with a worked solution
For the supplied comparison, differences are [0.04, 0.01, 0.04] and the mean is 0.03. A suitable conclusion is that these three invented pairs illustrate a positive average difference, not that a real method has been validated. For your own project, replace the invented pairs with recorded outcomes and document how they were obtained. An honest unresolved result is a valid starting point for the next experiment.
CLOSE YOUR NOTES
Can you explain it back?
These questions test the reasoning, not just a remembered phrase. Say your answer out loud or write a sentence first.
01Why keep negative outcomes in the report?
Reveal the reasoning for question 1
They reveal variability and failure conditions, and prevent the conclusion from being based only on favorable cases. Explain any exclusion rule and choose it independently of the desired result.
02What should you do when another person cannot reproduce your result?
Reveal the reasoning for question 2
Compare data versions, configuration, update order, numerical precision and evaluation conditions. Treat the mismatch as evidence to investigate, rather than assuming either implementation is correct.
TAKE THE IDEA WITH YOU
Three things to remember.
- Trace claims back to specific evidence and assumptions.
- Separate development choices from final evaluation.
- A clear, limited and reproducible conclusion is a worthwhile result.
Continue with one project from the ladder, or revisit the earliest lesson you could not explain without notes. Choose depth by the question you want to answer, not by the number of tools you can add.
Choose your next project →THE NEXT LAYER
Chapter depth map.
This introduction provides a first model. The following chapters are outlines for deeper future editions; their full treatments have not yet been published.
Read and question
25.1 Reading a paper · Outline
- 25.1.1 Problem and previous limitation
- 25.1.2 Method, equations and assumptions
- 25.1.3 Experimental design and actual results
- 25.1.4 Limitations, competing explanations and reproducibility
- 25.1.5 What changed and what remains unproven
25.2 From literature to hypothesis · Outline
- 25.2.1 Search primary literature and official resources
- 25.2.2 Track versions, datasets and negative results
- 25.2.3 Build a claim/evidence matrix
- 25.2.4 Form falsifiable hypotheses
- 25.2.5 Define scope and feasibility before implementation
Test and communicate
25.3 Experiments that answer a question · Outline
- 25.3.1 Baselines and ablation studies
- 25.3.2 Independent runs, uncertainty and effect sizes
- 25.3.3 Statistical significance versus practical importance
- 25.3.4 Benchmark leakage, confounders and compute parity
- 25.3.5 Reproduction manifests and release checklists
25.4 Writing and peer review · Outline
- 25.4.1 Methods precise enough to reproduce
- 25.4.2 Results figures and uncertainty
- 25.4.3 Limitations and alternative interpretations
- 25.4.4 Writing abstracts, papers and rebuttals
- 25.4.5 Peer review and publication are not guaranteed outcomes
Read with the original sources.
These readings support the concepts and deeper investigation. The worked numbers and illustrations above are teaching examples.
Primary paper
Shiu et al. (2024): a Drosophila computational brain model ↗
A connectome-based computational model of sensorimotor processing. Separate experimentally tested predictions from the model assumptions and untested behaviors.
Primary paper
Lin et al. (2024): network statistics of the fly connectome ↗
Study motif and network analysis with attention to filtering and null-model choices. Graph centrality is not automatically causal importance.