Cheatsheet

Decision Rules

When…	Do…	Because…
A task seems easy	Ask whether it is easy only for a human brain.	Tasks easy for us are often “astronomically difficult” for machines (chess vs dishwashing).
An AI beats humans at a task	Don’t conclude it solves the task the way humans do.	A calculator beats humans at arithmetic without understanding math.
A model divides the brain into stacked layers	Reject it.	Instinct, emotion and cognition span the whole brain (the myth of layers).
A stimulus’s meaning depends on context	Model valence as a function of internal state.	The same signal is good, bad or neutral depending on need.
An association fails once after a long record	Suppress it but expect it back.	The failure may be a fluke (spontaneous recovery).
A reward comes long after the actions	Learn from changes in predicted reward.	Reinforcing recent or all moves fails (temporal credit assignment).
A reward is fully expected	Expect no learning signal.	Prediction cancels the reward; dopamine stays at baseline.
An expected reward is omitted	Treat it as punishment, and an omitted punishment as reward.	This is disappointment and relief; it needs precise timing.
The first reward is far away	Add curiosity, not more random actions.	Random exploration fails (Montezuma’s Revenge).
A network must keep learning after deployment	Plan for catastrophic forgetting.	Common practice is to freeze and retrain from scratch.
Predictions agree and nothing has changed	Act on the habit.	Simulating costs time and energy.
Costs are close to benefits, or rules or needs change	Pause and simulate.	That is when model-free values are wrong.
An imitation-learned system fails after small errors	Infer the expert’s intent, or add a teacher who corrects in real time.	Direct copying never shows how to recover.
You test for theory of mind	Test what it thinks others know (false belief), not only what they want.	Dogs, attentive as they are, fail to track what their trainer knows.
You plan for a future need while in a strong state	Correct your forecast.	Your current state inflates the predicted need.
A signal was learned for a reward	Call it an imperative, not language.	Any model-free learner can do that; language needs declarative labels and grammar.
An LLM answers commonsense questions	Don’t infer a world model.	RLHF and chain-of-thought can produce answers without one.
You give an AI a goal	State the intended end state and its limits.	Requests are not well-defined goals (the paper-clip problem).
A skill won’t train	Change the curriculum before the learner.	Both evolution and AI found that easier.
A trait helps the group but costs the individual	Look for kin selection or punishment of defectors.	Group benefit alone is not evolutionarily stable.

Which Breakthrough Is This? (place any ability or AI technique)

• Turn toward good, away from bad; mood-like persistent states → #1 Steering (Ch 2–4)
• Learn by trial and error from reward; habits; curiosity; spatial maps → #2 Reinforcing (Ch 5–9)
• Plan by imagining; replay the past; ask what would have happened after another choice; skilled movement → #3 Simulating (Ch 10–14)
• Infer others’ intent and knowledge; learn new skills by watching; plan for future needs → #4 Mentalizing (Ch 15–18)
• Name things; use grammar; pass ideas across generations → #5 Speaking (Ch 19–22)
• Then check the layers below: an ability without its foundations is imitation, not the real thing (LLMs have #5-like output without #3 and #4).

Model-Free vs Model-Based

	Model-free	Model-based
Speed	Fast	Slow
Flexibility	Low; insensitive to devaluation once habitual	High; adapts when goals or rules change
Brain	Basal ganglia (vertebrates)	Neocortex + basal ganglia (mammals)
AI	TD-Gammon, Atari agents	AlphaZero
Use when	Familiar, stable situations	Close calls, new situations, changed needs

Thresholds & Numbers

• Eligibility traces credit cues from about 0–1 second before the outcome (Ch 4).
• Nematodes give up on inescapable stress after about two minutes (Ch 3).
• Habits: rats trained for 500 presses ignore devaluation; at 100 they don’t (Ch 13).
• Cooperation by direct relationships tops out near 150 people for a human-sized neocortex; common myths have no limit (Fig 19.1, Ch 19).
• Humans are born with 28 percent of adult brain size (chimpanzee 36, macaque 70) (Ch 21).
• The brain has 86 billion neurons and over 100 trillion connections (Ch 0).

Tells

• Behavior survives a moved start point or a removed cue → there is a world model, not a rule (Ch 9).
• A rat pauses and looks back and forth at a choice point → vicarious trial and error (Ch 12).
• It keeps working for a reward it no longer wants → a habit, with no simulation running (Ch 13).
• It copies irrelevant steps of a demonstration → over-imitation. Human children do it, and copy more the more they believe the teacher knows; chimps skip the irrelevant steps (Ch 19).
• Fluent answers that fail on fill-in-the-blank physics or minds → words without inner worlds (Ch 22).
• Impulsive choices when tired or stressed → the expensive aPFC is offline (Ch 13).