Patterns

Reverse-Engineer Through Evolution

When to use: a system is too complex to understand by inspecting its current form. How: start from the simplest ancestral version. Follow the changes forward and ask which new ability each change made possible and what survival problem forced it. Treat each living brain as “a time capsule” of earlier minds: a feature shared by all descendants of an ancestor was probably in that ancestor. Trade-offs: it tells one lineage’s story, and convergent evolution can mislead you. (Ch 0)

Two-Way Test Between AI and the Brain

When to use: judging a theory of how the brain computes something, or whether an AI system works like a brain. How: build the proposed algorithm. If it fails in a machine, the theory loses support. If it works and its properties match brain data (as TD learning matched dopamine), the theory gains support. Trade-offs: good performance alone proves nothing about mechanism; a calculator beats humans at arithmetic without understanding math. (Ch 0, 6)

Gradient Steering

When to use: navigating toward a goal with only a local signal and no map. How: keep going forward while the signal improves and turn when it worsens. Let neurons adapt to the baseline so a relative change fires the same at any level. Trade-offs: very cheap and robust, but the path circles in, and it needs a continuous signal. (Ch 2)

Persistent State After a Sparse Signal

When to use: signals are brief or patchy, and one detection predicts more nearby. How: let a cue switch the system into a mode (local search, escape, rest) that lasts after the cue fades, as affect does in worms and Dirt Detect does in a Roomba. Trade-offs: the system keeps searching or escaping after the cause is gone. (Ch 3)

Suppress, Don’t Delete

When to use: a learner faces rules that change. How: weaken an association quickly when it fails (extinction), let it return with time (spontaneous recovery), and relearn it faster when it works again (reacquisition). Trade-offs: the learner may briefly act on outdated rules. (Ch 4)

Cheap Credit Assignment Heuristics

When to use: deciding which of several cues caused an outcome with little computation. How: favor the cue from the last second or so (eligibility traces), the strongest cue (overshadowing), the novel cue (latent inhibition), and the cue already learned (blocking). Trade-offs: fails when the true cause is early, weak or familiar. (Ch 4)

Actor-Critic / Temporal Difference Learning

When to use: rewards are delayed or sparse, so the final outcome can’t say which earlier step earned it. How: train a critic to predict future reward at every step. Reinforce the actor by the change in that prediction, not by actual reward. Expected rewards then give no signal, and omitted rewards act as punishment. Trade-offs: looks circular and is unstable at first, but bootstraps (TD-Gammon). (Ch 6)

Curiosity Bonus

When to use: a large action space where the first reward is far away. How: make surprise or novelty reinforcing in itself, instead of adding more random actions. Trade-offs: drives pursuit of novel things with negative value, such as gambling and endless feeds. (Ch 8)

Build Known Structure into the Architecture (Inductive Bias)

When to use: data is scarce, or a property of the domain is certain. How: hard-code the property, as convolution hard-codes translation invariance. Trade-offs: the system handles only the invariances you built in; CNNs still fail on 3D rotation. (Ch 7)

Generative Replay

When to use: a learner must take in new patterns without forgetting old ones. How: replay or regenerate old experiences while training on new ones, as the hippocampus does. Trade-offs: costs extra computation and needs a memory or generator of past data. (Ch 7, 12)

Map-vs-Rules Test

When to use: checking whether an agent has a world model or only fixed rules. How: move its starting point or remove the goal cue, then see whether it takes new routes or shortcuts. Trade-offs: tests only spatial models. (Ch 9)

Hunch-Verifying Search

When to use: exhaustive search over options is impossible. How: let a fast model-free policy propose its top few options, simulate only those, and pick the best simulated outcome (AlphaZero; the aPFC). Trade-offs: cheap and reuses past learning, but misses options the policy never proposes. (Ch 13)

Uncertainty-Triggered Deliberation

When to use: choosing between a fast habit and slow planning. How: run the habit by default. Pause and simulate only when internal predictions disagree, such as when costs are close to benefits or conditions change. Trade-offs: saves effort, but heavy repetition removes the uncertainty, so habits persist after goals change. (Ch 13)

Train the Fast System in Imagination

When to use: real trials are costly or dangerous. How: let simulated outcomes reinforce the model-free learner as if they were real (“not by telling but by showing”). Trade-offs: a wrong world model trains wrong habits. (Ch 13)

Hierarchy of Simulation and Automation

When to use: designing control for complex, multi-step behavior. How: break high-level goals into subgoals. At each level, pair a slow, flexible simulator with a fast automatic path, and hand learned skills down to automation. Trade-offs: efficient, but if the middle level breaks, lower levels act without the overall goal. (Ch 14)

Model Yourself to Model Others

When to use: inferring another agent’s goals or knowledge, or your own future needs. How: simulate yourself in their situation, with their knowledge, and ask why you would act that way. For your future self, ask what you would want after the coming change. Trade-offs: your current state leaks in (projection bias; hungry shoppers buy too much). (Ch 16, 18)

Infer Intent Before Copying (Inverse RL, Active Teaching)

When to use: learning a skill from demonstrations. How: infer the reward or goal the expert is pursuing and copy only the steps that serve it, then learn by trial and error against it. Or let a teacher share control and correct errors as they happen. Trade-offs: inferring intent is harder than copying and can be wrong; active teaching costs the teacher’s time. Direct copying (ALVINN) never learns to recover from its own errors. (Ch 17)

Change the Curriculum Before the Learner

When to use: a learner fails to acquire a complex skill. How: order examples from simple to complex, or use self-play so the opponent stays evenly matched. Evolution produced language this way, through hardwired proto-conversation, joint attention and questions. Trade-offs: designing the curriculum takes work and can hide weaknesses in the learner. (Ch 20)

Simulation Probe for World Models

When to use: checking whether a system reasons from a world model or only predicts words. How: ask questions whose answers exist only in a simulation of physics, numbers or minds, and watch for the reflex answer (bat and ball). Trade-offs: published probes become training data, and chain-of-thought training can pass them without a world model. (Ch 22)

Intent Inference for Commands

When to use: before an agent optimizes any instruction. How: model the requester’s desired end state and the limits they didn’t state. Trade-offs: needs a model of the other mind; without one you get the paper-clip problem. (Ch 22)

Dependency-Chain Reasoning

When to use: predicting which abilities a system can gain next, or explaining why one is missing. How: list the layers an ability rests on. Speaking needs mentalizing, mentalizing needs simulating, simulating needs reinforcing, reinforcing needs valence. Trade-offs: biology may not map cleanly onto AI. (Ch 23)