Chapter 5: The Cambrian Explosion

Core Idea

About 500 million years ago, in the Cambrian, a small fishlike ancestor gave rise to the vertebrates and to a brain template that every vertebrate still shares. Bennett argues that the defining skill of this brain was the second breakthrough: reinforcement learning, which he defines as “the ability to learn arbitrary sequences of actions through trial and error.” Thorndike’s puzzle-box experiments showed that trial and error alone can produce complex, intelligent-looking behavior, and fish learn this way as well. Simple bilaterians such as nematodes cannot. The idea sounds easy, but AI researchers found it hard to make work, and that difficulty frames the next four chapters.

Frameworks Introduced

• The vertebrate brain template:
  • When to use: when you want to map a human brain structure onto a fish, reptile, or mouse, or to judge how old a brain function is.
  • How: every vertebrate embryo first forms three bulbs: forebrain, midbrain, and hindbrain. The forebrain then splits into two subsystems. One becomes the cortex and basal ganglia, and the other becomes the thalamus and hypothalamus. The result is six main structures in all vertebrates: cortex, basal ganglia, thalamus, hypothalamus, midbrain, and hindbrain.
  • Figure 5.3 shows this in three panels. Stage 1 has a forebrain, midbrain, hindbrain, and spinal cord. In stage 2 the forebrain has divided into “Cortex & basal ganglia” and “Thalamus & hypothalamus.” The adult fish, reptile, and mouse brains share the same parts (cortex, hypothalamus, tectum in the midbrain, and hindbrain), but the cortex is small in the fish and covers most of the mouse brain.
  • Figure 5.4 labels the brain of the first vertebrates from front to back. The forebrain holds the olfactory bulb (fed by neurons from the nose), the cortex, the basal ganglia, and the hypothalamus, with neurons from the eyes entering below. The midbrain holds the tectum. The hindbrain holds the cerebellum and runs into the spinal cord.
  • Why it works: apart from the cortex, which mammals modified (breakthrough #3), the circuitry of the human basal ganglia, thalamus, hypothalamus, midbrain, and hindbrain is “incredibly similar” to a fish’s. That is why a fish brain can teach you so much about a human one.
• Thorndike’s law of effect:
  • The law in his words: “Responses that produce a satisfying effect in a particular situation become more likely to occur again in that situation, and responses that produce a discomforting effect become less likely to occur again in that situation.”
  • How it works: the animal first performs random exploratory actions. Positive valence then reinforces the actions it just performed, and negative valence un-reinforces them. Psychologists later replaced “satisfying” and “discomforting” with “reinforcing” and “punishing,” because the old words suggested an inner feeling.
  • Failure mode: B. F. Skinner extended the law to claim that all behavior, even in humans, comes from trial and error. Bennett says breakthroughs #3, #4, and #5 show that Skinner was wrong, though trial and error still underlies a large part of learning.
• The comparative method (Thorndike’s Darwinist bet):
  • When to use: when you want to infer what an ancestral brain could do.
  • How: animals that share an ancestor should share the learning mechanisms they inherited from it. So test the same task on distant relatives. Cats, dogs, and birds share an ancestor from about 350 million years ago. Fish share one with them from about 500 million years ago. If fish pass the task and nematodes fail it, the ability probably arose in the first vertebrates.
  • Figure 5.2 shows the tree this method relies on. The first bilaterian splits into invertebrates (a nematode, an insect, and a slug) and the first vertebrate. The first vertebrate splits into jawless fish (the lamprey) and a branch that leads to bony fish, cartilaginous fish, reptiles, and mammals.

Key Concepts

• Cambrian period: the era from 540 to 485 million years ago. Large arthropods dominated it, and some grew over five feet long.
• Cambrian explosion: the largest expansion of animal diversity in Earth’s history. Bennett says steering in bilaterians sped up the arms race of predation that triggered it.
• Vertebrates: descendants of the fishlike ancestor, named for the vertebral column that protects the spinal cord. Early ones had fins, gills, a spinal cord, two eyes, nostrils, and a heart, and were only a few inches long.
• Lamprey: a jawless fish whose last common ancestor with humans was the first vertebrate, over 500 million years ago. Its brain shares most human brain structures. The part’s opening art, “Your brain 500 million years ago,” shows a lamprey-like fish with a row of gill openings chasing a tiny shrimp-like animal.
• Puzzle box: Thorndike’s cage with a latch, hidden button, or hoop that opened an escape door. Learning was measured as the time to escape across up to a hundred trials.
• Trial-and-error learning: learning by trying actions and keeping the ones that lead to good outcomes. It “often goes by another name: reinforcement learning.”
• Reinforcing and punishing: the later terms for Thorndike’s “satisfying” and “discomforting” outcomes.

Mental Models

• Use the fish brain as a first model of the human brain. In Bennett’s words, “If you want a crash course in how the human brain works, learning how the fish brain works will get you half of the way there.”
• Think of complex behavior as possibly the result of many small reinforced steps, before assuming it needs insight, imitation, or planning.
• Use a capability gap between vertebrates and simple bilaterians as evidence of when a skill evolved. A nematode cannot learn arbitrary action sequences, so the skill came with the vertebrates.
• Expect simple-sounding learning rules to be hard to build. This is a case where “our intuitions about what is intellectually easy and what is hard are mistaken.”

Anti-patterns

• Treating fish as dumb: the folk claim that fish remember for only three seconds is wrong. Fish remember trained tasks for months or even years, and “fish are far smarter than we give them credit for.”
• Assuming intelligent behavior needs insight or imitation: Thorndike showed that trial and error was enough for complex action sequences.
• Expecting imitation everywhere: Thorndike’s untrained cats did not improve by watching trained cats. Some animals can imitate, which is breakthrough #4.
• Over-extending trial and error: Skinner’s claim that it explains all behavior was wrong.

Rules of Thumb (author’s wording)

• “Responses that produce a satisfying effect in a particular situation become more likely to occur again in that situation.”
• “The second breakthrough was reinforcement learning: the ability to learn arbitrary sequences of actions through trial and error.”
• “A nematode cannot be trained to perform arbitrary sequences of actions; it will never learn to navigate through hoops to get food.”

Worked Example

Thorndike’s puzzle boxes (dissertation published in 1898, when he was twenty-three).

1. Harvard would not let Thorndike experiment on children, so he studied chickens, cats, and dogs instead. He expected shared learning principles, because these animals share an ancestor.
2. He built cages that opened when the animal pushed a latch, pressed a hidden button, or pulled a hoop. In some boxes he opened the door himself when the animal did a specific act, such as licking itself. Food sat outside the cage. Figure 5.5 shows one box: a slatted wooden crate with a door held by a latch and a hanging string loop.
3. A new cat tried many behaviors, such as scratching, pushing the ceiling, digging at the door, howling, squeezing through the bars, and pacing. Eventually it triggered the release by accident and ate the food.
4. Thorndike put the cat back again and again and timed each escape. Over many trials, the cat dropped its other behaviors and performed the escape action right away.
5. Figure 5.6 plots time to escape against trials for three boxes. In box 1 the door opens if the cat pulls a loop, and the curve falls quickly and smoothly. In box 2 the door opens if the cat licks itself, and the curve is noisy before it settles low. In box 3 the door opens after a sequence of movements, and the curve stays spiky longer before it flattens. In all three, escape time decays gradually.
6. A year later Thorndike tested fish. A fish started in bright light, which fish dislike, and had to pass through transparent walls with hidden gaps to reach the dark. It bumped into walls at first, then learned to swim straight through each gap. Later studies showed that fish learn to push buttons, use escape hatches in nets, and jump through hoops.

Key Takeaways

1. All vertebrates share a six-part brain template that dates to the first vertebrates more than 500 million years ago.
2. Thorndike’s law of effect says actions followed by good outcomes become more likely, and actions followed by bad outcomes become less likely.
3. Trial and error alone can produce complex sequences of behavior.
4. Fish learn through trial and error and remember for months or years, while nematodes and other simple bilaterians cannot learn arbitrary sequences.
5. Reinforcement learning only looks simple; making it work is the problem that chapters 6–9 address.

Connects To

• Ch 2–4: valence, the dopamine “wanting” signal, and associative learning in early bilaterians are the base that trial and error builds on.
• Ch 6: the first AI attempts at Thorndike-style learning failed because of the temporal credit assignment problem.
• Ch 7–9: pattern recognition, curiosity, and spatial maps are the other skills that came with the vertebrate brain.
• Ch 10–14: the mammalian cortex changes and learning by imagining are the reasons Skinner’s view falls short.
• Ch 15–18: imitation, which Thorndike’s cats lacked, appears in primates.