Chapter 9: The First Model of the World

Core Idea

Early vertebrates were the first animals to build an internal model of the world: a spatial map that records where things are relative to each other. Fish can find a remembered location from any starting point, while nematodes, ants, and bees follow fixed turning rules. The map depends on a vestibular sense from the semicircular canals, head-direction neurons in the hindbrain, and the medial cortex, which became the hippocampus in mammals. This first model was used for navigation, but it laid the ground for the next breakthrough, which is simulating.

Frameworks Introduced

• Map vs rules (how to tell if an animal has a world model):
  • When to use: when you need to decide whether an agent has learned a model of space or only a sequence of cue-triggered moves.
  • How: change the starting point, or remove the cue that marked the goal. An agent with a map still goes straight to the goal and takes shortcuts. An agent that learned only rules (“turn right at cue X, turn left at cue Y”) repeats its learned route.
  • Why it works: rules are tied to a fixed path, while a map gives the right direction to a target from any starting location.
• The ingredients of a spatial map:
  • Self-motion: the semicircular canals in the inner ear form three loops, one each for forward, sideways, and upward movement. Fluid in a canal moves only when the head moves in that direction, which gives the vestibular sense. Figure 9.1 zooms from a fish’s head to the inner ear, then to the semicircular canals, then to the sensory neurons that line them. The canals first appeared in early vertebrates.
  • Self vs world: the vestibular sense separates “something swimming toward me” from “me swimming toward something.” Both look the same to the eyes, but only one activates the canals.
  • Direction: head-direction neurons in the hindbrain fire only when the animal faces a certain direction. They combine visual and vestibular input into “a neural compass.”
  • Map: visual, vestibular, and head-direction signals meet in the medial cortex, where they are converted into a spatial map.
• The three areas of the early vertebrate cortex (Figure 9.2, a cross-section of a fish-like forebrain):
  • The lateral cortex recognized smells and later became the olfactory cortex in mammals.
  • The ventral cortex learned patterns of sights and sounds and later became the amygdala.
  • The medial cortex, folded into the middle, later became the hippocampus.

Key Concepts

• Spatial map: an internal model of where things are relative to other things. It is found in fish, reptiles, mice, monkeys, and humans, but not in nematodes.
• Vestibular sense: the sense of one’s own head movement, produced by the semicircular canals. Spinning too long overactivates it, which is why you feel dizzy after you stop.
• Head-direction neurons: hindbrain cells, found in species as different as fish and rats, that encode which way the animal is facing.
• Medial cortex (hippocampus): in fish it contains neurons that fire at specific locations, at the borders of the tank, or when facing specific directions. In humans and rats, the hippocampus has place cells.
• Hippocampal damage: fish with a damaged hippocampus can still learn to swim toward or away from cues, but they cannot remember locations. Damage also impairs navigation in lizards, rats, and humans.
• Internal model: a representation of the external world inside the brain. Bennett calls this the most important first of early vertebrates.

Mental Models

• Use the swivel-chair test on yourself: if you can guess which way you face after being turned with your eyes closed, your brain is tracking your own movement.
• Think of navigating your home in the dark as running on a stored map. The surprising part is not that it is hard but that it works at all.
• Think of object recognition and spatial mapping as possibly the same skill. Bennett speculates: “Perhaps the cortex is tuned to model 3D things—whether those things are objects or spatial maps.”
• Think of a world model as first being a practical tool. It let a fish remember where predators hid, how to get back to safety, and where food was.

Anti-patterns

• Mistaking a learned route for a map: ants that are moved onto the outbound path walk the whole loop again instead of turning around, because they follow turning rules.
• Treating the three-layered cortex as only auto-associative: it also handles invariance and builds models of space.
• Assuming advanced invertebrates must have maps: many, including bees and ants, fail spatial tasks.

Rules of Thumb (author’s wording)

• “Ants navigate by following a set of rules of when to turn where, not by constructing a map of space.”
• “It was also the first time a brain differentiated the self from the world.”
• “What began as a trick for remembering locations would go on to become much more.”

Worked Example

The fish and the twenty-five containers, compared with displaced ants.

1. Place a fish in a tank with a grid of twenty-five identical containers, and hide food in one. The fish inspects containers at random until it finds the food.
2. Remove the fish, put food back in the same container, and return the fish. After a few rounds it darts straight to that container.
3. Start the fish from different places. It still finds the container, so it has not learned a fixed rule such as always turning left at a certain object.
4. Leave all containers empty. The fish still goes to the right one, so it is not following the sight or smell of food.
5. The only clue left is the markings on the tank walls. The fish must be locating the container relative to those landmarks, which requires a spatial map.
6. Now compare ants. Ants use one path from nest to food and another path back. Take an ant that is carrying food home and put it on the outbound path. A fish in that position would turn around. The ant instead walks the entire loop again, because it learned a sequence of moves, not a map.

Key Takeaways

1. Early vertebrates built the first internal model of the world, a spatial map.
2. The map combines the vestibular sense, head-direction neurons, and visual input in the medial cortex, which later became the hippocampus.
3. A map lets an animal reach a goal from any starting point. Rules only repeat a route.
4. Tracking one’s place in a map requires telling self-motion apart from the motion of the world. This was the first separation of self from world.
5. This model of space is the base for simulating in mammals.

Connects To

• Ch 7: the same three-layered cortex recognizes patterns and handles invariance.
• Ch 8: curiosity drives exploration that fills in the map.
• Ch 10–14: breakthrough #3 (simulating) builds on this internal model, letting mammals learn by imagining.

Summary of Breakthrough #2: Reinforcing

About five hundred million years ago, our ancestors changed from wormlike bilaterians into fishlike vertebrates. Bennett groups the new brain structures and abilities around reinforcement learning:

• Dopamine became a temporal difference learning signal. This helped solve the temporal credit assignment problem and let animals learn through trial and error (Ch 6).
• The basal ganglia emerged as an actor-critic system. It generates the dopamine signal by predicting future rewards, and it uses that signal to reinforce and punish behaviors (Ch 6).
• Curiosity emerged as a necessary part of reinforcement learning, as the answer to the exploration-exploitation dilemma (Ch 8).
• The cortex emerged as an auto-associative network, which made pattern recognition possible (Ch 7).
• Precise timing emerged, so animals could learn not only what to do but when to do it (Ch 6).
• Perception of three-dimensional space emerged in the hippocampus and other structures, so animals could recognize where they were and remember where things were relative to each other (Ch 9).

This breakthrough rests on the first one. “Reinforcement learning is bootstrapped on simpler valence signals of good and bad.” The vertebrate brain is built on top of the steering system of bilaterians. “Without steering, there is no starting point for trial and error, no foundation on which to measure what to reinforce or un-reinforce.”

Figure 9.3 compares the two designs. The early bilaterian brain is a single loop: sensory input goes to valence, which produces a reflex motor response. The early vertebrate brain keeps that loop, with valence now in the hypothalamus. It adds two boxes. The cortex handles “Pattern & place recognition” and sends patterns and places to the basal ganglia. The basal ganglia handles “Trial & error learning.” A two-way link marked “Temporal difference signal via dopamine” joins the basal ganglia and the hypothalamus. The basal ganglia sends a “Learned response” to the motor system alongside the hypothalamus’s reflex response.

Next, early mammals developed a different kind of trial and error: “learning not by doing but by imagining” (Ch 10–14).