Chapter 17: Monkey Hammers and Self-Driving Cars

Core Idea

Primates are unusually good tool users, and different groups of the same species use tools differently. Bennett argues the reason is transmission, not invention: a skill only needs to be invented once if others can copy it. Many animals can use observation to select a behavior they already know. Primates can use observation to acquire a novel skill, and that takes theory of mind. Theory of mind lets teachers teach, keeps learners trying for years without reward, and lets learners tell an expert’s intended movements from accidental ones. Robotics found the same answer. Copying an expert’s actions directly (ALVINN) is brittle. Active teaching and inverse reinforcement learning, which infers the expert’s intent, work far better.

Frameworks Introduced

• Transmissibility beats ingenuity:
  • When to use: to explain why a group has a skill, or to judge how a skill spreads.
  • How: count how often a skill gets passed on, not how often it gets invented. If one chimp invents a termite-fishing stick, the whole group can learn it and pass it down for generations.
  • Why it works: invention is rare and copying is common. Without transmission, a clever individual’s skill dies with it.
• Selecting known skills versus acquiring novel skills through observation (book’s figure):

	Selecting known skills through observation	Acquiring novel skills through observation
Who	Many mammals, octopuses, fish, reptiles	Primates, some birds
What happens	The animal picks a behavior it already has after seeing another do it	The animal learns a movement it could not do before
Examples	Young mongooses copy their parent’s egg-opening method, but all of them can both bite and throw. Kittens use litter boxes only if their mother did, but all kittens can pee. Fish copy paths, not swimming.	Young chimps learn nut cracking and termite fishing by watching
What it needs	Simple reflexes (follow other fish, look where others look) or simulating a known action	Theory of mind, or at least a large benefit from it

• Three reasons novel-skill learning needs theory of mind:
  1. It enables active teaching. Teaching requires knowing what another mind does not know.
  2. It keeps learners focused over long periods. A chimp child practices nut cracking for years without success, because it understands that its mother has a skill it lacks. A rat gives up when copying doesn’t pay off soon.
  3. It separates intentional from unintentional movements, so the learner copies only what matters.
• Two fixes for brittle imitation learning in robotics:
  • Active teaching: an expert shares control with the AI and corrects its mistakes as they happen (Ross and Bagnell, 2009).
  • Inverse reinforcement learning: the AI first infers the reward function the expert seems to be optimizing (the “intent”), then learns by trial and error against that inferred reward (Abbeel, Coates and Ng, 2010). In standard reinforcement learning the reward function is hard-coded, not learned.
  • Failure mode of plain copying: the system has seen only correct behavior, so it has never seen a recovery from a mistake.

Key Concepts

• Primate tool use: chimp groups often show over twenty tool behaviors. With the possible exception of birds and elephants, only primates manufacture tools, for example by shortening, sharpening and stripping a stick. Wrasses, mongooses and sea otters typically have a single trick.
• Tool cultures: Goualougo chimps make termite-fishing rods differently from Gombe chimps. Some groups crack nuts with rocks, pound beehives with clubs, or swat flies with leafy twigs, and others don’t.
• Mirror neurons: neurons in premotor and motor cortex that fire both when a primate makes a movement and when it watches another make it. They were found by accident in 1990 in Giacomo Rizzolatti’s lab in Parma, when a monkey’s grasping neurons fired as a lab member bit into a sandwich.
• Mirror neurons as imagined movement (Bennett’s preferred reading): the neurons fire when a monkey hears a peanut break open without seeing it, and when a human seems to lift a box hidden behind a wall (but not if the monkey knows no box is there). An automatic mirror would need to see the movement.
• Action production and action recognition share circuits: patients who can’t mime using a toothbrush also struggle to recognize teeth-brushing in others. Inhibiting the hand area of premotor cortex impairs recognizing mimed hand actions but not mouth actions, and inhibiting the mouth area does the reverse.
• Sensorimotor understanding: the premotor cortex helps you infer the physical side of others’ actions, such as how heavy a box is from how someone lifts it. Inhibiting it impairs judging the box’s weight but not judging a bouncing ball’s weight.
• Imitation learning needs premotor cortex: novice guitarists’ premotor cortex activates far more when watching an unknown chord than a known one. Inhibiting premotor cortex impairs copying hand movements on a keyboard but not following a red dot to the same keys.
• ALVINN (Autonomous Land Vehicle in a Neural Network): Dean Pomerleau and Chuck Thorpe’s 1990 self-driving system at Carnegie Mellon, trained to map road images to steering positions.

Mental Models

• Think of tool use as culture: ask who they learned it from before asking how clever they are.
• Use “Why would I do this?” as the way primates read others’ movements: simulate yourself doing the movement, then infer the goal.
• Think of an expert’s demonstration as signal plus noise. Intent is the filter that removes the noise, for primates (the puzzle box) and for robots (the helicopter pilots who kept correcting small mistakes).
• Use active teaching when a learner’s own errors take it into states the expert never showed.

Anti-patterns

• Treating observational learning as uniquely primate: rats learn lever pushing by watching, dogs learn lever pulling, dolphins imitate, and fish and reptiles copy paths. What is special in primates is learning novel skills.
• Copying expert behavior directly: ALVINN could not recover from small errors and veered off the road. Ross’s direct-copy system was still crashing after a million frames of expert data.
• Copying every movement of a demonstration: the chimp in the puzzle-box study skipped the experimenter’s irrelevant wand tapping and box rotating.
• Assuming tool use is uniquely human: tool use is found in primates, elephants, mongooses, crows, octopuses and wrasses.

Rules of Thumb (author’s wording)

• “The ability to use tools is less about ingenuity and more about transmissibility.”
• “Ingenuity must occur only once if transmissibility occurs frequently”
• “Teaching is possible only with theory of mind.”
• “Directly copying expert behaviors turned out to be a dangerously brittle approach to imitation learning.”
• A novice cannot reliably learn a new motor skill only by watching an expert’s movements; “novices must also peer into an expert’s mind.”
• “This is why primates use hammers and rats do not.”

Worked Example

From ALVINN to inverse reinforcement learning.

1. In 1990 Pomerleau recorded video and his steering-wheel position while he drove, and trained a neural network to map each road image to his steering. Earlier autonomous cars in Thorpe’s group could go only a quarter of a mile per hour. After only a few minutes of observation, ALVINN steered well and drove Pomerleau from Pittsburgh to the Great Lakes on a real highway.
2. The flaw: ALVINN had only seen correct driving. Small errors cascaded into failures, often off the road.
3. Fix 1, active teaching (2009): Stephane Ross drove a simulated Mario Kart track and traded control with the AI. He did most of the driving at first and quickly recovered the AI’s mistakes, then handed over more control. It drove almost perfectly after only a handful of laps. Bennett compares this to a chimp mother correcting her daughter’s termite-fishing.
4. Fix 2, inverse reinforcement learning (2010): Pieter Abbeel, Adam Coates and Andrew Ng recorded expert remote-control inputs for helicopter aerobatics: flips in place, forward rolls, upside-down flight and loops. Direct copying did not work.
5. Their system first inferred the trajectories the experts intended, then learned to pursue them, rewarding and punishing itself with the inferred reward function.
6. Because it modeled intent, it filtered out the experts’ small mistakes and corrected its own. By 2010 it flew aerobatics on its own.
7. Lesson: roboticists and evolution converged. Learning a new skill from a demonstration works best when the learner infers what the demonstrator meant to do.

The primate side. In a 1987 study, young chimps who watched an adult use a T-bar rake to reach food learned it, and those who didn’t watch never did, even though they could see the food. In another set of studies, one chimpanzee, capuchin or marmoset was taught a new food-getting skill and returned to its group. Within a month almost the whole group used it, and the skill passed down through generations. A chimp that has not learned nut cracking by watching by age five will not learn it later. In the 1990s, Christophe Boesch reported chimp mothers cracking nuts in slow motion near their young. The mothers also checked that the young were watching, fixed mistakes by cleaning the anvil, and turned the hammer in the child’s hands.

Key Takeaways

1. Primate tool skills spread by observation, so tool use varies by group like a culture.
2. Selecting a known behavior by watching is common; acquiring a novel skill by watching is rare and depends on theory of mind.
3. Theory of mind helps in three ways: teaching, staying motivated for years without reward, and filtering intended movements from accidents.
4. Mirror neurons are best read as imagined movements, which let a primate simulate itself doing what it sees.
5. In AI, direct imitation is brittle; active teaching and inverse reinforcement learning, which infers the expert’s reward function, fix it.
6. Theory of mind evolved for politics and was then repurposed for imitation learning.

Connects To

• Ch 12 and Ch 14 (Breakthrough #3, Simulating): motor cortex activates during imagined movement, and mental rehearsal improves real performance.
• Ch 5–9 (Breakthrough #2, Reinforcing): inverse reinforcement learning reverses standard reinforcement learning by learning the reward function from behavior.
• Ch 15 and Ch 16: theory of mind and its basis in modeling your own mind.
• Ch 18: the third ability built on the same machinery, anticipating future needs.
• Ch 19–22 (Breakthrough #5, Speaking): language takes transmissibility much further.