Chapter 2: Avoiding Bad Data

Core Idea

Even with good questions, conversations drift into three kinds of bad data: compliments, fluff, and ideas. Get back to facts by deflecting compliments, anchoring fluff, and digging beneath ideas. Also stop producing bad data yourself through approval-seeking, pitching, and talking too much.

Frameworks Introduced

• The three types of bad data: Compliments; Fluff (generics, hypotheticals, and the future); Ideas.
  • Bad data gives false negatives (you think the idea is dead when it isn’t) and, more dangerously, false positives (you think you’re right when you aren’t).
• Deflecting compliments:
  • When to use: whenever you hear praise, especially at the end of a meeting.
  • How: if you pitched, apologize. Then ask about their current behaviour (“how are you dealing with this at the moment?”). Ignore the small compliments that keep appearing at the end of later answers.
  • Check: if you can’t say why they liked it, how much it would save them, how it fits their life, and what else they tried, then you have a compliment and not data.
  • Symptoms in the meeting: “Thanks!”, “I’m glad you like it.” Symptoms back at the office: “That meeting went really well.” “We’re getting a lot of positive feedback.” “Everybody I’ve talked to loves the idea.”
• Anchoring fluff:
  • Fluff comes in three shapes: generic claims (“I usually”, “I always”, “I never”); future-tense promises (“I would”, “I will”); hypothetical maybes (“I might”, “I could”).
  • How: ask when it last happened, ask them to talk you through it, and ask how they solved it and what else they tried.
  • Fluff-inducing questions: “Do you ever…”, “Would you ever…”, “What do you usually…”, “Do you think you…”, “Might you…”, “Could you see yourself…”. Asking them is not the mistake. Valuing the answers is. They can serve as a starting point: “Do you ever X?” → “All the time.” → “When’s the last time that happened?” → “Two weekends ago.” → “Can you talk me through that?”
  • The most dangerous fluff is “I would definitely buy that.” The author’s first startup believed it and lost about $10 million.
• Digging beneath ideas (and emotions):
  • When to use: when someone suggests features or ideas, or shows strong emotion.
  • How: write the request down but don’t add it to the to-do list. Ask about the motivation behind it.
  • Questions for feature requests: “Why do you want that?” “What would that let you do?” “How are you coping without it?” “Do you think we should push back the launch to add that feature, or is it something we could add later?” “How would that fit into your day?”
  • Questions for emotional signals: “Tell me more about that.” “That seems to really bug you — I bet there’s a story here.” “What makes it so awful?” “Why haven’t you been able to fix this already?” “You seem pretty excited about that — it’s a big deal?” “Why so happy?” “Go on.”
• Avoiding approval-seeking has two forms:
  • Fishing for compliments (deliberate): you’ve already decided and want a blessing. “I had an awesome idea for an app — do you like it?”
  • The Pathos Problem (accidental): you expose your ego, so people protect you with nice words. “So here’s that top-secret project I quit my job for… what do you think?” “I can take it — be honest and tell me what you really think!” Fix it by focusing on them and on specific past cases.
• Cut off pitches: when you notice you’re monologuing, apologize and return to what they were saying. If they want to hear about your idea, promise to tell them at the end so you don’t bias them first. Symptoms: “No no, I don’t think you get it…” “Yes, but it also does this!”
• Talk less: don’t interrupt to correct their understanding (“So it’s similar to…”) or to show you’ve already thought about a concern. Those moments show you how they see the world.

Key Concepts

• Fluff: generic, hypothetical, or future-tense statements that sound like data but aren’t.
• Sneaky compliment: praise tacked onto the end of an otherwise factual answer.
• Complainer vs customer: a complainer grumbles about a problem but has never looked for a solution, so won’t buy yours.
• Flipping to your side of the table: when an excited person starts suggesting ideas and features. It’s a good sign, but the ideas themselves are not data.
• Feature creep: the result of turning every request into a to-do item.
• Pitch mode: talking about your idea instead of learning about their problems.

Mental Models

• A compliment is a warning flag, not a result. It often means the person is trying to end the conversation.
• When people speak in generics, they describe who they want to be, not who they are. Specifics bring out the edge cases.
• When someone is noncommittal, put them to a decision. If they won’t solve their problem today, they won’t care about your solution tomorrow.
• A feature request is a symptom. The motivation behind it is the real data.
• Famous founders who openly ask for criticism (Elon Musk, Reid Hoffman, Gordon Ramsay) are not a model to copy. Nobody worries about their feelings, so their approach doesn’t transfer to you.

Anti-patterns

• Treating a compliment as validation: six months later you have zero customers.
• Hearing “that’s the worst part of my day” and moving on: any strong emotion deserves a dig.
• Holding someone hostage until they say they like it: their compliment is just a way to get you out of the room.
• Correcting them mid-sentence: you lose a view of their mental model, and it annoys people who are trying to help.

Rules of Thumb (author’s wording)

• Compliments are the fool’s gold of customer learning: shiny, distracting, and worthless.
• Ideas and feature requests should be understood, but not obeyed.
• If you’ve mentioned your idea, people will try to protect your feelings.
• Anyone will say your idea is great if you’re annoying enough about it.
• The more you’re talking, the worse you’re doing.

Worked Example

The MTV analytics request (the author’s company, Habit).

1. MTV said they needed analytics and reports for their campaigns. The team took the request at face value and built a polished self-serve dashboard with many ways to slice the data. MTV reacted with “ooh” and “ahh.”
2. MTV then called every Friday asking for a CSV of the week’s stats, so the team built CSV export. Then MTV asked for PDF, so the team built PDF export, which took longer.
3. Weeks later, MTV still called every Friday for the same report. Then they asked for their logo and colours on it.
4. Frustrated, the author finally asked why they wanted branding on reports they didn’t even use. The answer: nobody reads them; MTV’s own clients just like getting something emailed every week, and fancier would be nicer.
5. The real motivation was keeping MTV’s clients happy, not understanding data. The right solution was a few headline numbers sent automatically every Friday, possibly assembled by an intern. No dashboard was needed. About 90% of what was built was irrelevant, and the author lost about three months.

A second example: the loyalty-card complainer. A stranger complains about carrying many café loyalty cards.

• Bad: pitch your loyalty app and ask “Do you think you would use something like that?” You get “I would definitely use that,” which is pure fluff.
• Good: agree, then ask whether they’ve tried any loyalty apps. “Those exist?” Then “Why don’t you download it now?” “I’ll do it next time.” That means it isn’t a real problem.
• Other useful answers you could get instead: they tried several but need a different app per chain (you’d need critical mass of retailers first); the discounts are too small and feel like data collection (you’d need better perks and to address privacy); the existing app is too slow to use (you’d need to simplify and out-execute).

Key Takeaways

1. Expect compliments in almost every answer, and discard them all.
2. Translate every “usually”, “would”, and “might” into “when was the last time?”
3. Write feature requests down, then ask what the feature would let them do.
4. Dig into every strong emotion, whether anger, embarrassment, or excitement.
5. When you catch yourself pitching, apologize and return to their story.
6. Keep your idea and your ego out of the conversation until you’re ready to ask for commitments.

Connects To

• Ch 1: the Mom Test questions are the tools for deflecting and anchoring.
• Ch 3: a lukewarm answer is good learning; don’t pitch harder to change it.
• Ch 5: a product meeting that ends in a compliment has failed.
• Ch 8: the note symbols for emotions and feature requests help you remember to dig.