Why “do you like my idea?” is useless

CatChow, a small online shop for cat food, treats and litter, wants to launch a food subscription: a monthly delivery, a bit cheaper than buying in the shop. Before anyone builds it, the team needs to know whether customers really want it. The obvious plan is to call twenty customers, describe the subscription and ask whether they like it. Megan, the head of sales, has already tried a version of this at a pet expo: 38 of the 40 visitors at the booth said they'd subscribe.

Ask what that yes cost them. Nothing. They had free samples in hand, a smiling rep was asking, and yes is the quickest, friendliest way to end the chat. Such a yes tells you about people's manners, not about what they'll do with their money next month.

Asking more people doesn't help either. If the question invites a polite answer, 400 people give you 400 polite answers. The problem is the question, not the sample. What counts as evidence is behaviour that cost something: time, effort or money already spent on the problem.

It's tempting to think that a loyal customer like Olivia, who has bought from CatChow for two years, has no reason to mislead you. In fact she has a very good one: she likes you. You arrive excited about your idea, and saying “meh” feels rude, so the pleasant answer comes out instead of the true one.

The natural fix seems to be talking to strangers, people who don't care about your feelings. But strangers are polite too: Megan's expo visitors had never met her, and nearly all of them said yes. The problem isn't who you ask, it's what you ask. Nobody can honestly answer “Is my idea good?” Only the market can, when people pay.

So change the question, not the people. Ask about the person's own life and what has already happened: when they last had the problem, what they did, what it cost. There's no polite version of a fact. The next lesson turns this into three rules for questions that get honest answers, rules that work even with someone who loves you.

When customers mislead you, they're almost never lying on purpose, and seeing them as liars leads to the wrong fix.

Two things happen at once. First, the conversation is social: you're friendly and keen, and agreeing is the easiest way to keep it pleasant. Second, you've asked for a prediction: would they buy, use or pay in a future that doesn't exist yet? Nobody knows that, so they guess, and with someone friendly listening, they round up in your favour.

The striking part is that most people believe their own “I'd buy it” while they say it. Olivia isn't trying to end the call quickly; she genuinely pictures herself unpacking a monthly box. What's uncertain is the future, not her honesty.

That's why trying to catch customers out, pushing harder or asking “Are you sure?” doesn't help: a sincere guess stays a guess. What helps is asking about things that already happened, where there's nothing to guess and nothing to round up.

A false positive is evidence that says “go” when the real answer is “no”. It rarely appears in one step; it grows in a chain:

1. Someone pitches the idea and asks for an opinion.
2. The listener gives the kind answer: “I'd definitely sign up.”
3. The answer is logged as a “yes” in a spreadsheet.
4. Nineteen yeses out of twenty feel like proof, so the team orders packaging, signs a courier contract and plans a March launch.
5. At launch only a handful of customers subscribe, and three months of work are gone.

Each step looks reasonable on its own. That's what makes the chain dangerous: by step four nobody remembers that the “evidence” was plain politeness. The truth arrives only when people are asked to pay, which is the most expensive moment to learn it.

The cheapest place to break the chain is the first link. Change the question, and steps two to five never happen.

“Some data is better than no data” sounds sensible, and that's exactly the trap. Compare three situations:

• No data. The team knows it's guessing, so it stays careful: small tests, small bets, nothing it can't undo.
• False positive. A signal says “go” when the real answer is “no”. The team thinks it knows, so it commits budget, hires and signs contracts, and learns the truth only at launch. Its confidence went up; its chances didn't.
• False negative. The mirror image: the signal says “no” and the idea works anyway. In interviews this is the rarer mistake, because people are polite rather than harsh.

So a false positive isn't just weak evidence. It's misleading evidence that makes a team confident and wrong. Polite yeses to “Would you buy this?” are its classic source, which is why a badly asked question can cost more than no question at all.

While you're still on the call, sort every answer into two piles.

• Opinions: what someone thinks, likes or says they would do. “I love the idea” is an opinion. “I'd order every month” is a guess about the future, so it goes in the same pile. So does a general claim about yourself, such as “I usually plan my pet shopping carefully”: it describes who the person wants to be, and you can't check it.
• Facts: what actually happened. What the person did, bought, paid, forgot or complained about, ideally with a date, a place or a sum: “On Saturday I bought two 2 kg bags at the shop by my office.”

Only the second pile counts as evidence. Some words give an answer away: “would”, “definitely” and “always” tend to mark an opinion; “last month”, “on Saturday” and a sum paid tend to mark a fact. Notes full of enthusiasm and empty of events are still empty notes.

The most useful answer in a customer call often doesn't mention your product at all. Take “Last month we ran out on a Sunday, and I drove to a 24-hour pet shop.” Nothing in it is about CatChow's subscription, and that's exactly why you can trust it: nobody can flatter an idea they haven't heard.

Look how much one such fact tells you:

• the problem is real: it actually happened;
• it's recent: last month, not years ago;
• the person already pays for it, in time, fuel and a Sunday trip.

Compare “I'd order every month, for sure.” It's about your idea, so politeness can shape it, and it's about the future, so it costs nothing to say. A quick test for any answer: could the person have said it without knowing anything about your idea? If yes, it's probably evidence. If it only makes sense as a reaction to your pitch, it's probably an opinion.

Here's a real first call. Sofia pitched: “We're launching a subscription: food delivered every month, 10% cheaper. Do you like the idea?” Olivia replied: “Oh, that sounds lovely! Whiskers would be thrilled. I'd definitely sign up. Will there be treats in the box too?” Sofia wrote down “strong yes, wants treats”.

Count the facts: zero. Label each line as you hear it:

• Compliment: praise for the idea (“Sounds lovely!”). Pleasant but empty. “Whiskers would be thrilled” is the same kind of guess, made on the cat's behalf.
• Future promise: what someone says they will do (“I'd definitely sign up”). Easy to give, rarely kept.
• Feature request: an idea for the product (“Will there be treats too?”). Worth noting, but mostly a sign of a good conversation partner, not a reason to build.
• Fact from the past: something that really happened (“We ran out on a Sunday night”). The only one that counts as evidence.

The first three together are bad data: they feel like evidence, but they're reactions to your idea. Later lessons show what to do with each; for now, learn to spot them.

So what should Sofia have asked Olivia? Something about her life, not about CatChow's idea: “When did you last buy food for Whiskers? How did that go?” There's no polite lie for that: either there was a hassle or there wasn't. It works because the answer is a memory, not a judgement. There's nothing of yours in the question to praise, so she simply tells you what happened.

To rewrite any planned question, move it from the imagined future to a real, recent event, and leave the product out. Watch for three traps:

• Hypothetical: “Would you sign up if…?” asks for a guess.
• Pricing a future product: “How much would you pay?” gets a made-up number.
• Leading: “That must be annoying, right?” hands over the answer you want.

And if the story shows no hassle at all, that's not a failed call. You've learned in five minutes what a launch would teach you in five months: maybe buying food just isn't a problem for her. That's a saved budget.

Sooner or later you'll have to tell a colleague that their “validated” idea isn't validated. Megan has already told the CEO that 38 of 40 expo visitors want the subscription. Your goal is to help her, not to prove her wrong.

Keep it short and skip the lecture. Separate the people from the question: she's right that people were friendly, and wrong only about what friendliness proves. Anyone asking that question would have collected 38 yeses. Then offer a cheap, concrete next step, such as calling ten of those visitors and asking how they bought food last time.

The next step isn't cancelling the launch; it's checking properly. Start with the notes you already have, and read them as if a stranger wrote them. A page of “loves it” adds up to nothing, but one customer who complained at length about slow deliveries has given you a real event. Complaints about things that actually happened are gold. A tally like “3 out of 3 positive” is just the expo mistake in miniature.