What people say they will do, and what experts expect to work, are both unreliable guides. Finding out how people actually decide, then testing a strategy’s assumptions against it, is the part with evidence behind it.
- 01
RobustWhat people say they will do is only a partial guide to what they do. Intentions turn into action about half the time.
Sheeran & Webb (2016), a review drawing on a meta-analysis of 422 studies
- 02
RobustPeople overstate what they would pay. Stated willingness to pay runs 21% above real willingness to pay on average.
Schmidt & Bijmolt (2020), Journal of the Academy of Marketing Science; meta-analysis of 77 studies
- 03
RobustSpecialists cannot reliably predict what will work. Of 54 interventions designed by 30 scientists, 45% raised gym visits during the programme, 8% had a lasting effect, and forecasters failed to pick the winners.
Milkman et al. (2021), Nature; 61,293 gym members
- 04
ModerateStart-ups trained to treat their ideas as hypotheses and test them with customers were more likely to abandon or change weak ideas, and earned more revenue, than those that were not.
Camuffo, Cordova, Gambardella & Spina (2020), Management Science; randomised trial of 116 start-ups. With three further trials, 759 firms in total (Camuffo et al., 2024, Strategic Management Journal). The evidence is from start-ups, not established companies
- 05
WeakThe UK government requires this of its own services: teams must understand what users need, and its guidance calls for user research in every iteration.
GOV.UK Service Standard, point 1 (2019, updated 2022), and the Service Manual’s user research guidance. A required practice, not a measured outcome
Robust: meta-analysis, replication or very large samples. Moderate: one strong study, or several with limits. Weak: small studies, or run by an interested party.
/ Where the evidence stops
Research does not guarantee the outcome. Published behavioural effects shrink in practice: when two of the largest US nudge units reported every trial they ran, the average effect was 1.4 percentage points, against 8.7 in the academic literature (DellaVigna & Linos, 2022). Research can also study the wrong thing: New Coke was backed by nearly 200,000 taste tests. And we found no controlled study that measures the return on discovery research in established companies. So we promise fewer wrong bets, not a return multiple.