Suggest edit — Evaluating Rare Interventions

Title

Name

Note

---
title: "Evaluating Rare Interventions"
visibility: public
---

# Evaluating Rare Interventions

Category: [[probability|Probability]]

[Read the original document](https://docs.google.com/document/d/19RiyCUPKIS8Ynzfse0NoJBiSIHHYmWWmIeMWYhbcE9s/edit?usp=sharing&sa=D&ust=1596495076413000&usg=AOvVaw3fXFUyKoY-b1QYCg17wHzG)

---

Say some fantastic intervention comes along. 
Having clear long terms goals is an easy example.

A ton of people will survey the people they deem to have been successful and ask them, so what was your pathway?

Let’s take the simple heuristic of looking at the behavior of successful people and mimicking what they’ve done.

Say that by default, 95% of people don’t go through life with clear long term goals laid out in front of them.

5% do.

Say that this is one of the most effective interventions conceivable. If you set clear long term goals, you have a 100% chance of success. If you don’t, you have a 10% chance of success.

You will see that only a third of the final population has set clear long term goals.

This may convince you that you’re more likely to succeed than not by not setting clear long term goals, even though it pushes success probability from 10% to 100%!

This horrifying pattern repeats itself across almost every intervention, most of which are rare. When you look at the final succeeding population, averaging over their behavior is a truly awful heuristic.

Ask what happens by default, first and foremost. And discount it. Discount it deeply. Seeing it spread throughout the final, successful population says more about what the predominant behavior was than it says about whether that behavior is genuinely helpful.

I run into counterarguments to obviously valuable interventions (things like planning, being systematic, creating habits, wake time / bedtimes, social accountability, time and focus tracking) and they’re often backed by the sense that few successful people are doing them or advocate for them.

This is our emotional backing for almost all of our decisions - association.

All we tend to feel is whether or not we’ve seen the intervention and success co-occur.

But the step forward in thinking style is from associative mode to causal mode.
In causal mode, so many interventions will occur to you that (often) you’ve never heard anybody try before but which would obviously work if they were done.

This essay is being written during a 16h back-to-back-to-back writing session.
Obvious intervention.
I’ve heard of nobody who’s tried it.
But this wasn’t driven by associative thinking. It was driven by causal thinking.

The evaluation of interventions needs to be counterfactual - all you care about is what would have happened otherwise. Somebody much less successful than you may have an intervention which took them from clearly incapable of performing to performing, which will absolutely work for you if you can release yourself from associative mode. You need the interventions that explain the differences between who they are and who they would have been, rather than their overall average level.

Free your causal mind to power a rise never seen before.

---

*Source: [Original Google Doc](https://docs.google.com/document/d/19RiyCUPKIS8Ynzfse0NoJBiSIHHYmWWmIeMWYhbcE9s/edit?usp=sharing&sa=D&ust=1596495076413000&usg=AOvVaw3fXFUyKoY-b1QYCg17wHzG)*