Chapter 6: Why You Can’t Think Your Way Out
This chapter is a first draft written entirely by Claude from Aditya's notes and key points, and he has not yet rewritten it in his own voice. If you only want his prose, skip to the section marked "Original material."
Ultimately evil is done not so much by evil people, but by good people who do not know themselves and who do not probe deeply. —Reinhold Niebuhr
The road to hell is paved with good intentions. —Proverb
You noticed, at the very start, which part of you had to argue. This chapter is about what happens once that part is in charge.
I’m not against thinking. Thinking is how the part of you that cares gets anything done; it’s the hands. What thinking can’t do is sense what’s good, because that was never its job, and it can’t tell you whether it’s been recruited against the part that can. A proof can show that a plan follows from its premises. It can’t show that the premises weren’t chosen by a part of you that had already turned away. So when you ask “is this good?” and your mind hands you a proof, you haven’t learned whether it’s good. You’ve learned what the proof was built to show.
Here is the deeper thing, and I’d rather you felt it than agreed with it. Go back to the bus seat and the test. Under the same coarse description, making an exception of yourself, one was fine and one was not. A rule could try to tell them apart by adding conditions: need, consent, foreseeable harm. But each condition is one more thing the arguing part can satisfy while the deep part is away, and none of them is what actually told the two apart. What told them apart was whether the other person was in view. Now the stranger and the man who treated you with dignity: nearly the same words, opposite effects, and nothing in a transcript that would show which was which. In both cases the difference that decided it was the one the record couldn’t settle.
That’s the sense in which goodness can’t be formalized. Not because we haven’t found the right rules yet, but because whatever can be checked without looking isn’t where it lives. You can write “keep the other person in view” on the wall, and I’d recommend it. You can’t obey it. You can only do it, and no rule can tell you whether you did. Use rules as guidance; I do. Many of them came from the deep part, and they restrain real harm. But the moment you believe a rule has captured goodness, you’ve handed the arguing part something to satisfy to the letter while the deep part is turned away, and you’ve given yourself permission to stop looking, which is the one thing goodness can’t survive. Trusting the numbers to tell us whether we’re doing good is the same permission, moved outside the skull. And any system that learns what we value only from what we observably do inherits the ambiguity between an act of care and an act of self-deception that take the same outward form. Encoding our behavior is not yet understanding why anything matters.
There are two things we call righteous. One is the part of you that wants nothing more than to be of service to life. You can feel it when you’re connected to it. The other is an idea of serving life, with a proof attached: the best way to serve life is to destroy these people, and here is why. The feeling changes beneath your notice. You can only choose the counterfeit by forgetting what the real one feels like, the way you can only choose junk food when you’ve forgotten real food, and the further you follow it, the less of the real one you have left to compare it with.
So here is a test for any grand plan for doing good. Does it need you to feel contempt for what’s in front of you now, for the sake of some good that comes later? Then it isn’t coming from the part of you that cares, and it won’t do what its proof says it will. It’s why the inquiry after the disaster so often finds a memo: somebody raised it, and there was a reason on file for why it didn’t count. The reason wasn’t a lie. It convinced people. That’s the point. And it’s why the one clue I’d give anyone is this: when you’re acting from contempt, notice how sure you are that it’s good. The real thing doesn’t feel like that. It doesn’t need to be sure.
Three cases, and then a fourth.
Original material, to recover
Sally cares deeply about animals, and chooses to go vegan. Frustrated that her friends won’t see the harm they’re funding, she gets sharper with them—which feels like honesty, not cruelty. Eventually she alienates them, and comes to resent humanity in general.
Sam Bankman-Fried cares about effective altruism. He builds a trading empire to generate billions he plans to give away, living cheaply while he does it. When customer funds are needed to keep the thing alive, moving them feels responsible—that money will do more good in his hands than anywhere else. He defrauds those customers of billions and goes to prison.
Adolf Hitler sees his country humiliated after a war and sunk into poverty. He sets out to make his people strong and to protect them from what he’s certain will otherwise destroy them. To him, none of it is aggression. It’s defense. I think you know what happens next.
Let me be clear: what these specific cases have in common is the shape of the mistake, not the scope of the harm.
In each, there’s a kernel of truth in the person’s care. That kernel licensed them to overstep moral bounds, in increasingly severe degrees. None of it felt wrong, let alone evil—because it was for a “good cause.” And yet look at the result.
In every case, the person set their cause back. Sally wanted people to see what she sees; now they treat her cause with ridicule. Sam wanted to prove that enormous good could be funded at scale; now Effective Altruism is treated with more suspicion. Hitler wanted to save his people; he destroyed them instead. Same with the husband: he wanted less selfishness and got more of it.
But do any of these people blame themselves? Or does the ridicule and contempt they receive just make them more convinced that they’re right?
Which brings us to the fourth and most instructive example: us.