The Martial Art of Rationality

Author: Eliezer Yudkowsky. Link to original: (English).
Tags: lesswrong, Rationality Submitted by Remlin 11.03.2013. Public material.

Translations of this material:

into Russian: Боевое искусство рациональности. Translated in draft, editing and proof-reading required.
Submitted for translation by Remlin 11.03.2013


I often use the metaphor that rationality is the martial art of mind. You don't need huge, bulging muscles to learn martial arts - there's a tendency toward more athletic people being more likely to learn martial arts, but that may be a matter of enjoyment as much as anything else. Some human beings are faster or stronger than others; but a martial art does not train the variance between humans. A martial art just trains your muscles - if you have the human-universal complex machinery of a hand, with tendons and muscles in the appropriate places, then you can learn to make a fist.

How can you train a variance? What does it mean to train +2 standard deviations of muscle? It's equally unclear what it would mean to train an IQ of 132.

But if you have a brain, with cortical and subcortical areas in the appropriate places, you might be able to learn to use it properly. If you're a fast learner, you might learn faster - but the art of rationality isn't about that; it's about training brain machinery we all have in common.

Alas, our minds respond less readily to our will than our hands. Muscles are evolutionarily ancient subjects of neural control, while cognitive reflectivity is a comparatively more recent innovation. We shouldn't be surprised that muscles are easier to use than brains. But it is not wise to neglect the latter training because it is more difficult. It is not by bigger muscles that the human species rose to prominence upon Earth.

If you live in an urban area, you probably don't need to walk very far to find a martial arts dojo. Why aren't there dojos that teach rationality? One reason, perhaps, is that it's harder to verify skill. To rise a level in Tae Kwon Do, you might need to break a board of a certain width. If you succeed, all the onlookers can see and applaud. If you fail, your teacher can watch how you shape a fist, and check if you shape it correctly. If not, the teacher holds out a hand and makes a fist correctly, so that you can observe how to do so. Within martial arts schools, techniques of muscle have been refined and elaborated over generations. Techniques of rationality are harder to pass on, even to the most willing student. It is also harder to give impressive public exhibitions of rationality. This may partially explain why there are no rationality dojos as yet.

Very recently - in just the last few decades - the human species has acquired a great deal of new knowledge about human rationality. The most salient example would be the heuristics and biases program in experimental psychology. There is also the Bayesian systematization of probability theory and statistics; evolutionary psychology; social psychology. Experimental investigations of empirical human psychology; and theoretical probability theory to interpret what our experiments tell us; and evolutionary theory to explain the conclusions. These fields give us new focusing lenses through which to view the landscape of our own minds. With their aid, we may be able to see more clearly the muscles of our brains, the fingers of thought as they move. We have a shared vocabulary in which to describe problems and solutions. Humanity may finally be ready to synthesize the martial art of mind: to refine, share, systematize, and pass on techniques of personal rationality.

Such understanding as I have of rationality, I acquired in the course of wrestling with the challenge of Artificial General Intelligence (an endeavor which, to actually succeed, would require sufficient mastery of rationality to build a complete working rationalist out of toothpicks and rubber bands). In most ways the AI problem is enormously more demanding than the personal art of rationality, but in some ways it is actually easier. In the martial art of mind, we need to acquire the realtime procedural skill of pulling the right levers at the right time on a large, pre-existing thinking machine whose innards are not end-user-modifiable. Some of the machinery is optimized for evolutionary selection pressures that run directly counter to our declared goals in using it. Deliberately we decide that we want to seek only the truth; but our brains have hardwired support for rationalizing falsehoods. We can try to compensate for what we choose to regard as flaws of the machinery; but we can't actually rewire the neural circuitry. Nor may a martial artist plate titanium over his bones - not today, at any rate.

Trying to synthesize a personal art of rationality, using the science of rationality, may prove awkward: One imagines trying to invent a martial art using an abstract theory of physics, game theory, and human anatomy. But humans are not reflectively blind; we do have a native instinct for introspection. The inner eye is not sightless; but it sees blurrily, with systematic distortions. We need, then, to apply the science to our intuitions, to use the abstract knowledge to correct our mental movements and augment our metacognitive skills. We are not writing a computer program to make a string puppet execute martial arts forms; it is our own mental limbs that we must move. Therefore we must connect theory to practice. We must come to see what the science means, for ourselves, for our daily inner life.

And we must, above all, figure out how to communicate the skill; which may not be a matter for declarative statements alone. Martial artists spar with each other, execute standard forms, and are watched throughout by their teachers. Calculus students do homework, and check their answers. Olympic runners continually try to beat their best previous time, as measured by a stopwatch.

How to communicate procedural skills of rationality, or measure them, is probably the single largest open issue that stands between humanity and rationality dojos - at least it's the part of the problem that most baffles me. Meanwhile I lecture. So does anyone out there have ideas?