If I am going to answer what a Koan does I better answer first what a Koan is….
A Koan is a question that uses language or sometimes an utterance that is not a word and sometimes movement of the body to point at the nature of reality, i.e. emptiness of all appearances. In a Koan the roles of Questioner and Answer-er are reversed. Normally a person who does not know something, asks someone else who does know something for the answer to their question. In the case of a Koan it is a test from the master to the student to test their understanding of Buddhist enlightenment and their embodiment of it. So the one who is enlightened is asking a question of the one seeking enlightenment. But in that question there is also a pointing toward the nature of enlightenment as it is hidden in the current existential situation that the student and teacher are in at the moment. Many times the answer refers to previous Koans, to Sutras, to folk knowledge, or generally the Buddhist tradition.
To illustrate I am going to refer to a very rare movie called Satori. In this movie Satori is a monster that eats thoughts. Satori is following around a girl who is the only one who can see him. The movie is about how she deals with this monster and the fact that she cannot get rid of him, and stop him from following her around. So the main character will be talking to someone in a kitchen say, and Satori will be curled up on the cabinet, listening and up to his usual mischief. Now this is a folk image of what Buddhist enlightenment is like, and I think it gives us some insight into the nature of enlightenment in the Zen tradition which is that you get rid of all your thoughts.
Of course, this kind of Mind-Only idealism is in fact a departure from the middle way. Bodhidharma who brought Chan to China from India, supposedly, emphasized the Lankavatra Sutra which is an extremely idealist offshoot from the Yogacara and Tathagata schools of Buddhism. It interprets enlightenment as the stopping the thoughts of the mind. It took a while for Hui Neng to correct this doctrine, and thus we got a split between Northern (slow enlightenment) and Southern (fast enlightenment) schools of Chan Buddhism.
Koans were a practice that Chan added to Buddhism, and we can see that it replaces the Sutra as the center of attention in some schools of Zen. Renzai Zen concentrates on the Koan as the route to enlightenment via dialogue with the enlightened master. Soto Zen concentrates on meditation practices and the understanding of dialectics. However, we must not think that Zen Priests did not know the contents of the Sutras, but rather they assume that you know it, and need to go beyond it to experience enlightenment itself.
It was Renzai Buddhism that was the main branch imported into the USA by D.T. Suzuki and others, and so we think of Zen through that lens, but actually there is a variety of schools. Of those, to me the most interesting is Soto Zen which has more intellectual content, with sophisticated dialectical theory, and also that is the school of Dogen Kigen who was a genius. One particular chapter of the Sobogenzo is the one called ExistenceTime which is particularly significant.
Most Zen/Chan teachers assume the teachings of the Hua Yen school of Fa Tsang as a background for understanding Zen/Chan practice. This school and the Tien Tai school are attempting to find a middle ground between Taoism and Zen Buddhism. And the way that Hua Yen does that is to posit that emptiness is identical to the interpenetration of all things. That is to say they have formulated a positive characterization of the nature of the Form/Emptiness chiasm.
The concept of the Koan is that language can be a medium for transmission of an awareness of interpenetration, but it may be indicated by an interference or a movement, or anything that points to the inherent emptiness that makes interpenetration possible.
One way to think about it is via Quantum Mechanics, which itself is for us one huge KOAN. In QM there is entanglement and superposition. When we try to marry it with Relativity theory at the Planck scale we get contradictions, paradox and absurdity. But lets suppose for a moment that QM operates on a macro-scale as it does in Bose-Einstein condensates. Or we can appeal to Cooper pairs of electrons in Super-Conductivity that are entangled and moving through a lattice of the material that is cooled sufficiently for this phenomena to appear. These electrons interact with the vibratory phonons in the lattice in order to synchronize with the lattice and to avoid all resistance from imperfections in the lattice. These electrons probably use superposition to pick the path of least resistance. They use their entanglement to triangulate with the phonons in the lattice. Or anyway that is one theory.
Point is that at the quantum level there is an ultra-efficacy (hyper-efficient and hyper-effective) that allows Cooper Pairs to avoid all resistance at a certain temperature in certain materials. Now just for the heck of it lets say that a similar phenomena can operate on a macro level, sometimes that is called FLOW, when we get in a groove, and we do everything just right, and can handle what ever the environment can send at us. This is a basic theme in Adventure movies, they model Flow of the Hero’s as they navigate the obstacles in the imaginary world of the movie. It is like when you fly in your dreams. So such a case might even occur on an interpersonal level, say between enlightened master and student. The idea of Synchronicity and Psychoid phenomena in Jung points to this possibility.
So say you are with your teacher who is enlightened. Some situation comes along, and the teacher will use that situation to point at the nature of reality, usually by some pertinence statement that has multitudes of meanings, that one grasps all at once. If one is in the right state the transmission can occur where the state of consciousness of the teacher passes directly to the student as if it were some type of contagion. And in that moment the student realizes that the natural reality of his self has always been enlightened from the very beginning. The Koan is the means of this transmission, and these are collected in the Zen tradition and given to students as problems to solve, which they supposedly exhaust their reason on, and when they give up trying to understand with their mind they are then able to reach the goal of mindlessness that is seen my many as the goal of Zen.
However, in my view this is completely wrong. If we take Plato’s divided line as the model then it has two limits Paradox and the Supra-rational. Paradox (entanglement) is the extreme of DOXA which is appearance and opinion. Supra-rationality (superposition) is the extreme of RATIO which is our understanding. This NO-MIND doctrine throws you from one extreme to the other. The extreme of figuring things out rationally, to the extreme of pure appearance. Even though Buddhism is basically phenomenological, that does not mean that appearance in itself is enlightenment. This would not be the middle way. But since we fall into Paradox so naturally, the Koan is a sort of antidote that cancels the paradox with discontinuity and separation instead of mixture of incompatibles.
By confronting the student with a supra-rational indication, that counterbalances the confusion of the student which is basically a mixed state, when confronted with the paradoxical impossibly of enlightenment as a goal of the self. Because both India and China used Mass logics of Pervasion, these statements mean something essentially different from what we might think based on the set-based syllogistic logic we use in our reasoning. Blythe has collected the koan like statements in our tradition. But since he did not know the difference between paradox and supra-rationality, he included paradoxical statements in his collection, so there is some misunderstanding in what a Koan actually is in our interpretation of Zen/Chan tradition. We think they are paradoxes or absurdities when they are exactly the opposite for the most part. Of course, there are some Koans that are paradoxes, thrown in for good measure, but for the most part they are supra-rational statements.
So lets take the Flag Koan as an example. Is the flag moving in the wind or is it your mind? First we need to know that this points at the doctrine of the karmic function and the “substance” of consciousness which we see in the Awakening of Faith for example. So this is actually a very specific reference that is quite complex conceptually.
The flag is an appearance which flutters in the wind, and thus moves, and anything that moves generates contradiction from a logical point of view (nb. Zeno).
But more generally there is the Karmic Function and the “substance” consciousness (white light). The Karmic Function “moves the emptiness of consciousness interpreted as white light” producing phenomena in consciousness.
So something invisible (air as wind) is moving the substance of the flag (fabric). On the other hand the karmic function as the flux of consciousness is moving the empty substance of consciousness which at its root is a white light experience to produce phenomena in consciousness that correlates with the appearances to us of external objects in the world. Throw into that mix that Buddhists do not believe in the reality of anything outside of consciousness, and you get a deadly concoction, especially when asked what is moving, the external flag in the wind, or the internal karmic function that disturbs consciousness.
If you say it is the Flag outside being blown by the external wind you feel on your skin, then you are rejecting the inherent phenomenological position of Buddhism. But if you say that it is the mind moving, how can the mind move? If anything moves it generates contradictions, so if the mind was to move it would also generated contradictions if not paradoxes or absurdities. The question crosses the inward/outward boundary, and is a boundary violation either way.
But the answer is of course that the whole situation is nondual as apprehended by supra-rationality. So the flag is moving in the Wind, and the Karmic function is moving the white light of the basic “substance” of the mind simultaneously without interference. In the movement of the flag in the wind there is entanglement, and in the movement of the karmic function of the substance of consciousness there is entanglement, but between these entanglements there is supra-rational simultaneous non-interfering synchonistic and psychoid isomorphic mirroring. What is being pointed at is the mirror between inward and outward. Look into that mirror and you to will see your own nature. Inward exists as the outward, and outward exists as the inward. No more, no less.
The sound of one hand clapping, that is so famous is also pointing at the enantiomorphism of left and right, and how they become the same, if there is nondual experience. And so it goes, Koans point almost always directly at nondual experience of existence as emptiness, whose nature is ultimately interpenetration. When this is pointed out the mixture of confusion in our mindbody is clarified beyond all expectation.