Free Will, Willpower, and Randomness

Highlights from some recent conversations

There’s been a recent spate of discussions related to desire, utility, and free will. For example, take the apxhard article The Way You Think About Value is Making You Miserable.

In it, Mark advises his younger self to pay more attention to the present-moment value of his utility function—a measure of how good his life is—rather than its time derivative. In other words: be happy with what you have, instead of thinking about ways to get more and worrying about loss.

Your brain optimizes for the only way you use your utility function, which is to compute its gradient under various counterfactual scenarios. Instead of seeing the present as ‘one million utility units’ and a future state where you have washed a dish as ‘one million and one utility units’, it is computationally cheaper to simply compute the delta between the dish being clean, and it not, as being one.

This idea has been explored before, but it’s an interesting and important one. Mark does a great job articulating the theory and connecting it back to personal experience.

Dave at Pelorus has a rebuttal for Mark, which makes two good points:

• Mark’s prescriptivist advice to “get in the habit of computing your utility function on the world as a whole, in the present moment” is unrealistic—people simply value things getting better, and don’t like when they get worse.

• The suggestion that the reader pick a better utility function is itself a counterfactual scenario. Reading Mark’s article, we find ourselves wishing we were different, instead of appreciating our utility function as it is. The irony!

These ideas dance around a common logical trap related to free will: we have the power to choose, but do we choose what we choose? Can we willfully change our utility function? Schopenhauer said:

Man can do what he wills, but he cannot will what he wills.

The paradox was addressed in a recent essay from Paul Graham, What You (Want to)* Want. Paul points out that you can sometimes “will what you will”—e.g. some people can will themselves into liking broccoli. But as the title points out, there’s still an infinite regress: why do you want to want to like broccoli?

These questions are hard, maybe philosophically so. But I’d like to add a little more terminology to help us navigate.

Outline

• Does Free Will Exist?

• Willpower and Free Will

  • Some Definitions

  • Limitations

• Interpersonal Willpower

  • Shared Willpower

  • Intergenerational Willpower

• Injecting Randomness

• The Physics of Free Will

• At Home in the Mystery

Does Free Will Exist?

There’s an argument against the existence of free will that goes something like this:

• Everything is made of physics.

• Physical systems operate according to fixed laws, which might be deterministic or random.

  • If the laws are deterministic, then everything is decided ahead of time, and we can’t make any choices.

  • If the laws are random, then our behavior is just the result of random outcomes, rather than the result of “choice”.

• Therefore, free will is logically impossible. The concept itself is nonsensical.

I’m surprised¹ at how seriously some people take this argument. As The Contractualist points out, we have a clear sensation of free will. Denying its existence is like denying the existence of the color yellow. “It’s an illusion!” you say. “It’s just electromagnetic waves vibrating at 520 THz!” And yet I plainly see yellow.

The concept of free will doesn’t even make much sense without the sensation. What would it mean for a non-conscious being to have free will? Or for a conscious being to have free will, but feel completely unfree?

Years ago, in the throes of a strange LSD trip, I did in fact lose my sensation of free will. I felt like a passenger, watching my body move through pre-orchestrated behaviors. I could even foresee what I’d do next, but had no belief in my ability to modify that outcome. I felt trapped, like I was moving along the fixed track of a rollercoaster.

Was this a brief window into the truth of things? Was I just hallucinating? These are the wrong questions.

What I really want to know is: what was happening at a physiological or neurological level that so radically changed my sense of self? What physical reality undergirds my subjective experience of having will?

What is free will’s 520 THz equivalent?

Willpower and Free Will

Some Definitions

To approach this question, I like to divide the concept of Will into two opposing categories:

• Willpower—the ability to constrain your future behavior

• Free Will—the ability to act spontaneously, without deference to the past

(I’ll use capital letters to distinguish this category of Free Will from the more generic problem of free will.)

Both categories have an element in common: time. Willpower is expressed when there is a causal link between my former self and my present self; Free Will is expressed when my actions are temporally isolated.

To put it another way: Willpower is deterministic, while Free Will is random.

Expressing Willpower moves the needle towards determinism: I decide today that I’ll never drink again, or that I’ll work out every day, et voila—if I’m a person with strong Willpower, so it is.

In other areas I might be a “free spirit”—some days I read voraciously, other days I sit and watch TV. When I walk my dog, the wind and a passing mood have more influence over my route than pre-formed habits. Chaos takes over and my behavior becomes unpredictable.

This description of Will provides the perfect resolution to my LSD trip. It’s important to note that all my behavior during the experience occurred in loops—I would literally walk in circles, or perform the same minor behaviors (looking at my phone, then wiggling my leg, then stretching my arms) in a sort of ritualistic dance. My actions were perfectly rooted in the near-past, 100% deterministic. I’d stopped injecting any randomness into my behavior, and as a result got stuck looping around a strange local minimum.

Obviously these definitions don’t solve the mystery of free will. But at least we now have a concept of Will that’s quantifiable and measurable: to the degree a behavior is constrained by the past, it’s a product of Willpower. Free Will is the inverse: it’s a measure of the entropy² in our behavior.

Any action can be given a score along this Free Will ↔ Willpower spectrum using these definitions.

Limitations

Note that we’re stretching the common usage of “free will” and “willpower” a bit.

First, these definitions can be applied to any arbitrary physical system, not just people and animals³.

Defining Free Will as randomness also feels weird: when I choose coffee over tea, I have the sense that I’m actually weighing my options and picking one, not just flipping a coin. But from the outside, the final decision does appear to be random. I’m more likely to pick the “better” option, but sometimes I actually go with a down-weighted option on a whim. “Coffee sounds better…fuck it, I’m gonna make some tea.”⁴

And our definition of Willpower is probably too generic—there are plenty of deterministic behaviors that are more rote than driven. If I habitually have a beer everyday after work, is that really what we’d call “willpower”? Was my loopy LSD-fueled behavior “willpower”?

For the moment, let’s just take the above concepts of Free Will and Willpower as definitional, and see where they take us. Our first stop will be a strange detour into Interpersonal Willpower.

Interpersonal Willpower

Shared Willpower

Several years ago, I spent a month participating in a Gurdjieffian encounter group. They had a fantastic practice called “shared will”: everyone would set a quantifiable five-week goal for themselves—say, walking outside for an hour each day, cooking three meals a week, or reading two novels. We’d then report on our progress during weekly meetings.

This worked shockingly well: I cooked all the meals I said I would. Occasionally someone would come up a bit short, but I’d estimate we were 90% successful. And it wasn’t just a matter of accountability. There was an important sense of being all in it together—if I failed, then everyone else would be a bit more likely to fail the following week. I felt a moral responsibility toward the group.

There’s a very real way for us to anchor our Will in others. This is the reason we take wedding vows in front of a witness, often a large crowd of our friends and family—their involvement creates additional consequences for breaking up or cheating. Our vows sing determinism into the universe, and the audience plays a large role in ensuring we hold to the path we’ve chosen.

So I’d like to amend my definition of Willpower above: Willpower is a measure of how deeply constrained our behavior is in both time and space. Other people can prevent us from acting spontaneously.

Again, we’re stretching the definition of Willpower, which usually only applies to one person. But the interpersonal definition is a powerful one.

Intergenerational Willpower

An extension of Interpersonal Willpower is Intergenerational Willpower—many of our habits are carried over from values adopted by distant ancestors.

Most religions operate this way: converts promise to indoctrinate their children, who make the same promise in turn. Millennia ago, Jewish people promised to cut off the foreskin of their sons’ penises, and the practice is still nearly universal today. Even atheist Jews I know balk at the idea of breaking a 4000 year old tradition. They’d literally rather let a stranger take a knife to their newborn’s genitals.

Many of our actions are so deeply rooted in the past and in our peers that it’s hard to say we have much Free Will at all. Everything from drinking culture to gender roles can be viewed through this lens—we do these things because it’s how things are done. Separated from tradition, I doubt contemporary Jews would instate circumcision de novo. But they are compelled by the determination of their ancestors—by intergenerational Willpower.

Injecting Randomness

If so much of our behavior falls on the Willpower end of the spectrum, how can we reclaim our Free Will?

Everyone has an inner rebel, which will try and break out of socially- or self-imposed habits. But we stifle it to avoid being ostracized, or because we fear change, or out of plain inertia.

I often find myself performing the same behaviors (mostly, but not entirely, drug- and Internet-related) no matter how dissatisfying or destructive they might be. I can almost see myself from the outside, knowing I should change my behavior, but strangely unable to do so.

There’s a simple way to break out of these loops: inject randomness.

Here’s Nobel-winning physicist Roger Penrose talking about using a random number generator to reclaim his free will:

There’s this game [rock, paper, scissors]…and [my younger brother] used to wallop me at this game consistently. And I thought I was using free will to decide which I’m going to do. How does he know? Is he predicting the future or something?

So what I did is I went to a table of logarithms…and I worked out [a way to generate random moves]. So I’d play this game reading off the digits. And then I broke even.

It was as though my free will was not really much of a free will. I was just doing which of them came easily to me in the moment.

It’s ironic that the cure was for Penrose to robotically follow the instructions generated by his table of logarithms. But it allowed him to bypass the unconscious habits that made his actions predictable and caused him to lose.

This same strategy can be put to use in more practical ways.

For example: if you’re struggling to quit smoking cigarettes, try flipping a coin each time you feel like a smoke. If it lands heads, go ahead and light up. If it lands tails, flip the coin again hold off for a little while, even if it’s only 10 minutes. You’ll start to develop the muscle of resisting the habit, little by little. Over time you can tilt the odds away from smoking⁵.

More generally, seeking out novel experiences can help inject broad-based randomness into your life. Changing jobs, moving, or even just traveling to a new city can help kick you out of a local minimum and find a new normal.

More sophisticated methods for injecting randomness, like casting Tarot cards or I Ching readings, can be helpful as well. They combine sheer randomness with unconscious projection in a way that can be deeply helpful for breaking out of habitual thinking.⁶

One caveat: while these methods are great for escaping Willpower traps—where you feel compelled to act a certain way—they can have the perverse effect of robbing your Will altogether. Relying on randomness or divination to make every decision in your life can be addictive: it absolves you of responsibility for your actions and their consequences.

Appealing to coins or cards effectively outsources your Will to the universe; it’s important to reclaim it once you’ve broken out of the Willpower trap.

The Physics of Free Will

There’s a long history of controversial studies purporting to prove the non-existence of free will. They typically find that they can predict a subject’s behavior—say, pressing one of two buttons—based on physiological data, before the subject is aware of their choice.

Our terminology sidesteps the issue here. We might even see the results as evidence of free will: the subject’s behavior appears to remain unpredictable until milliseconds before the observed behavior. That’s a pretty weak spatiotemporal constraint.

We keep dancing around a paradox: as each moment goes by, many possible futures collapse down into a single reality. I can press either button, but at some point the muscles in my left forearm start twitching, and a left button press becomes inevitable.

This process is a structural feature of reality. There are outcomes that cannot, even in principle, be known ahead of time⁷. An electron, when measured, spontaneously decides whether to appear as spin-up or spin-down, and its wavefunction collapses. Prior to measurement, the result is fundamentally unknowable. Using the terminology above (and holding our nose a bit at the implication of panpsychism), we’d say the electron has a great deal of Free Will.

Here’s Penrose again, talking about the relationship between the collapse of the wave function and free will:

I’ve always been of two minds about [free will]. You see with physics, you might think well, it’s all determined—what you do now is determined by what happened yesterday, if I give you all the details of how the particles were moving.

But this isn’t quite true, because you have this curious thing in quantum mechanics: you have the collapse of the wave function. The Schrödinger equation chugs along and says what’s going to happen, and suddenly it doesn’t. Suddenly it jumps to this or that.

Usually people talk about this in terms of measurements. My view is that it’s happening all the time. When you have just lots of particles, and they get up to a certain mass level, it does one thing or another. Stuart Hameroff and I have built up this idea about consciousness, and we consider that consciousness comes about when these collapsed wave function things happen.

Now the question is does it collapse in a way which is random? That’s the way physics works, we consider it to be quite random. Or is it maybe entangled with all sorts of other things, and the decision…has some kind of content to it. Some mental content which is not altogether random. And that would be where the free will plays its role. Maybe that’s controlled by some deterministic laws which go deeper than the laws we know now. Or is there some actual freedom in what happens in the operation of the universe?

Penrose doesn’t quite agree with me here—in his view, the decision point is physically determined (by an accumulation of mass), but there is room for the ostensibly random outcome to be influenced by “mental content”.

I would argue that the outcome is indeed random, since in experiments we can know the exact probability of each outcome. Instead, I’d suggest that Will is what decides when the collapse happens—it initiates the transition from randomness to determinism, or prevents it and allows the probabilities to evolve further.

Regardless, both of us are sweeping some magic under the rug. We both refer to a Self which is independent of physical reality, but still influences it. I recognize this is nonsensical. But I’ve never seen a satisfactory resolution of the paradox.

At Home in the Mystery

It’s both physically likely and empirically obvious that our future is unwritten. Quantum mechanics guarantees that some amount of randomness is built into the universe. Or to put it another way, using the (somewhat weak) terminology above: we have Free Will.

There are also methods for expanding and collapsing the uncertainty of our future. We can make promises to clamp down on the space of possibilities, and we can inject randomness by traveling or flipping a coin.

But even if the ideas above play out—if it turns out that the sensation of free will is deeply linked with predictability, and that the wavefunction plays a role—there’s still a mystery. Where can freedom enter our equations? How does mind impact matter? What even is consciousness?

Dodging these questions by explaining them away—saying that free will and consciousness are just an illusion—would be intellectually dishonest. We have these sensations, and we should be asking how exactly they correspond to physical reality.

I can feel my own understanding of the problem evolving and becoming more sophisticated over time. It’s reasonable to think the same is happening for humanity as a whole, no matter how slow the progress seems.

These questions may ultimately turn out to be unanswerable—and that’s OK!—but it doesn’t make them any less important. I’m glad folks are paying attention.

1. To me, this argument is a reductio ad absurdum which undercuts the premise “Everything is made of physics.” We’ve also discussed Erwin Schrödinger’s compatibilist solution to the paradox.

2. Entropy in the information sense, not the physical sense. And in truth, Shannon entropy only applies to discrete systems. Assuming time is continuous, we’d have to appeal to its continuous analogue.

3. As a tentative panpsychist, I’m kind of cool with this.

4. See also: split brains and post hoc rationalization

5. Instead of a coin, I like to use the minutes on a digital clock. This is (a) more subtle, and (b) lets me fine-tune the odds (even numbers for a 50% chance, multiples of 3 for a 33% chance, etc).

6. I plan to dig into the psychological mechanics of divination tools like Tarot and the I Ching in a future article.

7. Specifically, experiments have shown that quantum systems violate the Bell Inequality, which proves that measurement outcomes can’t be determined by a “local hidden variable”. The proof is quite beautiful. See this lecture from Leonard Susskind or The Road to Reality by Roger Penrose for deeper explanations, but beware there is a good deal of undergraduate-level math.