Faith and Bayes' Theorem

Posted 2013-09-13 02:27 AM GMT

Occasionally on this blog I will write about my beliefs and the things I learn about the world based on those beliefs. To learn more about what I believe, please visit mormon.org.

Recently I learned about Bayes' Theorem, and I have found the implications to be overwhelming. This simple formula is used to:

  • Keep spam out of our email inboxes
  • Help us find our colleagues and friends
  • Help us find a movie to watch or a product to buy
  • Help self-driving cars know exactly where they are on the road

Bayes' Theorem has also helped me increase my faith. Before I explain that, though, here is a quick intro to how Bayes' Theorem works.

Bayes' Theorem

The concept is pretty easy to understand, though the formula is a bit confusing:

PX|Y= PY|XPX PY

This formula gives you the probability of X (the unknown) given Y (the observation). A few examples should help clear things up a bit. The example on Wikipedia is great. We'll work through another example here.

Let's say I'm going on a road trip from Las Vegas to Portland. The trip should take about 16 hours total, with 8 hours driving through Nevada. Let's say I fall asleep at the beginning of the trip, and wake up at a random time in the next 16 hours (we'll say the car is driving itself :). What is the probability that I am in Nevada? Given no additional information, it must be 0.5:

PNevada= 8 hours in Nevada 16 hours total = 0.5

P(California) and P(Oregon) are calculated the same way:

Probability
P(Nevada) 0.5
P(California) 0.23
P(Oregon) 0.27

Now, I sit up and look out the window and I see sagebrush. I have a little bit more information. Does my probability of being in Nevada change? Is it higher or lower?

First, we need the to know the probability of seeing sagebrush in the states we will pass through (I'm making this up based on a map from Wikipedia):

Probability of sagebrush given location
P(sagebrush|Nevada) 0.8
P(sagebrush|California) 0.4
P(sagebrush|Oregon) 0.5

There is a lot of sagebrush in Nevada.

Now our Bayes formula looks like this:

PNevada|sagebrush= Psagebrush|NevadaPNevada Psagebrush

We can fill in all the values on the right side of the equation, except P(sagebrush). We need to calculate the total probability of seeing sagebrush on the route, which is the sum of the probability of sagebrush in each state times the probability of being in that state:

Psagebrush = Psagebrush|NevadaPNevada + Psagebrush|CaliforniaPCalifornia + Psagebrush|OregonPOregon = 0.80.5+0.40.23+0.50.27=0.63

That means that sagebrush is visible during about 63% of our journey. Now, back to Bayes:

PNevada|sagebrush= 0.80.5 0.63 =0.635

When I woke up, the probability that I was in Nevada was just 0.5. But, when I sat up and saw sagebrush the probability increased to 0.635. A single observation, and my probability of being in Nevada increased by 27% ( (0.635 - 0.5)/0.5 = 0.27). If I also see a casino or an Elvis impersonator, the probability will keep going up. If I see a tree or rain, the probability will go down.

Faith

In the Book of Mormon, a prophet named Alma teaches about faith (Alma 32), and his explanation fits very well with Bayes' Theorem. Thinking of Bayes' Theorem in terms of faith helped me understand Bayes' Theorem. Then, thinking of faith in terms of Bayes' Theorem gave me a new perspective on faith.

From our previous example, P(Nevada) is called a prior belief. It is our belief of the probability prior to any observations. P(Nevada|sagebrush) is a posterior belief. There are two important caveats to Bayes' Theorem:

  1. The prior belief must be greater than 0 and less than 1, else the posterior belief will always be the same as the prior belief.
  2. The unknown (Nevada) and the observation (sagebrush) must be in some way connected. If not, the observation will have no effect on the posterior belief.

I first realized this while reading Nate Silver's book The Signal and the Noise, where he explains the concept using examples about politics and religion. These same caveats are critical to faith:

  1. Our prior belief has to be greater than 0 (we must at least have a desire to believe[1]), but less than 1 (if we already know something, there is no need for faith[2]).
  2. We have to be able to observe something that is correlated to the unknown, else our faith can't grow (this correlation and observation are promised by heavenly law[3]).

With Bayes' Theorem P(X|Y) will never reach 1 (or 0). You cannot have a perfect knowledge of something that can be only indirectly observed. However, after each observation, the posterior belief becomes the prior belief of the next observation, and as we continue to make observations we can eventually have very high confidence that our belief is the truth.

But, the posterior belief continues to approach the truth only if we are actively experimenting and making observations. In the context of faith, that means that we must act according to our faith, and consciously observe the result. The first time I fast and pray for something, and that thing happens, I may credit coincidence. After years of fasting and praying every month, if I carefully observe the result each time, I can see that coincidence has little to do with it. Faith comes from experimenting on the Word again and again, and always observing the result. Count your blessings, for it is mathematically proven to increase your faith!