[6 points] Prove Bayes’ Theorem. Briefly explain why it is useful for machine learning problems, i. e., by converting posterior probability to likelihood and prior probability.

Mathematically Bayes’ theorem is defined as

P(A\B)=P(B\A) ×P(A)

P(B)

Bayes theorem is defined as where A and B are events, P(A|B) is the conditional probability that event A occurs given that event B has already occurred (P(B|A) has the same meaning but with the roles of A and B reversed) and P(A) and P(B) are the marginal probabilities of event A and event B occurring respectively.

Step-by-step explanation: for example, picking a card from a pack of traditional playing cards. There are 52 cards in the pack, 26 of them are red and 26 are black. What is the probability of the card being a 4 given that we know the card is red?

To convert this into the math symbols that we see above we can say that event A is the event that the card picked is a 4 and event B is the card being red. Hence, P(A|B) in the equation above is P(4|red) in our example, and this is what we want to calculate. We previously worked out that this probability is equal to 1/13 (there 26 red cards and 2 of those are 4's) but let’s calculate this using Bayes’ theorem.

We need to find the probabilities for the terms on the right-hand side. They are:

P(B|A) = P(red|4) = 1/2

P(A) = P(4) = 4/52 = 1/13

P(B) = P(red) = 1/2

When we substitute these numbers into the equation for Bayes’ theorem above we get 1/13, which is the answer that we were expecting.