Learn practical skills, build real-world projects, and advance your career


Probability and Statistics


Conditional Probability:

Probabilities of events get modified when we have some partial information about the outcome.

Consider the random experiment of rolling a fair die, sample space Ω={1,2,3,4,5,6}\Omega=\{1,2,3,4,5,6\} and let us be interested in the following events here.
Let A={2,3,5}A=\{2,3,5\}\leftarrow die landing on a prime number
B={2,4,6}B=\{2,4,6\}\leftarrow die landing on a even number
C={1,3,5}C=\{1,3,5\}\leftarrow die landing on odd number

Intuitively we can see that P(AP(A given that BB already happened)=13)=\frac{1}{3}
Because when it is given that BB already happened, we think of only 2, 4, 6 as possible outcomes, so BB kind of becomes new reduced sample space, and for AA only 2 is favourable for being prime.
And P(AP(A given that BB already happened)=1312=P(A))=\frac{1}{3}\ne \frac{1}{2}=P(A)
That is when we know that B happened probability of A got reduced from 12\frac{1}{2} to 13\frac{1}{3}
similarly we can argue that P(AP(A given that CC already happened)=2312=P(A))=\frac{2}{3}\ne \frac{1}{2}=P(A)
That is when we know that C happened probability of A got increased from 12\frac{1}{2} to 23\frac{2}{3}

So when partial information about outcome is available , we might have to update the probability of event.
Question: How to update the probability of an event using partial information?

To get an intuition, let us focus on random experiment with finite sample space and equally likely outcomes.
Let Ω\Omega be a sample space, and A,BA,B ar events. and we are interested in determining probability of AA, when we have information that BB happened.
So, BB is now, new reduced sample space and only outcomes of AA that concern us are those (if any) that are also outcomes from BB
(i.e outcomes in ABA\cap B)
So it makes sense to define P(AP(A given that BB already happened)=ABB=\frac{|A\cap B|}{|B|}
and =ABB=ABΩBΩ=P(AB)P(B)=\frac{|A\cap B|}{|B|}=\frac{\frac{|A\cap B|}{|\Omega|}}{\frac{|B|}{|\Omega|}}=\frac{P(A\cap B)}{P(B)}
So, intuition suggests that when BB already happened,new probability of AA should be updated as P(AB)P(B)\frac{P(A\cap B)}{P(B)} (in terms of old probabilities)

Let us check this rule of updating probability with above die example that we considered.
P(AP(A given that BB happened)=P(AB)P(B)=1612=13)=\frac{P(A\cap B)}{P(B)}=\frac{\frac{1}{6}}{\frac{1}{2}}=\frac{1}{3} and P(AP(A given that CC happened)=P(AC)P(C)=1312=23)=\frac{P(A\cap C)}{P(C)}=\frac{\frac{1}{3}}{\frac{1}{2}}=\frac{2}{3}

Now let us show that the updated probabilities are indeed representing valid probabilities.
Let (Ω,F,P)(\Omega,\mathscr{F},P) be probability space with P(B)>0P(B)>0
Define a set function, PB:FRP_B:\mathscr{F}\rightarrow \mathbb{R}, PB(A)=P(AB)P(B)P_B(A)=\frac{P(A\cap B)}{P(B)}

claim: PBP_B is a probability (measure) function.
Here we need to check three axioms of probability for PBP_B function.

  • PB(A)=P(AB)P(B)0 AFP_B(A)=\frac{P(A\cap B)}{P(B)}\ge0~\forall A \in \mathscr{F} (P\because P already probability function P(AB)0, P(B)>0\Rightarrow P(A\cap B)\ge0 ,~P(B)>0 by Axiom 1)
    So Axiom 1 for PBP_B\checkmark

  • Pb(Ω)=P(ΩB)P(B)=P(B)P(B)=1P_b(\Omega)=\frac{P(\Omega \cap B)}{P(B)}=\frac{P(B)}{P(B)}=1 (BΩΩB=B\because B\subseteq \Omega \Rightarrow \Omega\cap B= B )
    So Axiom 2 for PBP_B\checkmark

  • Let A1,A2,A3,...,An,...A_1,A_2,A_3,...,A_n,... are mutually exclusive events in F\mathscr{F}

    PB(i=1Ai)=P((i=1Ai)B)P(B)P_B(\bigcup_{i=1}^{\infty}A_i)=\frac{P((\bigcup_{i=1}^{\infty}A_i)\cap B)}{P(B)}
    =P(i=1(AiB))P(B)=\frac{P(\bigcup_{i=1}^{\infty}(A_i\cap B))}{P(B)} (\because generalized distributive laws)
    =i=1P(AiB)P(B)=\frac{\sum_{i=1}^{\infty}P(A_i\cap B)}{P(B)} (A1B,A2B,A3B,.....\because A_1\cap B, A_2\cap B, A_3\cap B,..... also mutually exclusive and Axiom 3 for PP )
    =i=1P(AiB)P(B)=i=1PB(Ai)=\sum_{i=1}^{\infty}\frac{P(A_i\cap B)}{P(B)}=\sum_{i=1}^{\infty}P_B(A_i) (\because definition of PBP_B)
    So Axiom 3 for PBP_B\checkmark

    PB\therefore P_B is a probability measure

    For any AF,PB(A)A\in \mathscr{F}, P_B(A) is called conditional probability of AA given that BB already happened.
    In practice PB(A)P_B(A)is denoted by P(A/B)P(A/B) and read it as 'probability of A given B'
    P(A/B):=P(AB)P(B) ,P(B)>0P(A/B):=\frac{P(A\cap B)}{P(B)}~,P(B)>0

Example 1:A family has two children. What is the conditional probability that both are boys given that(i) family has a boy? (ii) elder one is boy?

solution: Let sample space Ω={(b,b),(b,g),(g,b),(g,g)}\Omega=\{(b,b),(b,g),(g,b),(g,g)\}
Let AA denote the event that family having both boys, i.e A={(b,b)}P(A)=14A=\{(b,b)\}\Rightarrow P(A)=\frac{1}{4},
BB denote the event that family having a boys, i.e B={(b,b),(b,g),(g,b)}P(B)=34B=\{(b,b),(b,g),(g,b)\}\Rightarrow P(B)=\frac{3}{4},
CC denote the event that family having elder one boy, i.e C={(b,b),(b,g)}P(C)=12C=\{(b,b),(b,g)\}\Rightarrow P(C)=\frac{1}{2}
(i). P(A/B)=P(AB)P(B)=1434=13P(A/B)=\frac{P(A\cap B)}{P(B)}=\frac{\frac{1}{4}}{\frac{3}{4}}=\frac{1}{3}

(ii). P(A/C)=P(AC)P(C)=1412=12P(A/C)=\frac{P(A\cap C)}{P(C)}=\frac{\frac{1}{4}}{\frac{1}{2}}=\frac{1}{2}
Note: Most people don't see that (i) and (ii) are actually different, i.e people approach (i) as (ii) and come up with answer 12\frac{1}{2},
please understand the difference.