MA2102

Probability and Statistics

Lecture-7

Conditional Probability:

Probabilities of events get modified when we have some partial information about the outcome.


Consider the random experiment of rolling a fair die, sample space $\Omega=\{1,2,3,4,5,6\}$ and let us be interested in the following events here.
Let $A=\{2,3,5\}\leftarrow$ die landing on a prime number
$B=\{2,4,6\}\leftarrow$ die landing on a even number
$C=\{1,3,5\}\leftarrow$ die landing on odd number


Intuitively we can see that $P(A$ given that $B$ already happened$)=\frac{1}{3}$
Because when it is given that $B$ already happened, we think of only 2, 4, 6 as possible outcomes, so $B$ kind of becomes new reduced sample space, and for $A$ only 2 is favourable for being prime.
And $P(A$ given that $B$ already happened$)=\frac{1}{3}\ne \frac{1}{2}=P(A)$
That is when we know that B happened probability of A got reduced from $\frac{1}{2}$ to $\frac{1}{3}$
similarly we can argue that $P(A$ given that $C$ already happened$)=\frac{2}{3}\ne \frac{1}{2}=P(A)$
That is when we know that C happened probability of A got increased from $\frac{1}{2}$ to $\frac{2}{3}$

So when partial information about outcome is available , we might have to update the probability of event.
Question: How to update the probability of an event using partial information?

To get an intuition, let us focus on random experiment with finite sample space and equally likely outcomes.
Let $\Omega$ be a sample space, and $A,B$ ar events. and we are interested in determining probability of $A$, when we have information that $B$ happened.
So, $B$ is now, new reduced sample space and only outcomes of $A$ that concern us are those (if any) that are also outcomes from $B$
(i.e outcomes in $A\cap B$)
So it makes sense to define $P(A$ given that $B$ already happened)$=\frac{|A\cap B|}{|B|}$
and $=\frac{|A\cap B|}{|B|}=\frac{\frac{|A\cap B|}{|\Omega|}}{\frac{|B|}{|\Omega|}}=\frac{P(A\cap B)}{P(B)}$
So, intuition suggests that when $B$ already happened,new probability of $A$ should be updated as $\frac{P(A\cap B)}{P(B)}$ (in terms of old probabilities)

Let us check this rule of updating probability with above die example that we considered.
$P(A$ given that $B$ happened$)=\frac{P(A\cap B)}{P(B)}=\frac{\frac{1}{6}}{\frac{1}{2}}=\frac{1}{3}$ and $P(A$ given that $C$ happened$)=\frac{P(A\cap C)}{P(C)}=\frac{\frac{1}{3}}{\frac{1}{2}}=\frac{2}{3}$

Now let us show that the updated probabilities are indeed representing valid probabilities.
Let $(\Omega,\mathscr{F},P)$ be probability space with $P(B)>0$
Define a set function, $P_B:\mathscr{F}\rightarrow \mathbb{R}$, $P_B(A)=\frac{P(A\cap B)}{P(B)}$

claim: $P_B$ is a probability (measure) function.
Here we need to check three axioms of probability for $P_B$ function.

• $P_B(A)=\frac{P(A\cap B)}{P(B)}\ge0~\forall A \in \mathscr{F}$ ($\because P$ already probability function $\Rightarrow P(A\cap B)\ge0 ,~P(B)>0$ by Axiom 1)
  So Axiom 1 for $P_B\checkmark$
  
  
• $P_b(\Omega)=\frac{P(\Omega \cap B)}{P(B)}=\frac{P(B)}{P(B)}=1$ ($\because B\subseteq \Omega \Rightarrow \Omega\cap B= B$ )
  So Axiom 2 for $P_B\checkmark$
  
  
• Let $A_1,A_2,A_3,...,A_n,...$ are mutually exclusive events in $\mathscr{F}$
  
  $P_B(\bigcup_{i=1}^{\infty}A_i)=\frac{P((\bigcup_{i=1}^{\infty}A_i)\cap B)}{P(B)}$
  $=\frac{P(\bigcup_{i=1}^{\infty}(A_i\cap B))}{P(B)}$ ($\because$ generalized distributive laws)
  $=\frac{\sum_{i=1}^{\infty}P(A_i\cap B)}{P(B)}$ ($\because A_1\cap B, A_2\cap B, A_3\cap B,.....$also mutually exclusive and Axiom 3 for $P$ )
  $=\sum_{i=1}^{\infty}\frac{P(A_i\cap B)}{P(B)}=\sum_{i=1}^{\infty}P_B(A_i)$ ($\because$ definition of $P_B$)
  So Axiom 3 for $P_B\checkmark$
  
  
  $\therefore P_B$ is a probability measure
  
  
  For any $A\in \mathscr{F}, P_B(A)$ is called conditional probability of $A$ given that $B$ already happened.
  In practice $P_B(A)$is denoted by $P(A/B)$ and read it as 'probability of A given B'
  $P(A/B):=\frac{P(A\cap B)}{P(B)}~,P(B)>0$
  

Example 1:A family has two children. What is the conditional probability that both are boys given that(i) family has a boy? (ii) elder one is boy?

solution: Let sample space $\Omega=\{(b,b),(b,g),(g,b),(g,g)\}$
Let $A$ denote the event that family having both boys, i.e $A=\{(b,b)\}\Rightarrow P(A)=\frac{1}{4}$,
$B$ denote the event that family having a boys, i.e $B=\{(b,b),(b,g),(g,b)\}\Rightarrow P(B)=\frac{3}{4}$,
$C$ denote the event that family having elder one boy, i.e $C=\{(b,b),(b,g)\}\Rightarrow P(C)=\frac{1}{2}$
(i). $P(A/B)=\frac{P(A\cap B)}{P(B)}=\frac{\frac{1}{4}}{\frac{3}{4}}=\frac{1}{3}$

(ii). $P(A/C)=\frac{P(A\cap C)}{P(C)}=\frac{\frac{1}{4}}{\frac{1}{2}}=\frac{1}{2}$
Note: Most people don't see that (i) and (ii) are actually different, i.e people approach (i) as (ii) and come up with answer $\frac{1}{2}$,
please understand the difference.