5.2 Probability

Detailed Theory: Probability

1. Basic Concepts and Terminology

1.1 What is Probability?

Probability is a branch of mathematics that deals with calculating the likelihood of events occurring. It quantifies uncertainty.

Key Idea: Probability is a number between 0 and 1 that represents how likely an event is to occur.

• 0 means the event is impossible

• 1 means the event is certain

• 0.5 means the event is equally likely to occur or not occur

1.2 Important Terms

a) Experiment

An action or process that leads to one of several possible outcomes.

Examples:

• Tossing a coin

• Rolling a die

• Drawing a card from a deck

b) Sample Space (S)

The set of all possible outcomes of an experiment.

Examples:

1. Tossing a coin: $S = \{H, T\}$ (H = Head, T = Tail)

2. Rolling a die: $S = \{1, 2, 3, 4, 5, 6\}$

3. Tossing two coins: $S = \{HH, HT, TH, TT\}$

c) Event (E)

A subset of the sample space (one or more outcomes).

Examples for die roll:

• Event A: Getting an even number = $\{2, 4, 6\}$

• Event B: Getting a number > 4 = $\{5, 6\}$

d) Types of Events

1. Simple Event: Single outcome (e.g., getting a 3 on die)

2. Compound Event: Multiple outcomes (e.g., getting even number)

3. Impossible Event: Event that cannot occur (probability = 0)

4. Sure/Certain Event: Event that always occurs (probability = 1)

5. Complementary Event (E' or E⁰): All outcomes NOT in E

---

2. Different Approaches to Probability

2.1 Classical (Theoretical) Probability

Used when all outcomes are equally likely.

Formula:

$P(E) = \frac{\text{Number of favorable outcomes}}{\text{Total number of outcomes}} = \frac{n(E)}{n(S)}$

Example: Probability of getting an even number when rolling a die:

$S = \{1, 2, 3, 4, 5, 6\} \Rightarrow n(S) = 6$

$E = \{2, 4, 6\} \Rightarrow n(E) = 3$

$P(E) = \frac{3}{6} = \frac{1}{2} = 0.5$

2.2 Empirical (Experimental) Probability

Based on actual experiments or observations.

Formula:

$P(E) = \frac{\text{Number of times event occurred}}{\text{Total number of trials}}$

Example: In 100 coin tosses, heads appeared 47 times.

Empirical probability of heads = $\frac{47}{100} = 0.47$

2.3 Axiomatic Probability

Based on three axioms (rules):

Axiom 1: $0 \leq P(E) \leq 1$ for any event E

Axiom 2: $P(S) = 1$ (probability of sample space is 1)

Axiom 3: For mutually exclusive events A and B: $P(A \cup B) = P(A) + P(B)$

---

3. Basic Probability Rules

3.1 Complement Rule

The probability that an event does NOT occur.

$P(E') = 1 - P(E)$

Example: Probability of NOT getting a 6 on a die roll:

$P(\text{not 6}) = 1 - P(\text{getting 6}) = 1 - \frac{1}{6} = \frac{5}{6}$

3.2 Addition Rule

For any two events A and B:

$P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Special Case: Mutually Exclusive Events

Events that cannot occur together (no common outcomes).

For mutually exclusive events: $P(A \cap B) = 0$

So: $P(A \cup B) = P(A) + P(B)$

Example: Probability of getting 2 OR 5 on a die roll:

These are mutually exclusive: $P(2 \cup 5) = P(2) + P(5) = \frac{1}{6} + \frac{1}{6} = \frac{2}{6} = \frac{1}{3}$

3.3 Multiplication Rule

For independent events (occurrence of one doesn't affect the other):

$P(A \cap B) = P(A) \times P(B)$

Example: Probability of getting heads on two consecutive coin tosses:

$P(H \text{ and } H) = P(H) \times P(H) = \frac{1}{2} \times \frac{1}{2} = \frac{1}{4}$

---

4. Conditional Probability

4.1 Definition

Probability of event A occurring given that event B has already occurred.

Notation: $P(A|B)$ (read as "probability of A given B")

4.2 Formula

$P(A|B) = \frac{P(A \cap B)}{P(B)}$, provided $P(B) \neq 0$

This can be rearranged as: $P(A \cap B) = P(B) \times P(A|B)$

4.3 Understanding with Example

In a class of 30 students: 18 girls, 12 boys. 6 girls and 4 boys wear glasses.

Let: A = student wears glasses, B = student is a girl

Find probability that a student wears glasses given they're a girl:

$P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{6/30}{18/30} = \frac{6}{18} = \frac{1}{3}$

Direct method: Among 18 girls, 6 wear glasses, so probability = $\frac{6}{18} = \frac{1}{3}$

4.4 Independent Events using Conditional Probability

Events A and B are independent if:

$P(A|B) = P(A)$ or $P(B|A) = P(B)$

This means knowing B occurred doesn't change probability of A.

Test for independence: A and B are independent if and only if: $P(A \cap B) = P(A) \times P(B)$

---

5. Bayes' Theorem

5.1 Statement

For events A and B with $P(B) \neq 0$:

$P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}$

5.2 Extended Form

If $B_1, B_2, \ldots, B_n$ form a partition of sample space, then:

$P(B_i|A) = \frac{P(A|B_i) \times P(B_i)}{\sum_{j=1}^{n} P(A|B_j) \times P(B_j)}$

5.3 Application Example

Medical Test Problem:

• Disease affects 1% of population (P(D) = 0.01)

• Test is 95% accurate: P(positive|D) = 0.95, P(negative|no D) = 0.95

• Find P(D|positive)

Solution using Bayes':

Let D = has disease, T+ = test positive

We want: $P(D|T+) = \frac{P(T+|D) \times P(D)}{P(T+)}$

$P(T+|D) = 0.95$, $P(D) = 0.01$

$P(T+) = P(T+|D)P(D) + P(T+|no D)P(no D)$ $= 0.95 \times 0.01 + 0.05 \times 0.99 = 0.0095 + 0.0495 = 0.059$

So: $P(D|T+) = \frac{0.95 \times 0.01}{0.059} = \frac{0.0095}{0.059} \approx 0.161$

Only 16.1% chance of having disease given positive test!

---

6. Random Variables

6.1 Definition

A variable whose values are determined by outcomes of a random experiment.

Notation: Capital letters (X, Y, Z) for random variables, lowercase (x, y, z) for specific values.

6.2 Types of Random Variables

a) Discrete Random Variable

Takes countable number of distinct values.

Examples:

• Number of heads in 3 coin tosses (0, 1, 2, 3)

• Number of students absent in a class

b) Continuous Random Variable

Takes infinitely many values in an interval (measurable).

Examples:

• Height of students

• Time taken to complete a task

• Temperature

6.3 Probability Distribution

For a discrete random variable X, lists all possible values and their probabilities.

Requirements:

1. $0 \leq P(X = x_i) \leq 1$ for all i

2. $\sum P(X = x_i) = 1$

Example: Probability distribution of number of heads in 2 coin tosses:

X (number of heads)	P(X)
0	1/4
1	1/2
2	1/4
Total	1

---

7. Expectation (Mean) and Variance

7.1 Expected Value (Mean)

Average value we expect if experiment is repeated many times.

For discrete random variable X:

$E(X) = \mu = \sum x_i \cdot P(X = x_i)$

Example: Expected number of heads in 2 coin tosses:

$E(X) = 0 \times \frac{1}{4} + 1 \times \frac{1}{2} + 2 \times \frac{1}{4} = 0 + 0.5 + 0.5 = 1$

We expect 1 head on average.

7.2 Properties of Expectation

1. $E(c) = c$ where c is constant

2. $E(cX) = cE(X)$

3. $E(X + Y) = E(X) + E(Y)$

4. $E(X - Y) = E(X) - E(Y)$

5. For independent X, Y: $E(XY) = E(X)E(Y)$

7.3 Variance

Measures how spread out the values are from the mean.

Formula: $\text{Var}(X) = E[(X - \mu)^2] = E(X^2) - [E(X)]^2$

Calculation formula: $\text{Var}(X) = \sum x_i^2 P(X = x_i) - [\sum x_i P(X = x_i)]^2$

7.4 Standard Deviation

Square root of variance: $\sigma = \sqrt{\text{Var}(X)}$

7.5 Properties of Variance

1. $\text{Var}(c) = 0$ where c is constant

2. $\text{Var}(cX) = c^2 \text{Var}(X)$

3. $\text{Var}(X + c) = \text{Var}(X)$

4. For independent X, Y: $\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y)$

---

8. Important Probability Distributions

8.1 Binomial Distribution

a) Conditions for Binomial Experiment

1. Fixed number of trials (n)

2. Each trial has exactly two outcomes: success or failure

3. Probability of success (p) is constant for each trial

4. Trials are independent

b) Probability Mass Function

For random variable X = number of successes in n trials:

$P(X = r) = \binom{n}{r} p^r (1-p)^{n-r}, \quad r = 0, 1, 2, \ldots, n$

where $\binom{n}{r} = \frac{n!}{r!(n-r)!}$ (binomial coefficient)

c) Mean and Variance

Mean: $\mu = np$

Variance: $\sigma^2 = np(1-p)$

Standard Deviation: $\sigma = \sqrt{np(1-p)}$

d) Example

Probability of getting exactly 3 heads in 5 coin tosses:

$n=5$, $p=0.5$, $r=3$

$P(X=3) = \binom{5}{3} (0.5)^3 (0.5)^2 = 10 \times 0.125 \times 0.25 = 10 \times 0.03125 = 0.3125$

8.2 Poisson Distribution

a) When to Use

Models number of events occurring in a fixed interval of time/space, when events occur independently at constant average rate.

Examples:

• Number of calls at a call center per hour

• Number of accidents at an intersection per day

b) Probability Mass Function

$P(X = r) = \frac{e^{-\lambda} \lambda^r}{r!}, \quad r = 0, 1, 2, \ldots$

where $\lambda$ = average number of events in the interval

c) Mean and Variance

Mean = Variance = $\lambda$

8.3 Normal Distribution (Gaussian Distribution)

a) Characteristics

• Bell-shaped curve

• Symmetric about mean

• Mean = median = mode

• Total area under curve = 1

b) Probability Density Function

$f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2}, \quad -\infty < x < \infty$

where $\mu$ = mean, $\sigma$ = standard deviation

c) Standard Normal Distribution

Special case with $\mu = 0$, $\sigma = 1$

Z-score: $z = \frac{x - \mu}{\sigma}$

Converts any normal distribution to standard normal.

d) Empirical Rule (68-95-99.7 Rule)

For normal distribution:

• About 68% of data falls within 1σ of mean

• About 95% of data falls within 2σ of mean

• About 99.7% of data falls within 3σ of mean

---

9. Permutations and Combinations in Probability

9.1 Fundamental Counting Principle

If event A can occur in m ways, and event B can occur in n ways, then:

• A AND B can occur in m × n ways

• A OR B can occur in m + n ways

9.2 Permutations

Arrangements where order matters.

Number of permutations of n distinct objects taken r at a time:

$P(n, r) = \frac{n!}{(n-r)!}$

Special case: All n objects: $P(n, n) = n!$

9.3 Combinations

Selections where order doesn't matter.

Number of combinations of n distinct objects taken r at a time:

$C(n, r) = \binom{n}{r} = \frac{n!}{r!(n-r)!}$

9.4 Application in Probability

Example: Probability of getting a specific poker hand.

---

10. Solved Examples

Example 1: Basic Probability

A bag contains 5 red, 3 blue, and 2 green balls. One ball is drawn at random. Find: a) P(red) b) P(not blue) c) P(red or green)

Solution: Total balls = 5 + 3 + 2 = 10

a) $P(\text{red}) = \frac{5}{10} = 0.5$

b) $P(\text{not blue}) = 1 - P(\text{blue}) = 1 - \frac{3}{10} = \frac{7}{10} = 0.7$

c) $P(\text{red or green}) = P(\text{red}) + P(\text{green}) = \frac{5}{10} + \frac{2}{10} = \frac{7}{10} = 0.7$

Example 2: Conditional Probability

In a class, 60% are girls. 40% of girls and 30% of boys wear glasses. If a student wears glasses, what's the probability they're a girl?

Solution: Let: G = girl, B = boy, W = wears glasses

Given: $P(G) = 0.6$, $P(B) = 0.4$, $P(W|G) = 0.4$, $P(W|B) = 0.3$

We want: $P(G|W)$

By Bayes': $P(G|W) = \frac{P(W|G)P(G)}{P(W)}$

$P(W) = P(W|G)P(G) + P(W|B)P(B)$ $= 0.4 \times 0.6 + 0.3 \times 0.4 = 0.24 + 0.12 = 0.36$

So: $P(G|W) = \frac{0.4 \times 0.6}{0.36} = \frac{0.24}{0.36} = \frac{2}{3} \approx 0.667$

Example 3: Binomial Distribution

A test has 10 multiple-choice questions, each with 4 options. If a student guesses randomly, find probability of: a) Exactly 5 correct b) At least 8 correct

Solution: $n=10$, $p = \frac{1}{4} = 0.25$ (probability of correct guess)

a) $P(X=5) = \binom{10}{5} (0.25)^5 (0.75)^5$

$= 252 \times 0.0009766 \times 0.2373 = 252 \times 0.0002318 = 0.0584$

b) $P(X \geq 8) = P(X=8) + P(X=9) + P(X=10)$

$P(X=8) = \binom{10}{8} (0.25)^8 (0.75)^2 = 45 \times 0.00001526 \times 0.5625 = 0.000386$ $P(X=9) = \binom{10}{9} (0.25)^9 (0.75)^1 = 10 \times 0.000003815 \times 0.75 = 0.0000286$ $P(X=10) = \binom{10}{10} (0.25)^{10} (0.75)^0 = 1 \times 0.000000954 = 0.000000954$

Sum = 0.000386 + 0.0000286 + 0.000000954 = 0.0004155

Example 4: Expected Value

A game: Roll a die. If you get 6, win $10. If you get 4 or 5, win $5. Otherwise, lose $3. What's expected value?

Solution: Let X = winnings

Outcome	X (winnings)	P(X)
6	$10	1/6
4 or 5	$5	2/6
1,2,3	-$3	3/6

$E(X) = 10 \times \frac{1}{6} + 5 \times \frac{2}{6} + (-3) \times \frac{3}{6}$ $= \frac{10}{6} + \frac{10}{6} - \frac{9}{6} = \frac{11}{6} \approx \$1.83$

Positive expected value means game is favorable on average.

---

11. Important Formulas Summary

11.1 Basic Probability

• $P(E) = \frac{n(E)}{n(S)}$

• $P(E') = 1 - P(E)$

• $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

• Mutually exclusive: $P(A \cap B) = 0$

11.2 Conditional Probability

• $P(A|B) = \frac{P(A \cap B)}{P(B)}$

• Independent events: $P(A \cap B) = P(A)P(B)$

11.3 Bayes' Theorem

• $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$

11.4 Random Variables

• Expected value: $E(X) = \sum x_i P(X = x_i)$

• Variance: $\text{Var}(X) = E(X^2) - [E(X)]^2$

11.5 Binomial Distribution

• $P(X=r) = \binom{n}{r} p^r (1-p)^{n-r}$

• Mean: $np$

• Variance: $np(1-p)$

11.6 Poisson Distribution

• $P(X=r) = \frac{e^{-\lambda} \lambda^r}{r!}$

• Mean = Variance = $\lambda$

11.7 Counting

• Permutations: $P(n,r) = \frac{n!}{(n-r)!}$

• Combinations: $C(n,r) = \frac{n!}{r!(n-r)!}$

---

12. Exam Tips and Common Mistakes

12.1 Common Mistakes to Avoid

1. Confusing "and" vs "or":

   • "A and B" means both occur (intersection)

   • "A or B" means at least one occurs (union)

2. Misapplying addition rule: Remember to subtract intersection unless events are mutually exclusive

3. Confusing independent vs mutually exclusive:

   • Independent: $P(A \cap B) = P(A)P(B)$

   • Mutually exclusive: $P(A \cap B) = 0$

4. Forgetting to check conditions for binomial/Poisson distributions

5. Probability > 1 or < 0: Impossible! Check calculations if this happens

12.2 Problem-Solving Strategy

1. Define events clearly: Write what each event represents

2. Identify what's asked: "Given that", "and", "or", etc.

3. Choose correct formula: Based on conditions

4. Check independence/mutual exclusivity

5. Draw Venn diagrams or tree diagrams for visualization

12.3 Quick Checks

1. Total probability always sums to 1

2. Conditional probability: $0 \leq P(A|B) \leq 1$

3. Complement: $P(A) + P(A') = 1$

4. Expected value interpretation: Long-run average

5. Variance: Always non-negative

---

13. Real-World Applications

13.1 Everyday Life

1. Weather forecasting: Probability of rain

2. Games of chance: Cards, dice, lotteries

3. Insurance: Calculating premiums based on risk

13.2 Science and Engineering

1. Quality control: Probability of defective items

2. Reliability engineering: Probability of system failure

3. Medical testing: Sensitivity and specificity

13.3 Finance

1. Stock market: Probability of price movements

2. Risk assessment: Probability of loan default

3. Portfolio management: Expected returns

This comprehensive theory covers all aspects of probability with detailed explanations and examples, making it easy to understand while being thorough enough for exam preparation.