Chapter 2 Probability of Events
2.1 Axioms of Kolmogorov
- Non-negativity Axiom
\[ \text{For any event A in our sample space, } \mathbb{P}(A)≥0 \]
- Unity Axiom
\[ \text{Normalization, also called the assumption of unit measure,}\\ \text{defines the probability of the sample space } \Omega \text{ as } \mathbb{P}(\Omega)=1 \]
- \(\sigma\)-additivity Axiom
\[ \text{Any countable sequence of disjoint (mutually exclusive) sets satisfies} \\ \mathbb{P}(\bigcup_{i=1}^{\infty}E_{i}) = \sum_{n=1}^{\infty}\mathbb{P}(E_{i}) \qquad\text{where }\ E_i\cap E_j = \varnothing, \quad \forall i\not=j \]
Additional Notes
These axioms allow us to derive the theorems and properties that we frequently come across in the study of probability.
Some authors tend to use finite additivity, rather than sigma additivity as the third axiom.
The Axioms can be stated in two axioms, rather than three:
\((1) \qquad \text{Axiom 1 }\mathbb{P(\varnothing) =0, P(\Omega)=1}, \\ (2) \qquad\sigma-\text{additivity}\)
2.2 Implied consequences of the Axioms
Demonstrating these immediate consequences illustrates the power of the third axiom, and its interaction with the remaining two axioms.
\[ \text{ Probability of the empty set} \\ \mathbb{P}(\varnothing) = 0 \tag{Theorem 2.1} \]
\[ \text{Finite Additivity} \\ \mathbb{P}(\bigcup_{n=1}^{N} E_i)=\sum_{n=1}^{N}\mathbb{P}(E_i) \tag{Theorem 2.2} \]
\[ \text{Monotonicity} \\ A\subseteq B \text{ then }\mathbb{P}(A)\le\mathbb{P}(B) \tag{Theorem 2.3} \]
- Understand that \(A\subseteq B\) means that if a occurs, then B must occur.
\[ \text{The numeric bound} \\ 0\le\mathbb{P}(E)\le1, \qquad\forall E\in F \tag{Theorem 2.3} \]
\[ \text{The complement rule} \\ \mathbb{P}(E^c) = \mathbb{P}(\Omega-E) = 1 - \mathbb{P}(E)\tag{Theorem 2.4} \]
\[ \text{Inclusion-Exclusion Principal} \\ P(A\cup B)=P(A) + P(B)-P(AB) \tag{Theorem 2.5} \\ \]
\[ \text{Generalized Inclusion-Exclusion Principle} \\ \mathbb{P}(\bigcup_{i=1}^n A_i) = \sum_{i=1}^{n} (-1)^{i+1} \\ \sum_{\Omega \subseteq} \mathbb{P}(\bigcup_{i \in \Omega} A_{i}) \tag{Theorem 2.5} \]
\[ \text{Equally Likely Theorem} \\ \mathbb{P}(A)=\frac{N(A)}{N}\tag{Theorem 2.7} \]
- This is the naive definition of probability
\[ \text{No Name Theorem} \\ \mathbb{P}(A) = P(AB) + P(AB^c) \tag{Theorem 2.8} \]
\[ \text{Boole's Inequality} \\ \mathbb{P}(\bigcup_{i=1}^{\infty}E_{i}) \le \sum_{n=1}^{\infty}\mathbb{P}(E_{i}) \]
2.3 About Sets, Sample Spaces, and events
- Sets are collections of distinct elements.
- A sample space is the set of disjoint, collectively exhaustive outcomes taken at a determined level of granularity.
- A sample space can be described as finite or infinite, discrete or continuous.
Sample Space Notation: A sample space can be described with notation below, the long bar, | is read as “such that”
\[ A\cup B=\{x| x∈A \cup x∈B\} \\ A^c=\{x|x\not\in A\} \]
2.3.1 Set Notation and LaTex code.
| Symbol | Name | LaTeX |
|---|---|---|
| \(\cup\) | Union | \cup |
| \(\cap\) | Intersection | \cap |
| \(\in\) | Belongs to | \in |
| \(\varnothing\) | Null set | \varnothing |
| \(\Omega\), \(S\) | Universal set | \Omega |
| \(\omega\), \(s\) | Atom, singleton | \omega |
| \(\subset\) | Proper subset | \subset |
| \(\subseteq\) | Subset, S is contained within A | \subseteq |
| \(\bigcup\) | infinitary union, generalized union, unified union | \bigcup |
| \(\bigcap\) | infinitary intersection, generalized intersection, unified intersection cap | \bigcap |
2.3.2 Some Set Definitions
| Name | Definition | Rule |
|---|---|---|
| Certainty | The probability of the sample space (the set of all possible outcomes) is 1, representing absolute certainty | \(P(\Omega) = 1\) |
| Impossibility | The probability of an impossible event is 0 | \(P(\varnothing) = 0\) |
| Mutual Exclusivity | Two events A and B cannot occur simultaneously, in terms of probability | \(P(A \cap B) = 0\) |
| Complement | The complement of event A is self-defined | \(P(A^c) = 1 - P(A)\) |
| Sum Rule | The probability of the union of two mutually exclusive events A and B | \(P(A \cup B) = P(A) + P(B)\) |
| Difference Rule | The probability of the difference between two events A and B | \(P(A - B) = P(A) - P(A \cap B)\) |
Note that Impossibility and Certainty are both themselves axioms