Chapter 5 Probability Distributions (Part 1)
5.1 Random Variables
The concept of random variables allows us to explore what is unknown, or governed by randomness. We define a random variable as a real-valued function that maps any given event/outcome of a random experiment to a real number. The most formal, axiomatic definition of a random variable involves measure theory.
\[ X:\Omega\rightarrow \mathbb{R} \]
The outcome of a coin toss can be represented by a random variable that takes the value 1 if the coin lands heads and 0 if it lands tails.
Depending on the type of data, a random variable is either discrete or continuous.
We can have several random variables defined on a sample space.
Support Set
An event/outcome of a random experiment that is a subset from our sample space.
The support set is the set of possible values of \(X\).
A random variable is not the same as it’s probability distribution:
- A probability distribution is more of a blueprint for all of the possible values that a random variable can take on, i.e. the support.
- There are many different mistakes that can be made by thinking random variable(s) abide by the same rules as something like a PMF.
5.2 Distribution Functions
Also called, Cumulative Distribution Functions. A distribution function characterizes a random variable, they are used as a way to encode information about a random variable.
More formally, a random variable \(X\) is a function \(F\) from \((-\infty, +\infty)\) to \(\mathbb{R}\) defined by\(F(t)=P(X \le t)\).
Properties
From the definition of a distribution function, the following properties are determined:
- \(F(t)\) is non-decreasing
- \(F(t)\) is right continuous
- Satisfies \(\lim_{t \rightarrow \infty}= 1\)
- Satisfies \(\lim_{t \rightarrow -\infty}= 0\)
If a function satisfies these properties then it is a distribution function of a random variable.
\[ F_X(t) = P(X \le t) = \sum_{\{x \in R_x | x\le t\}} p_X(x)\tag{Discrete} \]
\[ F_X(t) = P(X \le t) = \int_{-\infty}^{t} f_X(x) dx \tag{Continuous} \]
Additional Notes
- For x-values where \(F'(X)\) exists, \(F'(X) = f(x)\)
- For discrete random variables, CDF’s are step functions.
- A distribution function does not need to be continuous at all points, just right continuous.
5.3 Calculating Probability
Recall that the CDF can characterize a random variable, how is this done? Considered the events that are represented as the following inequalities, if \(\mathbb{P}(X \le t)\) is known for all \(t \in \mathbb{R}\) all of the following events can be calculated:
\[ (X \le a) \qquad (X < a) \qquad (X \ge a) \qquad (X > a) \qquad (X = a) \]
\[ (a \le X \le b) \qquad (a < X < b) \qquad (a < X \le b) \qquad (a \le X < b) \]
Discrete Case
The CDF is defined as \(F(X) = P(X \le x)\). Use these relations to find the following probabilities
| Probability of Event | CDF Calculation |
|---|---|
| \(P(X \ge a)\) | \(1 - F(a-)\) |
| \(P(X > a)\) | \(1 - F(a)\) |
| \(P(X < x)\) | \(F(a-)\) |
| \(P(X = a)\) | \(F(a)-F(a-)\) |
| \(P(a < X \le b)\) | \(F(b) - F(a)\) |
| \(P(a \le X < b)\) | \(F(b-) - F(a-)\) |
| \(P(a \le X \le b)\) | \(F(b) - F(a-)\) |
| \(P(a < X < b)\) | \(F(b-) - F(a)\) |
Continuous Case
The nature of continuous data makes calculating probabilities using the CDF much simpler. This is because of the way by which we integrate to calculate these continuous probabilities.
\[ P(a<X<b)=P(a\leq X<b)=P(a<X\leq b)=P(a\leq X\leq b)=\int_{a}^{b} f(t) \, dt \]
| Probability of Event | CDF Calculation |
|---|---|
| \(P(X = x) = 0\) | \(F(x) =0\) |
| \(P(a < X \leq b), \\ P(a \leq X < b), \\ P(a \leq X \leq b), \\P(a < X < b)\) | \(F(b) - F(a)\) |
| \(P(X > a)\) | \(1 - F(a)\) |
| \(P(X \geq a)\) | \(1 - F(a)\) |
| \(P(X < a)\) | \(F(a)\) |
5.4 Probability Mass Functions
Also called a PMF, probability function, or discrete probability function.
Defined as a real-valued function from support set of a random variable \(X\) to \(\mathbb{R}\), i.e. \(p: \mathbb{R_x} \rightarrow \mathbb{R}\).
\[ p_X(x) = P(X=x) = P(\{\omega \in \Omega | X(\omega)=x\}) \]
a proper PMF satisfies the properties
\[ p(x) \ge 0 \]
and
\[ \sum_{x \in R_x}p(x)=1 \]
5.5 Probability Density Functions
Defined as a real-valued function from \(\mathbb{R}\) to \(\mathbb{R}\), i.e. \(f: \mathbb{R} \rightarrow \mathbb{R}\)
a proper PDF satisfies the following properties
\[ \tag{1} f(x) \ge 0, \quad \forall x \in \mathbb{R} \]
and
\[ \int_{-\infty}^{\infty}f(x)dx=1 \tag{2} \]
Additional notes (PMF and PDF)
- The PMF is defined as the difference between consecutive CDF values
- The properties of the PMF and PDF show that they are probability measures, as shown by the Axioms of Kolmogorov.
5.6 Finding the value of c, a constant in a distribution.
When we are asked to find a constant value in order to define a function, This definition of a probability distribution to set the PMF or PDF (which ever applies) equal to 1. We then solve for the constant value. What does this tell us? I means that if that constant was any other value.. then the function in question would no longer be defined as a PDF or CDF.