SMaC: Statistics, Math, and Computing
APSTA-GE 2006: Applied Statistics for Social Science Research
Random Variables Are Not Random
It would be inconvenient to enumerate all possible events to describe a stochastic system
A random variable is a measurable function \(X:S\to\mathbb{R}\) that maps the sample space \(S\) into the real line
For each possible outcome \(s\in S\), the random variable assigns the value \(X(s)\)
This mapping is deterministic. The randomness comes from the experiment, not from the random variable (RV)
While \(\P(A)\) is defined for an event \(A\), \(\P(X)\) is not meaningful. The event \(\{s\in S:X(s)=x\}\) is abbreviated as \(\{X=x\}\), so we write \(\P(X=x)\)
Let \(X\) be the number of heads in two coin flips. If the outcome is \(s=HH\), then \(X(s)=2\), while \(S=\{TT,TH,HT,HH\}\)
- Random variable \(X\) for the number of heads in two flips
Geometric RV
- The geometric distribution is a discrete waiting-time distribution; the exponential distribution is its continuous analog
- Let \(X\) be the number of failures before the first success. Then \(X\sim\operatorname{Geometric}(\theta)\), where \(0<\theta\le1\) is the probability of success on each independent trial
- Example: We flip a fair coin until the first head appears
- If the sequence is T T T T H, then \(X=4\)
- The probability of this sequence is \((1/2)^4(1/2)=1/32\)
- The PMF is \(\P(X=x)=(1-\theta)^x\theta\) for \(x=0,1,2,\ldots\)
- To check if this is a valid PMF, we need to sum over all \(x\):
\[ \begin{align} \sum_{x = 0}^{\infty} \theta (1 - \theta)^x = \theta \sum_{x = 0}^{\infty} (1 - \theta)^x \\ \text{Let } u = 1 - \theta \\ \theta \sum_{x = 0}^{\infty} u^x = \theta \frac{1}{1-u} = \theta \frac{1}{1-1 + \theta} = \frac{\theta}{\theta} = 1 \end{align} \]
- The final step uses the geometric-series formula for \(|u|<1\)
- For the sequence T T T H, \(x=3\) failures. When \(\theta=1/2\), its probability is \((1/2)^4=1/16\)
- If \(\theta=1/3\), the probability of the same sequence is \((2/3)^3(1/3)=8/81\)
- The support is unbounded, the PMF tends to 0 as \(x\to\infty\), and the probabilities sum to 1
For \(|u|<1\), let \(S\) denote the infinite geometric series. Multiplying by \(u\) shifts every term one position:
\[ \begin{aligned} S &= 1+u+u^2+u^3+\cdots,\\ uS &= \phantom{1+{}}u+u^2+u^3+\cdots. \end{aligned} \]
Subtracting the second line from the first cancels the infinite tail:
\[ S-uS=1 \quad\Longrightarrow\quad \boxed{S=\frac{1}{1-u}}. \]
Poisson PMF
- Notice the location of \(\E(X) = \lambda\) in each plot
Continuous RVs and the Uniform
- We leave the discrete world and enter continuous RVs
- For a continuous RV, every point probability satisfies \(\P(X=x)=0\), so point probabilities do not characterize the distribution
- We use PDFs and obtain probabilities over intervals by integrating the density
- A PDF satisfies \(f_X(x)\ge0\) and \(\int_{-\infty}^{\infty}f_X(x)\,dx=1\)
\[ \begin{aligned} \P(a<X<b)&=\int_a^b f_X(x)\,dx,\\ F_X(x)&=\int_{-\infty}^{x}f_X(u)\,du. \end{aligned} \]
If \(X\sim\operatorname{Uniform}(\alpha,\beta)\) with \(\alpha<\beta\), its PDF is
\[ f_X(x)= \begin{cases} \dfrac{1}{\beta-\alpha}, & \alpha\le x\le\beta,\\[4pt] 0, & \text{otherwise}. \end{cases} \]
- Your Turn: Guess \(\E(X)\)
- Now derive it from \(\E(X)=\int_{\alpha}^{\beta}x f_X(x)\,dx\)
Normal RV
- If \(X\sim\operatorname{Normal}(\mu,\sigma)\), with \(\mu\in\mathbb{R}\) and \(\sigma>0\), then for \(x\in\mathbb{R}\) its PDF is
\[ f_X(x)=\frac{1}{\sqrt{2\pi}\,\sigma} \exp\!\left[-\frac12\left(\frac{x-\mu}{\sigma}\right)^2\right]. \]
- The negative quadratic in the exponent produces the bell shape
- The mean, median, and mode are all \(\mu\)
- The variance is \(\V(X)=\sigma^2\), and the standard deviation is \(\sigma\)
- Standardizing subtracts the mean and divides by the standard deviation
\[ Z=\frac{X-\mu}{\sigma}\sim\operatorname{Normal}(0,1), \qquad f_Z(z)=\frac{1}{\sqrt{2\pi}}e^{-z^2/2}. \]
Example: Heights of US Adults
- Under standard central-limit conditions, a standardized sum tends toward \(\operatorname{Normal}(0,1)\); equivalently, a large sum is approximately normal with its own mean and standard deviation
- Human height reflects many small contributions and is often approximately normal within groups
- The following values come from Teaching Statistics: A Bag of Tricks by Gelman and Nolan
mu_m <- 69.1 # mean height of US men in inches
sigma_m <- 2.9 # corresponding standard deviation
mu_w <- 63.7 # mean height of US women in inches
sigma_w <- 2.7 # corresponding standard deviation
x <- seq(50, 80, length.out = 1e3)
pdf_m <- dnorm(x, mean = mu_m, sd = sigma_m)
pdf_w <- dnorm(x, mean = mu_w, sd = sigma_w)
p <- ggplot(data.frame(x, pdf_m, pdf_w), aes(x, pdf_m))
p <- p + geom_line(linewidth = 0.2, color = "red") +
geom_line(aes(y = pdf_w), linewidth = 0.2, color = "blue") +
xlab("Height (in)") + ylab("") +
ggtitle("Distribution of heights of US adults") +
annotate("text", x = 73.5, y = 0.10, label = "Men", color = "red") +
annotate("text", x = 58, y = 0.10, label = "Women", color = "blue") +
theme(axis.text.y = element_blank(),
axis.ticks.y = element_blank())
print(p)
A Mixture of Normal Distributions
- A mixture of normal distributions is not generally normal, even when each component is normal
set.seed(2006)
n_m <- as.integer(1e5 * 0.48)
n_w <- as.integer(1e5 * 0.52)
h_m <- rnorm(n_m, mu_m, sigma_m)
h_w <- rnorm(n_w, mu_w, sigma_w)
h <- c(h_m, h_w)
p <- ggplot(data.frame(h), aes(x = h))
p + geom_density(linewidth = 0.2, bw = 0.5) +
xlab("Height (in)") + ylab("") + ylim(0, 0.15) +
ggtitle("Mixture of men's and women's height distributions") +
theme(
axis.text.y = element_blank(),
axis.ticks.y = element_blank()
)