📐 Concept diagram

Phase 10: Probability Theory

Subject 10-07: Continuous Random Variables

Prerequisites: 10-04 (Discrete Random Variables), 10-06 (Expectation), basic calculus (integration)

Learning Objectives

1. Define a continuous random variable via its probability density function (PDF) and cumulative distribution function (CDF)
2. Compute probabilities for continuous RVs using integration: P(a < X < b) = ∫ₐᵇ f(x) dx
3. State and apply the uniform distribution on [a, b] and the exponential distribution
4. Derive the CDF from the PDF by integration and recover the PDF by differentiation
5. Explain why P(X = c) = 0 for continuous RVs and why PDF values CAN exceed 1

Core Content

1. Definition: Continuous Random Variables

A random variable X is continuous if its CDF F_X(x) = P(X ≤ x) can be expressed as:

$F_X(x) = ∫_{−∞}^{x} f_X(t) dt
$

where f_X(x) is called the probability density function (PDF).

Properties of a valid PDF: 1. Non-negativity: f_X(x) ≥ 0 for all x 2. Normalization: ∫_{−∞}^{∞} f_X(x) dx = 1 3. Probability computation: P(a < X < b) = P(a ≤ X < b) = P(a < X ≤ b) = ∫ₐᵇ f_X(x) dx

Critical insight: For continuous RVs, P(X = c) = 0 for any single point c. This is because:

$P(X = c) = ∫_{c}^{c} f(x) dx = 0
$

Including or excluding endpoints never matters for continuous RVs.

Common misconception: "f_X(x) is the probability that X equals x." This is FALSE. The PDF at a point is NOT a probability — it's a density. Probabilities come from AREAS under the PDF. The PDF can exceed 1 (e.g., Uniform(0, 0.1) has PDF = 10 on [0, 0.1]).

CDF properties (continuous case): - F_X(x) is continuous everywhere (unlike discrete RVs where it has jumps) - F_X(x) is non-decreasing - lim_{x→−∞} F(x) = 0, lim_{x→∞} F(x) = 1 - f_X(x) = F'_X(x) wherever the derivative exists

2. Expectation and Variance for Continuous RVs

For a continuous RV X with PDF f(x):

Expected value:

$E[X] = ∫_{−∞}^{∞} x f(x) dx
$

(provided the integral converges absolutely)

LOTUS (continuous):

$E[g(X)] = ∫_{−∞}^{∞} g(x) f(x) dx
$

Variance:

$Var(X) = E[(X − μ)²] = ∫_{−∞}^{∞} (x − μ)² f(x) dx = E[X²] − (E[X])²
$

3. Uniform Distribution

X is uniformly distributed on [a, b] if its PDF is constant over that interval.

Notation: X ~ Uniform(a, b) or U(a, b)

PDF:

$f(x) = { 1/(b−a),   a ≤ x ≤ b
       { 0,          otherwise
$

CDF:

$F(x) = { 0,                       x < a
       { (x − a)/(b − a),         a ≤ x < b
       { 1,                       x ≥ b
$

Mean and Variance:

$E[X] = (a + b)/2    (midpoint — by symmetry)
Var(X) = (b − a)² / 12
$

Derivation of variance: E[X²] = ∫ₐᵇ x²/(b−a) dx = [x³/(3(b−a))]ₐᵇ = (b³−a³)/(3(b−a)) = (b²+ab+a²)/3 Var(X) = E[X²] − (E[X])² = (b²+ab+a²)/3 − ((a+b)/2)² = (b²+ab+a²)/3 − (a²+2ab+b²)/4 = (b−a)²/12

Key property: For X ~ U(0, 1), P(X ∈ [c, d]) = d − c for 0 ≤ c < d ≤ 1. The probability is proportional to length.

Universality of Uniform: If U ~ Uniform(0, 1) and F is any continuous CDF with inverse F⁻¹, then X = F⁻¹(U) has CDF F. This is the basis of inverse transform sampling (generating any distribution from a uniform random number generator).

4. Exponential Distribution

X has an exponential distribution with rate parameter λ > 0.

Notation: X ~ Exponential(λ) or Exp(λ)

PDF:

$f(x) = { λ e^{−λx},   x ≥ 0
       { 0,            x < 0
$

CDF:

$F(x) = { 1 − e^{−λx},   x ≥ 0
       { 0,              x < 0
$

Survival function: P(X > x) = e^{−λx}.

Mean and Variance:

$E[X] = 1/λ
Var(X) = 1/λ²
$

Derivation of mean (integration by parts):

$E[X] = ∫₀^{∞} x λ e^{−λx} dx
$

Let u = x, dv = λe^{−λx}dx. Then du = dx, v = −e^{−λx}.

$E[X] = [−x e^{−λx}]₀^{∞} + ∫₀^{∞} e^{−λx} dx = 0 + [−e^{−λx}/λ]₀^{∞} = 0 − (−1/λ) = 1/λ
$

Memoryless property (continuous): P(X > s + t | X > s) = P(X > t) = e^{−λt}. The exponential is the ONLY continuous distribution with this property.

Connection to Poisson process: As shown in 10-05, inter-arrival times in a Poisson process of rate λ are i.i.d. Exponential(λ).

Hazard rate: h(t) = f(t) / (1 − F(t)) = λ e^{−λt} / e^{−λt} = λ (constant). The exponential has constant hazard — the risk doesn't increase or decrease with time. This makes it appropriate for modeling lifetimes of items that don't "age."

Edge case: The exponential is a special case of the Gamma distribution: Exponential(λ) = Gamma(1, λ).

Key Terms

• 10 07 Continuous Random Variables
• 10-08 Normal (Gaussian) Distribution
• Answer: b.
• Answer: c.
• Answer: d.
• CDF of a continuous RV
• PDF at a point
• Subject 10-07: Continuous Random Variables
• continuous
• probability density function (PDF)

Worked Examples

Example 1: PDF from CDF

Let X have CDF F(x) = 1 − 1/x² for x ≥ 1, and 0 for x < 1.

(a) Find the PDF. (b) Verify it's a valid PDF. (c) Find P(2 < X < 4). (d) Find the median.

Solution:

(a) f(x) = F'(x) = d/dx (1 − 1/x²) = 2/x³ for x ≥ 1, 0 otherwise.

(b) ∫₁^{∞} 2/x³ dx = [−1/x²]₁^{∞} = 0 − (−1) = 1 ✓. Also f(x) ≥ 0 for x ≥ 1 ✓.

(c) P(2 < X < 4) = F(4) − F(2) = (1−1/16) − (1−1/4) = 15/16 − 3/4 = (15−12)/16 = 3/16 = 0.1875.

(d) Median m satisfies F(m) = 1/2. 1 − 1/m² = 1/2 → 1/m² = 1/2 → m = √2 ≈ 1.414.

Example 2: Exponential Waiting Time

A lightbulb's lifetime T ~ Exponential(λ = 0.001 per hour).

(a) What is the mean lifetime? (b) P(T > 2000 hours)? (c) P(500 < T < 1500)? (d) Given it has lasted 1000 hours, P(it lasts at least another 500)?

Solution:

(a) E[T] = 1/λ = 1/0.001 = 1000 hours.

(b) P(T > 2000) = e^{−0.001·2000} = e^{−2} ≈ 0.1353.

(c) P(500 < T < 1500) = F(1500) − F(500) = (1−e^{−1.5}) − (1−e^{−0.5}) = e^{−0.5} − e^{−1.5} ≈ 0.6065 − 0.2231 = 0.3834.

(d) By memoryless property: P(T > 1500 | T > 1000) = P(T > 500) = e^{−0.5} ≈ 0.6065. The additional lifetime is independent of the time already survived.

Example 3: Expectation of a Function via LOTUS

Let X ~ Uniform(0, 2). Find E[X³] and E[e^X].

Solution:

f(x) = 1/2 for 0 ≤ x ≤ 2.

E[X³] = ∫₀² x³ · (1/2) dx = (1/2) [x⁴/4]₀² = (1/2)(16/4) = 2.

E[e^X] = ∫₀² eˣ · (1/2) dx = (1/2)[eˣ]₀² = (e² − 1)/2 ≈ 3.1945.

Check: Is E[X³] = (E[X])³? E[X] = 1, 1³ = 1 ≠ 2. LOTUS ≠ naive substitution!

Quiz

Q1: For a continuous random variable X with PDF f(x), which statement is TRUE?

A) f(x) gives the probability that X equals x B) P(X = c) = 0 for any single point c C) f(x) can never exceed 1 D) P(a < X < b) = f(b) − f(a)

Correct: B)

• If you chose B: Correct! For continuous RVs, the probability at any single point is zero because ∫ₓˣ f(t) dt = 0. Probabilities come from areas under the PDF.
• If you chose A: The PDF at a point is a DENSITY, not a probability. It can exceed 1 (e.g., Uniform(0, 0.1) has PDF = 10).
• If you chose C: PDFs CAN exceed 1 — only the integral must equal 1, not the function values.
• If you chose D: P(a < X < b) = F(b) − F(a), not f(b) − f(a). The CDF gives probabilities, not the PDF directly.

Q2: X ~ Uniform(a, b). Its PDF is:

A) (b−a) on [a, b] B) 1/(b−a) on [a, b] C) x/(b−a) on [a, b] D) e^{−(x−a)/(b−a)}/(b−a)

Correct: B)

• If you chose B: Correct! The uniform PDF is constant at height 1/(b−a) so that the total area = (b−a) × 1/(b−a) = 1.
• If you chose A: This would give area (b−a)², not 1.
• If you chose C: This is a linear function, not constant.
• If you chose D: This is an exponential-like form, not uniform.

Q3: The memoryless property P(T > s + t | T > s) = P(T > t) characterizes which continuous distribution?

A) Normal B) Uniform C) Exponential D) Gamma

Correct: C)

• If you chose C: Correct! The Exponential(λ) is the ONLY continuous distribution with the memoryless property. P(T > s+t | T > s) = e^{−λt} = P(T > t).
• If you chose A: The normal distribution has memory — the hazard rate increases over time.
• If you chose B: The uniform distribution has increasing hazard rate (finite support).
• If you chose D: Gamma has the memoryless property only when α = 1 (i.e., it reduces to exponential).

Q4: For X ~ Exponential(λ), the CDF is F(x) = 1 − e^{−λx} for x ≥ 0. What is E[X]?

A) λ B) 1/λ C) λ² D) 1/λ²

Correct: B)

• If you chose B: Correct! E[Exponential(λ)] = 1/λ. Higher rate λ means shorter expected waiting time.
• If you chose A: This is E[Poisson(λ)], not Exponential.
• If you chose C: This is Var(Poisson(λ))... no, that's also λ. λ² is incorrect.
• If you chose D: This is Var(Exponential(λ)) = 1/λ².

Q5: The relationship between PDF f(x) and CDF F(x) is:

A) f(x) = F(x) B) f(x) = F'(x) where differentiable C) F(x) = f'(x) D) f(x) = 1/F(x)

Correct: B)

• If you chose B: Correct! The PDF is the derivative of the CDF: f(x) = F'(x). Conversely, F(x) = ∫₋∞ˣ f(t) dt.
• If you chose A: These are different functions — F(x) accumulates, f(x) is the rate of accumulation.
• If you chose C: This is backwards — the CDF is the integral of the PDF.
• If you chose D: There's no such relationship.

Q6: For X ~ Uniform(a, b), E[X] equals:

A) (b−a)/2 B) (a+b)/2 C) ab/2 D) √(ab)

Correct: B)

• If you chose B: Correct! The mean of a uniform distribution is the midpoint of the interval: (a+b)/2.
• If you chose A: This is half the width, not the midpoint.
• If you chose C: This is half the product — not a location measure.
• If you chose D: This is the geometric mean, not the arithmetic mean.

Practice Problems

1. Verify that f(x) = (3/8)x² for 0 ≤ x ≤ 2 is a valid PDF. Find the CDF.

2. Let X ~ Uniform(0, 10). Find: (a) P(3 < X < 7), (b) P(X > 8 | X > 5), (c) the 90th percentile.

3. Show that the exponential PDF integrates to 1. Then find the median in terms of λ.

4. Let T ~ Exponential(0.5). Find E[T], Var(T), P(T > 3), and P(T < 1).

5. A continuous RV X has CDF F(x) = x² for 0 ≤ x ≤ 1. Find the PDF, E[X], and Var(X).

6. If X ~ Uniform(a, b), derive the formula for P(c < X < d) where a ≤ c < d ≤ b.

7. Show that for an exponential RV with rate λ, the ratio P(T > t+s)/P(T > s) is independent of s, proving the memoryless property.

Summary

• A continuous RV has PDF f(x) ≥ 0 with ∫ f(x)dx = 1; probabilities are areas: P(a<X<b)=∫ₐᵇf(x)dx
• The CDF F(x)=∫_{−∞}ˣ f(t)dt is continuous; P(X=c)=0 for any c for continuous RVs, and the PDF CAN exceed 1 (it's density, not probability)
• Uniform(a,b) has constant PDF 1/(b−a), mean (a+b)/2, variance (b−a)²/12 — probability is proportional to interval length
• Exponential(λ) has PDF λe^{−λx} for x≥0, mean 1/λ, variance 1/λ², and the continuous memoryless property — inter-arrival times in a Poisson process
• LOTUS extends to continuous RVs: E[g(X)] = ∫ g(x) f(x) dx

Pitfalls

• Treating the PDF value f(x) as a probability. f(x) is a DENSITY, not P(X = x). For continuous RVs, P(X = c) = 0 for any point c. Probabilities come from integrating the PDF over an interval. The PDF CAN exceed 1 — e.g., Uniform(0, 0.1) has PDF = 10 everywhere on its support.
• Including or excluding endpoints in continuous probability calculations. For continuous RVs, P(a < X < b) = P(a ≤ X < b) = P(a < X ≤ b) because P(X = a) = P(X = b) = 0. You don't need to worry about strict vs. non-strict inequalities.
• Confusing the PDF and CDF. The CDF F(x) gives cumulative probability P(X ≤ x); the PDF f(x) = F'(x) gives the rate of change. To find interval probabilities, use F(b) − F(a), not f(b) − f(a). The PDF alone cannot answer "what is P(X < 3)?" — you need the area under it.
• Misapplying the memoryless property. Only the exponential distribution (continuous) and geometric distribution (discrete) have the memoryless property. If someone tells you a uniform or normal lifetime has the memoryless property, they're wrong — those distributions "age."
• Forgetting to check that a proposed PDF integrates to 1. Every time you derive or guess a PDF, always verify ∫ f(x) dx = 1. Missing normalization constants are one of the most common errors when working with continuous distributions.

Quiz

1. If X is a continuous RV, P(X = 3.5) = ? a) f(3.5) b) F(3.5) c) 0 d) 0.5 Answer: c. For any continuous RV, the probability at a single point is zero — it's an integral over a zero-width interval.

2. The PDF of a continuous RV tells you: a) The probability at each point b) The relative likelihood (density) — probabilities come from integrating it c) The cumulative probability d) The variance directly Answer: b. f(x) is density, not probability. P(x₁ < X < x₂) = ∫_{x₁}^{x₂} f(x) dx.

3. For X ~ Uniform(0, 4), P(1 ≤ X ≤ 3) = ? a) 1/4 b) 1/2 c) 3/4 d) 1 Answer: b. The interval length is 2 out of total length 4: 2/4 = 1/2.

4. The memoryless property of the exponential distribution means: a) f(s + t) = f(s) f(t) b) P(X > s + t | X > s) = P(X > t) c) The mean does not depend on the parameter d) The variance is one Answer: b. Conditional survival beyond s+t given survival beyond s equals unconditional survival beyond t.

5. Which CANNOT happen for a valid PDF? a) f(x) > 1 for some x b) f(x) = 0 for some x c) ∫ f(x) dx = 1 d) f(x) < 0 for some x Answer: d. A PDF must be non-negative everywhere. But it CAN exceed 1 — e.g., Uniform(0, 0.1) has density 10.

6. The CDF of a continuous RV is: a) Always a step function b) Always continuous c) Always greater than the PDF d) Always less than 1 Answer: b. For continuous RVs, the CDF is continuous. Discrete RVs have step-function CDFs.

7. If X ~ Exponential(λ), the hazard rate (instantaneous failure rate) is: a) Increasing in t b) Decreasing in t c) Constant = λ d) Zero Answer: c. h(t) = f(t)/(1−F(t)) = λe^{−λt}/e^{−λt} = λ. Constant hazard means the item doesn't "age."

8. For X ~ Uniform(0, θ), Var(X) = ? a) θ²/3 b) θ²/12 c) θ/2 d) θ²/4 Answer: b. Var(Uniform(0,θ)) = (θ−0)²/12 = θ²/12.

Next Steps

Continue to 10-08 Normal (Gaussian) Distribution to learn about the standard normal, general normal, Z-scores, the empirical rule, MGF, and the sum of independent normals.