📐 Concept diagram

Phase 10: Probability Theory

Subject 10-05: Poisson Process and Distribution

Prerequisites: 10-04 (Discrete Random Variables) — binomial, geometric distributions; limits

Learning Objectives

1. Derive the Poisson distribution as the limit of the binomial distribution when n → ∞, p → 0, with np = λ fixed
2. State and apply the Poisson PMF P(X = k) = λᵏ e^{−λ} / k! for modeling rare-event counts
3. Define the Poisson process and prove the distribution of counts follows Poisson(λt)
4. Derive the exponential distribution of inter-arrival times from the Poisson process and explain the memoryless property
5. Apply Poisson models to real-world phenomena: radioactive decay, customer arrivals, website hits, rare disease incidence

Core Content

1. Derivation: Poisson as the Limit of the Binomial

Consider Xₙ ~ Binomial(n, pₙ) where pₙ = λ/n for some fixed λ > 0 (so the expected number of events is constant at λ).

Theorem: As n → ∞, the Binomial(n, λ/n) PMF converges to the Poisson(λ) PMF.

Proof: For fixed k:

P(Xₙ = k) = C(n, k) (λ/n)ᵏ (1 − λ/n)^{n−k}

           = [n(n−1)...(n−k+1) / k!] · (λᵏ/nᵏ) · (1 − λ/n)ⁿ · (1 − λ/n)^{−k}

Break into factors:

1. n(n−1)...(n−k+1) / nᵏ → 1 as n → ∞ (for fixed k)
2. (1 − λ/n)ⁿ → e^{−λ} (classic limit: lim_{n→∞} (1 + x/n)ⁿ = eˣ)
3. (1 − λ/n)^{−k} → 1 as n → ∞ (for fixed k)

Therefore:

$lim_{n→∞} P(Xₙ = k) = (λᵏ / k!) · 1 · e^{−λ} · 1 = λᵏ e^{−λ} / k!
$

This is the Poisson PMF.

Notation: X ~ Poisson(λ)

When to use Poisson: Modeling counts of rare events in large populations — when n is large, p is small, and the events are approximately independent. The parameter λ = np is the expected count.

2. Poisson Distribution Properties

PMF:

P(X = k) = λᵏ e^{−λ} / k!    for k = 0, 1, 2, ...

Verification of summing to 1:

$Σ_{k=0}^{∞} λᵏ e^{−λ} / k! = e^{−λ} Σ_{k=0}^{∞} λᵏ/k! = e^{−λ} · e^{λ} = 1
$

Support: All non-negative integers {0, 1, 2, ...}

Key properties: - Mean: E[X] = λ - Variance: Var(X) = λ - Mode: ⌊λ⌋ (and λ−1 if λ is an integer) - Poisson is unique in having mean = variance

Shape: For small λ, PMF is skewed right with mode at 0. As λ grows, the distribution becomes more symmetric and bell-shaped (approaches Normal by CLT).

Additivity: If X₁ ~ Poisson(λ₁) and X₂ ~ Poisson(λ₂) are independent, then X₁ + X₂ ~ Poisson(λ₁ + λ₂).

Proof: Using generating functions, M_{X₁+X₂}(t) = M_{X₁}(t) M_{X₂}(t) = exp(λ₁(eᵗ−1)) · exp(λ₂(eᵗ−1)) = exp((λ₁+λ₂)(eᵗ−1)), which is the MGF of Poisson(λ₁+λ₂).

3. The Poisson Process

A Poisson process with rate λ > 0 is a counting process {N(t) : t ≥ 0} where N(t) is the number of events occurring in [0, t], satisfying:

1. N(0) = 0 (start at zero)
2. Independent increments: For disjoint intervals, the counts are independent
3. Stationary increments: The distribution of N(t) − N(s) depends only on t − s
4. Orderliness: P(N(h) = 1) = λh + o(h) and P(N(h) ≥ 2) = o(h) as h → 0

From these axioms, one can prove:

$N(t) ~ Poisson(λt)
$

Derivation sketch: Partition [0, t] into n subintervals of length t/n. As n → ∞, each subinterval can have at most one event (orderliness). The probability of an event in any subinterval is approximately λ·(t/n) = λt/n. The number of events is approximately Binomial(n, λt/n), which converges to Poisson(λt). The independence of increments gives independence across subintervals.

Applications: - Radioactive decay: λ = decay events per second - Call center: λ = calls per minute - Website traffic: λ = hits per second - Earthquake occurrences (with caveats — earthquakes may cluster)

4. Inter-Arrival Times: Exponential Distribution

Let T₁ be the time until the first event in a Poisson process with rate λ. Then:

P(T₁ > t) = P(N(t) = 0) = (λt)⁰ e^{−λt} / 0! = e^{−λt}

Therefore the CDF of T₁ is:

$F_{T₁}(t) = P(T₁ ≤ t) = 1 − e^{−λt},    t ≥ 0
$

This is the CDF of an Exponential(λ) random variable. Its PDF is:

$f_{T₁}(t) = λ e^{−λt},    t ≥ 0
$

More generally, all inter-arrival times T₁, T₂, T₃, ... (time between consecutive events) are i.i.d. Exponential(λ).

Memoryless property of Exponential:

P(T > s + t | T > s) = P(T > t) = e^{−λt}

Proof: P(T > s + t | T > s) = P(T > s + t) / P(T > s) = e^{−λ(s+t)} / e^{−λs} = e^{−λt} = P(T > t).

The exponential distribution is the ONLY continuous distribution with the memoryless property (just as the geometric is the only discrete one).

Relationship to Gamma: The waiting time until the r-th event, T₁ + T₂ + ... + Tᵣ, follows a Gamma(r, λ) distribution. This generalizes the exponential.

5. Poisson Approximation to Binomial

When n is large and p is small, Binomial(n, p) ≈ Poisson(np). A common rule of thumb: n ≥ 20 and p ≤ 0.05 (or n ≥ 100 and np ≤ 10).

Example: n = 1000, p = 0.002. Binomial calculation of P(X ≤ 2) requires summing large binomial coefficients. Poisson with λ = 2: P(X ≤ 2) = e^{−2}(1 + 2 + 2²/2) = e^{−2}(1 + 2 + 2) = 5e^{−2} ≈ 0.6767.

Key Terms

• Poisson PMF
• Poisson process

Worked Examples

Example 1: Poisson Limit of Binomial

Emails arrive from a list of 10,000 subscribers. Each independently opens an email with probability 0.0003. What is the probability that exactly 2 people open it? At most 2?

Solution:

n = 10000, p = 0.0003, λ = np = 3.

Using Poisson(3):

$P(X=2) = 3² e^{−3} / 2! = 9 e^{−3} / 2 ≈ 9 · 0.04979 / 2 ≈ 0.2240
$

Using exact binomial: C(10000,2)(0.0003)²(0.9997)^{9998} ≈ 0.2240 (very close).

P(X ≤ 2) = e^{−3}(1 + 3 + 9/2) = e^{−3} · 8.5 ≈ 0.4232.

Example 2: Poisson Process — Call Center

Calls arrive at a call center according to a Poisson process with rate λ = 4 calls per minute.

(a) Probability of exactly 6 calls in 2 minutes? (b) Probability of no calls in 30 seconds? (c) Time until the next call exceeds 45 seconds? (d) Given no calls in the first minute, probability of no calls in the next 30 seconds?

Solution:

(a) t = 2 minutes, λt = 8. P(N(2)=6) = 8⁶ e^{−8} / 6! = 262144 e^{−8} / 720 ≈ 122.1 e^{−8} ≈ 0.1221.

(b) t = 0.5, λt = 2. P(N(0.5)=0) = e^{−2} ≈ 0.1353.

(c) T ~ Exponential(4). P(T > 0.75) = e^{−4·0.75} = e^{−3} ≈ 0.04979.

(d) By memoryless property: P(T > 1.5 | T > 1) = P(T > 0.5) = e^{−4·0.5} = e^{−2} ≈ 0.1353.

Example 3: Additivity of Poisson

A website receives requests from two independent sources: Source A ~ Poisson(3) per minute, Source B ~ Poisson(2) per minute. Find: (a) P(total requests = 0 per minute), (b) P(exactly 1 request from A given total is 1).

Solution:

Total X = X_A + X_B ~ Poisson(5).

(a) P(X=0) = e^{−5} ≈ 0.00674

(b) P(X_A=1 | X=1) = P(X_A=1, X_B=0) / P(X=1) = [P(X_A=1) P(X_B=0)] / P(X=1) = [(3e^{−3})(e^{−2})] / (5e^{−5}) = 3/5 = 0.6.

Interestingly, given the total is 1, the source is A with probability proportional to its rate: 3/(3+2) = 3/5.

Quiz

Q1: The Poisson(λ) distribution arises as the limit of which distribution?

A) Geometric(p) as p → 0 B) Hypergeometric(N, K, n) as N → ∞ C) Binomial(n, p) as n → ∞, p → 0 with np = λ D) Negative Binomial(r, p) as r → ∞

Correct: C)

• If you chose C: Correct! When n is large and p is small but np stays constant at λ, the binomial PMF converges to λᵏ e^{−λ}/k!. This is the law of rare events.
• If you chose A: Geometric converges to Exponential in the continuous limit, not Poisson.
• If you chose B: Hypergeometric converges to Binomial (not Poisson) as N → ∞.
• If you chose D: Negative Binomial relates to Gamma, not Poisson in limits.

Q2: For X ~ Poisson(λ), which statement about its mean and variance is correct?

A) E[X] = λ, Var(X) = λ² B) E[X] = λ, Var(X) = λ C) E[X] = 1/λ, Var(X) = 1/λ D) Mean and variance are unrelated

Correct: B)

• If you chose B: Correct! The Poisson is uniquely characterized (among non-degenerate distributions) by having mean = variance = λ.
• If you chose A: Var(X) = λ² would imply the standard deviation equals the mean, which is not true for Poisson.
• If you chose C: This is the mean and variance of Exponential(λ), not Poisson.
• If you chose D: The equality of mean and variance is a defining Poisson property.

Q3: In a Poisson process with rate λ, the distribution of N(t), the count in time interval [0, t], is:

A) Poisson(λ) B) Poisson(λt) C) Exponential(λt) D) Binomial(t, λ)

Correct: B)

• If you chose B: Correct! The expected count is λ × t, so N(t) ~ Poisson(λt). The rate λ is per unit time; multiplying by t gives the parameter.
• If you chose A: This would be correct only for t = 1.
• If you chose C: Exponential is the distribution of inter-arrival times, not counts.
• If you chose D: N(t) is Poisson, not Binomial (though the process can be derived from a binomial limit).

Q4: The inter-arrival times in a Poisson process of rate λ follow:

A) Poisson(λ) B) Exponential(λ) C) Uniform(0, 1/λ) D) Normal(λ, λ)

Correct: B)

• If you chose B: Correct! T ~ Exponential(λ) with PDF λe^{−λt} for t ≥ 0. The exponential is the continuous analog of the geometric distribution.
• If you chose A: Poisson is for counts, not waiting times.
• If you chose C: Waiting times in a Poisson process are not uniform.
• If you chose D: Normal can take negative values; waiting times are non-negative.

Q5: The memoryless property of the exponential distribution is expressed as:

A) E[T] = 1/λ B) P(T > s + t | T > s) = P(T > t) C) T forgets its distribution D) Var(T) = 1/λ²

Correct: B)

• If you chose B: Correct! Given survival to time s, the probability of surviving an additional t is the same as the unconditional probability of surviving t. The exponential is the ONLY continuous distribution with this property.
• If you chose A: This is a true property but not the memoryless property.
• If you chose C: This is an anthropomorphic restatement, not the mathematical definition.
• If you chose D: This is the variance of Exponential(λ), not memorylessness.

Q6: If X₁ ~ Poisson(3) and X₂ ~ Poisson(2) are independent, then X₁ + X₂ ~:

A) Poisson(5) B) Poisson(6) C) Poisson(1) D) Approximately Poisson(2.5)

Correct: A)

• If you chose A: Correct! Independent Poisson random variables are additive: Poisson(λ₁) + Poisson(λ₂) = Poisson(λ₁ + λ₂). So 3 + 2 = 5.
• If you chose B: This would require multiplying rates, not adding them.
• If you chose C: This would be the difference, not the sum.
• If you chose D: This is the average, not the sum.

Q7: Which is NOT an assumption of the Poisson process?

A) N(0) = 0 B) Independent increments C) Stationary increments D) Events occur at regular fixed intervals

Correct: D)

• If you chose D: Correct! Poisson processes have RANDOM event times — the orderliness property only says that simultaneous events have probability zero, not that events are regularly spaced.
• If you chose A: The process starts at zero — a fundamental assumption.
• If you chose B: Counts in disjoint intervals are independent.
• If you chose C: The distribution depends only on interval length, not absolute time.

Practice Problems

1. Show that Σ_{k=0}^{∞} λᵏ e^{−λ} / k! = 1.

2. Let X ~ Poisson(4). Find P(X = 0), P(X = 3), P(X ≥ 5), and the mode.

3. Derive the exponential PDF from its CDF: f(t) = d/dt (1 − e^{−λt}) = λe^{−λt} for t ≥ 0.

4. A radioactive source emits alpha particles at a rate of 2 per second (Poisson process). Find: (a) P(exactly 3 particles in 2 seconds), (b) P(at least 1 particle in 0.5 seconds), (c) P(wait > 1 second for first particle).

5. Typographical errors in a book occur at a rate of 0.5 per page. What is the probability a 300-page book has fewer than 140 errors? (Use Normal approximation to Poisson or compute bounds.)

6. If X ~ Poisson(λ), show using the Taylor series for eˣ that the PMF sums to 1. Then derive the recurrence P(X = k+1) = [λ/(k+1)] P(X = k).

7. A Poisson process has rate λ = 5 events per hour. What is the distribution of the time between events? What is the expected number of events in 3 hours?

Summary

• The Poisson(λ) distribution (PMF: λᵏ e^{−λ} / k!) arises as the limit of Binomial(n, λ/n) as n → ∞, modeling counts of rare independent events
• A Poisson process with rate λ has N(t) ~ Poisson(λt), counts with independent stationary increments, and the orderliness property (at most one event per infinitesimal interval)
• Inter-arrival times in a Poisson process are i.i.d. Exponential(λ) with the continuous memoryless property P(T > s+t | T > s) = P(T > t)
• Poisson distributions are additive: sum of independent Poissons is Poisson with summed rates
• The Poisson approximates the binomial when n is large and p is small (typically n ≥ 20, p ≤ 0.05, or np ≤ 10)

Pitfalls

• Forgetting to scale λ by the time interval t in a Poisson process. The count N(t) is Poisson(λt), not Poisson(λ). If the rate is λ per unit time and you observe an interval of length t, the parameter is λt. A 2-minute window at rate 4/min has λt = 8, not 4.
• Confusing the Poisson distribution (for counts) with the Exponential distribution (for waiting times). In a Poisson process, the number of events N(t) ~ Poisson(λt), but the time until the NEXT event T ~ Exponential(λ). They answer different questions: "how many?" vs. "how long?".
• Applying the Poisson approximation to the binomial blindly. The approximation Binomial(n, p) ≈ Poisson(np) requires both n large AND p small (rule of thumb: n ≥ 20, p ≤ 0.05). For moderate p (e.g., p = 0.3), the binomial is distinctly symmetric and the Poisson approximation fails.
• Forgetting that the Poisson mean equals its variance. Poisson(λ) has E[X] = Var(X) = λ. If you estimate λ from data as the sample mean, the variance is NOT a free parameter — a sample variance much different from the mean suggests Poisson is the wrong model.
• Treating events in a Poisson process as evenly spaced. The orderliness property only guarantees no simultaneous events; it does NOT mean events are regular. Inter-arrival times are random (exponential), so clusters and gaps both occur naturally.

Quiz

1. The Poisson distribution is a limiting case of which distribution? a) Geometric b) Hypergeometric c) Binomial d) Uniform Answer: c. Binomial(n, p) → Poisson(λ) when n → ∞, p → 0 with np = λ.

2. For X ~ Poisson(λ), E[X] and Var(X) are: a) E[X] = λ, Var(X) = λ² b) E[X] = λ, Var(X) = λ c) E[X] = λ², Var(X) = λ d) E[X] = 1/λ, Var(X) = 1/λ² Answer: b. The Poisson is the only non-degenerate distribution with mean = variance.

3. In a Poisson process with rate λ, N(t) ~: a) Poisson(λ) b) Poisson(λt) c) Exponential(λ) d) Binomial(t, λ) Answer: b. The count in time interval of length t is Poisson(λt).

4. The memoryless property for the exponential distribution means: a) The distribution forgets its parameter b) P(T > s+t | T > s) = P(T > t) c) T has no memory d) λ = 0 Answer: b. The conditional probability of surviving an additional t given survival to s equals the unconditional probability of surviving t.

5. If X₁ ~ Poisson(2) and X₂ ~ Poisson(3) are independent, X₁ + X₂ ~: a) Poisson(6) b) Poisson(5) c) Approximately Poisson(2.5) d) Poisson(1) Answer: b. Independent Poissons add: λ₁ + λ₂ = 5.

6. For what value of k is P(X = k) maximized when X ~ Poisson(4.7)? a) 3 b) 4 c) 5 d) 4 and 5 Answer: b. Mode = ⌊λ⌋ = ⌊4.7⌋ = 4. When λ is not an integer, there is a unique mode at ⌊λ⌋.

7. The inter-arrival times in a Poisson process of rate λ follow: a) Poisson(λ) b) Exponential(λ) c) Uniform(0, 1/λ) d) Geometric(1/λ) Answer: b. The waiting time until the next event is Exponential(λ) with PDF λe^{−λt}.

8. Which is NOT an assumption of the Poisson process? a) N(0) = 0 b) Independent increments c) Stationary increments d) Events occur at fixed intervals Answer: d. Events occur randomly, not at fixed intervals. The orderliness property says only that simultaneous events have probability zero.

Next Steps

Continue to 10-06 Expectation of Discrete Random Variables to learn about E[X], linearity of expectation, LOTUS, variance, and covariance.