Maximum Likelihood Estimate of the Poisson Distribution
The Poisson distribution is a discrete probability model widely used to count the number of events occurring within a fixed interval of time or space. When we have observed data that we believe follows a Poisson process, a common goal is to estimate the unknown λ from the sample. It is defined by a single parameter, usually denoted by λ (lambda), which represents both the mean and the variance of the distribution. The Maximum Likelihood Estimate (MLE) provides a systematic way to obtain this estimate, delivering a value that maximizes the probability of observing the given data under the assumed model.
Introduction
In many real‑world scenarios—ranging from the number of emails received per hour to the count of defects on a manufactured part—events happen independently at a constant average rate. The Poisson distribution captures this behavior, but the true λ is rarely known. And the MLE approach fills this gap by turning observed counts into a best‑fit λ. This article walks through the derivation, calculation, and interpretation of the MLE for a Poisson distribution, and highlights why it is a preferred method in statistical inference Still holds up..
You'll probably want to bookmark this section.
How the Likelihood Function Is Built
The likelihood function measures how probable the observed sample is for different values of λ. For a sample of independent observations (x_1, x_2, \dots, x_n) drawn from a Poisson distribution, the probability mass function (PMF) for a single observation is
[ P(X = x_i \mid \lambda) = \frac{e^{-\lambda}\lambda^{x_i}}{x_i!}. ]
Because the observations are independent, the joint likelihood is the product of the individual probabilities:
[ L(\lambda) = \prod_{i=1}^{n} \frac{e^{-\lambda}\lambda^{x_i}}{x_i!} = e^{-n\lambda}\frac{\lambda^{\sum x_i}}{\prod x_i!} Simple as that..
The denominator (\prod x_i!) does not involve λ, so it can be treated as a constant when maximizing. The log‑likelihood simplifies the optimization:
[ \ell(\lambda) = \log L(\lambda) = -n\lambda + \Big(\sum_{i=1}^{n} x_i\Big)\log\lambda - \sum_{i=1}^{n}\log(x_i!). ]
Only the first two terms depend on λ, so the maximization reduces to
[ \ell(\lambda) = -n\lambda + S\log\lambda, ]
where (S = \sum_{i=1}^{n} x_i) is the total count observed.
Deriving the Maximum Likelihood Estimate
To find the λ that maximizes the likelihood, we differentiate the log‑likelihood with respect to λ and set the derivative to zero:
[ \frac{d\ell}{d\lambda} = -n + \frac{S}{\lambda} = 0. ]
Solving for λ gives
[ \hat{\lambda}_{\text{MLE}} = \frac{S}{n} = \bar{x}, ]
where (\bar{x}) is the sample mean. This result is intuitive: the best estimate of the Poisson rate is simply the average number of events observed per interval Turns out it matters..
A second derivative check confirms that this critical point is indeed a maximum:
[ \frac{d^2\ell}{d\lambda^2} = -\frac{S}{\lambda^2} < 0 \quad (\text{for } S>0), ]
so the likelihood function is concave, guaranteeing a unique maximum Simple, but easy to overlook..
Practical Steps to Compute the MLE
- Collect the data – Record the number of events in each observation interval (e.g., emails per hour, defects per batch).
- Calculate the sample mean – Sum all observed counts and divide by the number of intervals: (\bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i).
- Set the MLE – The estimated λ is (\hat{\lambda} = \bar{x}).
- Assess fit (optional) – Compare the observed frequencies with the expected Poisson probabilities using a histogram or a chi‑square goodness‑of‑fit test.
Because the MLE is just the sample mean, the computation is straightforward even for large datasets.
Properties of the Poisson MLE
- Consistency – As the sample size (n) grows, (\hat{\lambda}) converges in probability to the true λ.
- Asymptotic normality – For large n, the distribution of (\hat{\lambda}) approaches a normal distribution with mean λ and variance (\lambda/n).
- Efficiency – Among unbiased estimators, the MLE achieves the lowest possible variance (Cramér‑Rao lower bound) for the Poisson parameter.
- Bias – The MLE is actually unbiased: (E[\hat{\lambda}] = \lambda).
These desirable statistical properties make the MLE a strong choice for estimating the Poisson rate.
Scientific Explanation: Why the MLE Works for Poisson Data
The Poisson distribution belongs to the exponential family, a class of distributions whose likelihood functions have a convenient form. For exponential family members, the MLE often has a closed‑form solution, as we have seen. The derivation hinges on two key facts:
- Additivity of log‑likelihood – Independence allows us to sum log‑probabilities, turning a product into a sum.
- Differentiability – The log‑likelihood is smooth, enabling calculus‑based optimization.
Because the Poisson PMF involves only λ in the terms (e^{-\lambda}) and (\lambda^{x_i}), the log‑likelihood becomes a simple linear‑plus‑log function of λ. This structure yields the elegant result that the MLE equals the sample mean, a statistic that is both easy to compute and statistically efficient.
Example: Estimating Email Arrival Rate
Suppose you monitor the number of emails received each hour over a 10‑hour period and record the following counts:
[ 3, 5, 2, 4, 6, 3, 5, 4, 2, 3. ]
- Compute the total count: (S = 3+5+2+4+6+3+5+4+2+3 = 37).
- Find the sample mean: (\bar{x} = 37/10 = 3.7).
- Set the MLE: (\hat{\lambda} = 3.7).
Interpretation: On average, you can expect 3.7 emails per hour. Also, using this λ, you can predict probabilities such as (P(X=4) = e^{-3. 7}3.But 7^4/4! \approx 0.19).
Frequently Asked Questions (FAQ)
Q: Can the MLE be zero?
A: Yes, if all observed counts are zero, the sample mean is zero, giving (\hat{\lambda}=0). This is a valid estimate, indicating that no events were observed in the sampled intervals Surprisingly effective..
Q: What if the data contain zeros but the true λ is positive?
A: The MLE will still be the sample mean, which may underestimate λ if the sample size is small. In such cases, Bayesian approaches with prior information can provide a more stable estimate But it adds up..
Q: Is the MLE always the best estimator?
A: For large samples, the MLE is asymptotically optimal. For small samples, other estimators (e.g., method of moments) may have comparable performance, but the MLE remains a standard choice due to its efficiency and ease of computation.
Q: How do I test whether the Poisson model fits the data?
A: Use a goodness‑of‑fit test (e.g., chi‑square) comparing observed frequencies with expected Poisson probabilities based
Q: How do I compute the chi‑square statistic for a Poisson goodness‑of‑fit test?
A: After estimating λ̂ (usually the sample mean), calculate the expected frequency for each count k as
[ E_k = N , P(X=k;,\lambdâ) = N \frac{e^{-\lambdâ}\lambdâ^{,k}}{k!}, ]
where N is the total number of observations. Collapse cells with very low expected counts (commonly E < 5) to satisfy the chi‑square approximation. The test statistic is
[ \chi^2 = \sum_{k} \frac{(O_k - E_k)^2}{E_k}, ]
with Oₖ the observed frequency of count k. But under the null hypothesis that the data follow a Poisson(λ̂), χ² follows a chi‑square distribution with degrees of freedom equal to (number of collapsed categories − 1 − p), where p is the number of estimated parameters (here p = 1). A large p‑value indicates no evidence against the Poisson model.
Q: What if the data are over‑dispersed?
A: Over‑dispersion (variance > mean) violates the Poisson assumption. In such cases the MLE for λ remains the sample mean, but standard errors become unreliable. Practitioners often turn to a negative‑binomial model, which introduces an extra dispersion parameter, or they use a quasi‑Poisson approach that scales the variance by a factor estimated from the data.
Q: Can I obtain a confidence interval for λ without relying on asymptotics?
A: Yes. An exact Clopper‑Pearson–type interval can be derived from the Poisson likelihood. For an observed total count S in n observations, the lower and upper bounds (λL, λU) satisfy
[ P\bigl(\text{Poisson}(n\lambdaL) \ge S\bigr) = \frac{\alpha}{2},\qquad P\bigl(\text{Poisson}(n\lambdaU) \le S\bigr) = \frac{\alpha}{2}, ]
where α is the desired significance level (e.g.Day to day, , 0. Which means 05). In real terms, in practice, most software packages (R’s poisson. test, Python’s statsmodels) provide these intervals directly.
Practical Implementation
Below are concise snippets for estimating λ̂ and performing a chi‑square goodness‑of‑fit test in two popular environments And that's really what it comes down to..
R
# Example data: counts vector
counts <- c(3,5,2,4,6,3,5,4,2,3)
# MLE (sample mean)
lambda_hat <- mean(counts)
# Expected frequencies for counts 0:max(counts)
max_k <- max(counts)
expected <- sapply(0:max_k, function(k) {
sum(dpois(k, lambda_hat) * length(counts))
})
# Observed frequencies
observed <- table(factor(counts, levels = 0:max_k))
observed <- as.vector(observed)
# Collapse low expected cells (E < 5)
collapse_idx <- which(expected < 5)
if (length(collapse_idx) > 0) {
expected <- expected[!collapse_idx]
observed <- observed[!collapse_idx]
}
# Chi‑square test
chi2 <- sum((observed - expected)^2 / expected)
df <- length(observed) - 1 - 1 # subtract 1 for total N, 1 for estimated lambda
pval <- 1 - pchisq(chi2, df)
Python (statsmodels)
import numpy as np
import statsmodels.api as sm
from statsmodels.stats.poisson_contribution import poisson_gof
counts = np.array([3,5,2,4,6,3,5,4,2,3])
lambda_hat = counts.mean()
# Goodness‑of‑fit test
chi2, pval, dof = poisson_gof(counts, mu=lambda_hat)
Both scripts output the chi‑square statistic,
