Distributions You Must Know

The probability distributions every AI/ML engineer is expected to recognize, parameterize, and apply.

🟢 beginner5 min readprobability

Probability distributions model data generation processes. Discrete distributions include Bernoulli (binary trials), Binomial (k successes in N trials), Poisson (event rates in fixed intervals), and Categorical/Multinomial. Continuous distributions include Uniform, Gaussian/Normal (central limit theorem baseline), Exponential (time between events), and Beta/Dirichlet (priors over probabilities). Selecting the right parametric distribution dictates likelihood formulations and loss functions.

Summary Matrix of Essential Distributions

                             Probability Distributions
              ┌──────────────────────────┴──────────────────────────┐
              ▼                                                     ▼
     Discrete Distributions                               Continuous Distributions
  - Bernoulli(p): Single 0/1 flip                       - Gaussian(μ, σ²): Bell curve
  - Binomial(N, p): k successes in N                    - Exponential(λ): Wait times
  - Poisson(λ): Event counts per interval               - Beta(α, β): Prior over [0,1]
  - Categorical(p_1..p_K): K-class pick                 - Dirichlet(α_1..α_K): Prior over Simplex

Distribution	Support	Key Parameters	Mean $\mathbb{E}[X]$	Variance $\text{Var}(X)$	Primary ML Application
Bernoulli	$\{0, 1\}$	$p \in [0, 1]$	$p$	$p(1-p)$	Binary Classification, Dropout
Binomial	$\{0, \dots, N\}$	$N \in \mathbb{N}, p \in [0,1]$	$Np$	$Np(1-p)$	A/B Testing conversion counts
Poisson	$\{0, 1, 2, \dots\}$	$\lambda > 0$ (rate)	$\lambda$	$\lambda$	Call center arrivals, web traffic
Categorical	$\{1, \dots, K\}$	$p_1, \dots, p_K$	$\sum i p_i$	Matrix form	Multi-class Softmax classification
Gaussian (Normal)	$(-\infty, +\infty)$	$\mu, \sigma^2 > 0$	$\mu$	$\sigma^2$	MSE Loss, VAE latent space, Noise
Exponential	$[0, +\infty)$	$\lambda > 0$	$1/\lambda$	$1/\lambda^2$	Time-to-failure, Survival analysis
Beta	$[0, 1]$	$\alpha, \beta > 0$	$\frac{\alpha}{\alpha + \beta}$	Complex	Bayesian Prior over probabilities
Dirichlet	Probability Simplex	$\alpha_1, \dots, \alpha_K$	$\frac{\alpha_i}{\sum \alpha_k}$	Complex	Topic Modeling (LDA), Bayesian GMM

Deep Dives on Critical Distributions

1. Gaussian / Normal Distribution $\mathcal{N}(\mu, \sigma^2)$

Probability Density Function (PDF):

f(x) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left( -\frac{(x - \mu)^2}{2\sigma^2} \right)

Log-likelihood maximization yields Mean Squared Error (MSE).

2. Beta Distribution $\text{Beta}(\alpha, \beta)$

Supported on $[0, 1]$ . Shape depends on pseudo-counts $\alpha$ (successes) and $\beta$ (failures):

$\alpha = 1, \beta = 1 \implies \text{Uniform}(0, 1)$ .
$\alpha = 10, \beta = 2 \implies$ Skewed toward $1.0$ (High confidence success).

Used in Multi-Armed Bandits (Thompson Sampling) and Bayesian A/B testing.

3. Dirichlet Distribution $\text{Dirichlet}(\boldsymbol{\alpha})$

Multivariate generalization of the Beta distribution over probability vectors $\mathbf{p} = [p_1, \dots, p_K]$ where $\sum p_k = 1.0$ .

Core foundation for Latent Dirichlet Allocation (LDA) topic modeling.

Say this out loud

"Bernoulli models single 0/1 outcomes; Binomial models success counts in N trials; Poisson models event rates per time window with mean equal to variance λ. Gaussian N(μ, σ²) is the central limit baseline deriving MSE loss. Beta(α, β) models uncertainty over probabilities on [0,1] for Thompson Sampling, while Dirichlet generalizes Beta to multi-class probability vectors for topic modeling."

Follow-ups to expect

What is the Memoryless Property of the Exponential Distribution? $P(T > s + t \mid T > s) = P(T > t)$ . The probability of waiting an additional $t$ minutes does not depend on how long $s$ you have already been waiting.
What is the Heavy-Tailed Cauchy distribution? Ratio of two independent standard Gaussians. Has undefined mean and infinite variance, violating the Central Limit Theorem.

Check yourself

Question 1 of 3

Which continuous probability distribution is bounded strictly on [0, 1] and serves as the conjugate prior for Binomial / Bernoulli likelihoods in Bayesian modeling?

More in Math & Statistics

See all →

Bayes’ Theorem4 min Central Limit Theorem4 min Law of Large Numbers4 min