# KL Divergence Calculator

> Calculate the Kullback–Leibler divergence D(P‖Q) and D(Q‖P) between two discrete distributions, plus cross-entropy and Jensen–Shannon divergence.

Category: Statistics & probability · Applies to: Worldwide · Last updated: 2026-10-29
Interactive calculator: https://calculators.geekmidst.com/calculators/kl-divergence-calculator

KL divergence measures how much information is lost when Q is used to approximate P. It is zero only when the distributions match, and it is not symmetric.

Also called: Kullback–Leibler divergence calculator, Relative entropy calculator.

## Inputs
- Distribution P (true / observed) (default 0.36, 0.48, 0.16)
- Distribution Q (model / reference) (default 1, 1, 1)
- Logarithm base (options: Nats (ln), Bits (log₂), Hartleys (log₁₀); default e)
- Pseudo-count added to every category (optional) (default ; optional)

## Results
- D(P ‖ Q)
- D(Q ‖ P), reverse
- Jensen–Shannon divergence
- Cross-entropy H(P, Q)
- Entropy H(P)
- Unit

## Formula

```
D(P‖Q) = Σ p_i log(p_i ÷ q_i), with 0·log(0/q) = 0 and p·log(p/0) = ∞;  H(P, Q) = H(P) + D(P‖Q);  JS = ½D(P‖M) + ½D(Q‖M), M = (P + Q) ÷ 2
```

Where:
- p_i, q_i: probabilities of category i under P and Q
- M: the average of P and Q

## Worked example
1. Q normalises to ⅓ each.
2. D(P‖Q) = 0.36 ln(1.08) + 0.48 ln(1.44) + 0.16 ln(0.48) = 0.0277 + 0.1750 − 0.1174 = 0.0853 nats.
3. D(Q‖P) = ⅓[ln(⅓ ÷ 0.36) + ln(⅓ ÷ 0.48) + ln(⅓ ÷ 0.16)] = 0.0975 nats.

**Result:** Approximating the binomial(2, 0.4) distribution by a uniform one loses 0.0853 nats; the reverse direction gives 0.0975, showing KL is not symmetric.

## What the result means

Use D(P‖Q) with P as the data and Q as the model. Jensen–Shannon divergence is symmetric and always finite (at most ln 2 nats or 1 bit), so it is easier to compare across pairs.

## Assumptions and limits
- Both lists cover the same categories in the same order.
- Zero handling: categories where P is 0 add nothing; if Q is 0 where P is not, D(P‖Q) is infinite and the calculator asks for smoothing instead of returning a number.
- Discrete distributions only.
- Smoothing changes the result; report the pseudo-count you used.

## Questions

### Why is KL divergence not a distance?

It is not symmetric and does not satisfy the triangle inequality. Its square-root-of-JS cousin is a true metric.

### How does it relate to cross-entropy loss?

Cross-entropy H(P, Q) = H(P) + D(P‖Q). With P fixed by the data, minimising cross-entropy is the same as minimising KL divergence.

## Related calculators
- [Shannon Entropy Calculator](https://calculators.geekmidst.com/calculators/entropy-shannon-calculator)
- [Information Gain Calculator](https://calculators.geekmidst.com/calculators/information-gain-calculator)
- [Probability Calculator](https://calculators.geekmidst.com/calculators/probability-calculator)

Source: GM¹X CalcVerse — https://calculators.geekmidst.com/calculators/kl-divergence-calculator