KL divergence

Definition

Kullback-Leibler divergence (or distanct) between two probability density functions (PDF) $f$ and $g$ is defined as

$$D(f||g) = \int f(x) \log \frac{ f(x) }{ g(x) } dx.$$