Have you ever searched for a calculation on the Internet and then was disappointed when you couldn’t find it?

Yeah, me too. I was trying to find the KL divergence between a Gamma and a Normal random distribution, but eventually decided to compute it on my own.

So, if we consider the Gamma distribution, , and the Normal distribution, , then the KL of with respect to is:

The term is the entropy of ,

with being the Gamma function and being the digamma function. Also,

The (2) and (3) pieces, substituted in (1), provide the KL between the two distributions.

### Like this:

Like Loading...

*Related*