Skip to content


Integrating Dirichlet distributions

I've been learning a whole lot about PGMs and machine learning lately.  I don't consider it straying too far from my physics roots, in light of the fact that many juicy bits of contemporary AI, such as Markov random fields or Metropolis-Hastings sampling originated in physics.  This connection notwithstanding, my background doesn't give me that great of an advantage -- most of the time.  A few days ago, however, I was able to apply one delicious trick I knew in order to work out the integral of a Dirichlet distribution, and I can't help sharing it here.  This story has it all -- Fourier representation of the Dirac delta, Gamma functions, Laplace transforms, sandwiches, Bromwiches -- and yet it all fits into a pretty simple narrative.

Our story starts on a stormy summer night, with our protagonist grappling with the following question:  how on Earth do you take this integral?

\int_{\theta_1}\cdots\int_{\theta_k} \prod_{j=1}^k\theta_j^{\alpha-1} d\theta_1\cdots d\theta_k

Variables \theta_j here are multinomial parameters, and thus must be non-negative and sum to 1.  In a valiant (but ultimately futile) effort to avoid doing this integral by myself, I found this blog post -- which was a good start, and would allow me to casually drop terms like "integration over a simplex" or "k-fold Laplace convolution".  But it turns out that we can get away with something much simpler than this simplex business.  When faced with some constraint in an integrand, a physicist's instinct is to express it as a Dirac delta and integrate right over it.  For the problem above, this approach works like magic!  Without giving too much away, here's the punchline:

Want to know more?  Because I've been on a Xournal/youtube binge lately, I narrated this derivation and put it up for the world to see.  Here are the "slides"  and below is the actual video.  Enjoy!

Tagged with , .

One Response

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.

  1. nice job!! The video is entertaining as well. :)

Some HTML is OK

(required)

(required, but never shared)

or, reply to this post via trackback.