[mlpack] GSoC 2014 simulated annealing optimizer

[Ryan Curtin]

On Tue, Jul 01, 2014 at 11:46:43PM -0500, Zhihao Lou wrote:
> Hi Ryan,
> 
> I've noticed the SA hasn't been merged into the repo. Is there anything I
> can help to speed things up?

Hello Zhihao,

Please accept my apologies.  I had meant to take care of this, but then
I became busy with other things and forgot.  I have committed your
contribution to the trunk repository and it will be a part of the next
release.

I made some minor changes; a list of changes is given at
http://www.mlpack.org/trac/log/trunk/src/mlpack/core/optimizers/sa
and
http://www.mlpack.org/trac/log/trunk/src/mlpack/tests/sa_test.cpp

If any of the changes I have made are bad, please let me know -- you are
the expert on simulated annealing.  I have simply tried to refactor a
few things, sometimes for speed, sometimes for consistency with other
mlpack code.

I asked you some time back to refactor the move distribution into its
own template class, but I took a closer look at your email and the code,
and I think you are right that it doesn't really make sense (or would be
difficult) to use another distribution there, since MoveControl() is
derived for the univariate Laplace distribution.  So I moved the
LaplaceDistribution back into sa_impl.hpp, but also expanded on it a
little bit to make it a multivariate Laplace distribution with unit
covariance, and added a LaplaceDistribution to src/mlpack/core/dists/:

http://www.mlpack.org/trac/browser/trunk/src/mlpack/core/dists/laplace_distribution.hpp

Now I think it will be possible to use that LaplaceDistribution class as
a PDF for other machine learning algorithms; one example might be an
HMM.

Thank you so much for your contribution, and again, I am sorry it took
so long.

Thanks,

Ryan

-- 
Ryan Curtin    | "Lots of respectable people have been hit by
ryan at ratml.org | trains."  - Penny