[mlpack] GSoC 2014 simulated annealing optimizer

[Zhihao Lou]

Hi Ryan

On Fri, Apr 4, 2014 at 4:56 PM, Ryan Curtin <gth671b at mail.gatech.edu> wrote:

>
>  * Could we templatize the probability distribution from which the next
>    sample is chosen, and also the cooling schedule?  This way we can
>    have an interface that looks like this:
>
>    template<typename FunctionType, typename DistributionType, typename
>        CoolingScheduleType>
>    class SA;
>
>    This will allow more flexibility to the user in the exact way they
>    want to use simulated annealing.  You could take what you already
>    have, which is a uniform distribution and a geometric cooling
>    schedule, and split them out into a UniformDistribution and
>    GeometricCoolingSchedule class.
>

I'm certainly willing to do that. The problem is, however, how to design
the
interface between these separate classes and the main loop of the annealing.

For example, the geometric cooling schedule only needs current temperature
to calculate next step's temperature. The same is true for linear and
logarithmic
schedule, though I don't think anybody should use these two. But the other
major class of cooling schedules is adaptive schedules, which usually
require
additional information like the current value of the cost function (usually
for
calculating variance etc), and in Lam's schedule (see my comments in
MoveControl) requires the boolean whether last proposed move has been
accepted. So it is very hard to anticipate what information the cooling
schedule
will need. The solution I can think of is to pass the SA itself to the
schedule,
but this is ugly.

The other thing is that the actual amount of change move in generateMove()
is actually a double exponential (Laplace distribution) calculated from
uniform
intermediate unif. (I probably need more comments there.) The double
exponential distribution is related to the move control and the 0.44 value.
I'll suggest not to change this.



>
>  * Can you comment what is going on in GenerateMove() and MoveControl()
>    a little better?  I can sort of follow what you are doing, but it
>    takes a while to figure it out, and a couple informative comments could
>    make it much easier to read.
>

Sure. I'll going to work on these right now.


>
>  * I'd like to add your name to the list of contributors once I work
>    this in.  Do you mind if I do this?
>

That will be great!

Best regards,

Zhihao Lou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20140404/79992584/attachment-0003.html>