[mlpack] GSoC 2014 simulated annealing optimizer

[Ryan Curtin]

On Tue, Mar 25, 2014 at 04:06:12PM -0500, Zhihao Lou wrote:
> Hi Ryan,
> 
> I've got the  SA done. It currently can only handle unbounded search space,
> but otherwise it should work. Where should I check if the search space has
> hard boundary?
> 
> My code used element-wise arma::min(), which only available in Armadillo
> after version 3.930. Let me know if that's not acceptable. It is
> straightforward to loop through all elements manually.
> 
> Attached is my patch, which is generated by running
> svn diff > sa.diff
> I'm not sure this is the correct way of doing this. Let me know if there's
> more I have to do.

Hi Zhihao,

Thank you for the contribution.  It looks good.  However, a couple
notes:

 * Could we templatize the probability distribution from which the next
   sample is chosen, and also the cooling schedule?  This way we can
   have an interface that looks like this:

   template<typename FunctionType, typename DistributionType, typename
       CoolingScheduleType>
   class SA;

   This will allow more flexibility to the user in the exact way they
   want to use simulated annealing.  You could take what you already
   have, which is a uniform distribution and a geometric cooling
   schedule, and split them out into a UniformDistribution and
   GeometricCoolingSchedule class.

 * Can you comment what is going on in GenerateMove() and MoveControl()
   a little better?  I can sort of follow what you are doing, but it
   takes a while to figure it out, and a couple informative comments could
   make it much easier to read.

 * I'd like to add your name to the list of contributors once I work
   this in.  Do you mind if I do this?

If we can fix those things, I can work it into the trunk codebase.  I'll
probably add a few more tests, but that will only take a few moments.

Sorry that it took so long for me to respond to this; I've been more
busy than I would like...

Thanks,

Ryan

-- 
Ryan Curtin    | "Long live the new flesh."
ryan at ratml.org |   - Max