[mlpack] GSoC 2014 simulated annealing optimizer

[Zhihao Lou]

Ahh... You caught me right before I'm set for the independence day break.
I'll take a look at the code next week.

And yes it's my honor to be listed as one of the contributors.

Now onto the search space. I can think of two ways of working with bounded
search space.

One is to let the function being optimized simply return something like
DBL_MAX (or numeric_limits<double>::max() if you prefer c++ style) when any
parameter go outside of the search space. This way there is no change
needed for the annealing code. I've used this method in one of my problems
and I can report that at least in my problem, I haven't seen any instances
that repeatedly hit the boundary. In addition, it will allow the user to
switch between hard and soft bound (like a penalty term) with relatively
less changes.

The other way is to let the function somehow report its search space,
likely via a function call that returns 2 matrices, lowerBound and
upperBound, each has the same dimension as the function being
optimized.  I'm not sure how such boundary information is passed in other
optimizers. But once that is done, the move generation can cut off at the
boundary and alter the move distribution to one sided exponential while the
parameter in question is at the boundary.

Let me know your thoughts.

Best regards,

Zhihao Lou
PhD Candidate
Department of Computer Science
The University of Chicago


On Thu, Jul 3, 2014 at 12:14 PM, Ryan Curtin <gth671b at mail.gatech.edu>
wrote:

> On Fri, Apr 04, 2014 at 05:56:00PM -0400, Ryan Curtin wrote:
> > On Tue, Mar 25, 2014 at 04:06:12PM -0500, Zhihao Lou wrote:
> > > Hi Ryan,
> > >
> > > I've got the  SA done. It currently can only handle unbounded search
> space,
> > > but otherwise it should work. Where should I check if the search space
> has
> > > hard boundary?
>
> Ack, sorry to bombard you with multiple emails in a day, but I realized
> I had not said anything about a bounded search space.  I don't have a
> quick solution for this.  Do you have any ideas?  I'm sure we can figure
> something out, if that is support that you would use if it was added.
>
> Also, may I include your name/email/website on the list of contributors
> both in the code and on the website?
>
> Thanks,
>
> Ryan
>
> --
> Ryan Curtin    | "I'm just going to shoot you once!"
> ryan at ratml.org |   - Joseph Dunn
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20140703/30e07ecf/attachment-0003.html>