[mlpack] mlpack GSOC idea - Parallel Stochastic Optimization Methods

[Ryan Curtin]

On Sun, Mar 01, 2015 at 10:08:23AM -0800, Stephen Tu wrote:
> On Sat, Feb 28, 2015 at 6:48 AM, Chen Qan <kazenoyumechen at gmail.com> wrote:
> > Although the task seems possible, but I have some questions about this:
> >
> > 1. Is parallel SGD/SCD effective ?
> >     I found the following post
> > scikit-learn-parallelize-stochastic-gradient-descent
> > <http://stackoverflow.com/questions/21052050/scikit-learn-parallelize-stochastic-gradient-descent> says
> > that instead of using parallel SGD similar in [1], L-BFGS should be used
> > with warm start point getting from SGD.
> >
> 
> Great question, although the link you provide is not really a good counter
> example for the reasons discussed within. But nevertheless, a valid
> question. This is one of the biggest downfalls of the ML community, to be
> honest, that there are no good papers I can point to that really do an
> extensive empirical study of this stuff. Hence, if you are interested in
> running some benchmarks, you could help answer this question with your
> implementation :)

It's worth pointing out that we do have a benchmarking system that could
be used for this, should you be interested in running benchmarks:

https://github.com/zoq/benchmarks/

I use the system for gathering benchmarks on algorithms that I am
working on.

-- 
Ryan Curtin    | "Reprogram him!"
ryan at ratml.org |   - Master Control Program