[mlpack] Parallel Stochastic Potimazation Methods

[pvachill at ece.auth.gr]

Hello!

I have implemented a naive parallel edition of the serial sgd  
implementation that already exists in mlpack build, using pthreads. My  
goal is to make HOGWILD! algorithm. I send my code and  an example  
using the test_function. Before i continue implementing HOGWILD!  
algorithm i would like you to mention if i am in the correct way.I  
would also appreciate if someone could send me a sparse dataset as  
mentioned at this paper:
http://papers.nips.cc/paper/4390-hogwild-a-lock-free-approach-to-parallelizing-stochastic-gradient-descent.pdf.

 From what i have already understand, i could make some suggestions  
which are still a bit of fuzzy in my mind:
1)Create stepSize an abstract class that would have different  
implementations for different approaches
2)Change Optimize member function to template function for different  
approaches
3)Create functions that don't use datasets as an abstract class and a  
second abstract class for those who use datasets.
4)Also, change sgd to abstract sgd by the same reasoning.

Thanks,

Axilleas

Quoting Ryan Curtin <ryan at ratml.org>:

> On Fri, Mar 11, 2016 at 07:59:08PM +0200, pvachill at ece.auth.gr wrote:
>>
>> Hi
>>
>> I am a 5th year undergraduate Electrical Engineer and Computer Engineer
>> student at Aristotle University. My name is Axilleas Pasias and i am from
>> Greece. I have background in Pthreads, OpenMP, MPI and Cuda.I also have some
>> knowledge  in convex optimization. I would like you to recommend me some
>> papers about Parallel stochastic optimization methods to get in touch with
>> the idea.
>
> Hi Axilleas,
>
> Here are some interesting papers.
>
> http://papers.nips.cc/paper/4390-hogwild-a-lock-free-approach-to-parallelizing-stochastic-gradient-descent.pdf
> http://papers.nips.cc/paper/4006-parallelized-stochastic-gradient-descent.pdf
> http://leon.bottou.org/publications/pdf/compstat-2010.pdf
>
> Note that for mlpack, the desired scenario is a multi-core single
> machine, not a distributed system.
>
> I hope that this is helpful.  Let me know if I can clarify anything.
>
> Thanks,
>
> Ryan
>
> --
> Ryan Curtin    | "And they say there is no fate, but there is: it's
> ryan at ratml.org | what you create." - Minister


-------------- next part --------------
A non-text attachment was scrubbed...
Name: Parallel_SGD_An_Attempt.tar.gz
Type: application/x-compressed-tar
Size: 50488 bytes
Desc: not available
URL: <http://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20160316/6665edf4/attachment-0002.bin>