[mlpack] Guidance for the GSOC for Deep learning modules and tree traversers

[Marcus Edel]

Hello nikhil,

that sounds good. It would probably be useful to read many papers on different
basic neural network models you like to implement over the summer, including the
design guidelines and the current neural network code (take a look at the FFN
class) that details the abstractions on which mlpack network models are based:

- https://github.com/mlpack/mlpack/wiki/DesignGuidelines
- https://github.com/mlpack/mlpack/blob/master/src/mlpack/methods/ann/ffn.hpp
- https://github.com/mlpack/mlpack/blob/master/src/mlpack/methods/ann/ffn_impl.hpp

And here are some papers on neural network models. A good theoretical
understanding of what these models do and why they work is a necessity to be
able to implement these well.

Restricted Boltzmann Machines (RBM)
- https://www.cs.toronto.edu/~hinton/absps/guideTR.pdf
- http://deeplearning.net/tutorial/rbm.html

Deep Belief Networks (DBN)
- http://www.cs.toronto.edu/~rsalakhu/papers/science.pdf
- http://deeplearning.net/tutorial/DBN.html

Radial Basis Function Networks (RBFN)
- http://www.cc.gatech.edu/~isbell/tutorials/rbf-intro.pdf

Bidrectional Recurrent networks (BRN)
Note: mlpack provides already an implementation for recurrent network
- http://www.di.ufpe.br/~fnj/RNA/bibliografia/BRNN.pdf

Convolutional Auto-Encoders (CAE)
- http://people.idsia.ch/~masci/papers/2011_icann.pdf

Hopfield neural networks (HNN)
- http://page.mi.fu-berlin.de/rojas/neural/chapter/K13.pdf

Keep in mind that you don't have to implement all of this models; A good project
will select a handful of architectures and implement them with tests and
documentation. Writing good tests is often the hardest part, so keep that in
mind when you create your project timeline.

Let me know if you have any more questions that I can answer.

Thanks
Marcus

> On 21 Feb 2016, at 12:50, Nikhil Yadala <nikhil.yadala at gmail.com> wrote:
> 
> hi marcus,
> 
>       Thanks for the reply. As you have said i have gone through the information in the above links. I would like to stick for the implementation of deep learning modules project.
> 
>     I have known the basics of deep learning and i have read some chapters of the book you have mentioned.Should I be knowing all the concepts of deep learning?, if yes, i guess i should complete reading the book, Nevertheless I am now good at understanding the things given in the language of deep learning.Could you tell me what is exactly required, so that i would concentrate on that topic to a deeper extent.?
> 
> thanks,
> nikhil yadala.
> 
> On Wed, Feb 17, 2016 at 8:01 PM, Marcus Edel <marcus.edel at fu-berlin.de <mailto:marcus.edel at fu-berlin.de>> wrote:
> Hello Nikhil,
> 
> What I really like about the "Essential Deep Learning Modules" project is that
> it offers the chance to learn about various fundamental deep learning models
> from a practical perspective. As the usual suspects are the most commonly used
> algorithms by the community, it is likely that people and developer will execute
> your code or use the code that you wrote as basis for their own structures.
> 
> Since you already build mlpack, one place to start is to look at the tutorials
> and try to compile and run simple mlpack programs.
> 
> http://www.mlpack.org/tutorials.html <http://www.mlpack.org/tutorials.html>
> 
> Then, you could look at the list of issues on Github and maybe you can find an
> easy and interesting bug to solve.
> 
> https://github.com/mlpack/mlpack/issues <https://github.com/mlpack/mlpack/issues>
> 
> The issues are labeled with difficulty, so you may be able to find some
> which are suited to your level of experience with mlpack.
> 
> The literature for the Essential Deep Learning Modules project depends on the
> network models you like to implement over the summer. But here are some
> hopefully useful links to get information about the basics:
> 
> - "Deep learning reading list" (http://deeplearning.net/reading-list/ <http://deeplearning.net/reading-list/>)
> - "Neural Networks for Machine Learning" by Geoffrey Hinton (https://www.coursera.org/course/neuralnets <https://www.coursera.org/course/neuralnets>)
> - "Deep learning" by Yoshua Bengio, Ian Goodfellow and Aaron Courville (http://www.deeplearningbook.org/ <http://www.deeplearningbook.org/>)
> 
> If you have found network models you're interested in, you
> could start reading about them and we can go from there.
> 
> Since Ryan is the mentor of the "Improvement of tree traversers" and the expert,
> I go and just make a quote here:
> 
> "A good place to start is by working through the mlpack tutorials and
> making sure you can get mlpack to compile and understand how to use it.
> Once you've done that, you should probably read about the ideas behind
> dual-tree algorithms, since your interest is in the 'improvement of tree
> traversers' project.  You might start here:
> 
> http://machinelearning.wustl.edu/mlpapers/paper_files/icml2013_curtin13.pdf <http://machinelearning.wustl.edu/mlpapers/paper_files/icml2013_curtin13.pdf>
> 
> There are a lot of references in that document, and you should probably
> read most of them to get a good idea of what is going on (especially the
> cover tree paper by Langford)."
> 
> Since you said you are working on a python to c++ translator, you might be also
> interested in the automatic bindings project. That project would give a way to
> automatically generate MATLAB, Python, and R bindings, which would be much
> easier to maintain than having to maintain each binding individually:
> 
> https://github.com/mlpack/mlpack/wiki/SummerOfCodeIdeas#automatic-bindings <https://github.com/mlpack/mlpack/wiki/SummerOfCodeIdeas#automatic-bindings>
> 
> Let me know if you have any more questions that I can answer.
> 
> Thanks!
> Marcus
> 
>> On 16 Feb 2016, at 22:40, Nikhil Yadala <nikhil.yadala at gmail.com <mailto:nikhil.yadala at gmail.com>> wrote:
>> 
>> Hi,
>> 
>>            I am nikhil yadala, pursuing 2nd year of my Btech at dept of CSE at Indian Institute of Technology Guwahati( IIT Guwahati). I am very much interested in machine learning particularly in computational biology. Currently i am doing research in Gene profiling using deep learning methods.
>> 
>>           I am interested in two of the projects that are floated as potential ones for the GSOC,2016
>>           1)Improvement of Tree traversers
>>           2)Essential Deep Learning Modules
>> I have enough expertise in C,C++,python,Currently iam developing a PYTHON TO C++ TRNSLATOR In c++.
>> I would be glad if any one over there would guide me as to get enough knowledge to get going with these projects. I have already built mlpack over my ubuntu and started to understand the code.
>> 
>> Thanks,
>> NIkhil yadala
>> 
>> _______________________________________________
>> mlpack mailing list
>> mlpack at cc.gatech.edu <mailto:mlpack at cc.gatech.edu>
>> https://mailman.cc.gatech.edu/mailman/listinfo/mlpack <https://mailman.cc.gatech.edu/mailman/listinfo/mlpack>
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20160222/9e41bfb9/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5233 bytes
Desc: not available
URL: <http://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20160222/9e41bfb9/attachment-0002.bin>