[mlpack] [GSoC'18] Reinforcement Learning

[Marcus Edel]

Hello Vaibhav,

> As far as Zenodo is concerned, I was majorly involved in discussions regarding
> the design (both UI and backend) of the Researchers's Profile Project - to be
> rendered using a dedicated page. It involved proposing various possible Database
> Schemas, UI Designs via mockups and proposed various mechanism for message
> passing by juxtaposing their merits/demerits. I also resolved some issues within
> the code-base.

That sounds like a lot of fun, thanks for the update on this.

> Currently, I am going through the codebase of MLPACK to better understand the
> code structure along with present algorithms & implementations. This would help
> to gain some insights as to  What all algorithms can be added straightaway ? ,
> Does the existing implementations have scope of improvement? (and similar
> questions).

There is an open PR: https://github.com/mlpack/mlpack/pull/1091 that might be
interesting, I should point out that this is not an easy one, but any progress
on that would be great.

Let me know if I should clarify anything.

Thanks,
Marcus

> On 15. Feb 2018, at 21:50, VAIBHAV GUPTA <guptavaibhav18197 at gmail.com> wrote:
> 
> Hi Marcus,
> 
> Thanks for the response. 
> 
> What did you do at ZENODO?
> 
> As far as Zenodo is concerned, I was majorly involved in discussions regarding the design (both UI and backend) of the Researchers's Profile Project - to be rendered using a dedicated page. It involved proposing various possible Database Schemas, UI Designs via mockups and proposed various mechanism for message passing by juxtaposing their merits/demerits. I also resolved some issues within the code-base.
> 
> Currently, I am going through the codebase of MLPACK to better understand the code structure along with present algorithms & implementations. This would help to gain some insights as to 
> What all algorithms can be added straightaway ? , 
> Does the existing implementations have scope of improvement? (and similar questions).
> 
> Thank you
> Vaibhav
> 
> 
> On Wed, Feb 14, 2018 at 3:38 PM, Marcus Edel <marcus.edel at fu-berlin.de <mailto:marcus.edel at fu-berlin.de>> wrote:
> Hello Vaibhav,
> 
> welcome, thanks for getting in touch.
> 
>> My research domain is Artificial Intelligence, specifically Reinforcement
>> Learning & Multi-agent Systems in Machine Learning Lab, IIIT Hyderabad. I am
>> doing my research under Prof. Praveen Paruchuri and Prof. Balaraman Ravindaran.
>> I have past open source experience of contributing to ZENODO(CERN). Also, I was
>> selected as intern in The Linux Foundation where my project revolved around
>> coming up with various performance metrics for object storage.
> 
> That's sounds really interesting, what did you do at ZENODO, if you don't mind
> to share that information.
> 
>> I have gone through the project idea list of mlpack and found the project idea
>> Reinforcement Learning really interesting. I have read papers on Double DQN /
>> Playing Atari with deep reinforcement learning and have fairly good
>> understanding of these. Attached is the exhaustive list of papers that I have
>> implemented and read as part of research work.  I am an enthusiast in
>> reinforcement learning and am ready to read and learn on the go as the need be.
>> 
>> Since I am new to mlpack please let me know as to how can I get started. Also
>> since, there are no relevant tickets open at this time, please suggest me know
>> how to proceed.
> 
> Getting familiar with the codebase especially
> src/mlpack/methods/reinforcement_learning/ should be the first step. Running the
> tests: (rl_components_test.cpp) 'bin/mlpack_test -t RLComponentsTest' and
> (q_learning_test.cpp) 'bin/mlpack_test -t QLearningTest' should help to
> understand the overall structure.
> 
> If you like you can work on a simple RL method like (stochastic) Policy
> Gradients and use that to jump into the codebase, but don't feel obligated.
> 
> Also, the methods listed on the ideas page are just suggestions, so if you have
> an interesting method in mind you like to work on, let me know.
> 
> Thanks,
> Marcus
> 
>> On 13. Feb 2018, at 22:17, VAIBHAV GUPTA <guptavaibhav18197 at gmail.com <mailto:guptavaibhav18197 at gmail.com>> wrote:
>> 
>> Hello everyone,
>> 
>> My name is Vaibhav Gupta. I am a 3rd year undergraduate student pursuing my B.Tech in Computer Science and M.S by research in IIIT Hyderabad, India. 
>> 
>> My research domain is Artificial Intelligence, specifically Reinforcement Learning & Multi-agent Systems in Machine Learning Lab, IIIT Hyderabad. I am doing my research under Prof. Praveen Paruchuri <https://scholar.google.com/citations?user=ILUqgKEAAAAJ&hl=en> and Prof. Balaraman Ravindaran <https://scholar.google.co.in/citations?user=nGUcGrYAAAAJ&hl=en>. I have past open source experience of contributing to ZENODO(CERN). Also, I was selected as intern in The Linux Foundation where my project revolved around coming up with various performance metrics for object storage.
>> 
>> I have good understanding of neural networks and (as a part of my academic project). I have also implemented <https://github.com/guptavaibhav18197/student-teacher-transfer-learning> the paper - Distilling the knowledge in Neural Network <https://arxiv.org/abs/1503.02531> in which we try to transfer the learning of a larger network(teacher) to a relatively smaller network(student) making use of the logits of the teacher network. 
>> 
>> Currently, I am doing research in Reinforcement learning (Transfer learning) and trying to come up with a state granular confidence metric in simultaneously learning heterogeneous agents. I have sound knowledge of many prominent algorithms used in Reinforcement Learning.
>> 
>> I have a sound background in data structures and algorithms and have qualified twice for ACM ICPC regionals. I have secured good rank in other programming contests too. I have good understanding of C++ having done all my competitive programming and several different projects using it.
>> 
>> I have gone through the project idea list of mlpack and found the project idea Reinforcement Learning really interesting. I have read papers on Double DQN /  Playing Atari with deep reinforcement learning and have fairly good understanding of these. Attached is the exhaustive list of papers that I have implemented and read as part of research work.  I am an enthusiast in reinforcement learning and am ready to read and learn on the go as the need be.
>> 
>> Since I am new to mlpack please let me know as to how can I get started. Also since, there are no relevant tickets open at this time, please suggest me know how to proceed.
>> 
>> Thanks 
>> Vaibhav Gupta
>> <Honours Project.pdf>_______________________________________________
>> mlpack mailing list
>> mlpack at lists.mlpack.org <mailto:mlpack at lists.mlpack.org>
>> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack <http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20180216/155bbed8/attachment.html>