[mlpack] Greetings GSOC 19 : Idea Reinforcement Learning

[Ryan Curtin]

On Fri, Mar 08, 2019 at 04:31:55AM +0530, Rohan Raj wrote:
> Hello Ryan, Marcus and fellow contributors of MLPACK,
> 
> I am Rohan Raj (Github : mirraaj) <https://github.com/mirraaj>,
> undergraduate student from Indian Institute of Technology (IIT) Guwahati. I
> am writing this email to you to express my interests in becoming a part of
> *MLPACK* for the coming *Google Summer of Codes 2019.*
> 
> I sincerely congratulate Mlpack for being accepted as a mentor organization
> for the coming Google Summer of Codes 2019. I am interested in
> reinforcement learning project for the coming year. In particular, I plan
> to implement Rainbow and PPO for the coming coding season.
> 
> My tentative schedule is present below,
> 
> Week 1-6 : Implement different Rainbow DQN functions
> 
> Week 6-10 : PPO Algorithm
> 
> Week 11-12  Bug fixing and final submission.
> 
> I believe it is really important to test any function/feature added to the
> mlpack codebase. I have been working on RL and Mlpack for quite a long time
> and I personally think it is difficult to reproduce result sometimes. It is
> also a time taking procedure to stabilize statistical test results on
> mlpack codebase. Hence I would like to go ahead with 2 algorithms so that I
> get proper time to test the algorithms on different environments.
> 
> Please let me know your valuable inputs to this short proposal. I will
> definitely add the details of the project in my actual proposal.

Hi Rohan,

Thanks for the congratulations and we're happy to have you involved.
Although I am not a reinforcement learning expert and I won't be the
mentor for that project, I will at least say that two weeks set aside
for 'bug fixing' is a bit vague---it's definitely hard to predict when
you'll have bugs, but as you prepare your proposal I'd encourage you to
spend a bit of time thinking about how you will write the tests to catch
all potential bugs you might have during implementation.

You're right that testing is a very important part, so often when I am
reviewing proposals, I look for a lot of detail about how the proposed
algorithm will be implemented and things of this nature.

I hope this is helpful. :)

Thanks!

Ryan

-- 
Ryan Curtin    | "None of your mailman friends can hear you."
ryan at ratml.org |   - Alpha