[mlpack] GSoC-2021

Thu Mar 11 11:19:20 EST 2021

Hello Gopi,

thanks for the clarification, so to me, this sounds like different reward functions but in
the same environment. So I guess the way I would integrate such a task into the existing
codebase is to add a separate task for each scenario. Maybe you have another idea?

> Regarding the first idea, I will soon implement a basic structure and make a PR, I will
> also send a detailed mail of what I am planning regarding the pre-processing tool.

Sounds good.

Thanks,
Marcus

> On 10. Mar 2021, at 01:09, Gopi Manohar Tatiraju <deathcoderx at gmail.com> wrote:
> 
> Heyy Marcus Edel,
> 
> Thanks for your feedback. 
> 
> When we frame trading as an RL problem on the surface it seems like the goal of the agent is to maximize the net worth. But there are many ways to reach this goal and there are different groups of people who work on different principles. 
> 
> Let's compare some:
> Day trader: The goal of any day trader is to maximize his profit but also minimize the risk (Trading 101: Always cap your losses). So for this use-case, we want to encourage the agent to use something called stop-loss. So more reward should be given to trades that are made with stop-loss rather than to the trades which are made without stop-loss. This will make sure that our agents learn to cover their losses, which is very important in a real-world scenario.
> Institutional Traders: These guys consider VWAP(Volume Weighted Average Pricing) as the best price on which they can acquire the stocks. So regardless of what the current price is these guys always try to buy at VWAP only. So for cases like this, we can polarize for not following VWAP, thus making it understandable that VWAP is the best price.
> 
> Different reward_schemes will be tailored for different use-cases. Based on how one wants to trade he can choose different reward schemes.
> 
> Regarding the first idea, I will soon implement a basic structure and make a PR, I will also send a detailed mail of what I am planning regarding the pre-processing tool.
> 
> Let me know if you have any more doubts regarding reward_schemes or anything else.
> 
> Thanks,
> Gopi
> 
> On Wed, Mar 10, 2021 at 5:37 AM Marcus Edel <marcus.edel at fu-berlin.de <mailto:marcus.edel at fu-berlin.de>> wrote:
> Hello Gopi M. Tatiraju,
> 
> thanks for reaching out; I like both ideas, I can see the first idea would
> integrate perfectly into the preprocessing pipeline; that said, it would be
> useful to discuss the project's scope in more detail. Specifically, what
> functionality you like to add, in #2727 you already implemented some
> features, so I'm curious to hear what other features you have in mind.
> 
> The RL idea sounds interesting as well, and I think could also fit into the
> RL codebase that is already there. I'm curious what do you mean with
> "rewards schemes"?
> 
> Thanks,
> Marcus
> 
>> On 9. Mar 2021, at 14:55, Gopi Manohar Tatiraju <deathcoderx at gmail.com <mailto:deathcoderx at gmail.com>> wrote:
>> 
>> Hello mlpack,
>> 
>> I am Gopi Manohar Tatiraju currently in my final year of Engineering from India. 
>> 
>> I've been working on mlpack for quite some time now. I've tried to contribute and learn from the community. I've received ample support from the community which made learning really fun.
>> 
>> Now, as GSoC is back with its 2021 edition, I want to take this opportunity to learn from the mentors and contribute to the community.
>> 
>> I am planning to contribute to mlapck under GSoC 2021. Currently, I am working on creating a pandas dataframe-like class that can be used to analyze the datasets in a better way. 
>> 
>> Having a class like this would help in working with datasets as ml is not only about the model but about data as well. 
>> 
>> I have a pr already open for this: https://github.com/mlpack/mlpack/pull/2727 <https://github.com/mlpack/mlpack/pull/2727> 
>> 
>> I wanted to know if I can work on this in GSoC? As it was not listed on the idea page, but I think this would be a start to something useful and big.
>> 
>> If this idea doesn't seem workable right now, I want to implement RL Environments for Trading and some working examples for each env.
>>  
>> What all exactly I am planning to implement are the building blocks of any RL system:
>> rewards schemes
>> action schemes
>> env
>> 
>> Fin-Tech is a growing field, and there is a lot of application of Deep-Q Learning there. 
>> 
>> I am planning to implement different strategies like Bull-Sell-Hold, Long only, Short only...
>> This will make example-repo rich in terms of DRL examples...
>> We can even build a small backtesting module that can be used to run backtest on our predictions.
>> 
>> There are some libraries that are currently working on such models in python, we can use it as a reference to go forward.
>> FinRL: https://github.com/AI4Finance-LLC/FinRL-Library <https://github.com/AI4Finance-LLC/FinRL-Library>
>> 
>> Planning to implement: 
>> 
>> Different types of envs for different kind of financial tasks:
>> single stock trading env
>> multi stock trading env
>> portfolio selection env
>> Some example env in python: https://github.com/AI4Finance-LLC/FinRL-Library/tree/master/finrl/env <https://github.com/AI4Finance-LLC/FinRL-Library/tree/master/finrl/env>
>> 
>> Different types of action_schemes:
>> make only long trades
>> make only short trades
>> make both long and short
>> BHS(Buy Hold Sell)
>> Example action_schemes: https://github.com/tensortrade-org/tensortrade/blob/master/tensortrade/env/default/actions.py <https://github.com/tensortrade-org/tensortrade/blob/master/tensortrade/env/default/actions.py>
>> 
>> We can see class BHS, SimpleOrder, etc.
>> 
>> Different types of reward_schemes:
>> simple reward
>> risk-adjusted reward
>> position based reward
>> 
>> For the past 3 months, I've been working as an ML Researcher in a Fin-Tech startup and have worked on this only. 
>>  
>> I would love to hear your feedback and suggestions.
>> 
>> Regards.
>> Gopi M. Tatiraju
>> _______________________________________________
>> mlpack mailing list
>> mlpack at lists.mlpack.org <mailto:mlpack at lists.mlpack.org>
>> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack <http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20210311/0761ba26/attachment-0001.htm>