[mlpack] Fwd: Variational Autoencoders and Reinforcement Learning

[arjun k]

---------- Forwarded message ----------
From: arjun k <arjun.k018 at gmail.com>
Date: Thu, Mar 15, 2018 at 7:51 PM
Subject: Re: [mlpack] Variational Autoencoders and Reinforcement Learning
To: Marcus Edel <marcus.edel at fu-berlin.de>


My idea was to create a framework for VAE where the user can specify an
architecture and we can integrate it with the current structure of mlpack.
This would be useful for many users that are concerned about the low-level
implementation details. Additionally, an option for using
pretrained network can be added for standard datasets. As mentioned testing
would be an issue as testing of generative models are not that evolved.
There is a concept of inception score which can be used for testing the
network. This type of testing would be useful for future implementation of
any generative models. I am still thinking about where in mlpack would this
be integrated into.

Thanks,
Arjun.

On Wed, Mar 14, 2018 at 11:29 AM, Marcus Edel <marcus.edel at fu-berlin.de>
wrote:

> Hello Arjun,
>
> thanks for the feedback on this one. Agreed, this could end up as an
> additional
> feature, I think we could keep that in mind for the interface if we like
> to go
> for it.
>
> Thanks,
> Marcus
>
> On 14. Mar 2018, at 03:03, arjun k <arjun.k018 at gmail.com> wrote:
>
> Hi,
>
> The paper looks interesting, the idea to introduce RL to relieve the lack
> of data is good. But what I found is that it makes some assumptions about
> the data that is that the latent representation can be divided into
> disentangled and non-interpretable variable. Usually what happens is these
> assumptions do not scale well to different data. Otherwise overall the
> model looks promising and would be interesting implement. Maybe we could
> add this as a feature to main VAE framework(like an alternative for use in
> semi-supervised learning scenarios) since VAE as of itself is unsupervised.
> Let me know what you think. Thank you,
>
> Arjun
>
> On Tue, Mar 13, 2018 at 7:29 PM, Marcus Edel <marcus.edel at fu-berlin.de>
> wrote:
>
>> Hello Arjun,
>>
>> Thank you, Marcus, for the quick reply.  That clarifies the doubts I had.
>> I am
>> interested in both the projects, reinforcement learning and variational
>> autoencoders with almost equal importance to both. So is there any way
>> that I
>> can involve in both the projects. Maybe focus on one and have some
>> involvement
>> in the other?. In that case, how would I write a proposal to this
>> effect(write
>> as two separate proposals or mention my interest in both under one
>> proposal)?
>>
>>
>> I think we could combine both ideas, something like
>> https://arxiv.org/abs/1709.05047 could work, let me know if that would
>> is an
>> option you are interested in.
>>
>> Thanks,
>> Marcus
>>
>> On 12. Mar 2018, at 23:45, arjun k <arjun.k018 at gmail.com> wrote:
>>
>> Hi,
>>
>> Thank you, Marcus, for the quick reply.  That clarifies the doubts I had.
>> I am interested in both the projects, reinforcement learning and
>> variational autoencoders with almost equal importance to both. So is there
>> any way that I can involve in both the projects. Maybe focus on one and
>> have some involvement in the other?. In that case, how would I write a
>> proposal to this effect(write as two separate proposals or mention my
>> interest in both under one proposal)?
>>
>> On Mon, Mar 12, 2018 at 9:41 AM, Marcus Edel <marcus.edel at fu-berlin.de>
>> wrote:
>>
>>> Hello Arjun,
>>>
>>> welcome and thanks for getting in touch.
>>>
>>> I am Arjun, currently pursuing my Master's in Computer Science at the
>>> University
>>> of Massachusetts, Amherst, I came across the project on variational
>>> autoencoders
>>> and Reinforcement learning project and they look very interesting. Hope
>>> I am not
>>> too late.
>>>
>>>
>>> The application phase opens today, so you are not too late.
>>>
>>> I am more interested in the reinforcement learning project as it
>>> involves some
>>> research in a field that I am working on and would like to get involved.
>>> As I
>>> understand, coding up an algorithm and implementing it in a single game
>>> would
>>> not be much of an issue. How many algorithms are proposed to be
>>> benchmarked
>>> against each other? Is there any new idea that is being tested or the
>>> research
>>> component is the benchmark alone?
>>>
>>>
>>> Keep in mind each method has to be tested and documented, which takes
>>> time, so
>>> my recommendation is to focus on one or two (depending on the method).
>>> The
>>> research component is two-fold, you could compare different algorithms or
>>> improve/ extend the method you are working on e.g. by integrating
>>> another search
>>> strategy, but this isn't a must, the focus is to extend the existing
>>> framework.
>>>
>>> In the variational encoders I am quite familiar with generative modeling
>>> having
>>> worked on some research projects myself(https://arxiv.org/abs/1802.07401),
>>> As we
>>> can make variational encoders is just a training procedure, how
>>> abstracted are
>>> you intending the implementation to be. Should the framework allow the
>>> user to
>>> be able to customize the underlying neural network and add additional
>>> features
>>> or is it highly abstracted with no control over the underlying
>>> architecture and
>>> only able to use VAE as a black box?
>>>
>>>
>>> Ideally, a user can modify the model structure based on the existing
>>> infrastructure, providing a black box, is something that naturally
>>> results from
>>> the first idea. And could be realized in the form of a specific model
>>> something
>>> like: https://github.com/mlpack/models/tree/master/Kaggle/DigitRec
>>> ognizer
>>>
>>> I hope anything I said was helpful, let me know if I should clarify
>>> anything.
>>>
>>> Thanks,
>>> Marcus
>>>
>>> On 11. Mar 2018, at 22:23, arjun k <arjun.k018 at gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> I am Arjun, currently pursuing my Master's in Computer Science at the
>>> University of Massachusetts, Amherst, I came across the project on
>>> variational autoencoders and Reinforcement learning project and they look
>>> very interesting. Hope I am not too late.
>>>
>>> I am more interested in the reinforcement learning project as it
>>> involves some research in a field that I am working on and would like to
>>> get involved. As I understand, coding up an algorithm and implementing it
>>> in a single game would not be much of an issue. How many algorithms are
>>> proposed to be benchmarked against each other? Is there any new idea that
>>> is being tested or the research component is the benchmark alone?
>>>
>>> In the variational encoders I am quite familiar with generative modeling
>>> having worked on some research projects myself(https://arxiv.org/abs/1
>>> 802.07401), As we can make variational encoders is just a training
>>> procedure, how abstracted are you intending the implementation to be.
>>> Should the framework allow the user to be able to customize the underlying
>>> neural network and add additional features or is it highly abstracted with
>>> no control over the underlying architecture and only able to use VAE as a
>>> black box?
>>>
>>> Thank you,
>>> Arjun Karuvally,
>>> College of Information and Computer Science,
>>> University of Massachusetts, Amherst.
>>> _______________________________________________
>>> mlpack mailing list
>>> mlpack at lists.mlpack.org
>>> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20180315/2549c3ee/attachment.html>