[mlpack] GSoC'20

[Yihan Wang]

Hi Marcus,

I have got familiar with the code base and the listed papers. And I think
these are the modifications we should add to the original CMA-ES.
* ACM-ES
We can add a surrogate model into the original algorithm. We check whether
the model is given during the iteration, and call it if so.
* IPOP-CMA-ES
The key idea is to restart the algorithm with a larger population when
terminated.
* Active-CMA-ES
The key idea is to introduce new update functions to variance matrix and
other parameters. And I think we can extract the update functions in the
main algorithm as separated functions which can be overwritten in derived
classes.

Here are steps that I think we need to do:
Phase #1
1. making initialization, iteration and update rules separated functions.
2. implement IPOP-CMA-ES, which will be easier on basis of step #1.
3. test and test preparation for the other two algorithms
4. documentation

Phase #2
1. implement Active-CMA-ES
2. test and documentation

Phase #3
1. implement ACM-ES
2. test and documentation

Please let me know if there is any suggestion on the plan.

Thanks,
Yihan


Marcus Edel <marcus.edel at fu-berlin.de> 于2020年3月9日周一 上午7:08写道：

> Hello Yihan,
>
> > * Enhance CMA-ES I have began to check the references listed, and I have
> a
> > question related to the current mlpack. Currently is there an original
> CMA-ES
> > algorithm in the mlpack? If there is none, I can begin from the original
> > implementation.
>
> All mlpack optimizers are in another repository including the CMA-ES
> optimizer:
> https://github.com/mlpack/ensmallen and
>
> https://github.com/mlpack/ensmallen/tree/master/include/ensmallen_bits/cmaes
> .
>
> > * Implement the Transformer in mlpack I think what we need to do is first
> > implement an attention layer and then the transformer itself. For
> testing, we
> > can compare the result with results got from pytorch or so.
>
> Agreed, mlpack doesn't implement an attention layer.
>
> Let me know if I should clarify anything.
>
> Thanks,
> Marcus
>
> > On 8. Mar 2020, at 07:54, Yihan Wang <wangyihan617 at gmail.com> wrote:
> >
> > Hi all,
> >
> > I am Yihan Wang, a final year student from Tsinghua University, with
> more than a year's research experience in machine learning algorithms. I am
> interested in participating in this year's GSoC. In particular I am
> interested in these two topics.
> >
> > * Enhance CMA-ES
> > I have began to check the references listed, and I have a question
> related to the current mlpack. Currently is there an original CMA-ES
> algorithm in the mlpack? If there is none, I can begin from the original
> implementation.
> >
> > * Implement the Transformer in mlpack
> > I think what we need to do is first implement an attention layer and
> then the transformer itself. For testing, we can compare the result with
> results got from pytorch or so.
> >
> > Is there any suggestion related to these two ideas?
> >
> > Best,
> > Yihan
> > _______________________________________________
> > mlpack mailing list
> > mlpack at lists.mlpack.org
> > http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20200316/8f42449e/attachment.htm>