[mlpack] MVU Bug Fix GSOC

[LI Xuran]

Hi Ryan,

Thanks a lot for the advice and sorry for late reply. I spent the past two days to went through most of the papers and it really helps. Apart from the previous idea:

1.generate random simple dataset, and compare normal MVU using mlpack Primal dualsolver  on it. compare it with the result of MVU +LRSDP.
2.write unit tests and substitution for the original code in LRSDP, check their correctness over processing of the above datasets.

I also have some new ideas coming up:

1.it is mentioned in multiple papers that the selection of parameters for the penalty, knn, and learning rate etc is critical for the convergence of the algorithm. Thus, I think it would be helpful to either implement a dynamical parameter adjusting algorithm or to check manually on the variables in the cases where the algorithm is not convergent.
2. there are several propositions and characteristics that a functioning LRSDP should meet mentioned in the papers.(i.e.nonzero duality gap and XS==0 for optimal solution etc) I am making a list of these properties and the goal is that to implement tests to check that these properties would be met in each iteration.
3. a not recommended alternative would be to change the implementation in LRSDP, for example, change the optimization criteria to search for the maximum distance of furthest neighbor, or to integrate a dual approach into the original code. The attractiveness of such action is that it would provide a theoretical guarantee that the algorithm would converge and have an optimal result(while by using LRSDP alone we only have optimal results on most cases), but I don't quite like it because it might bring side effects and deficiency on runtime. Also, I think it is not guaranteed to work as it was only proposed as an alternative in the papers. So I am currently thinking of taking up this approach as a last alternative if no bug is found in previous sections.

So that is all I had for now, Would you think some of it worth a try? I will read the rest of the pdf tomorrow, and change my proposal according to your suggestion.

Best Wishes,
Daniel Li


________________________________
From: LI Xuran
Sent: 18 March 2018 20:15:57
To: mlpack at lists.mlpack.org
Subject: Re: MVU Bug Fix GSOC


Dear Ryan,


 I am currently working on my proposal for the Fixes to MVU and low-rank semidefinite programs and have come up with the following ideas:
1.generate random simple dataset, and compare normal MVU with MVU +LRSDP on it. do visualization of the procedure and the result in 2d/3d.
2. write unit tests and substitution for the original code in mlpack's MVU implementation and check their correctness over processing of the above datasets.
3. base on the observation of the result of 1 and 2, create datasets that particularly points out the issue ... and check step by step on that sample
4.(or maybe datasets with a special property such that it should always converge by an implementation of MVU + LRSDP  and check if the expected result  is met )

do you think any of the above ideas worth a try?

Thanks!

Daniel Li


________________________________
From: LI Xuran
Sent: 17 March 2018 17:47:09
To: mlpack at lists.mlpack.org
Subject: MVU Bug Fix GSOC


Hello Ryan,

I am Daniel Li, a second-year student studying Artificial in the University of Edinburgh. I write fluent c++ code and is interested in taking up the quest to fix bugs regarding MVU and semidefinite programming in mlpack. I've read  about scalable semidefinite manifold learning and other articles and set up mlpack on my own computer. Could you give me some advice as for where to start my research on the project as I familiar myself with the code base? Also is it a good idea to implement the MVU with dual-tree algorithm to compare with the  current version of MVU using LRSDP?

Thanks!

Daniel Li
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20180321/f8cc281c/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20180321/f8cc281c/attachment-0001.ksh>