[mlpack] GSoC query

[Manish Kumar]

Thanks Ryan for helping me out. I got little anxious by learning the fact
that LMNN+LRSDP may not work and was thinking that my GSoC project may go
futile.

I do want to continue LMNN+LRSDP. I also see a good opportunity to do
something new while working on LMNN+LRSDP and that's why I never planned to
leave it but was looking for a perfect time for a project like this.

I will stick to the proposal and will try to achieve the results.

I just want to propose a little modification to the proposal. Tell me, if
that sounds fair.
I went through BoostMetric literature and find it as a significant
improvement over LMNN. So, can we at least include BoostMetric as a part of
the project apart from LMNN. I am sure that it will be a pretty good
addition and currently except author's implementation there is no other
implementation out there. Moreover, existing implementation takes the
exponential loss into consideration and we may include implementation based
on logistic loss function which shows improvement over the exponential one.
If that's okay, I will make some small changes to timeline and proposal.
This way we could have some pretty awesome metric learning algorithms.

On 16 March 2018 at 19:50, Ryan Curtin <ryan at ratml.org> wrote:

> On Fri, Mar 16, 2018 at 12:52:00PM +0530, Manish Kumar wrote:
> > Hello,
> > I am Manish Kumar (IRC: manish7294). After yesterday's discussion on
> IRC, I
> > went looking for the other optimal reliable options related to metric
> > learning. And have picked out some relatively comparable alternatives to
> > LMNN by going through literature in detail.
> >
> > 1. BoostMetric
> > <https://pdfs.semanticscholar.org/65af/3d9b9424cebb1054aac6f
> 71bf2e39a3b1994.pdf>(It
> > belongs to the category of supervised learning. It takes LMNN background
> as
> > its basis and tries to improve it by incorporating alternative
> exponential
> > loss function and a different optimization technique. Overall it combines
> > the characteristics of boosting and metric learning and has claimed to
> > outperform LMNN. See page 7 of literature for the results.)
> >
> > 2. ITML <http://www.cs.utexas.edu/users/pjain/pubs/metriclearning_
> icml.pdf>
> > (This one belongs to unsupervised-category and requires the external
> > knowledge of similar and dissimilar data points which acts as
> constraints.
> > Though constraints can be generated on the basis of labels, subsequently
> > shifting ITML to the supervised category. This one has shown results
> > comparable to LMNN as well.)
> >
> > After discussions, I realized that it will not be a good idea to propose
> > something that doesn't ensure to work at the end. So, for the time
> being, I
> > have decided to put LMNN with LRSDP at the halt and continue it from the
> > same point in near future as a commendable test project.
> >
> > At this point, I may need to re-design my proposal. So, I humbly request
> > you to give your feedback on my thought. I intend to include the
> > implementation of these two state-of-art algorithms, if favorable. I
> expect
> > them to give a solid boost to metric learning algorithms.
>
> Hi Manish,
>
> I didn't mean to imply that LMNN might not be useful.  We already have
> an SDP solver so it should be possible to at least implement LMNN to use
> the existing SDP solver and I think it is expected that that will work
> just fine.  So I don't think your proposal needs to be redefined, but I
> do think maybe a good "first step" is to get LMNN working with mlpack's
> PrimalDualSolver for SDPs (found in src/mlpack/core/optimizers/sdp/).
> Then the rest of the project can be making it work with LRSDPs.  This
> way, if we do have success with LRSDPs, I think that we might have some
> interesting results that could be published at some workshop.  And if
> there is no success with LRSDPs, it is not a huge issue since we already
> have LMNN working with the regular SDP solver.
>
> Here's another paper discussing acceleration of LMNN, but I think that
> focuses on accelerating the nearest neighbor search step, not the actual
> solution of the SDP:
>
> https://dl.acm.org/citation.cfm?id=1390302
>
> If you'd rather work on BoostMetric and ITML, please feel free to adjust
> your proposal for that, but I definitely don't want you to get the
> impression that LMNN+LRSDP is not a good project---I think it is just
> fine, assuming that we can at least have an implementation that will
> work with a regular SDP solver for the case where we can't get the LRSDP
> to converge.
>
> I hope this helps clarify.
>
> Thanks,
>
> Ryan
>
> --
> Ryan Curtin    | "That rug really tied the room together."
> ryan at ratml.org |   - The Dude
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20180316/1bbc0415/attachment.html>