[mlpack] Implementing MDS

[Ryan Curtin]

On Sat, Mar 07, 2015 at 02:12:28AM +0530, Dhawal Arora wrote:
> Hi Ryan,
> 
> Sorry couldn't keep up with the superLU lately. Its kinda going a bit slow
> here right now as my exams are going on. Sad that mlpack couldn't make it
> through this time, badly wanted to complete this manifold stuff this
> summer. But i still would like to contribute. Had a look at the metrics,
> using mahalonobis for euclidean distance although working to implement some
> other distances too.
> 
> following this paper(for MDS) although i've read the paper on the ideas
> page:
> http://www.math.uwaterloo.ca/~aghodsib/courses/f06stat890/readings/tutorial_stat890.pdf
> 
> a proof(pg 16) in his paper helped me switch to above implementation.
> http://www.lcayton.com/resexam.pdf
> 
> Also Is it over with eigs_sym_pair() ? Could you please tell where
> eigs_sym_pair() would fit in here? Right now in MDS i am using eigs_pair
> and eigs_sym of armadillo.
> Also not getting any clear idea or paper for non-metric MDS. If you could
> help on this. Will then probably try testing it in comparison to scikit.

Hi Dhawal,

I'm sorry for the slow response.  Things have been slow with SuperLU,
but I almost have a wrapper ready for submission upstream to Armadillo.
I will let you know when that's done.

It's fine to use eig_pair() and eig_sym() (eig, not eigs), but this will
be slower: you only need a few of the top eigenvalues and eigenvectors
of the input matrix.  So you can implement MDS just fine for now, and
MDS itself doesn't need eigs_pair() -- you should be able to just use
eigs_sym() to get a few of the eigenvectors.  I think that only some of
the other spectral manifold learning algorithms need eigs_sym_pair().

I think that just using eigendecomposition you should be able to
implement classical MDS and metric MDS, but I think that non-metric MDS
will have to be an entirely different algorithm.  I'm not an MDS expert,
so I don't have any references handy for non-metric MDS, unfortunately.

Anyway, let me know if I can help further. :)

Thanks,

Ryan

-- 
Ryan Curtin    | "Somebody dropped a bag on the sidewalk."
ryan at ratml.org |   - Kit