[mlpack] regression using sparse matrices/vectors

[Ryan Curtin]

On Tue, Dec 10, 2013 at 01:09:08PM -0500, Darryl Reeves wrote:
> Hi Ryan,
> 
> Thanks for the quick response. This is helpful. I really like the project
> and was worried that I might have to go in a different direction to find a
> solution for my problem. It is totally fine if the parameters of my model
> are not sparse. However, due to the size of the data that I'm working with,
> its not feasible for me to do the regression using the dense representation
> of the predictor matrix. Can you provide some direction on which code
> changes would be required to templatize the regression algorithms?

Refactoring LARS (least-angle regression) would be difficult but
refactoring LinearRegression (or ridge regression, they are basically
the same) would be much simpler.  The catch, though, which I
unfortunately didn't realize in my last email, is that arma::sp_mat (the
sparse matrix class) does not support either the QR decomposition or
inv() / solve().  This is something I am working on over time, but
there's nothing in place quite yet.

The current plan for Armadillo is to release version 4 shortly after New
Year's, and it's my job to get qr() and inv() and solve() in there by
then.   Once that's done I can refactor the LinearRegression class in
like five minutes.  If you wanted LARS and not LinearRegression, that'll
take a fair amount of refactoring thought, though.

So anyway, if you can wait a few weeks, everything will be ready to go,
but if not your best bet is probably MATLAB's sparse matrix support and
its sparse linear equation solver to do linear regression or ridge
regression.  See
http://www.mathworks.com/help/matlab/math/sparse-matrix-operations.html#f6-9169
for some more info on that.

Thanks,

Ryan

-- 
Ryan Curtin       | "You can think about it... but don't do it."
ryan at igglybob.com |   - Sheriff Justice