mlpack  master

An implementation of LARS, a stage-wise homotopy-based algorithm for l1-regularized linear regression (LASSO) and l1+l2 regularized linear regression (Elastic Net). More...

Public Member Functions

 LARS (const bool useCholesky=false, const double lambda1=0.0, const double lambda2=0.0, const double tolerance=1e-16)
 Set the parameters to LARS. More...

 
 LARS (const bool useCholesky, const arma::mat &gramMatrix, const double lambda1=0.0, const double lambda2=0.0, const double tolerance=1e-16)
 Set the parameters to LARS, and pass in a precalculated Gram matrix. More...

 
mlpack_deprecated LARS (const arma::mat &data, const arma::vec &responses, const bool transposeData=true, const bool useCholesky=false, const double lambda1=0.0, const double lambda2=0.0, const double tolerance=1e-16)
 Set the parameters to LARS and run training. More...

 
mlpack_deprecated LARS (const arma::mat &data, const arma::vec &responses, const bool transposeData, const bool useCholesky, const arma::mat &gramMatrix, const double lambda1=0.0, const double lambda2=0.0, const double tolerance=1e-16)
 Set the parameters to LARS, pass in a precalculated Gram matrix, and run training. More...

 
 LARS (const arma::mat &data, const arma::rowvec &responses, const bool transposeData=true, const bool useCholesky=false, const double lambda1=0.0, const double lambda2=0.0, const double tolerance=1e-16)
 Set the parameters to LARS and run training. More...

 
 LARS (const arma::mat &data, const arma::rowvec &responses, const bool transposeData, const bool useCholesky, const arma::mat &gramMatrix, const double lambda1=0.0, const double lambda2=0.0, const double tolerance=1e-16)
 Set the parameters to LARS, pass in a precalculated Gram matrix, and run training. More...

 
const std::vector< size_t > & ActiveSet () const
 Access the set of active dimensions. More...

 
const arma::vec & Beta () const
 Access the solution coefficients. More...

 
const std::vector< arma::vec > & BetaPath () const
 Access the set of coefficients after each iteration; the solution is the last element. More...

 
const std::vector< double > & LambdaPath () const
 Access the set of values for lambda1 after each iteration; the solution is the last element. More...

 
const arma::mat & MatUtriCholFactor () const
 Access the upper triangular cholesky factor. More...

 
mlpack_deprecated void Predict (const arma::mat &points, arma::vec &predictions, const bool rowMajor=false) const
 Predict y_i for each data point in the given data matrix, using the currently-trained LARS model (so make sure you run Regress() first). More...

 
void Predict (const arma::mat &points, arma::rowvec &predictions, const bool rowMajor=false) const
 Predict y_i for each data point in the given data matrix using the currently-trained LARS model. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const unsigned int)
 Serialize the LARS model. More...

 
mlpack_deprecated void Train (const arma::mat &data, const arma::vec &responses, arma::vec &beta, const bool transposeData=true)
 Run LARS. More...

 
mlpack_deprecated void Train (const arma::mat &data, const arma::vec &responses, const bool transposeData=true)
 Run LARS. More...

 
void Train (const arma::mat &data, const arma::rowvec &responses, arma::vec &beta, const bool transposeData=true)
 Run LARS. More...

 
void Train (const arma::mat &data, const arma::rowvec &responses, const bool transposeData=true)
 Run LARS. More...

 

Detailed Description

An implementation of LARS, a stage-wise homotopy-based algorithm for l1-regularized linear regression (LASSO) and l1+l2 regularized linear regression (Elastic Net).

Let $ X $ be a matrix where each row is a point and each column is a dimension and let $ y $ be a vector of responses.

The Elastic Net problem is to solve

\[ \min_{\beta} 0.5 || X \beta - y ||_2^2 + \lambda_1 || \beta ||_1 + 0.5 \lambda_2 || \beta ||_2^2 \]

where $ \beta $ is the vector of regression coefficients.

If $ \lambda_1 > 0 $ and $ \lambda_2 = 0 $, the problem is the LASSO. If $ \lambda_1 > 0 $ and $ \lambda_2 > 0 $, the problem is the elastic net. If $ \lambda_1 = 0 $ and $ \lambda_2 > 0 $, the problem is ridge regression. If $ \lambda_1 = 0 $ and $ \lambda_2 = 0 $, the problem is unregularized linear regression.

Note: This algorithm is not recommended for use (in terms of efficiency) when $ \lambda_1 $ = 0.

For more details, see the following papers:

@article{efron2004least,
title={Least angle regression},
author={Efron, B. and Hastie, T. and Johnstone, I. and Tibshirani, R.},
journal={The Annals of statistics},
volume={32},
number={2},
pages={407--499},
year={2004},
publisher={Institute of Mathematical Statistics}
}
@article{zou2005regularization,
title={Regularization and variable selection via the elastic net},
author={Zou, H. and Hastie, T.},
journal={Journal of the Royal Statistical Society Series B},
volume={67},
number={2},
pages={301--320},
year={2005},
publisher={Royal Statistical Society}
}

Definition at line 89 of file lars.hpp.

Constructor & Destructor Documentation

◆ LARS() [1/6]

LARS ( const bool  useCholesky = false,
const double  lambda1 = 0.0,
const double  lambda2 = 0.0,
const double  tolerance = 1e-16 
)

Set the parameters to LARS.

Both lambda1 and lambda2 default to 0.

Parameters
useCholeskyWhether or not to use Cholesky decomposition when solving linear system (as opposed to using the full Gram matrix).
lambda1Regularization parameter for l1-norm penalty.
lambda2Regularization parameter for l2-norm penalty.
toleranceRun until the maximum correlation of elements in (X^T y) is less than this.

◆ LARS() [2/6]

LARS ( const bool  useCholesky,
const arma::mat &  gramMatrix,
const double  lambda1 = 0.0,
const double  lambda2 = 0.0,
const double  tolerance = 1e-16 
)

Set the parameters to LARS, and pass in a precalculated Gram matrix.

Both lambda1 and lambda2 default to 0.

Parameters
useCholeskyWhether or not to use Cholesky decomposition when solving linear system (as opposed to using the full Gram matrix).
gramMatrixGram matrix.
lambda1Regularization parameter for l1-norm penalty.
lambda2Regularization parameter for l2-norm penalty.
toleranceRun until the maximum correlation of elements in (X^T y) is less than this.

◆ LARS() [3/6]

mlpack_deprecated LARS ( const arma::mat &  data,
const arma::vec &  responses,
const bool  transposeData = true,
const bool  useCholesky = false,
const double  lambda1 = 0.0,
const double  lambda2 = 0.0,
const double  tolerance = 1e-16 
)

Set the parameters to LARS and run training.

Both lambda1 and lambda2 are set by default to 0.

Parameters
dataInput data.
responsesA vector of targets.
transposeDataShould be true if the input data is column-major and false otherwise.
useCholeskyWhether or not to use Cholesky decomposition when solving linear system (as opposed to using the full Gram matrix).
lambda1Regularization parameter for l1-norm penalty.
lambda2Regularization parameter for l2-norm penalty.
toleranceRun until the maximum correlation of elements in (X^T y) is less than this.

◆ LARS() [4/6]

mlpack_deprecated LARS ( const arma::mat &  data,
const arma::vec &  responses,
const bool  transposeData,
const bool  useCholesky,
const arma::mat &  gramMatrix,
const double  lambda1 = 0.0,
const double  lambda2 = 0.0,
const double  tolerance = 1e-16 
)

Set the parameters to LARS, pass in a precalculated Gram matrix, and run training.

Both lambda1 and lambda2 are set by default to 0.

Parameters
dataInput data.
responsesA vector of targets.
transposeDataShould be true if the input data is column-major and false otherwise.
useCholeskyWhether or not to use Cholesky decomposition when solving linear system (as opposed to using the full Gram matrix).
gramMatrixGram matrix.
lambda1Regularization parameter for l1-norm penalty.
lambda2Regularization parameter for l2-norm penalty.
toleranceRun until the maximum correlation of elements in (X^T y) is less than this.

◆ LARS() [5/6]

LARS ( const arma::mat &  data,
const arma::rowvec &  responses,
const bool  transposeData = true,
const bool  useCholesky = false,
const double  lambda1 = 0.0,
const double  lambda2 = 0.0,
const double  tolerance = 1e-16 
)

Set the parameters to LARS and run training.

Both lambda1 and lambda2 are set by default to 0.

Parameters
dataInput data.
responsesA vector of targets.
transposeDataShould be true if the input data is column-major and false otherwise.
useCholeskyWhether or not to use Cholesky decomposition when solving linear system (as opposed to using the full Gram matrix).
lambda1Regularization parameter for l1-norm penalty.
lambda2Regularization parameter for l2-norm penalty.
toleranceRun until the maximum correlation of elements in (X^T y) is less than this.

◆ LARS() [6/6]

LARS ( const arma::mat &  data,
const arma::rowvec &  responses,
const bool  transposeData,
const bool  useCholesky,
const arma::mat &  gramMatrix,
const double  lambda1 = 0.0,
const double  lambda2 = 0.0,
const double  tolerance = 1e-16 
)

Set the parameters to LARS, pass in a precalculated Gram matrix, and run training.

Both lambda1 and lambda2 are set by default to 0.

Parameters
dataInput data.
responsesA vector of targets.
transposeDataShould be true if the input data is column-major and false otherwise.
useCholeskyWhether or not to use Cholesky decomposition when solving linear system (as opposed to using the full Gram matrix).
gramMatrixGram matrix.
lambda1Regularization parameter for l1-norm penalty.
lambda2Regularization parameter for l2-norm penalty.
toleranceRun until the maximum correlation of elements in (X^T y) is less than this.

Member Function Documentation

◆ ActiveSet()

const std::vector<size_t>& ActiveSet ( ) const
inline

Access the set of active dimensions.

Definition at line 320 of file lars.hpp.

◆ Beta()

const arma::vec& Beta ( ) const
inline

Access the solution coefficients.

Definition at line 327 of file lars.hpp.

◆ BetaPath()

const std::vector<arma::vec>& BetaPath ( ) const
inline

Access the set of coefficients after each iteration; the solution is the last element.

Definition at line 324 of file lars.hpp.

◆ LambdaPath()

const std::vector<double>& LambdaPath ( ) const
inline

Access the set of values for lambda1 after each iteration; the solution is the last element.

Definition at line 331 of file lars.hpp.

◆ MatUtriCholFactor()

const arma::mat& MatUtriCholFactor ( ) const
inline

Access the upper triangular cholesky factor.

Definition at line 334 of file lars.hpp.

References LARS::serialize().

◆ Predict() [1/2]

mlpack_deprecated void Predict ( const arma::mat &  points,
arma::vec &  predictions,
const bool  rowMajor = false 
) const

Predict y_i for each data point in the given data matrix, using the currently-trained LARS model (so make sure you run Regress() first).

If the data matrix is row-major (as opposed to the usual column-major format for mlpack matrices), set rowMajor = true to avoid an extra transpose.

Parameters
pointsThe data points to regress on.
predictionsy, which will contained calculated values on completion.

◆ Predict() [2/2]

void Predict ( const arma::mat &  points,
arma::rowvec &  predictions,
const bool  rowMajor = false 
) const

Predict y_i for each data point in the given data matrix using the currently-trained LARS model.

Parameters
pointsThe data points to regress on.
predictionsy, which will contained calculated values on completion.
rowMajorShould be true if the data points matrix is row-major and false otherwise.

◆ serialize()

void serialize ( Archive &  ar,
const unsigned  int 
)

Serialize the LARS model.

Referenced by LARS::MatUtriCholFactor().

◆ Train() [1/4]

mlpack_deprecated void Train ( const arma::mat &  data,
const arma::vec &  responses,
arma::vec &  beta,
const bool  transposeData = true 
)

Run LARS.

The input matrix (like all mlpack matrices) should be column-major – each column is an observation and each row is a dimension. However, because LARS is more efficient on a row-major matrix, this method will (internally) transpose the matrix. If this transposition is not necessary (i.e., you want to pass in a row-major matrix), pass 'false' for the transposeData parameter.

Parameters
dataColumn-major input data (or row-major input data if rowMajor = true).
responsesA vector of targets.
betaVector to store the solution (the coefficients) in.
transposeDataSet to false if the data is row-major.

◆ Train() [2/4]

mlpack_deprecated void Train ( const arma::mat &  data,
const arma::vec &  responses,
const bool  transposeData = true 
)

Run LARS.

The input matrix (like all mlpack matrices) should be column-major – each column is an observation and each row is a dimension. However, because LARS is more efficient on a row-major matrix, this method will (internally) transpose the matrix. If this transposition is not necessary (i.e., you want to pass in a row-major matrix), pass 'false' for the transposeData parameter.

Parameters
dataInput data.
responsesA vector of targets.
transposeDataShould be true if the input data is column-major and false otherwise.

◆ Train() [3/4]

void Train ( const arma::mat &  data,
const arma::rowvec &  responses,
arma::vec &  beta,
const bool  transposeData = true 
)

Run LARS.

The input matrix (like all mlpack matrices) should be column-major – each column is an observation and each row is a dimension. However, because LARS is more efficient on a row-major matrix, this method will (internally) transpose the matrix. If this transposition is not necessary (i.e., you want to pass in a row-major matrix), pass 'false' for the transposeData parameter.

Parameters
dataColumn-major input data (or row-major input data if rowMajor = true).
responsesA vector of targets.
betaVector to store the solution (the coefficients) in.
transposeDataSet to false if the data is row-major.

◆ Train() [4/4]

void Train ( const arma::mat &  data,
const arma::rowvec &  responses,
const bool  transposeData = true 
)

Run LARS.

The input matrix (like all mlpack matrices) should be column-major – each column is an observation and each row is a dimension. However, because LARS is more efficient on a row-major matrix, this method will (internally) transpose the matrix. If this transposition is not necessary (i.e., you want to pass in a row-major matrix), pass 'false' for the transposeData parameter.

Parameters
dataInput data.
responsesA vector of targets.
transposeDataShould be true if the input data is column-major and false otherwise.

The documentation for this class was generated from the following file: