mlpack  2.2.5
LogisticRegression< MatType > Class Template Reference

The LogisticRegression class implements an L2-regularized logistic regression model, and supports training with multiple optimizers and classification. More...

Public Member Functions

 LogisticRegression (const MatType &predictors, const arma::Row< size_t > &responses, const double lambda=0)
 Construct the LogisticRegression class with the given labeled training data. More...

 
 LogisticRegression (const MatType &predictors, const arma::Row< size_t > &responses, const arma::vec &initialPoint, const double lambda=0)
 Construct the LogisticRegression class with the given labeled training data. More...

 
 LogisticRegression (const size_t dimensionality, const double lambda=0)
 Construct the LogisticRegression class without performing any training. More...

 
template
<
template
<
typename
>
class
OptimizerType
>
 LogisticRegression (OptimizerType< LogisticRegressionFunction< MatType >> &optimizer)
 Construct the LogisticRegression class with the given labeled training data. More...

 
template
<
typename
VecType
>
size_t Classify (const VecType &point, const double decisionBoundary=0.5) const
 Classify the given point. More...

 
void Classify (const MatType &dataset, arma::Row< size_t > &labels, const double decisionBoundary=0.5) const
 Classify the given points, returning the predicted labels for each point. More...

 
void Classify (const MatType &dataset, arma::mat &probabilities) const
 Classify the given points, returning class probabilities for each point. More...

 
double ComputeAccuracy (const MatType &predictors, const arma::Row< size_t > &responses, const double decisionBoundary=0.5) const
 Compute the accuracy of the model on the given predictors and responses, optionally using the given decision boundary. More...

 
double ComputeError (const MatType &predictors, const arma::Row< size_t > &responses) const
 Compute the error of the model. More...

 
const double & Lambda () const
 Return the lambda value for L2-regularization. More...

 
double & Lambda ()
 Modify the lambda value for L2-regularization. More...

 
const arma::vec & Parameters () const
 Return the parameters (the b vector). More...

 
arma::vec & Parameters ()
 Modify the parameters (the b vector). More...

 
mlpack_deprecated void Predict (const MatType &predictors, arma::Row< size_t > &responses, const double decisionBoundary=0.5) const
 Predict the responses to a given set of predictors. More...

 
template
<
typename
Archive
>
void Serialize (Archive &ar, const unsigned int)
 Serialize the model. More...

 
template
<
template
<
typename
>
class
OptimizerType
=
mlpack::optimization::L_BFGS
>
void Train (const MatType &predictors, const arma::Row< size_t > &responses)
 Train the LogisticRegression model on the given input data. More...

 
template
<
template
<
typename
>
class
OptimizerType
=
mlpack::optimization::L_BFGS
>
void Train (OptimizerType< LogisticRegressionFunction< MatType >> &optimizer)
 Train the LogisticRegression model with the given instantiated optimizer. More...

 

Detailed Description


template
<
typename
MatType
=
arma::mat
>

class mlpack::regression::LogisticRegression< MatType >

The LogisticRegression class implements an L2-regularized logistic regression model, and supports training with multiple optimizers and classification.

The class supports different observation types via the MatType template parameter; for instance, logistic regression can be performed on sparse datasets by specifying arma::sp_mat as the MatType parameter.

Template Parameters
MatTypeType of data matrix.

Definition at line 34 of file logistic_regression.hpp.

Constructor & Destructor Documentation

◆ LogisticRegression() [1/4]

LogisticRegression ( const MatType &  predictors,
const arma::Row< size_t > &  responses,
const double  lambda = 0 
)

Construct the LogisticRegression class with the given labeled training data.

This will train the model. Optionally, specify lambda, which is the penalty parameter for L2-regularization. If not specified, it is set to 0, which results in standard (unregularized) logistic regression.

It is not possible to set a custom optimizer with this constructor. Either use a constructor that does not train and call Train() with a custom optimizer type, or use the constructor that takes an instantiated optimizer. (This unfortunate situation is a language restriction of C++.)

Parameters
predictorsInput training variables.
responsesOutputs resulting from input training variables.
lambdaL2-regularization parameter.

◆ LogisticRegression() [2/4]

LogisticRegression ( const MatType &  predictors,
const arma::Row< size_t > &  responses,
const arma::vec &  initialPoint,
const double  lambda = 0 
)

Construct the LogisticRegression class with the given labeled training data.

This will train the model. Optionally, specify lambda, which is the penalty parameter for L2-regularization. If not specified, it is set to 0, which results in standard (unregularized) logistic regression.

It is not possible to set a custom optimizer with this constructor. Either use a constructor that does not train and call Train() with a custom optimizer type, or use the constructor that takes an instantiated optimizer. (This unfortunate situation is a language restriction of C++.)

Parameters
predictorsInput training variables.
responsesOutputs results from input training variables.
initialPointInitial model to train with.
lambdaL2-regularization parameter.

◆ LogisticRegression() [3/4]

LogisticRegression ( const size_t  dimensionality,
const double  lambda = 0 
)

Construct the LogisticRegression class without performing any training.

The dimensionality of the data (which will be used to set the size of the parameters vector) must be specified, and all of the parameters in the model will be set to 0. Note that the dimensionality may be changed later by directly modifying the parameters vector (using Parameters()).

Parameters
dimensionalityDimensionality of the data.
lambdaL2-regularization parameter.

◆ LogisticRegression() [4/4]

LogisticRegression ( OptimizerType< LogisticRegressionFunction< MatType >> &  optimizer)

Construct the LogisticRegression class with the given labeled training data.

This will train the model. This overload takes an already instantiated optimizer (which holds the LogisticRegressionFunction error function, which must also be instantiated), so that the optimizer can be configured before the training is run by this constructor. The predictors and responses and initial point are all taken from the error function contained in the optimizer.

Parameters
optimizerInstantiated optimizer with instantiated error function.

Member Function Documentation

◆ Classify() [1/3]

size_t Classify ( const VecType &  point,
const double  decisionBoundary = 0.5 
) const

Classify the given point.

The predicted label is returned. Optionally, specify the decision boundary; logistic regression returns a value between 0 and 1. If the value is greater than the decision boundary, the response is taken to be 1; otherwise, it is 0. By default the decision boundary is 0.5.

Parameters
pointPoint to classify.
decisionBoundaryDecision boundary (default 0.5).
Returns
Predicted label of point.

Referenced by LogisticRegression< MatType >::Lambda().

◆ Classify() [2/3]

void Classify ( const MatType &  dataset,
arma::Row< size_t > &  labels,
const double  decisionBoundary = 0.5 
) const

Classify the given points, returning the predicted labels for each point.

Optionally, specify the decision boundary; logistic regression returns a value between 0 and 1. If the value is greater than the decision boundary, the response is taken to be 1; otherwise, it is 0. By default the decision boundary is 0.5.

Parameters
datasetSet of points to classify.
labelsPredicted labels for each point.
decisionBoundaryDecision boundary (default 0.5).

◆ Classify() [3/3]

void Classify ( const MatType &  dataset,
arma::mat &  probabilities 
) const

Classify the given points, returning class probabilities for each point.

Parameters
datasetSet of points to classify.
probabilitiesClass probabilities for each point (output).

◆ ComputeAccuracy()

double ComputeAccuracy ( const MatType &  predictors,
const arma::Row< size_t > &  responses,
const double  decisionBoundary = 0.5 
) const

Compute the accuracy of the model on the given predictors and responses, optionally using the given decision boundary.

The responses should be either 0 or 1. Logistic regression returns a value between 0 and 1. If the value is greater than the decision boundary, the response is taken to be 1; otherwise, it is 0. By default, the decision boundary is 0.5.

The accuracy is returned as a percentage, between 0 and 100.

Parameters
predictorsInput predictors.
responsesVector of responses.
decisionBoundaryDecision boundary (default 0.5).
Returns
Percentage of responses that are predicted correctly.

Referenced by LogisticRegression< MatType >::Lambda().

◆ ComputeError()

double ComputeError ( const MatType &  predictors,
const arma::Row< size_t > &  responses 
) const

Compute the error of the model.

This returns the negative objective function of the logistic regression log-likelihood function. For the model to be optimal, the negative log-likelihood function should be minimized.

Parameters
predictorsInput predictors.
responsesVector of responses.

Referenced by LogisticRegression< MatType >::Lambda().

◆ Lambda() [1/2]

const double& Lambda ( ) const
inline

Return the lambda value for L2-regularization.

Definition at line 149 of file logistic_regression.hpp.

◆ Lambda() [2/2]

◆ Parameters() [1/2]

const arma::vec& Parameters ( ) const
inline

Return the parameters (the b vector).

Definition at line 144 of file logistic_regression.hpp.

◆ Parameters() [2/2]

arma::vec& Parameters ( )
inline

Modify the parameters (the b vector).

Definition at line 146 of file logistic_regression.hpp.

◆ Predict()

mlpack_deprecated void Predict ( const MatType &  predictors,
arma::Row< size_t > &  responses,
const double  decisionBoundary = 0.5 
) const

Predict the responses to a given set of predictors.

The responses will be either 0 or 1. Optionally, specify the decision boundary; logistic regression returns a value between 0 and 1. If the value is greater than the decision boundary, the response is taken to be 1; otherwise, it is 0. By default the decision boundary is 0.5.

This method is deprecated—you should use Classify() instead.

Parameters
predictorsInput predictors.
responsesVector to put output predictions of responses into.
decisionBoundaryDecision boundary (default 0.5).

Referenced by LogisticRegression< MatType >::Lambda().

◆ Serialize()

void Serialize ( Archive &  ar,
const unsigned  int 
)

Serialize the model.

Referenced by LogisticRegression< MatType >::Lambda().

◆ Train() [1/2]

void Train ( const MatType &  predictors,
const arma::Row< size_t > &  responses 
)

Train the LogisticRegression model on the given input data.

By default, the L-BFGS optimization algorithm is used, but others can be specified (such as mlpack::optimization::SGD).

This will use the existing model parameters as a starting point for the optimization. If this is not what you want, then you should access the parameters vector directly with Parameters() and modify it as desired.

Template Parameters
OptimizerTypeType of optimizer to use to train the model.
Parameters
predictorsInput training variables.
responsesOutputs results from input training variables.

◆ Train() [2/2]

void Train ( OptimizerType< LogisticRegressionFunction< MatType >> &  optimizer)

Train the LogisticRegression model with the given instantiated optimizer.

Using this overload allows configuring the instantiated optimizer before training is performed.

Note that the initial point of the optimizer (optimizer.Function().GetInitialPoint()) will be used as the initial point of the optimization, overwriting any existing trained model. If you don't want to overwrite the existing model, set optimizer.Function().GetInitialPoint() to the current parameters vector, accessible via Parameters().

Parameters
optimizerInstantiated optimizer with instantiated error function.

The documentation for this class was generated from the following file: