mlpack

🔗 BayesianLinearRegression

The BayesianLinearRegression class implements a Bayesian ridge regression model for numerical data that optimally tunes the regularization strength to the given data. The class offers configurable functionality and template parameters to control the data type used for storing the model.

Simple usage example:

// Train a Bayesian linear regression model on random data and make predictions.

// All data and responses are uniform random; this uses 10 dimensional data.
// Replace with a data::Load() call or similar for a real application.
arma::mat dataset(10, 1000, arma::fill::randu); // 1000 points.
arma::rowvec responses = arma::randn<arma::rowvec>(1000);
arma::mat testDataset(10, 500, arma::fill::randu); // 500 test points.

mlpack::BayesianLinearRegression blr;  // Step 1: create model.
blr.Train(dataset, responses);         // Step 2: train model.
arma::rowvec predictions;
blr.Predict(testDataset, predictions); // Step 3: use model to predict.

// Print some information about the test predictions.
std::cout << arma::accu(predictions > 0.6) << " test points predicted to have"
    << " responses greater than 0.6." << std::endl;
std::cout << arma::accu(predictions < 0) << " test points predicted to have "
    << "negative responses." << std::endl;

More examples...

See also:

🔗 Constructors

Constructor Parameters:

name type description default
data arma::mat Column-major training matrix. (N/A)
responses arma::rowvec Training responses (e.g. values to predict). Should have length data.n_cols. (N/A)
centerData bool Whether to center the data before learning. true
scaleData bool Whether to scale the data to unit variance before learning. false
maxIterations size_t Maximum number of iterations for convergence. 50
tolerance double Tolerance for convergence of the model. 1e-4

As an alternative to passing centerData, scaleData, maxIterations, or tolerance, they can each be set or accessed with standalone methods:

🔗 Training

If training is not done as part of the constructor call, it can be done with the Train() function:

Types of each argument are the same as in the table for constructors above.

Notes:

🔗 Prediction

Once a LinearRegression model is trained, the Predict() member function can be used to make predictions for new data.

Prediction Parameters:

usage name type description
single-point point arma::vec Single point for prediction.
single-point prediction double& double to store predicted value into.
single-point stddev double& double to store standard deviation of predicted value into.
       
multi-point data arma::mat Set of column-major points for classification.
multi-point predictions arma::rowvec& Vector of doubles to store predictions into. Will be set to length data.n_cols.
multi-point stddevs arma::rowvec& Vector of doubles to store standard deviations of predictions into. Will be set to length data.n_cols.

🔗 Other Functionality

🔗 Simple Examples

See also the simple usage example for a trivial usage of the BayesianLinearRegression class.

Train a Bayesian linear regression model in the constructor on weighted data, compute the RMSE with RMSE(), and save the model.

// See https://datasets.mlpack.org/admission_predict.csv.
arma::mat data;
mlpack::data::Load("admission_predict.csv", data, true);

// See https://datasets.mlpack.org/admission_predict.responses.csv.
arma::rowvec responses;
mlpack::data::Load("admission_predict.responses.csv", responses, true);

// Generate random instance weights for each point, in the range 0.5 to 1.5.
arma::rowvec weights(data.n_cols, arma::fill::randu);
weights += 0.5;

// Train Bayesian linear regression model.  The data will be both centered and
// scaled to have unit variance.
mlpack::BayesianLinearRegression blr(data, responses, true, true);

// Now compute the RMSE on the training set.
std::cout << "RMSE on the training set: " << blr.RMSE(data, responses)
    << "." << std::endl;

// Finally, save the model with the name "blr".
mlpack::data::Save("blr_model.bin", "blr", blr, true);

Load a saved Bayesian linear regression model and print some information about it, then make some predictions individually for random points.

mlpack::BayesianLinearRegression blr;

// Load the model named "blr" from "lr_model.bin".
mlpack::data::Load("blr_model.bin", "blr", blr, true);

// Print some information about the model.
const size_t dimensionality = blr.Omega().n_elem;
if (dimensionality == 0)
{
  std::cout << "The model in `blr_model.bin` has not been trained."
      << std::endl;
  return 0;
}

std::cout << "Information on the BayesianLinearRegression model in "
    << "'blr_model.bin':" << std::endl;
std::cout << " - Data was centered when training: "
    << (blr.CenterData() ? std::string("yes") : std::string("no")) << "."
    << std::endl;
std::cout << " - Data was scaled to unit variance when training: "
    << (blr.ScaleData() ? std::string("yes") : std::string("no")) << "."
    << std::endl;
std::cout << " - Model intercept: " << blr.ResponsesOffset() << "."
    << std::endl;
std::cout << " - Precision of Gaussian prior: " << blr.Alpha() << "."
    << std::endl;
std::cout << " - Precision of model: " << blr.Beta() << "." << std::endl;

// Now make a prediction for three random points.
for (size_t t = 0; t < 3; ++t)
{
  arma::vec randomPoint(dimensionality, arma::fill::randu);
  double prediction, stddev;
  blr.Predict(randomPoint, prediction, stddev);

  std::cout << "Prediction for random point " << t << ": " << prediction
      << " +/- " << stddev << "." << std::endl;
}

🔗 Advanced Functionality: Different Element Types

The BayesianLinearRegression class has one template parameter that can be used to control the element type of the model. The full signature of the class is:

BayesianLinearRegression<ModelMatType>

ModelMatType specifies the type of matrix used for the internal representation of model parameters. Any matrix type that implements the Armadillo API can be used; however, the matrix should be dense, as in general BayesianLinearRegression will produce models that are not sparse.

The example below trains a Bayesian linear regression model on 32-bit floating point data.

// Create random, sparse 100-dimensional data.
arma::fmat dataset(100, 5000, arma::fill::randu);

// Generate noisy responses from random data.
arma::fvec trueWeights(100, arma::fill::randu);
arma::frowvec responses = trueWeights.t() * dataset +
    0.01 * arma::randu<arma::frowvec>(5000) /* noise term */;

mlpack::BayesianLinearRegression<arma::fmat> blr;
blr.ScaleData() = true;
blr.MaxIterations() = 75;

blr.Train(dataset, responses);

// Compute the RMSE on the training set and a random test set.
arma::fmat testDataset(100, 1000, arma::fill::randu);

arma::frowvec testResponses = trueWeights.t() * testDataset +
    0.01 * arma::randu<arma::frowvec>(1000) /* noise term */;

std::cout << "RMSE on training set: "
    << blr.RMSE(dataset, responses) << "." << std::endl;
std::cout << "RMSE on test set:     "
    << blr.RMSE(testDataset, testResponses) << "." << std::endl;