mlpack

🔗 Perceptron

The Perceptron class implements the simple perceptron classifier originally implemented by Frank Rosenblatt in 1958. The perceptron is a linear classifier, and can be understood as a trivial neural network with one neuron that uses the step function as an activation function. mlpack’s implementation of the Perceptron class also offers several template parameters that can be used to control the behavior of the perceptron.

Perceptrons are useful for classifying points with discrete labels (i.e., 0, 1, 2). Because they are simple classifiers, they are also useful as weak learners for the AdaBoost boosting classifier.

Simple usage example:

// Train a perceptron on random numeric data and predict labels on test data:

// All data and labels are uniform random; 10 dimensional data, 5 classes.
// Replace with a data::Load() call or similar for a real application.
arma::mat dataset(10, 1000, arma::fill::randu);
arma::Row<size_t> labels =
    arma::randi<arma::Row<size_t>>(1000, arma::distr_param(0, 4));
arma::mat testDataset(10, 500, arma::fill::randu); // 500 test points.

mlpack::Perceptron p;                 // Step 1: create model.
p.Train(dataset, labels, 5);          // Step 2: train model.
arma::Row<size_t> predictions;
p.Classify(testDataset, predictions); // Step 3: classify points.

// Print some information about the test predictions.
std::cout << arma::accu(predictions == 1) << " test points classified as class "
    << "1." << std::endl;

More examples...

See also:

🔗 Constructors

Construct a Perceptron object using one of the constructors below. Defaults and types are detailed in the Constructor Parameters section below.

Forms:

Constructor Parameters:

name type description default
data arma::mat Column-major training matrix. (N/A)
datasetInfo data::DatasetInfo Dataset information, specifying type information for each dimension. (N/A)
labels arma::Row<size_t> Training labels, between 0 and numClasses - 1 (inclusive). Should have length data.n_cols. (N/A)
weights arma::rowvec Weights for each training point. Should have length data.n_cols. (N/A)
numClasses size_t Number of classes in the dataset. (N/A)
dimensionality size_t Dimensionality of data (only used if an initialized but untrained model is desired). (N/A)
maxIterations size_t Maximum number of iterations during training. Can also be set with MaxIterations(). 1000

As an alternative to passing maxIterations, it can be set with a standalone method. The following function can be used before calling Train() to set the maximum number of iterations:

🔗 Training

If training is not done as part of the constructor call, it can be done with one of the following versions of the Train() member function:

Types of each argument are the same as in the table for constructors above.

Notes:

🔗 Classification

Once a Perceptron is trained, the Classify() member function can be used to make class predictions for new data.

Note: perceptrons do not provide any measure resembling probabilities during classification, and thus a version of Classify() that computes class probabilities is not available.

Classification Parameters:

usage name type description
single-point point arma::vec Single point for classification.
       
multi-point data arma::mat Set of column-major points for classification.
multi-point predictions arma::Row<size_t>& Vector of size_ts to store class prediction into. Will be set to length data.n_cols.

🔗 Other Functionality

For complete functionality, the source code can be consulted. Each method is fully documented.

🔗 Simple Examples

See also the simple usage example for a trivial use of Perceptron.

Train a perceptron multiple times, incrementally, with custom hyperparameters, and save the resulting model to disk.

// See https://datasets.mlpack.org/iris.csv.
arma::mat dataset;
mlpack::data::Load("iris.csv", dataset, true);
// See https://datasets.mlpack.org/iris.labels.csv.
arma::Row<size_t> labels;
mlpack::data::Load("iris.labels.csv", labels, true);

// Create a Perceptron object.
mlpack::Perceptron p;
// Set the maximum number of iterations to 100.  (This can also be done in the
// constructor.)
p.MaxIterations() = 100;

// Train the model for up to 100 iterations.
p.Train(dataset, labels, 3);

// Now, compute and print accuracy on the training set.
arma::Row<size_t> predictions;
p.Classify(dataset, predictions);
std::cout << "Training set accuracy after 100 iterations: "
    << (100.0 * double(arma::accu(labels == predictions)) / labels.n_elem)
    << "\%." << std::endl;

// Train for another 250 iterations and compute training set accuracy again.
p.Train(dataset, labels, 3, 250);
p.Classify(dataset, predictions);
std::cout << "Training set accuracy after 350 iterations: "
    << (100.0 * double(arma::accu(labels == predictions)) / labels.n_elem)
    << "\%." << std::endl;

// Save the perceptron to disk for later use.
mlpack::data::Save("perceptron.bin", "perceptron", p);

Load a saved perceptron from disk and print information about it.

mlpack::Perceptron p;
// This call assumes a perceptron called "p" has already been saved to
// `perceptron.bin` with `data::Save()`.
mlpack::data::Load("perceptron.bin", "p", p, true);

if (p.NumClasses() > 0)
{
  std::cout << "The perceptron in `perceptron.bin` was trained on "
      << p.NumClasses() << " classes." << std::endl;
  std::cout << "The dimensionality of the perceptron model is "
      << p.Weights().n_rows << "." << std::endl;
  std::cout << "The bias weights for each class are:" << std::endl;
  for (size_t i = 0; i < p.NumClasses(); ++i)
    std::cout << "  - Class " << i << ": " << p.Biases()[i] << std::endl;
}
else
{
  std::cout << "The perceptron in `perceptron.bin` has not been trained."
      << std::endl;
}

🔗 Advanced Functionality: Template Parameters

The Perceptron class also supports several template parameters, which can be used for custom behavior. The full signature of the class is as follows:

Perceptron<LearnPolicy,
           WeightInitializationPolicy,
           MatType>

LearnPolicy

// You can use this as a starting point for implementation.
class CustomLearnPolicy
{
  // Update the weights and biases in the `weights` matrix and the `biases`
  // vector given that the model currently classified `trainingPoint` as having
  // the label `incorrectClass`, when in reality it has the label
  // `correctClass`.  If `instanceWeight` is given, it specifies the instance
  // weight for the given `trainingPoint`.
  //
  // `VecType` will be an Armadillo-like vector type.  It will be a column from
  // the training data matrix (`data`) given to `Train()` or to the constructor.
  //
  // `eT` is the element type of the Perceptron (e.g. `float`, `double`).
  template<typename VecType, typename eT>
  void UpdateWeights(const VecType& trainingPoint,
                     arma::Mat<eT>& weights,
                     arma::Col<eT>& biases,
                     const size_t incorrectClass,
                     const size_t correctClass,
                     const double instanceWeight = 1.0);
};

WeightInitializationPolicy

// You can use this as a starting point for implementation.
class CustomWeightInitializationPolicy
{
  // Initialize the `weights` matrix and `biases` vector, given that the model
  // will have dimensionality of `numFeatures` (that is, the training data
  // matrix will have `numFeatures` rows), and the training data has
  // `numClasses` classes.
  //
  // The initialized `weights` matrix should have `numFeatures` rows and
  // `numClasses` columns, and the initialized `biases` vector should have
  // `numClasses` elements.
  //
  // `eT` specifies the element type of the weights and biases; it may be
  // `double`, `float`, or another floating-point type.
  template<typename eT>
  inline static void Initialize(arma::Mat<eT>& weights,
                                arma::Col<eT>& biases,
                                const size_t numFeatures,
                                const size_t numClasses)
  {
    weights.randu(numFeatures, numClasses);
    biases.randu(numClasses);
  }
};

MatType

🔗 Advanced Functionality Examples

Train a Perceptron with random initialization, instead of zero initialization of weights.

// 1000 random points in 10 dimensions.
arma::mat dataset(10, 1000, arma::fill::randu);
// Random labels for each point, totaling 5 classes.
arma::Row<size_t> labels =
    arma::randi<arma::Row<size_t>>(1000, arma::distr_param(0, 4));

// Train in the constructor.  Weights will be initialized randomly.
mlpack::Perceptron<mlpack::SimpleWeightUpdate,
                   mlpack::RandomPerceptronInitialization> p(
    dataset, labels, 5);

// Create test data (500 points).
arma::mat testDataset(10, 500, arma::fill::randu);
arma::Row<size_t> predictions;
p.Classify(testDataset, predictions);
// Now `predictions` holds predictions for the test dataset.

// Print some information about the test predictions.
std::cout << arma::accu(predictions == 1) << " test points classified as class "
    << "1." << std::endl;

Train a Perceptron on sparse 32-bit floating point data.


// 1000 sparse random points in 100 dimensions, with 1% nonzero elements.
arma::sp_fmat dataset;
dataset.sprandu(100, 1000, 0.01);
// Random labels for each point, totaling 5 classes.
arma::Row<size_t> labels =
    arma::randi<arma::Row<size_t>>(1000, arma::distr_param(0, 4));

// Train in the constructor.
mlpack::Perceptron p(dataset, labels, 5);

// Create test data (500 points).
arma::sp_fmat testDataset;
testDataset.sprandu(100, 500, 0.01);
arma::Row<size_t> predictions;
p.Classify(testDataset, predictions);
// Now `predictions` holds predictions for the test dataset.

// Print some information about the test predictions.
std::cout << arma::accu(predictions == 1) << " test points classified as class "
    << "1." << std::endl;