mlpack

🔗 LocalCoordinateCoding

The LocalCoordinateCoding class implements local coordinate coding, a variation of sparse coding with dictionary learning. Local coordinate coding is a form of representation learning, and can be used to represent each point in a dataset as a linear combination of a few nearby atoms in the learned dictionary.

Simple usage example:

// Create a random dataset with 100 points in 40 dimensions, and then a random
// test dataset with 50 points.
arma::mat data(40, 100, arma::fill::randn);
arma::mat testData(40, 50, arma::fill::randn);

// Perform local coordinate coding with 20 atoms and an L1 penalty of 0.1.
mlpack::LocalCoordinateCoding lcc(20, 0.1); // Step 1: create object.
double objective = lcc.Train(data);         // Step 2: learn dictionary.
arma::mat codes;
lcc.Encode(testData, codes);                // Step 3: encode new data.

// Print some information about the test encoding.
std::cout << "Average density of encoded test data: "
    << 100.0 * arma::mean(arma::sum(codes != 0)) / codes.n_rows << "\%."
    << std::endl;

More examples...

See also:

🔗 Constructors

Constructor Parameters:

name type description default
data arma::mat Column-major training matrix. (N/A)
atoms size_t Number of atoms in dictionary. (N/A)
lambda double L1 regularization penalty. Used in both Train() and Encode() steps. 0.0
maxIter size_t Maximum number of iterations for dictionary learning. 0 means no limit. 0
tol double Objective function tolerance for terminating dictionary learning. 0.01

As an alternative to passing atoms, lambda, maxIter, or tol, these can be set with a standalone method. The following functions can be used before calling Train():

Caveats:

🔗 Training

If training the dictionary is not done as part of the constructor call, it can be done with one of the following versions of the Train() member function:

🔗 Encoding

Once a LocalCoordinateCoding model has a trained dictionary, the Encode() member function can be used to encode new data points.

After encoding, the original data can be recovered (approximately) as lcc.Dictionary() * data.

🔗 Other Functionality

🔗 Simple Examples

See also the simple usage example for a trivial usage of the LocalCoordinateCoding class.

Train a local coordinate coding model on the cloud dataset and print the reconstruction error.

// See https://datasets.mlpack.org/cloud.csv.
arma::mat dataset;
mlpack::data::Load("cloud.csv", dataset, true);

mlpack::LocalCoordinateCoding lcc;
lcc.Atoms() = 50;
lcc.Lambda() = 1e-5;
lcc.MaxIterations() = 25;
lcc.Train(dataset);

// Encode the training dataset.
arma::mat codes;
lcc.Encode(dataset, codes);

std::cout << "Input matrix size: " << dataset.n_rows << " x " << dataset.n_cols
    << "." << std::endl;
std::cout << "Codes matrix size: " << codes.n_rows << " x " << codes.n_cols
    << "." << std::endl;

// Reconstruct the original matrix.
arma::mat recon = lcc.Dictionary() * codes;
double error = std::sqrt(arma::norm(dataset - recon, "fro") / dataset.n_elem);
std::cout << "RMSE of reconstructed matrix: " << error << "." << std::endl;

Train a local coordinate coding model on the iris dataset and save the model to disk.

// See https://datasets.mlpack.org/iris.train.csv.
arma::mat dataset;
mlpack::data::Load("iris.train.csv", dataset, true);

// Train the model in the constructor.
mlpack::LocalCoordinateCoding lcc(dataset,
                                  10 /* atoms */,
                                  0.1 /* L1 penalty */);

// Save the model to disk.
mlpack::data::Save("lcc.bin", "lcc", lcc);

Train a local coordinate coding model on the satellite dataset, trying several different regularization parameters and checking the objective value on a held-out test dataset.

// See https://datasets.mlpack.org/satellite.train.csv.
arma::mat trainData;
mlpack::data::Load("satellite.train.csv", trainData, true);
// See https://datasets.mlpack.org/satellite.test.csv.
arma::mat testData;
mlpack::data::Load("satellite.test.csv", testData, true);

for (double lambdaPow = -6; lambdaPow <= -2; lambdaPow += 1)
{
  const double lambda = std::pow(10.0, lambdaPow);
  mlpack::LocalCoordinateCoding lcc(50 /* atoms */);
  lcc.Lambda() = lambda;
  lcc.MaxIterations() = 25; // Keep iterations low so this runs relatively fast.

  const double trainObj = lcc.Train(trainData);

  // Compute the objective on the test set.
  arma::mat codes;
  lcc.Encode(testData, codes);
  const double testObj = lcc.Objective(testData, codes);

  std::cout << "Lambda: " << std::setfill(' ') << std::setw(3) << lambda
      << "; ";
  std::cout << "training set objective: " << std::setw(6) << trainObj << "; ";
  std::cout << "test set objective: " << std::setw(6) << testObj << "."
      << std::endl;
}

🔗 Advanced Functionality: Template Parameters

The LocalCoordinateCoding class has one class template parameter that can be used for custom behavior. The full signature of the class is:

LocalCoordinateCoding<MatType>

In addition, the constructors and Train() functions have a template parameter DictionaryInitializer that can be used for custom behavior.

MatType: Different Element Types

MatType specifies the type of matrix used for training data and internal representation of the dictionary. Any matrix type that implements the Armadillo API can be used. The example below trains a local coordinate coding model on 32-bit floating point data.

// See https://datasets.mlpack.org/cloud.csv.
arma::fmat dataset;
mlpack::data::Load("cloud.csv", dataset, true);

mlpack::LocalCoordinateCoding<arma::fmat> lcc;
lcc.Atoms() = 30;
lcc.Lambda() = 1e-5;
lcc.MaxIterations() = 100;
lcc.Train(dataset);

// Encode the training dataset.
arma::fmat codes;
lcc.Encode(dataset, codes);

std::cout << "Input matrix size: " << dataset.n_rows << " x " << dataset.n_cols
    << "." << std::endl;
std::cout << "Codes matrix size: " << codes.n_rows << " x " << codes.n_cols
    << "." << std::endl;

// Reconstruct the original matrix.
arma::fmat recon = lcc.Dictionary() * codes;
double error = std::sqrt(arma::norm(dataset - recon, "fro") / dataset.n_elem);
std::cout << "RMSE of reconstructed matrix: " << error << "." << std::endl;

DictionaryInitializer: Different Dictionary Initialization Strategies

The DictionaryInitializer template class specifies the strategy to be used to initialize the dictionary when Train() is called.

Note: none of the classes above have any members, and as such it is not necessary to use the constructor or Train() variants that take an initialized initializer object. That would only be necessary for a custom DictionaryInitializer class that stored internal members.

The example below uses NothingInitializer to set a specific initial dictionary.

// See https://datasets.mlpack.org/satellite.train.csv.
arma::mat trainData;
mlpack::data::Load("satellite.train.csv", trainData, true);

const size_t atoms = 25;
const double lambda = 1e-5;
const size_t maxIterations = 50;

// Use a uniform random matrix as the initial dictionary.
arma::mat initialDictionary(trainData.n_rows, atoms, arma::fill::randu);

mlpack::LocalCoordinateCoding lcc(atoms, lambda, maxIterations);
lcc.Dictionary() = initialDictionary;

const double obj = lcc.Train<mlpack::NothingInitializer>(trainData);
std::cout << "Training set objective: " << obj << "." << std::endl;

// You can use this as a starting point for implementation.
class CustomDictionaryInitializer
{
 public:
  // Initialize the dictionary to have the given number of atoms, given the
  // dataset.  MatType will be the matrix type used by the local coordinate
  // coding model (e.g. `arma::mat`, `arma::fmat`, etc.).
  template<typename MatType>
  void Initialize(const MatType& data,
                  const size_t atoms,
                  MatType& dictionary);
};