mlpack  2.2.5
LocalCoordinateCoding Class Reference

An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom. More...

Public Member Functions

template
<
typename
DictionaryInitializer
=
sparse_coding::DataDependentRandomInitializer
>
 LocalCoordinateCoding (const arma::mat &data, const size_t atoms, const double lambda, const size_t maxIterations=0, const double tolerance=0.01, const DictionaryInitializer &initializer=DictionaryInitializer())
 Set the parameters to LocalCoordinateCoding, and train the dictionary. More...

 
 LocalCoordinateCoding (const size_t atoms, const double lambda, const size_t maxIterations=0, const double tolerance=0.01)
 Set the parameters to LocalCoordinateCoding. More...

 
size_t Atoms () const
 Get the number of atoms. More...

 
size_t & Atoms ()
 Modify the number of atoms. More...

 
const arma::mat & Dictionary () const
 Accessor for dictionary. More...

 
arma::mat & Dictionary ()
 Mutator for dictionary. More...

 
void Encode (const arma::mat &data, arma::mat &codes)
 Code each point via distance-weighted LARS. More...

 
double Lambda () const
 Get the L1 regularization parameter. More...

 
double & Lambda ()
 Modify the L1 regularization parameter. More...

 
size_t MaxIterations () const
 Get the maximum number of iterations. More...

 
size_t & MaxIterations ()
 Modify the maximum number of iterations. More...

 
double Objective (const arma::mat &data, const arma::mat &codes, const arma::uvec &adjacencies) const
 Compute objective function given the list of adjacencies. More...

 
void OptimizeDictionary (const arma::mat &data, const arma::mat &codes, const arma::uvec &adjacencies)
 Learn dictionary by solving linear system. More...

 
template
<
typename
Archive
>
void Serialize (Archive &ar, const unsigned int)
 Serialize the model. More...

 
double Tolerance () const
 Get the objective tolerance. More...

 
double & Tolerance ()
 Modify the objective tolerance. More...

 
template
<
typename
DictionaryInitializer
=
sparse_coding::DataDependentRandomInitializer
>
void Train (const arma::mat &data, const DictionaryInitializer &initializer=DictionaryInitializer())
 Run local coordinate coding. More...

 

Detailed Description

An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom.

Let d be the number of dimensions in the original space, m the number of training points, and k the number of atoms in the dictionary (the dimension of the learned feature space). The training data X is a d-by-m matrix where each column is a point and each row is a dimension. The dictionary D is a d-by-k matrix, and the sparse codes matrix Z is a k-by-m matrix. This program seeks to minimize the objective: min_{D,Z} ||X - D Z||_{Fro}^2

  • lambda sum_{i=1}^m sum_{j=1}^k dist(X_i,D_j)^2 Z_i^j where lambda > 0.

This problem is solved by an algorithm that alternates between a dictionary learning step and a sparse coding step. The dictionary learning step updates the dictionary D by solving a linear system (note that the objective is a positive definite quadratic program). The sparse coding step involves solving a large number of weighted l1-norm regularized linear regression problems problems; this can be done efficiently using LARS, an algorithm that can solve the LASSO (paper below).

The papers are listed below.

@incollection{NIPS2009_0719,
title = {Nonlinear Learning using Local Coordinate Coding},
author = {Kai Yu and Tong Zhang and Yihong Gong},
booktitle = {Advances in Neural Information Processing Systems 22},
editor = {Y. Bengio and D. Schuurmans and J. Lafferty and C. K. I. Williams
and A. Culotta},
pages = {2223--2231},
year = {2009}
}
@article{efron2004least,
title={Least angle regression},
author={Efron, B. and Hastie, T. and Johnstone, I. and Tibshirani, R.},
journal={The Annals of statistics},
volume={32},
number={2},
pages={407--499},
year={2004},
publisher={Institute of Mathematical Statistics}
}

Definition at line 79 of file lcc.hpp.

Constructor & Destructor Documentation

◆ LocalCoordinateCoding() [1/2]

LocalCoordinateCoding ( const arma::mat &  data,
const size_t  atoms,
const double  lambda,
const size_t  maxIterations = 0,
const double  tolerance = 0.01,
const DictionaryInitializer &  initializer = DictionaryInitializer() 
)

Set the parameters to LocalCoordinateCoding, and train the dictionary.

This constructor will also initialize the dictionary using the given DictionaryInitializer before training.

If you want to initialize the dictionary to a custom matrix, consider either writing your own DictionaryInitializer class (with void Initialize(const arma::mat& data, arma::mat& dictionary) function), or call the constructor that does not take a data matrix, then call Dictionary() to set the dictionary matrix to a matrix of your choosing, and then call Train() with sparse_coding::NothingInitializer (i.e. Train<sparse_coding::NothingInitializer>(data)).

Parameters
dataData matrix.
atomsNumber of atoms in dictionary.
lambdaRegularization parameter for weighted l1-norm penalty.
maxIterationsMaximum number of iterations for training (0 runs until convergence).
toleranceTolerance for the objective function.

◆ LocalCoordinateCoding() [2/2]

LocalCoordinateCoding ( const size_t  atoms,
const double  lambda,
const size_t  maxIterations = 0,
const double  tolerance = 0.01 
)

Set the parameters to LocalCoordinateCoding.

This constructor will not train the model, and a subsequent call to Train() will be required before the model can encode points with Encode().

Parameters
atomsNumber of atoms in dictionary.
lambdaRegularization parameter for weighted l1-norm penalty.
maxIterationsMaximum number of iterations for training (0 runs until convergence).
toleranceTolerance for the objective function.

Member Function Documentation

◆ Atoms() [1/2]

size_t Atoms ( ) const
inline

Get the number of atoms.

Definition at line 173 of file lcc.hpp.

◆ Atoms() [2/2]

size_t& Atoms ( )
inline

Modify the number of atoms.

Definition at line 175 of file lcc.hpp.

◆ Dictionary() [1/2]

const arma::mat& Dictionary ( ) const
inline

Accessor for dictionary.

Definition at line 178 of file lcc.hpp.

◆ Dictionary() [2/2]

arma::mat& Dictionary ( )
inline

Mutator for dictionary.

Definition at line 180 of file lcc.hpp.

◆ Encode()

void Encode ( const arma::mat &  data,
arma::mat &  codes 
)

Code each point via distance-weighted LARS.

Parameters
dataMatrix containing points to encode.
codesOutput matrix to store codes in.

◆ Lambda() [1/2]

double Lambda ( ) const
inline

Get the L1 regularization parameter.

Definition at line 183 of file lcc.hpp.

◆ Lambda() [2/2]

double& Lambda ( )
inline

Modify the L1 regularization parameter.

Definition at line 185 of file lcc.hpp.

◆ MaxIterations() [1/2]

size_t MaxIterations ( ) const
inline

Get the maximum number of iterations.

Definition at line 188 of file lcc.hpp.

◆ MaxIterations() [2/2]

size_t& MaxIterations ( )
inline

Modify the maximum number of iterations.

Definition at line 190 of file lcc.hpp.

◆ Objective()

double Objective ( const arma::mat &  data,
const arma::mat &  codes,
const arma::uvec &  adjacencies 
) const

Compute objective function given the list of adjacencies.

◆ OptimizeDictionary()

void OptimizeDictionary ( const arma::mat &  data,
const arma::mat &  codes,
const arma::uvec &  adjacencies 
)

Learn dictionary by solving linear system.

Parameters
adjacenciesIndices of entries (unrolled column by column) of the coding matrix Z that are non-zero (the adjacency matrix for the bipartite graph of points and atoms)

◆ Serialize()

void Serialize ( Archive &  ar,
const unsigned  int 
)

Serialize the model.

Referenced by LocalCoordinateCoding::Tolerance().

◆ Tolerance() [1/2]

double Tolerance ( ) const
inline

Get the objective tolerance.

Definition at line 193 of file lcc.hpp.

◆ Tolerance() [2/2]

double& Tolerance ( )
inline

Modify the objective tolerance.

Definition at line 195 of file lcc.hpp.

References LocalCoordinateCoding::Serialize().

◆ Train()

void Train ( const arma::mat &  data,
const DictionaryInitializer &  initializer = DictionaryInitializer() 
)

Run local coordinate coding.

Parameters
nIterationsMaximum number of iterations to run algorithm.
objToleranceTolerance of objective function. When the objective function changes by a value lower than this tolerance, the optimization terminates.

The documentation for this class was generated from the following file:
  • src/mlpack/methods/local_coordinate_coding/lcc.hpp