mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer > Class Template Reference

An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom. More...

List of all members.

Public Member Functions
	LocalCoordinateCoding (const arma::mat &data, const size_t atoms, const double lambda)
	Set the parameters to LocalCoordinateCoding.
const arma::mat &	Codes () const
	Accessor the codes.
arma::mat &	Codes ()
	Modify the codes.
const arma::mat &	Data () const
	Access the data.
arma::mat &	Dictionary ()
	Mutator for dictionary.
const arma::mat &	Dictionary () const
	Accessor for dictionary.
void	Encode (const size_t maxIterations=0, const double objTolerance=0.01)
	Run local coordinate coding.
double	Objective (arma::uvec adjacencies) const
	Compute objective function given the list of adjacencies.
void	OptimizeCode ()
	Code each point via distance-weighted LARS.
void	OptimizeDictionary (arma::uvec adjacencies)
	Learn dictionary by solving linear system.
Private Attributes
size_t	atoms
	Number of atoms in dictionary.
arma::mat	codes
	Codes (columns are points).
const arma::mat &	data
	Data matrix (columns are points).
arma::mat	dictionary
	Dictionary (columns are atoms).
double	lambda
	l1 regularization term.

---

Detailed Description

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>
class mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >

An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom.

Let d be the number of dimensions in the original space, m the number of training points, and k the number of atoms in the dictionary (the dimension of the learned feature space). The training data X is a d-by-m matrix where each column is a point and each row is a dimension. The dictionary D is a d-by-k matrix, and the sparse codes matrix Z is a k-by-m matrix. This program seeks to minimize the objective: min_{D,Z} ||X - D Z||_{Fro}^2 + lambda sum_{i=1}^m sum_{j=1}^k dist(X_i,D_j)^2 Z_i^j where lambda > 0.

This problem is solved by an algorithm that alternates between a dictionary learning step and a sparse coding step. The dictionary learning step updates the dictionary D by solving a linear system (note that the objective is a positive definite quadratic program). The sparse coding step involves solving a large number of weighted l1-norm regularized linear regression problems problems; this can be done efficiently using LARS, an algorithm that can solve the LASSO (paper below).

The papers are listed below.

 @incollection{NIPS2009_0719,
   title = {Nonlinear Learning using Local Coordinate Coding},
   author = {Kai Yu and Tong Zhang and Yihong Gong},
   booktitle = {Advances in Neural Information Processing Systems 22},
   editor = {Y. Bengio and D. Schuurmans and J. Lafferty and C. K. I. Williams
       and A. Culotta},
   pages = {2223--2231},
   year = {2009}
 }

 @article{efron2004least,
   title={Least angle regression},
   author={Efron, B. and Hastie, T. and Johnstone, I. and Tibshirani, R.},
   journal={The Annals of statistics},
   volume={32},
   number={2},
   pages={407--499},
   year={2004},
   publisher={Institute of Mathematical Statistics}
 }

Definition at line 89 of file lcc.hpp.

---

Constructor & Destructor Documentation

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::LocalCoordinateCoding	(	const arma::mat &	data,
		const size_t	atoms,
		const double	lambda
	)

Set the parameters to LocalCoordinateCoding.

Parameters:

	data	Data matrix.
	atoms	Number of atoms in dictionary.
	lambda	Regularization parameter for weighted l1-norm penalty.

---

Member Function Documentation

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Codes

(

)

const [inline]

Accessor the codes.

Definition at line 85 of file lcc.hpp.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Codes

(

)

[inline]

Modify the codes.

Definition at line 87 of file lcc.hpp.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Data

(

)

const [inline]

Access the data.

Definition at line 77 of file lcc.hpp.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Dictionary

(

)

[inline]

Mutator for dictionary.

Definition at line 82 of file lcc.hpp.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Dictionary

(

)

const [inline]

Accessor for dictionary.

Definition at line 80 of file lcc.hpp.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

void mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Encode	(	const size_t	maxIterations = 0,
		const double	objTolerance = 0.01
	)

Run local coordinate coding.

Parameters:

	nIterations	Maximum number of iterations to run algorithm.
	objTolerance	Tolerance of objective function. When the objective function changes by a value lower than this tolerance, the optimization terminates.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

double mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Objective

(

arma::uvec

adjacencies

)

const

Compute objective function given the list of adjacencies.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

void mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::OptimizeCode

(

)

Code each point via distance-weighted LARS.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

void mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::OptimizeDictionary

(

arma::uvec

adjacencies

)

Learn dictionary by solving linear system.

Parameters:

adjacencies

Indices of entries (unrolled column by column) of the coding matrix Z that are non-zero (the adjacency matrix for the bipartite graph of points and atoms)

---

Member Data Documentation

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

size_t mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::atoms [private]

Number of atoms in dictionary.

Definition at line 91 of file lcc.hpp.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

arma::mat mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::codes [private]

Codes (columns are points).

Definition at line 100 of file lcc.hpp.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::data [private]

Data matrix (columns are points).

Definition at line 94 of file lcc.hpp.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

arma::mat mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::dictionary [private]

Dictionary (columns are atoms).

Definition at line 97 of file lcc.hpp.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

double mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::lambda [private]

l1 regularization term.

Definition at line 103 of file lcc.hpp.

---

The documentation for this class was generated from the following file:

• src/mlpack/methods/local_coordinate_coding/lcc.hpp