SparseAutoencoder Class Reference

A sparse autoencoder is a neural network whose aim to learn compressed representations of the data, typically for dimensionality reduction, with a constraint on the activity of the neurons in the network. More...

Public Member Functions

template
<
typename
OptimizerType
=
ens::L_BFGS
>
 SparseAutoencoder (const arma::mat &data, const size_t visibleSize, const size_t hiddenSize, const double lambda=0.0001, const double beta=3, const double rho=0.01, OptimizerType optimizer=OptimizerType())
 Construct the sparse autoencoder model with the given training data. More...

 
void Beta (const double b)
 Sets the KL divergence parameter. More...

 
double Beta () const
 Gets the KL divergence parameter. More...

 
void GetNewFeatures (arma::mat &data, arma::mat &features)
 Transforms the provided data into the representation learned by the sparse autoencoder. More...

 
void HiddenSize (const size_t hidden)
 Sets size of the hidden layer. More...

 
size_t HiddenSize () const
 Gets the size of the hidden layer. More...

 
void Lambda (const double l)
 Sets the L2-regularization parameter. More...

 
double Lambda () const
 Gets the L2-regularization parameter. More...

 
void Rho (const double r)
 Sets the sparsity parameter. More...

 
double Rho () const
 Gets the sparsity parameter. More...

 
void Sigmoid (const arma::mat &x, arma::mat &output) const
 Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))]. More...

 
void VisibleSize (const size_t visible)
 Sets size of the visible layer. More...

 
size_t VisibleSize () const
 Gets size of the visible layer. More...

 

Detailed Description

A sparse autoencoder is a neural network whose aim to learn compressed representations of the data, typically for dimensionality reduction, with a constraint on the activity of the neurons in the network.

Sparse autoencoders can be stacked together to learn a hierarchy of features, which provide a better representation of the data for classification. This is a method used in the recently developed field of deep learning. More technical details about the model can be found on the following webpage:

http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Tutorial

An example of how to use the interface is shown below:

arma::mat data; // Data matrix.
const size_t vSize = 64; // Size of visible layer, depends on the data.
const size_t hSize = 25; // Size of hidden layer, depends on requirements.
// Train the model using default options.
SparseAutoencoder encoder1(data, vSize, hSize);
const size_t numBasis = 5; // Parameter required for L-BFGS algorithm.
const size_t numIterations = 100; // Maximum number of iterations.
// Use an instantiated optimizer for the training.
SparseAutoencoderFunction saf(data, vSize, hSize);
L_BFGS<SparseAutoencoderFunction> optimizer(saf, numBasis, numIterations);
SparseAutoencoder<L_BFGS> encoder2(optimizer);
arma::mat features1, features2; // Matrices for storing new representations.
// Get new representations from the trained models.
encoder1.GetNewFeatures(data, features1);
encoder2.GetNewFeatures(data, features2);

This implementation allows the use of arbitrary mlpack optimizers via the OptimizerType template parameter.

Definition at line 63 of file sparse_autoencoder.hpp.

Constructor & Destructor Documentation

◆ SparseAutoencoder()

SparseAutoencoder ( const arma::mat &  data,
const size_t  visibleSize,
const size_t  hiddenSize,
const double  lambda = 0.0001,
const double  beta = 3,
const double  rho = 0.01,
OptimizerType  optimizer = OptimizerType() 
)

Construct the sparse autoencoder model with the given training data.

This will train the model. The parameters 'lambda', 'beta' and 'rho' can be set optionally. Changing these parameters will have an effect on regularization and sparsity of the model.

Template Parameters
OptimizerTypeThe optimizer to use.
Parameters
dataInput data with each column as one example.
visibleSizeSize of input vector expected at the visible layer.
hiddenSizeSize of input vector expected at the hidden layer.
lambdaL2-regularization parameter.
betaKL divergence parameter.
rhoSparsity parameter.

Member Function Documentation

◆ Beta() [1/2]

void Beta ( const double  b)
inline

Sets the KL divergence parameter.

Definition at line 147 of file sparse_autoencoder.hpp.

◆ Beta() [2/2]

double Beta ( ) const
inline

Gets the KL divergence parameter.

Definition at line 153 of file sparse_autoencoder.hpp.

◆ GetNewFeatures()

void GetNewFeatures ( arma::mat &  data,
arma::mat &  features 
)

Transforms the provided data into the representation learned by the sparse autoencoder.

The function basically performs a feedforward computation using the learned weights, and returns the hidden layer activations.

Parameters
dataMatrix of the provided data.
featuresThe hidden layer representation of the provided data.

◆ HiddenSize() [1/2]

void HiddenSize ( const size_t  hidden)
inline

Sets size of the hidden layer.

Definition at line 123 of file sparse_autoencoder.hpp.

◆ HiddenSize() [2/2]

size_t HiddenSize ( ) const
inline

Gets the size of the hidden layer.

Definition at line 129 of file sparse_autoencoder.hpp.

◆ Lambda() [1/2]

void Lambda ( const double  l)
inline

Sets the L2-regularization parameter.

Definition at line 135 of file sparse_autoencoder.hpp.

◆ Lambda() [2/2]

double Lambda ( ) const
inline

Gets the L2-regularization parameter.

Definition at line 141 of file sparse_autoencoder.hpp.

◆ Rho() [1/2]

void Rho ( const double  r)
inline

Sets the sparsity parameter.

Definition at line 159 of file sparse_autoencoder.hpp.

◆ Rho() [2/2]

double Rho ( ) const
inline

Gets the sparsity parameter.

Definition at line 165 of file sparse_autoencoder.hpp.

◆ Sigmoid()

void Sigmoid ( const arma::mat &  x,
arma::mat &  output 
) const
inline

Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))].

Parameters
xMatrix of real values for which we require the sigmoid activation.

Definition at line 105 of file sparse_autoencoder.hpp.

◆ VisibleSize() [1/2]

void VisibleSize ( const size_t  visible)
inline

Sets size of the visible layer.

Definition at line 111 of file sparse_autoencoder.hpp.

◆ VisibleSize() [2/2]

size_t VisibleSize ( ) const
inline

Gets size of the visible layer.

Definition at line 117 of file sparse_autoencoder.hpp.


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-3.2.1/src/mlpack/methods/sparse_autoencoder/sparse_autoencoder.hpp