SparseAutoencoderFunction Class Reference

This is a class for the sparse autoencoder objective function. More...

Public Member Functions

 SparseAutoencoderFunction (const arma::mat &data, const size_t visibleSize, const size_t hiddenSize, const double lambda=0.0001, const double beta=3, const double rho=0.01)
 Construct the sparse autoencoder objective function with the given parameters. More...

 
void Beta (const double b)
 Sets the KL divergence parameter. More...

 
double Beta () const
 Gets the KL divergence parameter. More...

 
double Evaluate (const arma::mat &parameters) const
 Evaluates the objective function of the sparse autoencoder model using the given parameters. More...

 
const arma::mat & GetInitialPoint () const
 Return the initial point for the optimization. More...

 
void Gradient (const arma::mat &parameters, arma::mat &gradient) const
 Evaluates the gradient values of the objective function given the current set of parameters. More...

 
void HiddenSize (const size_t hidden)
 Sets size of the hidden layer. More...

 
size_t HiddenSize () const
 Gets the size of the hidden layer. More...

 
const arma::mat InitializeWeights ()
 Initializes the parameters of the model to suitable values. More...

 
void Lambda (const double l)
 Sets the L2-regularization parameter. More...

 
double Lambda () const
 Gets the L2-regularization parameter. More...

 
void Rho (const double r)
 Sets the sparsity parameter. More...

 
double Rho () const
 Gets the sparsity parameter. More...

 
void Sigmoid (const arma::mat &x, arma::mat &output) const
 Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))]. More...

 
void VisibleSize (const size_t visible)
 Sets size of the visible layer. More...

 
size_t VisibleSize () const
 Gets size of the visible layer. More...

 

Detailed Description

This is a class for the sparse autoencoder objective function.

It can be used to create learning models like self-taught learning, stacked autoencoders, conditional random fields (CRFs), and so forth.

Definition at line 26 of file sparse_autoencoder_function.hpp.

Constructor & Destructor Documentation

◆ SparseAutoencoderFunction()

SparseAutoencoderFunction ( const arma::mat &  data,
const size_t  visibleSize,
const size_t  hiddenSize,
const double  lambda = 0.0001,
const double  beta = 3,
const double  rho = 0.01 
)

Construct the sparse autoencoder objective function with the given parameters.

Parameters
dataThe data matrix.
visibleSizeSize of input vector expected at the visible layer.
hiddenSizeSize of input vector expected at the hidden layer.
lambdaL2-regularization parameter.
betaKL divergence parameter.
rhoSparsity parameter.

Member Function Documentation

◆ Beta() [1/2]

void Beta ( const double  b)
inline

Sets the KL divergence parameter.

Definition at line 124 of file sparse_autoencoder_function.hpp.

◆ Beta() [2/2]

double Beta ( ) const
inline

Gets the KL divergence parameter.

Definition at line 130 of file sparse_autoencoder_function.hpp.

◆ Evaluate()

double Evaluate ( const arma::mat &  parameters) const

Evaluates the objective function of the sparse autoencoder model using the given parameters.

The cost function has terms for the reconstruction error, regularization cost and the sparsity cost. The objective function takes a low value when the model is able to reconstruct the data well using weights which are low in value and when the average activations of neurons in the hidden layers agrees well with the sparsity parameter 'rho'.

Parameters
parametersCurrent values of the model parameters.

◆ GetInitialPoint()

const arma::mat& GetInitialPoint ( ) const
inline

Return the initial point for the optimization.

Definition at line 85 of file sparse_autoencoder_function.hpp.

◆ Gradient()

void Gradient ( const arma::mat &  parameters,
arma::mat &  gradient 
) const

Evaluates the gradient values of the objective function given the current set of parameters.

The function performs a feedforward pass and computes the error in reconstructing the data points. It then uses the backpropagation algorithm to compute the gradient values.

Parameters
parametersCurrent values of the model parameters.
gradientMatrix where gradient values will be stored.

◆ HiddenSize() [1/2]

void HiddenSize ( const size_t  hidden)
inline

Sets size of the hidden layer.

Definition at line 100 of file sparse_autoencoder_function.hpp.

◆ HiddenSize() [2/2]

size_t HiddenSize ( ) const
inline

Gets the size of the hidden layer.

Definition at line 106 of file sparse_autoencoder_function.hpp.

◆ InitializeWeights()

const arma::mat InitializeWeights ( )

Initializes the parameters of the model to suitable values.

◆ Lambda() [1/2]

void Lambda ( const double  l)
inline

Sets the L2-regularization parameter.

Definition at line 112 of file sparse_autoencoder_function.hpp.

◆ Lambda() [2/2]

double Lambda ( ) const
inline

Gets the L2-regularization parameter.

Definition at line 118 of file sparse_autoencoder_function.hpp.

◆ Rho() [1/2]

void Rho ( const double  r)
inline

Sets the sparsity parameter.

Definition at line 136 of file sparse_autoencoder_function.hpp.

◆ Rho() [2/2]

double Rho ( ) const
inline

Gets the sparsity parameter.

Definition at line 142 of file sparse_autoencoder_function.hpp.

◆ Sigmoid()

void Sigmoid ( const arma::mat &  x,
arma::mat &  output 
) const
inline

Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))].

Parameters
xMatrix of real values for which we require the sigmoid activation.

Definition at line 79 of file sparse_autoencoder_function.hpp.

◆ VisibleSize() [1/2]

void VisibleSize ( const size_t  visible)
inline

Sets size of the visible layer.

Definition at line 88 of file sparse_autoencoder_function.hpp.

◆ VisibleSize() [2/2]

size_t VisibleSize ( ) const
inline

Gets size of the visible layer.

Definition at line 94 of file sparse_autoencoder_function.hpp.


The documentation for this class was generated from the following file: