LSTM< InputDataType, OutputDataType > Class Template Reference

Implementation of the LSTM module class. More...

Public Member Functions

 LSTM ()
 Create the LSTM object. More...

 
 LSTM (const size_t inSize, const size_t outSize, const size_t rho=std::numeric_limits< size_t >::max())
 Create the LSTM layer object using the specified parameters. More...

 
 LSTM (const LSTM &layer)
 Copy constructor. More...

 
 LSTM (LSTM &&)
 Move constructor. More...

 
template
<
typename
InputType
,
typename
ErrorType
,
typename
GradientType
>
void Backward (const InputType &input, const ErrorType &gy, GradientType &g)
 Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...

 
OutputDataType const & Delta () const
 Get the delta. More...

 
OutputDataType & Delta ()
 Modify the delta. More...

 
template
<
typename
InputType
,
typename
OutputType
>
void Forward (const InputType &input, OutputType &output)
 Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

 
template
<
typename
InputType
,
typename
OutputType
>
void Forward (const InputType &input, OutputType &output, OutputType &cellState, bool useCellState=false)
 Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

 
template
<
typename
InputType
,
typename
ErrorType
,
typename
GradientType
>
void Gradient (const InputType &input, const ErrorType &error, GradientType &gradient)
 
OutputDataType const & Gradient () const
 Get the gradient. More...

 
OutputDataType & Gradient ()
 Modify the gradient. More...

 
size_t InputShape () const
 Get the shape of the input. More...

 
size_t InSize () const
 Get the number of input units. More...

 
LSTMoperator= (const LSTM &layer)
 Copy assignment operator. More...

 
LSTMoperator= (LSTM &&layer)
 Move assignment operator. More...

 
OutputDataType const & OutputParameter () const
 Get the output parameter. More...

 
OutputDataType & OutputParameter ()
 Modify the output parameter. More...

 
size_t OutSize () const
 Get the number of output units. More...

 
OutputDataType const & Parameters () const
 Get the parameters. More...

 
OutputDataType & Parameters ()
 Modify the parameters. More...

 
void Reset ()
 
void ResetCell (const size_t size)
 
size_t Rho () const
 Get the maximum number of steps to backpropagate through time (BPTT). More...

 
size_t & Rho ()
 Modify the maximum number of steps to backpropagate through time (BPTT). More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the layer. More...

 
size_t WeightSize () const
 Get the size of the weights. More...

 

Detailed Description


template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::LSTM< InputDataType, OutputDataType >

Implementation of the LSTM module class.

The implementation corresponds to the following algorithm:

\begin{eqnarray} i &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ f &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ z &=& tanh(W \cdot x + W \cdot h + b) \\ c &=& f \odot c + i \odot z \\ o &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ h &=& o \odot tanh(c) \end{eqnarray}

Note that if an LSTM layer is desired as the first layer of a neural network, an IdentityLayer should be added to the network as the first layer, and then the LSTM layer should be added.

For more information, see the following.

@article{Graves2013,
author = {Alex Graves and Abdel{-}rahman Mohamed and Geoffrey E. Hinton},
title = {Speech Recognition with Deep Recurrent Neural Networks},
journal = CoRR},
year = {2013},
url = {http://arxiv.org/abs/1303.5778},
}
See also
FastLSTM for a faster LSTM version which combines the calculation of the input, forget, output gates and hidden state in a single step.
Template Parameters
InputDataTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 79 of file layer_types.hpp.

Constructor & Destructor Documentation

◆ LSTM() [1/4]

LSTM ( )

Create the LSTM object.

◆ LSTM() [2/4]

LSTM ( const size_t  inSize,
const size_t  outSize,
const size_t  rho = std::numeric_limits< size_t >::max() 
)

Create the LSTM layer object using the specified parameters.

Parameters
inSizeThe number of input units.
outSizeThe number of output units.
rhoMaximum number of steps to backpropagate through time (BPTT).

◆ LSTM() [3/4]

LSTM ( const LSTM< InputDataType, OutputDataType > &  layer)

Copy constructor.

◆ LSTM() [4/4]

LSTM ( LSTM< InputDataType, OutputDataType > &&  )

Move constructor.

Member Function Documentation

◆ Backward()

void Backward ( const InputType &  input,
const ErrorType &  gy,
GradientType &  g 
)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters
inputThe propagated input activation.
gyThe backpropagated error.
gThe calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const
inline

Get the delta.

Definition at line 171 of file lstm.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )
inline

Modify the delta.

Definition at line 173 of file lstm.hpp.

◆ Forward() [1/2]

void Forward ( const InputType &  input,
OutputType &  output 
)

Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.

◆ Forward() [2/2]

void Forward ( const InputType &  input,
OutputType &  output,
OutputType &  cellState,
bool  useCellState = false 
)

Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.
cellStateCell state of the LSTM.
useCellStateUse the cellState passed in the LSTM cell.

◆ Gradient() [1/3]

void Gradient ( const InputType &  input,
const ErrorType &  error,
GradientType &  gradient 
)

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const
inline

Get the gradient.

Definition at line 176 of file lstm.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )
inline

Modify the gradient.

Definition at line 178 of file lstm.hpp.

◆ InputShape()

size_t InputShape ( ) const
inline

Get the shape of the input.

Definition at line 190 of file lstm.hpp.

References LSTM< InputDataType, OutputDataType >::serialize().

◆ InSize()

size_t InSize ( ) const
inline

Get the number of input units.

Definition at line 181 of file lstm.hpp.

◆ operator=() [1/2]

LSTM& operator= ( const LSTM< InputDataType, OutputDataType > &  layer)

Copy assignment operator.

◆ operator=() [2/2]

LSTM& operator= ( LSTM< InputDataType, OutputDataType > &&  layer)

Move assignment operator.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const
inline

Get the output parameter.

Definition at line 166 of file lstm.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )
inline

Modify the output parameter.

Definition at line 168 of file lstm.hpp.

◆ OutSize()

size_t OutSize ( ) const
inline

Get the number of output units.

Definition at line 184 of file lstm.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const
inline

Get the parameters.

Definition at line 161 of file lstm.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )
inline

Modify the parameters.

Definition at line 163 of file lstm.hpp.

◆ Reset()

void Reset ( )

◆ ResetCell()

void ResetCell ( const size_t  size)

◆ Rho() [1/2]

size_t Rho ( ) const
inline

Get the maximum number of steps to backpropagate through time (BPTT).

Definition at line 156 of file lstm.hpp.

◆ Rho() [2/2]

size_t& Rho ( )
inline

Modify the maximum number of steps to backpropagate through time (BPTT).

Definition at line 158 of file lstm.hpp.

◆ serialize()

void serialize ( Archive &  ar,
const uint32_t   
)

Serialize the layer.

Referenced by LSTM< InputDataType, OutputDataType >::InputShape().

◆ WeightSize()

size_t WeightSize ( ) const
inline

Get the size of the weights.

Definition at line 187 of file lstm.hpp.


The documentation for this class was generated from the following files:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/layer_types.hpp
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/lstm.hpp