LSTMType< MatType > Class Template Reference

Implementation of the LSTM module class. More...

Inheritance diagram for LSTMType< MatType >:

Public Member Functions

 LSTMType ()
 Create the LSTM object. More...

 
 LSTMType (const size_t outSize)
 Create the LSTM layer object using the specified parameters. More...

 
 LSTMType (const LSTMType &other)
 Copy the given LSTMType object. More...

 
 LSTMType (LSTMType &&other)
 Take ownership of the given LSTMType object's data. More...

 
virtual ~LSTMType ()
 
void Backward (const MatType &input, const MatType &gy, MatType &g)
 Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...

 
void ClearRecurrentState (const size_t bpttSteps, const size_t batchSize)
 Reset the recurrent state of the LSTM layer, and allocate enough space to hold bpttSteps of previous passes with a batch size of batchSize. More...

 
LSTMTypeClone () const
 Clone the LSTMType object. This handles polymorphism correctly. More...

 
void ComputeOutputDimensions ()
 Given a properly set InputDimensions(), compute the output dimensions. More...

 
void Forward (const MatType &input, MatType &output)
 Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

 
void Gradient (const MatType &input, const MatType &error, MatType &gradient)
 Computing the gradient of the layer with respect to its own input. More...

 
LSTMTypeoperator= (const LSTMType &other)
 Copy the given LSTMType object. More...

 
LSTMTypeoperator= (LSTMType &&other)
 Take ownership of the given LSTMType object's data. More...

 
const MatType & Parameters () const
 Get the parameters. More...

 
MatType & Parameters ()
 Modify the parameters. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the layer. More...

 
void SetWeights (typename MatType::elem_type *weightsPtr)
 Reset the layer parameter. More...

 
size_t WeightSize () const
 Get the total number of trainable parameters. More...

 
- Public Member Functions inherited from RecurrentLayer< MatType >
 RecurrentLayer ()
 Create the RecurrentLayer. More...

 
 RecurrentLayer (const RecurrentLayer &other)
 Copy the given RecurrentLayer. More...

 
 RecurrentLayer (RecurrentLayer &&other)
 Take ownership of the given RecurrentLayer. More...

 
virtual ~RecurrentLayer ()
 
size_t CurrentStep () const
 Get the current step index to use in a forward or backward pass. More...

 
size_t & CurrentStep ()
 Modify the current step index to use in a forward or backward pass. More...

 
bool HasPreviousStep () const
 If Forward() or Backward() has been called since ClearRecurrentState(), this will return true. More...

 
RecurrentLayeroperator= (const RecurrentLayer &other)
 Copy the given RecurrentLayer. More...

 
RecurrentLayeroperator= (RecurrentLayer &&other)
 Take ownership of the given RecurrentLayer. More...

 
size_t PreviousStep () const
 Get the previous step index, representing the value of CurrentStep() in the previous call to Forward() or Backward(). More...

 
size_t & PreviousStep ()
 Modify the previous step index, representing the value of CurrentStep() in the previous call to Forward() or Backward(). More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the recurrent layer. More...

 
- Public Member Functions inherited from Layer< MatType >
 Layer ()
 Default constructor. More...

 
 Layer (const Layer &layer)
 Copy constructor. This is not responsible for copying weights! More...

 
 Layer (Layer &&layer)
 Move constructor. This is not responsible for moving weights! More...

 
virtual ~Layer ()
 Default deconstructor. More...

 
virtual void CustomInitialize (MatType &, const size_t)
 Override the weight matrix of the layer. More...

 
virtual void Forward (const MatType &, const MatType &)
 Takes an input and output object, and computes the corresponding loss of the layer. More...

 
const std::vector< size_t > & InputDimensions () const
 Get the input dimensions. More...

 
std::vector< size_t > & InputDimensions ()
 Modify the input dimensions. More...

 
virtual double Loss ()
 Get the layer loss. More...

 
virtual Layeroperator= (const Layer &layer)
 Copy assignment operator. This is not responsible for copying weights! More...

 
virtual Layeroperator= (Layer &&layer)
 Move assignment operator. This is not responsible for moving weights! More...

 
const std::vector< size_t > & OutputDimensions ()
 Get the output dimensions. More...

 
virtual size_t OutputSize () final
 Get the number of elements in the output from this layer. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the layer. More...

 
virtual bool const & Training () const
 Get whether the layer is currently in training mode. More...

 
virtual bool & Training ()
 Modify whether the layer is currently in training mode. More...

 

Additional Inherited Members

- Protected Attributes inherited from Layer< MatType >
std::vector< size_t > inputDimensions
 Logical input dimensions of each point. More...

 
std::vector< size_t > outputDimensions
 Logical output dimensions of each point. More...

 
bool training
 If true, the layer is in training mode; otherwise, it is in testing mode. More...

 
bool validOutputDimensions
 This is true if ComputeOutputDimensions() has been called, and outputDimensions can be considered to be up-to-date. More...

 

Detailed Description


template
<
typename
MatType
=
arma::mat
>

class mlpack::ann::LSTMType< MatType >

Implementation of the LSTM module class.

The implementation corresponds to the following algorithm:

\begin{eqnarray} i &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ f &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ z &=& tanh(W \cdot x + W \cdot h + b) \\ c &=& f \odot c + i \odot z \\ o &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ h &=& o \odot tanh(c) \end{eqnarray}

Note that if an LSTM layer is desired as the first layer of a neural network, an IdentityLayer should be added to the network as the first layer, and then the LSTM layer should be added.

For more information, see the following.

@article{Graves2013,
author = {Alex Graves and Abdel{-}rahman Mohamed and Geoffrey E. Hinton},
title = {Speech Recognition with Deep Recurrent Neural Networks},
journal = CoRR},
year = {2013},
url = {http://arxiv.org/abs/1303.5778},
}
See also
FastLSTM for a faster LSTM version which combines the calculation of the input, forget, output gates and hidden state in a single step.
Template Parameters
MatTypeMatrix representation to accept as input and use for computation.

Definition at line 59 of file lstm.hpp.

Constructor & Destructor Documentation

◆ LSTMType() [1/4]

LSTMType ( )

Create the LSTM object.

Referenced by LSTMType< MatType >::Clone().

◆ LSTMType() [2/4]

LSTMType ( const size_t  outSize)

Create the LSTM layer object using the specified parameters.

Parameters
outSizeThe number of output units.
rhoMaximum number of steps to backpropagate through time (BPTT).

◆ LSTMType() [3/4]

LSTMType ( const LSTMType< MatType > &  other)

Copy the given LSTMType object.

◆ LSTMType() [4/4]

LSTMType ( LSTMType< MatType > &&  other)

Take ownership of the given LSTMType object's data.

◆ ~LSTMType()

Member Function Documentation

◆ Backward()

void Backward ( const MatType &  input,
const MatType &  gy,
MatType &  g 
)
virtual

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters
inputThe propagated input activation.
gyThe backpropagated error.
gThe calculated gradient.

Reimplemented from Layer< MatType >.

Referenced by LSTMType< MatType >::~LSTMType().

◆ ClearRecurrentState()

void ClearRecurrentState ( const size_t  bpttSteps,
const size_t  batchSize 
)
virtual

Reset the recurrent state of the LSTM layer, and allocate enough space to hold bpttSteps of previous passes with a batch size of batchSize.

Parameters
bpttStepsNumber of steps of history to allocate space for.
batchSizeBatch size to prepare for.

Implements RecurrentLayer< MatType >.

Referenced by LSTMType< MatType >::~LSTMType().

◆ Clone()

LSTMType* Clone ( ) const
inlinevirtual

Clone the LSTMType object. This handles polymorphism correctly.

Implements Layer< MatType >.

Definition at line 74 of file lstm.hpp.

References LSTMType< MatType >::LSTMType(), and LSTMType< MatType >::operator=().

◆ ComputeOutputDimensions()

void ComputeOutputDimensions ( )
inlinevirtual

Given a properly set InputDimensions(), compute the output dimensions.

Reimplemented from Layer< MatType >.

Definition at line 145 of file lstm.hpp.

References Layer< MatType >::inputDimensions, Layer< MatType >::outputDimensions, and LSTMType< MatType >::serialize().

◆ Forward()

void Forward ( const MatType &  input,
MatType &  output 
)
virtual

Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.

Reimplemented from Layer< MatType >.

Referenced by LSTMType< MatType >::~LSTMType().

◆ Gradient()

void Gradient ( const MatType &  ,
const MatType &  ,
MatType &   
)
virtual

Computing the gradient of the layer with respect to its own input.

This is returned in gradient.

The layer parameters (weights and biases) are updated accordingly using the computed gradient not by the layer itself, instead they are updated by the network that holds the instantiated layer.

Parameters
*(input) The input parameter used for calculating the gradient.
*(error) The calculated error.
*(gradient) The calculated gradient.

Reimplemented from Layer< MatType >.

Referenced by LSTMType< MatType >::~LSTMType().

◆ operator=() [1/2]

LSTMType& operator= ( const LSTMType< MatType > &  other)

Copy the given LSTMType object.

Referenced by LSTMType< MatType >::Clone().

◆ operator=() [2/2]

LSTMType& operator= ( LSTMType< MatType > &&  other)

Take ownership of the given LSTMType object's data.

◆ Parameters() [1/2]

const MatType& Parameters ( ) const
inlinevirtual

Get the parameters.

Reimplemented from Layer< MatType >.

Definition at line 134 of file lstm.hpp.

◆ Parameters() [2/2]

MatType& Parameters ( )
inlinevirtual

Modify the parameters.

Reimplemented from Layer< MatType >.

Definition at line 136 of file lstm.hpp.

◆ serialize()

void serialize ( Archive &  ar,
const uint32_t   
)

Serialize the layer.

Referenced by LSTMType< MatType >::ComputeOutputDimensions().

◆ SetWeights()

void SetWeights ( typename MatType::elem_type *  weightsPtr)
virtual

Reset the layer parameter.

The method is called to assign the allocated memory to the internal learnable parameters.

Reimplemented from Layer< MatType >.

Referenced by LSTMType< MatType >::~LSTMType().

◆ WeightSize()

size_t WeightSize ( ) const
inlinevirtual

Get the total number of trainable parameters.

Reimplemented from Layer< MatType >.

Definition at line 139 of file lstm.hpp.


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/lstm.hpp