FastLSTM< InputDataType, OutputDataType > Class Template Reference

An implementation of a faster version of the Fast LSTM network layer. More...

Public Types

typedef OutputDataType::elem_type ElemType
 
typedef InputDataType::elem_type InputElemType
 

Public Member Functions

 FastLSTM ()
 Create the Fast LSTM object. More...

 
 FastLSTM (const size_t inSize, const size_t outSize, const size_t rho=std::numeric_limits< size_t >::max())
 Create the Fast LSTM layer object using the specified parameters. More...

 
template
<
typename
InputType
,
typename
ErrorType
,
typename
GradientType
>
void Backward (const InputType &&input, ErrorType &&gy, GradientType &&g)
 Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...

 
OutputDataType const & Delta () const
 Get the delta. More...

 
OutputDataType & Delta ()
 Modify the delta. More...

 
template
<
typename
InputType
,
typename
OutputType
>
void Forward (InputType &&input, OutputType &&output)
 Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

 
template
<
typename
InputType
,
typename
ErrorType
,
typename
GradientType
>
void Gradient (InputType &&input, ErrorType &&error, GradientType &&gradient)
 
OutputDataType const & Gradient () const
 Get the gradient. More...

 
OutputDataType & Gradient ()
 Modify the gradient. More...

 
OutputDataType const & OutputParameter () const
 Get the output parameter. More...

 
OutputDataType & OutputParameter ()
 Modify the output parameter. More...

 
OutputDataType const & Parameters () const
 Get the parameters. More...

 
OutputDataType & Parameters ()
 Modify the parameters. More...

 
void Reset ()
 
void ResetCell (const size_t size)
 
size_t Rho () const
 Get the maximum number of steps to backpropagate through time (BPTT). More...

 
size_t & Rho ()
 Modify the maximum number of steps to backpropagate through time (BPTT). More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const unsigned int)
 Serialize the layer. More...

 

Detailed Description


template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::FastLSTM< InputDataType, OutputDataType >

An implementation of a faster version of the Fast LSTM network layer.

Basically by combining the calculation of the input, forget, output gates and hidden state in a single step. The standard formula changes as follows:

\begin{eqnarray} i &=& sigmoid(W \cdot x + W \cdot h + b) \\ f &=& sigmoid(W \cdot x + W \cdot h + b) \\ z &=& tanh(W \cdot x + W \cdot h + b) \\ c &=& f \cdot c + i \cdot z \\ o &=& sigmoid(W \cdot x + W \cdot h + b) \\ h &=& o \cdot tanh(c) \end{eqnarray}

Note that FastLSTM network layer does not use peephole connections between the cell and gates.

For more information, see the following.

@article{Hochreiter1997,
author = {Hochreiter, Sepp and Schmidhuber, J\"{u}rgen},
title = {Long Short-term Memory},
journal = {Neural Comput.},
year = {1997}
}
See also
LSTM for a standard implementation of the LSTM layer.
Template Parameters
InputDataTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 61 of file fast_lstm.hpp.

Member Typedef Documentation

◆ ElemType

typedef OutputDataType::elem_type ElemType

Definition at line 66 of file fast_lstm.hpp.

◆ InputElemType

typedef InputDataType::elem_type InputElemType

Definition at line 65 of file fast_lstm.hpp.

Constructor & Destructor Documentation

◆ FastLSTM() [1/2]

FastLSTM ( )

Create the Fast LSTM object.

◆ FastLSTM() [2/2]

FastLSTM ( const size_t  inSize,
const size_t  outSize,
const size_t  rho = std::numeric_limits< size_t >::max() 
)

Create the Fast LSTM layer object using the specified parameters.

Parameters
inSizeThe number of input units.
outSizeThe number of output units.
rhoMaximum number of steps to backpropagate through time (BPTT).

Member Function Documentation

◆ Backward()

void Backward ( const InputType &&  input,
ErrorType &&  gy,
GradientType &&  g 
)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters
inputThe propagated input activation.
gyThe backpropagated error.
gThe calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const
inline

Get the delta.

Definition at line 147 of file fast_lstm.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )
inline

Modify the delta.

Definition at line 149 of file fast_lstm.hpp.

◆ Forward()

void Forward ( InputType &&  input,
OutputType &&  output 
)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.

◆ Gradient() [1/3]

void Gradient ( InputType &&  input,
ErrorType &&  error,
GradientType &&  gradient 
)

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const
inline

Get the gradient.

Definition at line 152 of file fast_lstm.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )
inline

Modify the gradient.

Definition at line 154 of file fast_lstm.hpp.

References FastLSTM< InputDataType, OutputDataType >::serialize().

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const
inline

Get the output parameter.

Definition at line 142 of file fast_lstm.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )
inline

Modify the output parameter.

Definition at line 144 of file fast_lstm.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const
inline

Get the parameters.

Definition at line 137 of file fast_lstm.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )
inline

Modify the parameters.

Definition at line 139 of file fast_lstm.hpp.

◆ Reset()

void Reset ( )

◆ ResetCell()

void ResetCell ( const size_t  size)

◆ Rho() [1/2]

size_t Rho ( ) const
inline

Get the maximum number of steps to backpropagate through time (BPTT).

Definition at line 132 of file fast_lstm.hpp.

◆ Rho() [2/2]

size_t& Rho ( )
inline

Modify the maximum number of steps to backpropagate through time (BPTT).

Definition at line 134 of file fast_lstm.hpp.

◆ serialize()

void serialize ( Archive &  ar,
const unsigned  int 
)

Serialize the layer.

Referenced by FastLSTM< InputDataType, OutputDataType >::Gradient().


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-3.2.1/src/mlpack/methods/ann/layer/fast_lstm.hpp