Implementation of the LSTM module class. More...

Inheritance diagram for LSTMType< MatType >:

Public Member Functions
	LSTMType ()
	Create the LSTM object. More...

	LSTMType (const size_t outSize)
	Create the LSTM layer object using the specified parameters. More...

	LSTMType (const LSTMType &other)
	Copy the given LSTMType object. More...

	LSTMType (LSTMType &&other)
	Take ownership of the given LSTMType object's data. More...

virtual	~LSTMType ()

void	Backward (const MatType &input, const MatType &gy, MatType &g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...

void	ClearRecurrentState (const size_t bpttSteps, const size_t batchSize)
	Reset the recurrent state of the LSTM layer, and allocate enough space to hold `bpttSteps` of previous passes with a batch size of `batchSize`. More...

LSTMType *	Clone () const
	Clone the LSTMType object. This handles polymorphism correctly. More...

void	ComputeOutputDimensions ()
	Given a properly set InputDimensions(), compute the output dimensions. More...

void	Forward (const MatType &input, MatType &output)
	Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

void	Gradient (const MatType &input, const MatType &error, MatType &gradient)
	Computing the gradient of the layer with respect to its own input. More...

LSTMType &	operator= (const LSTMType &other)
	Copy the given LSTMType object. More...

LSTMType &	operator= (LSTMType &&other)
	Take ownership of the given LSTMType object's data. More...

const MatType &	Parameters () const
	Get the parameters. More...

MatType &	Parameters ()
	Modify the parameters. More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

void	SetWeights (typename MatType::elem_type *weightsPtr)
	Reset the layer parameter. More...

size_t	WeightSize () const
	Get the total number of trainable parameters. More...

Public Member Functions inherited from RecurrentLayer< MatType >
	RecurrentLayer ()
	Create the RecurrentLayer. More...

	RecurrentLayer (const RecurrentLayer &other)
	Copy the given RecurrentLayer. More...

	RecurrentLayer (RecurrentLayer &&other)
	Take ownership of the given RecurrentLayer. More...

virtual	~RecurrentLayer ()

size_t	CurrentStep () const
	Get the current step index to use in a forward or backward pass. More...

size_t &	CurrentStep ()
	Modify the current step index to use in a forward or backward pass. More...

bool	HasPreviousStep () const
	If Forward() or Backward() has been called since ClearRecurrentState(), this will return true. More...

RecurrentLayer &	operator= (const RecurrentLayer &other)
	Copy the given RecurrentLayer. More...

RecurrentLayer &	operator= (RecurrentLayer &&other)
	Take ownership of the given RecurrentLayer. More...

size_t	PreviousStep () const
	Get the previous step index, representing the value of CurrentStep() in the previous call to Forward() or Backward(). More...

size_t &	PreviousStep ()
	Modify the previous step index, representing the value of CurrentStep() in the previous call to Forward() or Backward(). More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the recurrent layer. More...

Public Member Functions inherited from Layer< MatType >
	Layer ()
	Default constructor. More...

	Layer (const Layer &layer)
	Copy constructor. This is not responsible for copying weights! More...

	Layer (Layer &&layer)
	Move constructor. This is not responsible for moving weights! More...

virtual	~Layer ()
	Default deconstructor. More...

virtual void	CustomInitialize (MatType &, const size_t)
	Override the weight matrix of the layer. More...

virtual void	Forward (const MatType &, const MatType &)
	Takes an input and output object, and computes the corresponding loss of the layer. More...

const std::vector< size_t > &	InputDimensions () const
	Get the input dimensions. More...

std::vector< size_t > &	InputDimensions ()
	Modify the input dimensions. More...

virtual double	Loss ()
	Get the layer loss. More...

virtual Layer &	operator= (const Layer &layer)
	Copy assignment operator. This is not responsible for copying weights! More...

virtual Layer &	operator= (Layer &&layer)
	Move assignment operator. This is not responsible for moving weights! More...

const std::vector< size_t > &	OutputDimensions ()
	Get the output dimensions. More...

virtual size_t	OutputSize () final
	Get the number of elements in the output from this layer. More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

virtual bool const &	Training () const
	Get whether the layer is currently in training mode. More...

virtual bool &	Training ()
	Modify whether the layer is currently in training mode. More...

Additional Inherited Members
Protected Attributes inherited from Layer< MatType >
std::vector< size_t >	inputDimensions
	Logical input dimensions of each point. More...

std::vector< size_t >	outputDimensions
	Logical output dimensions of each point. More...

bool	training
	If true, the layer is in training mode; otherwise, it is in testing mode. More...

bool	validOutputDimensions
	This is `true` if `ComputeOutputDimensions()` has been called, and `outputDimensions` can be considered to be up-to-date. More...

Detailed Description

template
<
typename
MatType
=
arma::mat
>

class mlpack::ann::LSTMType< MatType >

Implementation of the LSTM module class.

The implementation corresponds to the following algorithm:

$\begin{eqnarray} i &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ f &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ z &=& tanh(W \cdot x + W \cdot h + b) \\ c &=& f \odot c + i \odot z \\ o &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ h &=& o \odot tanh(c) \end{eqnarray}$

Note that if an LSTM layer is desired as the first layer of a neural network, an IdentityLayer should be added to the network as the first layer, and then the LSTM layer should be added.

For more information, see the following.

@article{Graves2013,
  author  = {Alex Graves and Abdel{-}rahman Mohamed and Geoffrey E. Hinton},
  title   = {Speech Recognition with Deep Recurrent Neural Networks},
  journal = CoRR},
  year    = {2013},
  url     = {http://arxiv.org/abs/1303.5778},
}

See also: FastLSTM for a faster LSTM version which combines the calculation of the input, forget, output gates and hidden state in a single step.

Template Parameters

MatType Matrix representation to accept as input and use for computation.

Definition at line 59 of file lstm.hpp.

Constructor & Destructor Documentation

◆ LSTMType() [1/4]

LSTMType ( )

Create the LSTM object.

Referenced by LSTMType< MatType >::Clone().

◆ LSTMType() [2/4]

LSTMType ( const size_t outSize )

Create the LSTM layer object using the specified parameters.

Parameters

outSize	The number of output units.
rho	Maximum number of steps to backpropagate through time (BPTT).

◆ LSTMType() [3/4]

LSTMType ( const LSTMType< MatType > & other )

Copy the given LSTMType object.

◆ LSTMType() [4/4]

LSTMType ( LSTMType< MatType > && other )

Take ownership of the given LSTMType object's data.

◆ ~LSTMType()

virtual ~LSTMType ( )

inlinevirtual

Definition at line 85 of file lstm.hpp.

References LSTMType< MatType >::Backward(), LSTMType< MatType >::ClearRecurrentState(), LSTMType< MatType >::Forward(), LSTMType< MatType >::Gradient(), and LSTMType< MatType >::SetWeights().

Member Function Documentation

◆ Backward()

void Backward	(	const MatType &	input,
		const MatType &	gy,
		MatType &	g
	)

virtual

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters

input	The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

Reimplemented from Layer< MatType >.

Referenced by LSTMType< MatType >::~LSTMType().

◆ ClearRecurrentState()

void ClearRecurrentState	(	const size_t	bpttSteps,
		const size_t	batchSize
	)

virtual

Reset the recurrent state of the LSTM layer, and allocate enough space to hold bpttSteps of previous passes with a batch size of batchSize.

Parameters

bpttSteps	Number of steps of history to allocate space for.
batchSize	Batch size to prepare for.

Implements RecurrentLayer< MatType >.

Referenced by LSTMType< MatType >::~LSTMType().

◆ Clone()

LSTMType* Clone ( ) const

inlinevirtual

Clone the LSTMType object. This handles polymorphism correctly.

Implements Layer< MatType >.

Definition at line 74 of file lstm.hpp.

References LSTMType< MatType >::LSTMType(), and LSTMType< MatType >::operator=().

◆ ComputeOutputDimensions()

void ComputeOutputDimensions ( )

inlinevirtual

Given a properly set InputDimensions(), compute the output dimensions.

Reimplemented from Layer< MatType >.

Definition at line 145 of file lstm.hpp.

References Layer< MatType >::inputDimensions, Layer< MatType >::outputDimensions, and LSTMType< MatType >::serialize().

◆ Forward()

void Forward	(	const MatType &	input,
		MatType &	output
	)

virtual

Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

Reimplemented from Layer< MatType >.

Referenced by LSTMType< MatType >::~LSTMType().

◆ Gradient()

void Gradient	(	const MatType &	,
		const MatType &	,
		MatType &
	)

virtual

Computing the gradient of the layer with respect to its own input.

This is returned in gradient.

The layer parameters (weights and biases) are updated accordingly using the computed gradient not by the layer itself, instead they are updated by the network that holds the instantiated layer.

Parameters

*	(input) The input parameter used for calculating the gradient.
*	(error) The calculated error.
*	(gradient) The calculated gradient.

Reimplemented from Layer< MatType >.

Referenced by LSTMType< MatType >::~LSTMType().

◆ operator=() [1/2]

LSTMType& operator= ( const LSTMType< MatType > & other )

Copy the given LSTMType object.

Referenced by LSTMType< MatType >::Clone().

◆ operator=() [2/2]

LSTMType& operator= ( LSTMType< MatType > && other )

Take ownership of the given LSTMType object's data.

◆ Parameters() [1/2]

const MatType& Parameters ( ) const

inlinevirtual

Get the parameters.

Reimplemented from Layer< MatType >.

Definition at line 134 of file lstm.hpp.

◆ Parameters() [2/2]

MatType& Parameters ( )

inlinevirtual

Modify the parameters.

Reimplemented from Layer< MatType >.

Definition at line 136 of file lstm.hpp.

◆ serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the layer.

Referenced by LSTMType< MatType >::ComputeOutputDimensions().

◆ SetWeights()

void SetWeights ( typename MatType::elem_type * weightsPtr )

virtual

Reset the layer parameter.

The method is called to assign the allocated memory to the internal learnable parameters.

Reimplemented from Layer< MatType >.

Referenced by LSTMType< MatType >::~LSTMType().

◆ WeightSize()

size_t WeightSize ( ) const

inlinevirtual

Get the total number of trainable parameters.

Reimplemented from Layer< MatType >.

Definition at line 139 of file lstm.hpp.

The documentation for this class was generated from the following file:

/home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/lstm.hpp

Public Member Functions

Additional Inherited Members

Detailed Description

template<typenameMatType=arma::mat> class mlpack::ann::LSTMType< MatType >

Constructor & Destructor Documentation

◆ LSTMType() [1/4]

◆ LSTMType() [2/4]

◆ LSTMType() [3/4]

◆ LSTMType() [4/4]

◆ ~LSTMType()

Member Function Documentation

◆ Backward()

◆ ClearRecurrentState()

◆ Clone()

◆ ComputeOutputDimensions()

◆ Forward()

◆ Gradient()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ serialize()

◆ SetWeights()

◆ WeightSize()

template
<
typename
MatType
=
arma::mat
>

class mlpack::ann::LSTMType< MatType >