An implementation of a faster version of the Fast LSTM network layer. More...

Inheritance diagram for FastLSTMType< InputType, OutputType >:

Public Types
typedef InputType::elem_type	InputET

typedef OutputType::elem_type	OutputET

Public Member Functions
	FastLSTMType ()
	Create the FastLSTMType object. More...

	FastLSTMType (const FastLSTMType &layer)
	Copy Constructor. More...

	FastLSTMType (FastLSTMType &&layer)
	Move Constructor. More...

	FastLSTMType (const size_t inSize, const size_t outSize, const size_t rho=std::numeric_limits< size_t >::max())
	Create the Fast LSTM layer object using the specified parameters. More...

void	Backward (const InputType &input, const OutputType &gy, OutputType &g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...

FastLSTMType *	Clone () const
	Clone the FastLSTMType object. This handles polymorphism correctly. More...

void	Forward (const InputType &input, OutputType &output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

void	Gradient (const InputType &input, const OutputType &error, OutputType &gradient)
	Calculate the gradient using the output delta and the input activation. More...

size_t	InSize () const
	Get the number of input units. More...

FastLSTMType &	operator= (const FastLSTMType &layer)
	Copy assignment operator. More...

FastLSTMType &	operator= (FastLSTMType &&layer)
	Move assignment operator. More...

const std::vector< size_t >	OutputDimensions () const

size_t	OutSize () const
	Get the number of output units. More...

OutputType const &	Parameters () const
	Get the parameters. More...

OutputType &	Parameters ()
	Modify the parameters. More...

void	Reset ()
	Reset the layer parameter. More...

void	ResetCell (const size_t size)
	Resets the cell to accept a new input. More...

size_t	Rho () const
	Get the maximum number of steps to backpropagate through time (BPTT). More...

size_t &	Rho ()
	Modify the maximum number of steps to backpropagate through time (BPTT). More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

size_t	WeightSize () const
	Get the size of the weight matrix. More...

Public Member Functions inherited from Layer< InputType, OutputType >
	Layer ()
	Default constructor. More...

	Layer (const Layer &layer)
	Copy constructor. This is not responsible for copying weights! More...

	Layer (Layer &&layer)
	Move constructor. This is not responsible for moving weights! More...

virtual	~Layer ()
	Default deconstructor. More...

virtual void	Backward (const InputType &, const InputType &, InputType &)
	Performs a backpropagation step through the layer, with respect to the given input. More...

virtual void	ComputeOutputDimensions ()
	Compute the output dimensions. More...

virtual void	CustomInitialize (InputType &, const size_t)
	Override the weight matrix of the layer. More...

virtual void	Forward (const InputType &, InputType &)
	Takes an input object, and computes the corresponding output of the layer. More...

virtual void	Forward (const InputType &, const InputType &)
	Takes an input and output object, and computes the corresponding loss of the layer. More...

virtual void	Gradient (const InputType &, const InputType &, InputType &)
	Computing the gradient of the layer with respect to its own input. More...

const std::vector< size_t > &	InputDimensions () const
	Get the input dimensions. More...

std::vector< size_t > &	InputDimensions ()
	Modify the input dimensions. More...

virtual double	Loss ()
	Get the layer loss. More...

virtual Layer &	operator= (const Layer &layer)
	Copy assignment operator. This is not responsible for copying weights! More...

virtual Layer &	operator= (Layer &&layer)
	Move assignment operator. This is not responsible for moving weights! More...

const std::vector< size_t > &	OutputDimensions ()
	Get the output dimensions. More...

virtual size_t	OutputSize () final
	Get the number of elements in the output from this layer. More...

void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

virtual void	SetWeights (typename InputType ::elem_type *)
	Reset the layer parameter. More...

virtual bool const &	Training () const
	Get whether the layer is currently in training mode. More...

virtual bool &	Training ()
	Modify whether the layer is currently in training mode. More...

Additional Inherited Members
Protected Attributes inherited from Layer< InputType, OutputType >
std::vector< size_t >	inputDimensions
	Logical input dimensions of each point. More...

std::vector< size_t >	outputDimensions
	Logical output dimensions of each point. More...

bool	training
	If true, the layer is in training mode; otherwise, it is in testing mode. More...

bool	validOutputDimensions
	This is `true` if `ComputeOutputDimensions()` has been called, and `outputDimensions` can be considered to be up-to-date. More...

Detailed Description

template
<
typename
InputType
=
arma::mat
,
typename
OutputType
=
arma::mat
>

class mlpack::ann::FastLSTMType< InputType, OutputType >

An implementation of a faster version of the Fast LSTM network layer.

Basically by combining the calculation of the input, forget, output gates and hidden state in a single step. The standard formula changes as follows:

$\begin{eqnarray} i &=& sigmoid(W \cdot x + W \cdot h + b) \\ f &=& sigmoid(W \cdot x + W \cdot h + b) \\ z &=& tanh(W \cdot x + W \cdot h + b) \\ c &=& f \cdot c + i \cdot z \\ o &=& sigmoid(W \cdot x + W \cdot h + b) \\ h &=& o \cdot tanh(c) \end{eqnarray}$

Note that FastLSTM network layer does not use peephole connections between the cell and gates.

Note also that if a FastLSTM layer is desired as the first layer of a neural network, an IdentityLayer should be added to the network as the first layer, and then the FastLSTM layer should be added.

For more information, see the following.

@article{Hochreiter1997,
  author  = {Hochreiter, Sepp and Schmidhuber, J\"{u}rgen},
  title   = {Long Short-term Memory},
  journal = {Neural Comput.},
  year    = {1997},
  url     = {https://www.bioinf.jku.at/publications/older/2604.pdf}
}

See also: LSTM for a standard implementation of the LSTM layer.

Template Parameters

InputType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 68 of file fast_lstm.hpp.

Member Typedef Documentation

◆ InputET

typedef InputType::elem_type InputET

Definition at line 72 of file fast_lstm.hpp.

◆ OutputET

typedef OutputType::elem_type OutputET

Definition at line 73 of file fast_lstm.hpp.

Constructor & Destructor Documentation

◆ FastLSTMType() [1/4]

FastLSTMType ( )

Create the FastLSTMType object.

Referenced by FastLSTMType< InputType, OutputType >::Clone().

◆ FastLSTMType() [2/4]

FastLSTMType ( const FastLSTMType< InputType, OutputType > & layer )

Copy Constructor.

◆ FastLSTMType() [3/4]

FastLSTMType ( FastLSTMType< InputType, OutputType > && layer )

Move Constructor.

◆ FastLSTMType() [4/4]

FastLSTMType	(	const size_t	inSize,
		const size_t	outSize,
		const size_t	rho = `std::numeric_limits< size_t >::max()`
	)

Create the Fast LSTM layer object using the specified parameters.

Parameters

inSize	The number of input units.
outSize	The number of output units.
rho	Maximum number of steps to backpropagate through time (BPTT).

Member Function Documentation

◆ Backward()

void Backward	(	const InputType &	input,
		const OutputType &	gy,
		OutputType &	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters

input	The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

Referenced by FastLSTMType< InputType, OutputType >::Clone().

◆ Clone()

FastLSTMType* Clone ( ) const

inlinevirtual

Clone the FastLSTMType object. This handles polymorphism correctly.

Implements Layer< InputType, OutputType >.

Definition at line 102 of file fast_lstm.hpp.

References FastLSTMType< InputType, OutputType >::Backward(), FastLSTMType< InputType, OutputType >::FastLSTMType(), FastLSTMType< InputType, OutputType >::Forward(), FastLSTMType< InputType, OutputType >::Gradient(), FastLSTMType< InputType, OutputType >::Reset(), FastLSTMType< InputType, OutputType >::ResetCell(), and core::v2::size().

◆ Forward()

void Forward	(	const InputType &	input,
		OutputType &	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

Referenced by FastLSTMType< InputType, OutputType >::Clone().

◆ Gradient()

void Gradient	(	const InputType &	input,
		const OutputType &	error,
		OutputType &	gradient
	)

Calculate the gradient using the output delta and the input activation.

Parameters

input	The input parameter used for calculating the gradient.
error	The calculated error.
gradient	The calculated gradient.

Referenced by FastLSTMType< InputType, OutputType >::Clone().

◆ InSize()

size_t InSize ( ) const

inline

Get the number of input units.

Definition at line 161 of file fast_lstm.hpp.

◆ operator=() [1/2]

FastLSTMType& operator= ( const FastLSTMType< InputType, OutputType > & layer )

Copy assignment operator.

◆ operator=() [2/2]

FastLSTMType& operator= ( FastLSTMType< InputType, OutputType > && layer )

Move assignment operator.

◆ OutputDimensions()

const std::vector<size_t> OutputDimensions ( ) const

inline

Definition at line 172 of file fast_lstm.hpp.

References core::v2::data(), Layer< InputType, OutputType >::inputDimensions, and FastLSTMType< InputType, OutputType >::serialize().

◆ OutSize()

size_t OutSize ( ) const

inline

Get the number of output units.

Definition at line 164 of file fast_lstm.hpp.

◆ Parameters() [1/2]

OutputType const& Parameters ( ) const

inlinevirtual

Get the parameters.

Reimplemented from Layer< InputType, OutputType >.

Definition at line 156 of file fast_lstm.hpp.

◆ Parameters() [2/2]

OutputType& Parameters ( )

inlinevirtual

Modify the parameters.

Reimplemented from Layer< InputType, OutputType >.

Definition at line 158 of file fast_lstm.hpp.

◆ Reset()

void Reset ( )

Reset the layer parameter.

Referenced by FastLSTMType< InputType, OutputType >::Clone().

◆ ResetCell()

void ResetCell ( const size_t size )

Resets the cell to accept a new input.

This breaks the BPTT chain starts a new one.

Parameters

size	The current maximum number of steps through time.

Referenced by FastLSTMType< InputType, OutputType >::Clone().

◆ Rho() [1/2]

size_t Rho ( ) const

inline

Get the maximum number of steps to backpropagate through time (BPTT).

Definition at line 151 of file fast_lstm.hpp.

◆ Rho() [2/2]

size_t& Rho ( )

inline

Modify the maximum number of steps to backpropagate through time (BPTT).

Definition at line 153 of file fast_lstm.hpp.

◆ serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the layer.

Referenced by FastLSTMType< InputType, OutputType >::OutputDimensions().

◆ WeightSize()

size_t WeightSize ( ) const

inlinevirtual

Get the size of the weight matrix.

Reimplemented from Layer< InputType, OutputType >.

Definition at line 167 of file fast_lstm.hpp.

The documentation for this class was generated from the following file:

/home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/not_adapted/fast_lstm.hpp

Public Types

Public Member Functions

Additional Inherited Members

Detailed Description

template<typenameInputType=arma::mat,typenameOutputType=arma::mat> class mlpack::ann::FastLSTMType< InputType, OutputType >

Member Typedef Documentation

◆ InputET

◆ OutputET

Constructor & Destructor Documentation

◆ FastLSTMType() [1/4]

◆ FastLSTMType() [2/4]

◆ FastLSTMType() [3/4]

◆ FastLSTMType() [4/4]

Member Function Documentation

◆ Backward()

◆ Clone()

◆ Forward()

◆ Gradient()

◆ InSize()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ OutputDimensions()

◆ OutSize()

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ Reset()

◆ ResetCell()

◆ Rho() [1/2]

◆ Rho() [2/2]

◆ serialize()

◆ WeightSize()

template
<
typename
InputType
=
arma::mat
,
typename
OutputType
=
arma::mat
>

class mlpack::ann::FastLSTMType< InputType, OutputType >