Definition of a standard recurrent neural network container. More...

Inheritance diagram for RNN< OutputLayerType, InitializationRuleType, MatType >:

Public Member Functions
	RNN (const size_t bpttSteps=0, const bool single=false, OutputLayerType outputLayer=OutputLayerType(), InitializationRuleType initializeRule=InitializationRuleType())
	Create the RNN object. More...

	RNN (const RNN &)
	Copy constructor. More...

	RNN (RNN &&)
	Move constructor. More...

	~RNN ()
	Destroy the RNN and release any memory it is holding. More...

template<typename LayerType , typename... Args>
void	Add (Args... args)
	Add a new module to the model. More...

void	Add (Layer< MatType > *layer)
	Add a new module to the model. More...

size_t	BPTTSteps () const
	Return the number of steps allowed for BPTT. More...

size_t &	BPTTSteps ()
	Modify the number of steps allowed for BPTT. More...

MatType::elem_type	Evaluate (const arma::Cube< typename MatType::elem_type > &predictors, const arma::Cube< typename MatType::elem_type > &responses)
	Evaluate the recurrent network with the given predictors and responses. More...

MatType::elem_type	Evaluate (const MatType &parameters)
	Evaluate the recurrent network with the given parameters. More...

MatType::elem_type	Evaluate (const MatType &parameters, const size_t begin, const size_t batchSize)
	Evaluate the recurrent network with the given parameters, but using only a number of data points. More...

template < typename GradType >
MatType::elem_type	EvaluateWithGradient (const MatType &parameters, GradType &gradient)
	Evaluate the recurrent network with the given parameters. More...

template < typename GradType >
MatType::elem_type	EvaluateWithGradient (const MatType &parameters, const size_t begin, GradType &gradient, const size_t batchSize)
	Evaluate the recurrent network with the given parameters, but using only a number of data points. More...

template < typename GradType >
void	Gradient (const MatType &parameters, const size_t begin, GradType &gradient, const size_t batchSize)
	Evaluate the gradient of the recurrent network with the given parameters, and with respect to only a number of points in the dataset. More...

std::vector< size_t > &	InputDimensions ()
	Set the logical dimensions of the input. More...

const std::vector< size_t > &	InputDimensions () const
	Get the logical dimensions of the input. More...

const std::vector< Layer< MatType > * > &	Network () const
	Get the network model. More...

std::vector< Layer< MatType > * > &	Network ()
	Modify the network model. More...

size_t	NumFunctions () const
	Return the number of separable functions (the number of predictor points). More...

RNN &	operator= (const RNN &)
	Copy operator. More...

RNN &	operator= (RNN &&)
	Move assignment operator. More...

const MatType &	Parameters () const
	Return the initial point for the optimization. More...

MatType &	Parameters ()
	Modify the initial point for the optimization. More...

void	Predict (arma::Cube< typename MatType::elem_type > predictors, arma::Cube< typename MatType::elem_type > &results, const size_t batchSize=128)
	Predict the responses to a given set of predictors. More...

void	Reset (const size_t inputDimensionality=0)
	Reset the stored data of the network entirely. More...

void	ResetData (arma::Cube< typename MatType::elem_type > predictors, arma::Cube< typename MatType::elem_type > responses)
	Prepare the network for the given data. More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the model. More...

void	SetNetworkMode (const bool training)
	Set all the layers in the network to training mode, if `training` is `true`, or set all the layers in the network to testing mode, if `training` is `false`. More...

void	Shuffle ()
	Note: this function is implement so that it can be used by ensmallen's optimizers. More...

template<typename OptimizerType , typename... CallbackTypes>
MatType::elem_type	Train (arma::Cube< typename MatType::elem_type > predictors, arma::Cube< typename MatType::elem_type > responses, OptimizerType &optimizer, CallbackTypes &&... callbacks)
	Train the recurrent network on the given input data using the given optimizer. More...

template<typename OptimizerType = ens::RMSProp, typename... CallbackTypes>
MatType::elem_type	Train (arma::Cube< typename MatType::elem_type > predictors, arma::Cube< typename MatType::elem_type > responses, CallbackTypes &&... callbacks)
	Train the recurrent network on the given input data. More...

size_t	WeightSize ()

Detailed Description

template
<
typename
OutputLayerType
=
NegativeLogLikelihood
,
typename
InitializationRuleType
=
RandomInitialization
,
typename
MatType
=
arma::mat
>

class mlpack::ann::RNN< OutputLayerType, InitializationRuleType, MatType >

Definition of a standard recurrent neural network container.

A recurrent neural network can handle recurrent layers (i.e. RecurrentLayers), which hold internal state and are passed sequences of data as inputs.

As opposed to the standard FFN, which takes data in a matrix format where each column is a data point, the RNN takes a cube format where each column is a data point and each slice is a time step.

Template Parameters

OutputLayerType	The output layer type used to evaluate the network.
InitializationRuleType	Rule used to initialize the weight matrix.

Definition at line 24 of file forward_decls.hpp.

Constructor & Destructor Documentation

◆ RNN() [1/3]

RNN	(	const size_t	bpttSteps = `0`,
		const bool	single = `false`,
		OutputLayerType	outputLayer = `OutputLayerType()`,
		InitializationRuleType	initializeRule = `InitializationRuleType()`
	)

Create the RNN object.

Optionally, specify which initialize rule and performance function should be used.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Parameters

bpttSteps	Number of time steps to use for BPTT (backpropagation through time) when training.
single	If true, then the network will expect only a single timestep for responses. (That is, every input sequence only has one single output; so, `responses.n_slices` should be 1 when calling `Train()`.)
outputLayer	Output layer used to evaluate the network.
initializeRule	Optional instantiated InitializationRule object for initializing the network parameter.

◆ RNN() [2/3]

RNN ( const RNN< OutputLayerType, InitializationRuleType, MatType > & )

Copy constructor.

◆ RNN() [3/3]

RNN ( RNN< OutputLayerType, InitializationRuleType, MatType > && )

Move constructor.

◆ ~RNN()

~RNN ( )

Destroy the RNN and release any memory it is holding.

Member Function Documentation

◆ Add() [1/2]

void Add ( Args... args )

inline

Add a new module to the model.

Parameters

args	The layer parameter.

Definition at line 83 of file rnn.hpp.

◆ Add() [2/2]

void Add ( Layer< MatType > * layer )

inline

Add a new module to the model.

Parameters

layer The Layer to be added to the model.

Definition at line 90 of file rnn.hpp.

◆ BPTTSteps() [1/2]

size_t BPTTSteps ( ) const

inline

Return the number of steps allowed for BPTT.

Definition at line 209 of file rnn.hpp.

◆ BPTTSteps() [2/2]

size_t& BPTTSteps ( )

inline

Modify the number of steps allowed for BPTT.

Definition at line 211 of file rnn.hpp.

◆ Evaluate() [1/3]

MatType::elem_type Evaluate	(	const arma::Cube< typename MatType::elem_type > &	predictors,
		const arma::Cube< typename MatType::elem_type > &	responses
	)

Evaluate the recurrent network with the given predictors and responses.

This functions is usually used to monitor progress while training.

Parameters

predictors	Input variables.
responses	Target outputs for input variables.

Referenced by RNN< OutputLayerType, InitializationRuleType, CustomLayers... >::SetNetworkMode().

◆ Evaluate() [2/3]

MatType::elem_type Evaluate ( const MatType & parameters )

Evaluate the recurrent network with the given parameters.

This function is usually called by the optimizer to train the model.

Parameters

parameters Matrix model parameters.

◆ Evaluate() [3/3]

MatType::elem_type Evaluate	(	const MatType &	parameters,
		const size_t	begin,
		const size_t	batchSize
	)

Evaluate the recurrent network with the given parameters, but using only a number of data points.

This is useful for optimizers such as SGD, which require a separable objective function.

Note that the network may return different results depending on the mode it is in (see SetNetworkMode()).

Parameters

parameters	Matrix model parameters.
begin	Index of the starting point to use for objective function evaluation.
batchSize	Number of points to be passed at a time to use for objective function evaluation.

◆ EvaluateWithGradient() [1/2]

MatType::elem_type EvaluateWithGradient	(	const MatType &	parameters,
		GradType &	gradient
	)

Evaluate the recurrent network with the given parameters.

This function is usually called by the optimizer to train the model. This just calls the overload of EvaluateWithGradient() with batchSize = 1.

Parameters

parameters	Matrix model parameters.
gradient	Matrix to output gradient into.

Referenced by RNN< OutputLayerType, InitializationRuleType, CustomLayers... >::SetNetworkMode().

◆ EvaluateWithGradient() [2/2]

MatType::elem_type EvaluateWithGradient	(	const MatType &	parameters,
		const size_t	begin,
		GradType &	gradient,
		const size_t	batchSize
	)

Evaluate the recurrent network with the given parameters, but using only a number of data points.

This is useful for optimizers such as SGD, which require a separable objective function.

Parameters

parameters	Matrix model parameters.
begin	Index of the starting point to use for objective function evaluation.
gradient	Matrix to output gradient into.
batchSize	Number of points to be passed at a time to use for objective function evaluation.

◆ Gradient()

void Gradient	(	const MatType &	parameters,
		const size_t	begin,
		GradType &	gradient,
		const size_t	batchSize
	)

Evaluate the gradient of the recurrent network with the given parameters, and with respect to only a number of points in the dataset.

This is useful for optimizers such as SGD, which require a separable objective function.

Parameters

parameters	Matrix of the model parameters to be optimized.
begin	Index of the starting point to use for objective function gradient evaluation.
gradient	Matrix to output gradient into.
batchSize	Number of points to be processed as a batch for objective function gradient evaluation.

Referenced by RNN< OutputLayerType, InitializationRuleType, CustomLayers... >::SetNetworkMode().

◆ InputDimensions() [1/2]

std::vector<size_t>& InputDimensions ( )

inline

Set the logical dimensions of the input.

Train() and Predict() expect data to be passed such that one point corresponds to one column, but this data is allowed to be an arbitrary higher-order tensor.

So, if the input is meant to be 28x28x3 images, then the input data to Train() or Predict() should have 28*28*3 = 2352 rows, and InputDImensions() should be set to { 28, 28, 3}. Then, the layers of the network will interpret each input point as a 3-dimensional image instead of a 1-dimensional vector.

If InputDimensions() is left unset before training, the data will be assumed to be a 1-dimensional vector.

Definition at line 196 of file rnn.hpp.

◆ InputDimensions() [2/2]

const std::vector<size_t>& InputDimensions ( ) const

inline

Get the logical dimensions of the input.

Definition at line 198 of file rnn.hpp.

◆ Network() [1/2]

const std::vector<Layer<MatType>*>& Network ( ) const

inline

Get the network model.

Definition at line 93 of file rnn.hpp.

◆ Network() [2/2]

std::vector<Layer<MatType>*>& Network ( )

inline

Modify the network model.

Be careful! If you change the structure of the network or parameters for layers, its state may become invalid, and the next time it is used for any operation the parameters will be reset.

Don't add any layers like this; use Add() instead.

Definition at line 105 of file rnn.hpp.

◆ NumFunctions()

size_t NumFunctions ( ) const

inline

Return the number of separable functions (the number of predictor points).

Definition at line 326 of file rnn.hpp.

◆ operator=() [1/2]

RNN& operator= ( const RNN< OutputLayerType, InitializationRuleType, MatType > & )

Copy operator.

◆ operator=() [2/2]

RNN& operator= ( RNN< OutputLayerType, InitializationRuleType, MatType > && )

Move assignment operator.

◆ Parameters() [1/2]

const MatType& Parameters ( ) const

inline

Return the initial point for the optimization.

Definition at line 204 of file rnn.hpp.

◆ Parameters() [2/2]

MatType& Parameters ( )

inline

Modify the initial point for the optimization.

Definition at line 206 of file rnn.hpp.

◆ Predict()

void Predict	(	arma::Cube< typename MatType::elem_type >	predictors,
		arma::Cube< typename MatType::elem_type > &	results,
		const size_t	batchSize = `128`
	)

Predict the responses to a given set of predictors.

The responses will reflect the output of the given output layer as returned by the output layer function.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Parameters

predictors	Input predictors.
results	Matrix to put output predictions of responses into.
batchSize	Batch size to use for prediction.

Referenced by RNN< OutputLayerType, InitializationRuleType, CustomLayers... >::Network().

◆ Reset()

void Reset ( const size_t inputDimensionality = 0 )

Reset the stored data of the network entirely.

This reset all weights of each layer using InitializationRuleType, and prepares the network to accept a (flat 1-d) input size of inputDimensionality (if passed), or whatever input size has been set with InputDimensions().

This also resets the mode of the network to prediction mode (not training mode). See SetNetworkMode() for more information.

Referenced by RNN< OutputLayerType, InitializationRuleType, CustomLayers... >::BPTTSteps().

◆ ResetData()

void ResetData	(	arma::Cube< typename MatType::elem_type >	predictors,
		arma::Cube< typename MatType::elem_type >	responses
	)

Prepare the network for the given data.

This function won't actually trigger training process.

Parameters

predictors	Input data variables.
responses	Outputs results from input data variables.

Referenced by RNN< OutputLayerType, InitializationRuleType, CustomLayers... >::NumFunctions().

◆ serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the model.

Referenced by RNN< OutputLayerType, InitializationRuleType, CustomLayers... >::SetNetworkMode().

◆ SetNetworkMode()

void SetNetworkMode ( const bool training )

inline

Set all the layers in the network to training mode, if training is true, or set all the layers in the network to testing mode, if training is false.

Definition at line 229 of file rnn.hpp.

◆ Shuffle()

void Shuffle ( )

Note: this function is implement so that it can be used by ensmallen's optimizers.

It's not generally meant to be used otherwise.

Shuffle the order of function visitation. (This is equivalent to shuffling the dataset during training.)

Referenced by RNN< OutputLayerType, InitializationRuleType, CustomLayers... >::NumFunctions().

◆ Train() [1/2]

MatType::elem_type Train	(	arma::Cube< typename MatType::elem_type >	predictors,
		arma::Cube< typename MatType::elem_type >	responses,
		OptimizerType &	optimizer,
		CallbackTypes &&...	callbacks
	)

Train the recurrent network on the given input data using the given optimizer.

This will use the existing model parameters as a starting point for the optimization. If this is not what you want, then you should access the parameters vector directly with Parameters() and modify it as desired.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Template Parameters

OptimizerType	Type of optimizer to use to train the model.
CallbackTypes	Types of Callback Functions.

Parameters

predictors	Input training variables.
responses	Outputs results from input training variables.
optimizer	Instantiated optimizer used to train the model.
callbacks	Callback function for ensmallen optimizer `OptimizerType`. See https://www.ensmallen.org/docs.html#callback-documentation.

Returns: The final objective of the trained model (NaN or Inf on error).

Referenced by RNN< OutputLayerType, InitializationRuleType, CustomLayers... >::Network().

◆ Train() [2/2]

MatType::elem_type Train	(	arma::Cube< typename MatType::elem_type >	predictors,
		arma::Cube< typename MatType::elem_type >	responses,
		CallbackTypes &&...	callbacks
	)

Train the recurrent network on the given input data.

By default, the RMSProp optimization algorithm is used, but others can be specified (such as ens::SGD).

This will use the existing model parameters as a starting point for the optimization. If this is not what you want, then you should access the parameters vector directly with Parameters() and modify it as desired.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Template Parameters

OptimizerType Type of optimizer to use to train the model.

Parameters

predictors Input training variables.

Template Parameters

CallbackTypes Types of Callback Functions.

Parameters

responses	Outputs results from input training variables.
callbacks	Callback function for ensmallen optimizer `OptimizerType`. See https://www.ensmallen.org/docs.html#callback-documentation.

Returns: The final objective of the trained model (NaN or Inf on error).

◆ WeightSize()

size_t WeightSize ( )

inline

Definition at line 180 of file rnn.hpp.

The documentation for this class was generated from the following files:

/home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/forward_decls.hpp
/home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/rnn.hpp

Public Member Functions

Detailed Description

template<typenameOutputLayerType=NegativeLogLikelihood,typenameInitializationRuleType=RandomInitialization,typenameMatType=arma::mat> class mlpack::ann::RNN< OutputLayerType, InitializationRuleType, MatType >

Constructor & Destructor Documentation

◆ RNN() [1/3]

◆ RNN() [2/3]

◆ RNN() [3/3]

◆ ~RNN()

Member Function Documentation

◆ Add() [1/2]

◆ Add() [2/2]

◆ BPTTSteps() [1/2]

◆ BPTTSteps() [2/2]

◆ Evaluate() [1/3]

◆ Evaluate() [2/3]

◆ Evaluate() [3/3]

◆ EvaluateWithGradient() [1/2]

◆ EvaluateWithGradient() [2/2]

◆ Gradient()

◆ InputDimensions() [1/2]

◆ InputDimensions() [2/2]

◆ Network() [1/2]

◆ Network() [2/2]

◆ NumFunctions()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ Predict()

◆ Reset()

◆ ResetData()

◆ serialize()

◆ SetNetworkMode()

◆ Shuffle()

◆ Train() [1/2]

◆ Train() [2/2]

◆ WeightSize()

template
<
typename
OutputLayerType
=
NegativeLogLikelihood
,
typename
InitializationRuleType
=
RandomInitialization
,
typename
MatType
=
arma::mat
>

class mlpack::ann::RNN< OutputLayerType, InitializationRuleType, MatType >