FFN< OutputLayerType, InitializationRuleType, MatType > Class Template Reference

Implementation of a standard feed forward network. More...

Inheritance diagram for FFN< OutputLayerType, InitializationRuleType, MatType >:

Public Member Functions

 FFN (OutputLayerType outputLayer=OutputLayerType(), InitializationRuleType initializeRule=InitializationRuleType())
 Create the FFN object. More...

 
 FFN (const FFN &other)
 Copy constructor. More...

 
 FFN (FFN &&other)
 Move constructor. More...

 
template<typename LayerType , typename... Args>
void Add (Args... args)
 Add a new layer to the model. More...

 
void Add (Layer< MatType > *layer)
 Add a new layer to the model. More...

 
MatType::elem_type Backward (const MatType &inputs, const MatType &targets, MatType &gradients)
 Perform a manual backward pass of the data. More...

 
MatType::elem_type Evaluate (const MatType &predictors, const MatType &responses)
 Evaluate the feedforward network with the given predictors and responses. More...

 
MatType::elem_type Evaluate (const MatType &parameters)
 Note: this function is implemented so that it can be used by ensmallen's optimizers. More...

 
MatType::elem_type Evaluate (const MatType &parameters, const size_t begin, const size_t batchSize)
 Note: this function is implemented so that it can be used by ensmallen's optimizers. More...

 
MatType::elem_type EvaluateWithGradient (const MatType &parameters, MatType &gradient)
 Note: this function is implemented so that it can be used by ensmallen's optimizers. More...

 
MatType::elem_type EvaluateWithGradient (const MatType &parameters, const size_t begin, MatType &gradient, const size_t batchSize)
 Note: this function is implemented so that it can be used by ensmallen's optimizers. More...

 
void Forward (const MatType &inputs, MatType &results)
 Perform a manual forward pass of the data. More...

 
void Forward (const MatType &inputs, MatType &results, const size_t begin, const size_t end)
 Perform a manual partial forward pass of the data. More...

 
void Gradient (const MatType &parameters, const size_t begin, MatType &gradient, const size_t batchSize)
 Note: this function is implemented so that it can be used by ensmallen's optimizers. More...

 
std::vector< size_t > & InputDimensions ()
 Set the logical dimensions of the input. More...

 
const std::vector< size_t > & InputDimensions () const
 Get the logical dimensions of the input. More...

 
const std::vector< Layer< MatType > * > & Network () const
 Get the layers of the network. More...

 
std::vector< Layer< MatType > * > & Network ()
 Modify the network model. More...

 
size_t NumFunctions () const
 Note: this function is implemented so that it can be used by ensmallen's optimizers. More...

 
FFNoperator= (const FFN &other)
 Copy operator. More...

 
FFNoperator= (FFN &&other)
 Move assignment operator. More...

 
const MatType & Parameters () const
 Return the current set of weights. More...

 
MatType & Parameters ()
 Modify the current set of weights. More...

 
void Predict (MatType predictors, MatType &results, const size_t batchSize=128)
 Predict the responses to a given set of predictors. More...

 
void Reset (const size_t inputDimensionality=0)
 Reset the stored data of the network entirely. More...

 
void ResetData (MatType predictors, MatType responses)
 Prepare the network for training on the given data. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the model. More...

 
void SetNetworkMode (const bool training)
 Set all the layers in the network to training mode, if training is true, or set all the layers in the network to testing mode, if training is false. More...

 
void Shuffle ()
 Note: this function is implemented so that it can be used by ensmallen's optimizers. More...

 
template<typename OptimizerType , typename... CallbackTypes>
MatType::elem_type Train (MatType predictors, MatType responses, OptimizerType &optimizer, CallbackTypes &&... callbacks)
 Train the feedforward network on the given input data using the given optimizer. More...

 
template<typename OptimizerType = ens::RMSProp, typename... CallbackTypes>
MatType::elem_type Train (MatType predictors, MatType responses, CallbackTypes &&... callbacks)
 Train the feedforward network on the given input data. More...

 
size_t WeightSize ()
 

Detailed Description


template
<
typename
OutputLayerType
=
NegativeLogLikelihood
,
typename
InitializationRuleType
=
RandomInitialization
,
typename
MatType
=
arma::mat
>

class mlpack::ann::FFN< OutputLayerType, InitializationRuleType, MatType >

Implementation of a standard feed forward network.

Any layer that inherits from the base Layer class can be added to this model. For recursive neural networks, see the RNN class.

In general, a network can be created by using the Add() method to add layers to the network. Then, training can be performed with Train(), and data points can be passed through the trained network with Predict().

Although the actual types passed as input will be matrix objects with one data point per column, each data point can be a tensor of arbitrary shape. If data points are not 1-dimensional vectors, then set the shape of the input with InputDimensions() before calling Train().

More granular functionality is available with Forward(), Backward(), and Evaluate(), or even by accessing the individual layers directly with Network()`.

Template Parameters
OutputLayerTypeThe output layer type used to evaluate the network.
InitializationRuleTypeRule used to initialize the weight matrix.
MatTypeType of matrix to be given as input to the network.
MatTypeType of matrix to be produced as output from the last layer.

Definition at line 58 of file ffn.hpp.

Constructor & Destructor Documentation

◆ FFN() [1/3]

FFN ( OutputLayerType  outputLayer = OutputLayerType(),
InitializationRuleType  initializeRule = InitializationRuleType() 
)

Create the FFN object.

Optionally, specify which initialize rule and performance function should be used.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Parameters
outputLayerOutput layer used to evaluate the network.
initializeRuleOptional instantiated InitializationRule object for initializing the network parameter.

◆ FFN() [2/3]

FFN ( const FFN< OutputLayerType, InitializationRuleType, MatType > &  other)

Copy constructor.

◆ FFN() [3/3]

FFN ( FFN< OutputLayerType, InitializationRuleType, MatType > &&  other)

Move constructor.

Member Function Documentation

◆ Add() [1/2]

void Add ( Args...  args)
inline

Add a new layer to the model.

Parameters
argsThe layer parameter.

Definition at line 92 of file ffn.hpp.

◆ Add() [2/2]

void Add ( Layer< MatType > *  layer)
inline

Add a new layer to the model.

Note that any trainable weights of this layer will be reset! (Any constant parameters are kept.)

Parameters
layerThe Layer to be added to the model.

Definition at line 104 of file ffn.hpp.

◆ Backward()

MatType::elem_type Backward ( const MatType &  inputs,
const MatType &  targets,
MatType &  gradients 
)

Perform a manual backward pass of the data.

Forward() and Backward() should be used as a pair, and they are designed mainly for advanced users. You should try to use Predict() and Train() instead, if you can.

Parameters
inputsInputs of current pass.
targetsThe training target.
gradientsComputed gradients.
Returns
Training error of the current pass.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::Parameters().

◆ Evaluate() [1/3]

MatType::elem_type Evaluate ( const MatType &  predictors,
const MatType &  responses 
)

Evaluate the feedforward network with the given predictors and responses.

This functions is usually used to monitor progress while training.

Parameters
predictorsInput variables.
responsesTarget outputs for input variables.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::Parameters().

◆ Evaluate() [2/3]

MatType::elem_type Evaluate ( const MatType &  parameters)

Note: this function is implemented so that it can be used by ensmallen's optimizers.

It's not generally meant to be used otherwise.

Evaluate the feedforward network with the given parameters.

Parameters
parametersMatrix model parameters.

◆ Evaluate() [3/3]

MatType::elem_type Evaluate ( const MatType &  parameters,
const size_t  begin,
const size_t  batchSize 
)

Note: this function is implemented so that it can be used by ensmallen's optimizers.

It's not generally meant to be used otherwise.

Evaluate the feedforward network with the given parameters, but using only a number of data points. This is useful for optimizers such as SGD, which require a separable objective function.

Note that the network may return different results depending on the mode it is in (see SetNetworkMode()).

Parameters
parametersMatrix model parameters.
beginIndex of the starting point to use for objective function evaluation.
batchSizeNumber of points to be passed at a time to use for objective function evaluation.

◆ EvaluateWithGradient() [1/2]

MatType::elem_type EvaluateWithGradient ( const MatType &  parameters,
MatType &  gradient 
)

Note: this function is implemented so that it can be used by ensmallen's optimizers.

It's not generally meant to be used otherwise.

Evaluate the feedforward network with the given parameters. This function is usually called by the optimizer to train the model. This just calls the overload of EvaluateWithGradient() with batchSize = 1.

Parameters
parametersMatrix model parameters.
gradientMatrix to output gradient into.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::Parameters().

◆ EvaluateWithGradient() [2/2]

MatType::elem_type EvaluateWithGradient ( const MatType &  parameters,
const size_t  begin,
MatType &  gradient,
const size_t  batchSize 
)

Note: this function is implemented so that it can be used by ensmallen's optimizers.

It's not generally meant to be used otherwise.

Evaluate the feedforward network with the given parameters, but using only a number of data points. This is useful for optimizers such as SGD, which require a separable objective function.

Parameters
parametersMatrix model parameters.
beginIndex of the starting point to use for objective function evaluation.
gradientMatrix to output gradient into.
batchSizeNumber of points to be passed at a time to use for objective function evaluation.

◆ Forward() [1/2]

void Forward ( const MatType &  inputs,
MatType &  results 
)

Perform a manual forward pass of the data.

Forward() and Backward() should be used as a pair, and they are designed mainly for advanced users. You should try to use Predict() and Train(), if you can.

Parameters
inputsThe input data.
resultsThe predicted results.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::Parameters().

◆ Forward() [2/2]

void Forward ( const MatType &  inputs,
MatType &  results,
const size_t  begin,
const size_t  end 
)

Perform a manual partial forward pass of the data.

This function is meant for the cases when users require a forward pass only through certain layers and not the entire network. Forward() and Backward() should be used as a pair, and they are designed mainly for advanced users. You should try to use Predict() and Train(), if you can.

Parameters
inputsThe input data for the specified first layer.
resultsThe predicted results from the specified last layer.
beginThe index of the first layer.
endThe index of the last layer.

◆ Gradient()

void Gradient ( const MatType &  parameters,
const size_t  begin,
MatType &  gradient,
const size_t  batchSize 
)

Note: this function is implemented so that it can be used by ensmallen's optimizers.

It's not generally meant to be used otherwise.

Evaluate the gradient of the feedforward network with the given parameters, and with respect to only a number of points in the dataset. This is useful for optimizers such as SGD, which require a separable objective function.

Parameters
parametersMatrix of the model parameters to be optimized.
beginIndex of the starting point to use for objective function gradient evaluation.
gradientMatrix to output gradient into.
batchSizeNumber of points to be processed as a batch for objective function gradient evaluation.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::Parameters().

◆ InputDimensions() [1/2]

std::vector<size_t>& InputDimensions ( )
inline

Set the logical dimensions of the input.

Train() and Predict() expect data to be passed such that one point corresponds to one column, but this data is allowed to be an arbitrary higher-order tensor.

So, if the input is meant to be 28x28x3 images, then the input data to Train() or Predict() should have 28*28*3 = 2352 rows, and InputDimensions() should be set to { 28, 28, 3 }. Then, the layers of the network will interpret each input point as a 3-dimensional image instead of a 1-dimensional vector.

If InputDimensions() is left unset before training, the data will be assumed to be a 1-dimensional vector.

Definition at line 219 of file ffn.hpp.

◆ InputDimensions() [2/2]

const std::vector<size_t>& InputDimensions ( ) const
inline

Get the logical dimensions of the input.

Definition at line 227 of file ffn.hpp.

◆ Network() [1/2]

const std::vector<Layer<MatType>*>& Network ( ) const
inline

Get the layers of the network.

Definition at line 111 of file ffn.hpp.

◆ Network() [2/2]

std::vector<Layer<MatType>*>& Network ( )
inline

Modify the network model.

Be careful! If you change the structure of the network or parameters for layers, its state may become invalid, and the next time it is used for any operation the parameters will be reset.

Don't add any layers like this; use Add() instead.

Definition at line 123 of file ffn.hpp.

◆ NumFunctions()

size_t NumFunctions ( ) const
inline

Note: this function is implemented so that it can be used by ensmallen's optimizers.

It's not generally meant to be used otherwise.

Return the number of separable functions (the number of predictor points).

Definition at line 413 of file ffn.hpp.

◆ operator=() [1/2]

FFN& operator= ( const FFN< OutputLayerType, InitializationRuleType, MatType > &  other)

Copy operator.

◆ operator=() [2/2]

FFN& operator= ( FFN< OutputLayerType, InitializationRuleType, MatType > &&  other)

Move assignment operator.

◆ Parameters() [1/2]

const MatType& Parameters ( ) const
inline

Return the current set of weights.

These are linearized: this contains the weights of every layer.

Definition at line 231 of file ffn.hpp.

◆ Parameters() [2/2]

MatType& Parameters ( )
inline

Modify the current set of weights.

These are linearized: this contains the weights of every layer. Be careful! If you change the shape of parameters to something incorrect, it may be re-initialized the next time a forward pass is done.

Definition at line 236 of file ffn.hpp.

◆ Predict()

void Predict ( MatType  predictors,
MatType &  results,
const size_t  batchSize = 128 
)

Predict the responses to a given set of predictors.

The responses will be the output of the output layer when predictors is passed through the whole network (OutputLayerType).

Parameters
predictorsInput predictors.
resultsMatrix to put output predictions of responses into.
batchSizeBatch size to use for prediction.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::Network().

◆ Reset()

void Reset ( const size_t  inputDimensionality = 0)

Reset the stored data of the network entirely.

This resets all weights of each layer using InitializationRuleType, and prepares the network to accept a (flat 1-d) input size of inputDimensionality (if passed), or whatever input size has been set with InputDimensions().

This also resets the mode of the network to prediction mode (not training mode). See SetNetworkMode() for more information.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::Parameters().

◆ ResetData()

void ResetData ( MatType  predictors,
MatType  responses 
)

Prepare the network for training on the given data.

This function won't actually trigger the training process, and is generally only useful internally.

Parameters
predictorsInput data variables.
responsesOutputs results from input data variables.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::NumFunctions().

◆ serialize()

void serialize ( Archive &  ar,
const uint32_t   
)

◆ SetNetworkMode()

void SetNetworkMode ( const bool  training)

Set all the layers in the network to training mode, if training is true, or set all the layers in the network to testing mode, if training is false.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::Parameters().

◆ Shuffle()

void Shuffle ( )

Note: this function is implemented so that it can be used by ensmallen's optimizers.

It's not generally meant to be used otherwise.

Shuffle the order of function visitation. (This is equivalent to shuffling the dataset during training.)

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::NumFunctions().

◆ Train() [1/2]

MatType::elem_type Train ( MatType  predictors,
MatType  responses,
OptimizerType &  optimizer,
CallbackTypes &&...  callbacks 
)

Train the feedforward network on the given input data using the given optimizer.

If no parameters have ever been set (e.g. if Parameters() is an empty matrix), or if the parameters' size does not match the number of weights needed for the current input size (as given by predictors and optionally set further by InputDimensions()), then the network will be initialized using InitializeRuleType.

If parameters are the right size for the given predictors and InputDimensions(), then the existing parameters will be used as a starting point. (If you want to reinitialize, first call Reset().)

Template Parameters
OptimizerTypeType of optimizer to use to train the model.
CallbackTypesTypes of Callback Functions.
Parameters
predictorsInput training variables.
responsesOutputs results from input training variables.
optimizerInstantiated optimizer used to train the model.
callbacksCallback function for ensmallen optimizer OptimizerType. See https://www.ensmallen.org/docs.html#callback-documentation.
Returns
The final objective of the trained model (NaN or Inf on error).

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers... >::Network().

◆ Train() [2/2]

MatType::elem_type Train ( MatType  predictors,
MatType  responses,
CallbackTypes &&...  callbacks 
)

Train the feedforward network on the given input data.

By default, the RMSProp optimization algorithm is used, but others can be specified (such as ens::SGD).

If no parameters have ever been set (e.g. if Parameters() is an empty matrix), or if the parameters' size does not match the number of weights needed for the current input size (as given by predictors and optionally set further by InputDimensions()), then the network will be initialized using InitializeRuleType.

If parameters are the right size for the given predictors and InputDimensions(), then the existing parameters will be used as a starting point. (If you want to reinitialize, first call Reset().)

Template Parameters
OptimizerTypeType of optimizer to use to train the model.
Parameters
predictorsInput training variables.
Template Parameters
CallbackTypesTypes of Callback Functions.
Parameters
responsesOutputs results from input training variables.
callbacksCallback function for ensmallen optimizer OptimizerType. See https://www.ensmallen.org/docs.html#callback-documentation.
Returns
The final objective of the trained model (NaN or Inf on error).

◆ WeightSize()


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/ffn.hpp