SequentialType< InputType, OutputType, Residual > Class Template Reference

Implementation of the Sequential class. More...

Inheritance diagram for SequentialType< InputType, OutputType, Residual >:

Public Member Functions

 SequentialType ()
 Create the Sequential object. More...

 
 SequentialType (const bool ownsLayers)
 Create the Sequential object using the specified parameters. More...

 
 SequentialType (const SequentialType &layer)
 Copy constructor. More...

 
 ~SequentialType ()
 Destroy the Sequential object. More...

 
void Backward (const InputType &, const OutputType &gy, OutputType &g)
 Ordinary feed backward pass of a neural network, using 3rd-order tensors as input, calculating the function f(x) by propagating x backwards through f. More...

 
SequentialTypeClone () const
 Clone the SequentialType object. This handles polymorphism correctly. More...

 
void Forward (const InputType &input, OutputType &output)
 Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

 
void Gradient (const InputType &input, const OutputType &error, OutputType &)
 Calculate the gradient using the output delta and the input activation. More...

 
size_t InputShape () const
 
SequentialTypeoperator= (const SequentialType &layer)
 Copy assignment operator. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the layer. More...

 
- Public Member Functions inherited from MultiLayer< InputType, OutputType >
 MultiLayer ()
 Create an empty MultiLayer that holds no layers of its own. More...

 
 MultiLayer (const MultiLayer &other)
 Copy the given MultiLayer. More...

 
 MultiLayer (MultiLayer &&other)
 Take ownership of the layers of the given MultiLayer. More...

 
virtual ~MultiLayer ()
 Virtual destructor: delete all held layers. More...

 
void Add (Args... args)
 Add a new module to the model. More...

 
void Add (Layer< InputType > *layer)
 Add a new module to the model. More...

 
virtual void Backward (const InputType &input, const InputType &gy, InputType &g)
 Perform a backward pass with the given data. More...

 
virtual void ComputeOutputDimensions ()
 Compute the output dimensions of the MultiLayer using InputDimensions(). More...

 
virtual void CustomInitialize (InputType &W, const size_t elements)
 Initialize the weight matrix of the layer. More...

 
virtual void Forward (const InputType &input, InputType &output)
 Perform a forward pass with the given input data. More...

 
void Forward (const InputType &input, InputType &output, const size_t start, const size_t end)
 Perform a forward pass with the given input data, but only on a subset of the layers in the MultiLayer. More...

 
virtual void Gradient (const InputType &input, const InputType &error, InputType &gradient)
 Compute the gradients of each layer. More...

 
virtual double Loss () const
 Compute the loss that should be added to the objective. More...

 
const std::vector< Layer< InputType > *> & Network () const
 Get the network (series of layers) held by this MultiLayer. More...

 
std::vector< Layer< InputType > *> & Network ()
 Modify the network (series of layers) held by this MultiLayer. More...

 
MultiLayeroperator= (const MultiLayer &other)
 Copy the given MultiLayer. More...

 
MultiLayeroperator= (MultiLayer &&other)
 Take ownership of the given MultiLayer. More...

 
void serialize (Archive &ar, const uint32_t)
 Serialize the MultiLayer. More...

 
virtual void SetWeights (typename InputType ::elem_type *weightsPtr)
 Set the weights of the layer to use the memory given as weightsPtr. More...

 
virtual size_t WeightSize () const
 Return the number of weights in the MultiLayer. More...

 
- Public Member Functions inherited from Layer< InputType >
 Layer ()
 Default constructor. More...

 
 Layer (const Layer &layer)
 Copy constructor. This is not responsible for copying weights! More...

 
 Layer (Layer &&layer)
 Move constructor. This is not responsible for moving weights! More...

 
virtual ~Layer ()
 Default deconstructor. More...

 
virtual void Forward (const InputType &, const InputType &)
 Takes an input and output object, and computes the corresponding loss of the layer. More...

 
const std::vector< size_t > & InputDimensions () const
 Get the input dimensions. More...

 
std::vector< size_t > & InputDimensions ()
 Modify the input dimensions. More...

 
virtual double Loss ()
 Get the layer loss. More...

 
virtual Layeroperator= (const Layer &layer)
 Copy assignment operator. This is not responsible for copying weights! More...

 
virtual Layeroperator= (Layer &&layer)
 Move assignment operator. This is not responsible for moving weights! More...

 
const std::vector< size_t > & OutputDimensions ()
 Get the output dimensions. More...

 
virtual size_t OutputSize () final
 Get the number of elements in the output from this layer. More...

 
virtual const InputType & Parameters () const
 Get the parameters. More...

 
virtual InputType & Parameters ()
 Set the parameters. More...

 
void serialize (Archive &ar, const uint32_t)
 Serialize the layer. More...

 
virtual bool const & Training () const
 Get whether the layer is currently in training mode. More...

 
virtual bool & Training ()
 Modify whether the layer is currently in training mode. More...

 

Additional Inherited Members

- Protected Member Functions inherited from MultiLayer< InputType, OutputType >
void InitializeBackwardPassMemory (const size_t batchSize)
 Initialize memory that will be used by each layer for the backwards pass, assuming that the input will have the given batchSize. More...

 
void InitializeForwardPassMemory (const size_t batchSize)
 Initialize memory that will be used by each layer for the forward pass, assuming that the input will have the given batchSize. More...

 
void InitializeGradientPassMemory (InputType &gradient)
 Initialize memory for the gradient pass. More...

 
- Protected Attributes inherited from MultiLayer< InputType, OutputType >
size_t inSize
 
InputType layerDeltaMatrix
 This matrix stores all of the backwards pass results of each layer when Backward() is called. More...

 
std::vector< InputType > layerDeltas
 These are aliases of layerDeltaMatrix for each layer. More...

 
std::vector< InputType > layerGradients
 Gradient aliases for each layer. More...

 
InputType layerOutputMatrix
 This matrix stores all of the outputs of each layer when Forward() is called. More...

 
std::vector< InputType > layerOutputs
 These are aliases of layerOutputMatrix for each layer. More...

 
std::vector< Layer< InputType > *> network
 The internally-held network. More...

 
size_t totalInputSize
 
size_t totalOutputSize
 
- Protected Attributes inherited from Layer< InputType >
std::vector< size_t > inputDimensions
 Logical input dimensions of each point. More...

 
std::vector< size_t > outputDimensions
 Logical output dimensions of each point. More...

 
bool training
 If true, the layer is in training mode; otherwise, it is in testing mode. More...

 
bool validOutputDimensions
 This is true if ComputeOutputDimensions() has been called, and outputDimensions can be considered to be up-to-date. More...

 

Detailed Description


template<typename InputType = arma::mat, typename OutputType = arma::mat, bool Residual = false>
class mlpack::ann::SequentialType< InputType, OutputType, Residual >

Implementation of the Sequential class.

The sequential class works as a feed-forward fully connected network container which plugs various layers together.

This class can also be used as a container for a residual block. In that case, the sizes of the input and output matrices of this class should be equal. A typedef has been added for use as a Residual<> class.

For more information, refer the following paper.

@article{He15,
author = {Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun},
title = {Deep Residual Learning for Image Recognition},
year = {2015},
url = {https://arxiv.org/abs/1512.03385},
eprint = {1512.03385},
}

Note: If this class is used as the first layer of a network, it should be preceded by IdentityLayer<>.

Note: This class should at least have two layers for a call to its Gradient() function.

Template Parameters
InputTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
ResidualIf true, use the object as a Residual block.

Definition at line 61 of file sequential.hpp.

Constructor & Destructor Documentation

◆ SequentialType() [1/3]

Create the Sequential object.

Referenced by SequentialType< InputType, OutputType >::Clone().

◆ SequentialType() [2/3]

SequentialType ( const bool  ownsLayers)

Create the Sequential object using the specified parameters.

Parameters
ownsLayersIf true, then this module will delete its layers when deallocated.

◆ SequentialType() [3/3]

SequentialType ( const SequentialType< InputType, OutputType, Residual > &  layer)

Copy constructor.

◆ ~SequentialType()

Destroy the Sequential object.

Member Function Documentation

◆ Backward()

void Backward ( const InputType &  ,
const OutputType &  gy,
OutputType &  g 
)

Ordinary feed backward pass of a neural network, using 3rd-order tensors as input, calculating the function f(x) by propagating x backwards through f.

Using the results from the feed forward pass.

Parameters
*(input) The propagated input activation.
gyThe backpropagated error.
gThe calculated gradient.

Referenced by SequentialType< InputType, OutputType >::Clone().

◆ Clone()

SequentialType* Clone ( ) const
inlinevirtual

Clone the SequentialType object. This handles polymorphism correctly.

Reimplemented from MultiLayer< InputType, OutputType >.

Definition at line 87 of file sequential.hpp.

◆ Forward()

void Forward ( const InputType &  input,
OutputType &  output 
)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.

Referenced by SequentialType< InputType, OutputType >::Clone().

◆ Gradient()

void Gradient ( const InputType &  input,
const OutputType &  error,
OutputType &   
)

Calculate the gradient using the output delta and the input activation.

Parameters
inputThe input parameter used for calculating the gradient.
errorThe calculated error.
gradientThe calculated gradient.

Referenced by SequentialType< InputType, OutputType >::Clone().

◆ InputShape()

size_t InputShape ( ) const

◆ operator=()

SequentialType& operator= ( const SequentialType< InputType, OutputType, Residual > &  layer)

Copy assignment operator.

◆ serialize()

void serialize ( Archive &  ar,
const uint32_t   
)

Serialize the layer.

Referenced by SequentialType< InputType, OutputType >::Clone().


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/not_adapted/sequential.hpp