A "multi-layer" is a layer that is a wrapper around other layers. More...

Inheritance diagram for MultiLayer< MatType >:

Public Member Functions

 MultiLayer ()
 Create an empty MultiLayer that holds no layers of its own. More...

 
 MultiLayer (const MultiLayer &other)
 Copy the given MultiLayer. More...

 
 MultiLayer (MultiLayer &&other)
 Take ownership of the layers of the given MultiLayer. More...

 
virtual ~MultiLayer ()
 Virtual destructor: delete all held layers. More...

 
template<typename LayerType , typename... Args>
void Add (Args... args)
 Add a new module to the model. More...

 
void Add (Layer< MatType > *layer)
 Add a new module to the model. More...

 
virtual void Backward (const MatType &input, const MatType &gy, MatType &g)
 Perform a backward pass with the given data. More...

 
virtual MultiLayerClone () const
 Create a copy of the MultiLayer (this is safe for polymorphic use). More...

 
virtual void ComputeOutputDimensions ()
 Compute the output dimensions of the MultiLayer using InputDimensions(). More...

 
virtual void CustomInitialize (MatType &W, const size_t elements)
 Initialize the weight matrix of the layer. More...

 
virtual void Forward (const MatType &input, MatType &output)
 Perform a forward pass with the given input data. More...

 
void Forward (const MatType &input, MatType &output, const size_t start, const size_t end)
 Perform a forward pass with the given input data, but only on a subset of the layers in the MultiLayer. More...

 
virtual void Gradient (const MatType &input, const MatType &error, MatType &gradient)
 Compute the gradients of each layer. More...

 
virtual double Loss () const
 Compute the loss that should be added to the objective. More...

 
const std::vector< Layer< MatType > * > & Network () const
 Get the network (series of layers) held by this MultiLayer. More...

 
std::vector< Layer< MatType > * > & Network ()
 Modify the network (series of layers) held by this MultiLayer. More...

 
MultiLayeroperator= (const MultiLayer &other)
 Copy the given MultiLayer. More...

 
MultiLayeroperator= (MultiLayer &&other)
 Take ownership of the given MultiLayer. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the MultiLayer. More...

 
virtual void SetWeights (typename MatType::elem_type *weightsPtr)
 Set the weights of the layer to use the memory given as weightsPtr. More...

 
virtual size_t WeightSize () const
 Return the number of weights in the MultiLayer. More...

 
- Public Member Functions inherited from Layer< MatType >
 Layer ()
 Default constructor. More...

 
 Layer (const Layer &layer)
 Copy constructor. This is not responsible for copying weights! More...

 
 Layer (Layer &&layer)
 Move constructor. This is not responsible for moving weights! More...

 
virtual ~Layer ()
 Default deconstructor. More...

 
virtual void Forward (const MatType &, const MatType &)
 Takes an input and output object, and computes the corresponding loss of the layer. More...

 
const std::vector< size_t > & InputDimensions () const
 Get the input dimensions. More...

 
std::vector< size_t > & InputDimensions ()
 Modify the input dimensions. More...

 
virtual double Loss ()
 Get the layer loss. More...

 
virtual Layeroperator= (const Layer &layer)
 Copy assignment operator. This is not responsible for copying weights! More...

 
virtual Layeroperator= (Layer &&layer)
 Move assignment operator. This is not responsible for moving weights! More...

 
const std::vector< size_t > & OutputDimensions ()
 Get the output dimensions. More...

 
virtual size_t OutputSize () final
 Get the number of elements in the output from this layer. More...

 
virtual const MatType & Parameters () const
 Get the parameters. More...

 
virtual MatType & Parameters ()
 Set the parameters. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the layer. More...

 
virtual bool const & Training () const
 Get whether the layer is currently in training mode. More...

 
virtual bool & Training ()
 Modify whether the layer is currently in training mode. More...

 

Protected Member Functions

void InitializeBackwardPassMemory (const size_t batchSize)
 Initialize memory that will be used by each layer for the backwards pass, assuming that the input will have the given batchSize. More...

 
void InitializeForwardPassMemory (const size_t batchSize)
 Initialize memory that will be used by each layer for the forward pass, assuming that the input will have the given batchSize. More...

 
void InitializeGradientPassMemory (MatType &gradient)
 Initialize memory for the gradient pass. More...

 

Protected Attributes

size_t inSize
 
MatType layerDeltaMatrix
 This matrix stores all of the backwards pass results of each layer when Backward() is called. More...

 
std::vector< MatType > layerDeltas
 These are aliases of layerDeltaMatrix for each layer. More...

 
std::vector< MatType > layerGradients
 Gradient aliases for each layer. More...

 
MatType layerOutputMatrix
 This matrix stores all of the outputs of each layer when Forward() is called. More...

 
std::vector< MatType > layerOutputs
 These are aliases of layerOutputMatrix for each layer. More...

 
std::vector< Layer< MatType > * > network
 The internally-held network. More...

 
size_t totalInputSize
 
size_t totalOutputSize
 
- Protected Attributes inherited from Layer< MatType >
std::vector< size_t > inputDimensions
 Logical input dimensions of each point. More...

 
std::vector< size_t > outputDimensions
 Logical output dimensions of each point. More...

 
bool training
 If true, the layer is in training mode; otherwise, it is in testing mode. More...

 
bool validOutputDimensions
 This is true if ComputeOutputDimensions() has been called, and outputDimensions can be considered to be up-to-date. More...

 

Detailed Description


template
<
typename
MatType
>

class mlpack::ann::MultiLayer< MatType >

A "multi-layer" is a layer that is a wrapper around other layers.

It passes the input through all of its child layers sequentially, returning the output from the last layer.

It's likely not very useful to use this layer directly; instead, this layer is meant as a base class for use by other layers that must store and use multiple layers.

Template Parameters
MatTypeMatrix representation to accept as input and use for computation.

Definition at line 34 of file multi_layer.hpp.

Constructor & Destructor Documentation

◆ MultiLayer() [1/3]

Create an empty MultiLayer that holds no layers of its own.

Be sure to add layers with Add() before using!

Referenced by MultiLayer< InputType, OutputType >::Clone().

◆ MultiLayer() [2/3]

MultiLayer ( const MultiLayer< MatType > &  other)

Copy the given MultiLayer.

◆ MultiLayer() [3/3]

MultiLayer ( MultiLayer< MatType > &&  other)

Take ownership of the layers of the given MultiLayer.

◆ ~MultiLayer()

virtual ~MultiLayer ( )
inlinevirtual

Virtual destructor: delete all held layers.

Definition at line 53 of file multi_layer.hpp.

Member Function Documentation

◆ Add() [1/2]

void Add ( Args...  args)
inline

Add a new module to the model.

Parameters
argsThe layer parameter.

Definition at line 167 of file multi_layer.hpp.

Referenced by DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType >::DuelingDQN().

◆ Add() [2/2]

void Add ( Layer< MatType > *  layer)
inline

Add a new module to the model.

Parameters
layerThe Layer to be added to the model.

Definition at line 180 of file multi_layer.hpp.

◆ Backward()

virtual void Backward ( const MatType &  input,
const MatType &  gy,
MatType &  g 
)
virtual

Perform a backward pass with the given data.

gy is expected to be the propagated error from the subsequent layer (or output), input is expected to be the output from this layer when Forward() was called, and g will store the propagated error from this layer (to be passed to the previous layer as gy).

It is expected that g has the correct size already (e.g., number of rows equal to OutputSize() of the previous layer, and number of columns equal to input.n_cols).

This function is expected to be called for the same input data as Forward() was just called for.

Parameters
inputOutput of Forward().
gyPropagated error from next layer.
gMatrix to store propagated error in for previous layer.

Reimplemented from Layer< MatType >.

Reimplemented in ConcatType< MatType >, AddMergeType< MatType >, and AddMergeType< InputType, OutputType >.

Referenced by MultiLayer< InputType, OutputType >::Clone().

◆ Clone()

◆ ComputeOutputDimensions()

virtual void ComputeOutputDimensions ( )
virtual

Compute the output dimensions of the MultiLayer using InputDimensions().

This computes the dimensions of each layer held by the MultiLayer, and the output dimensions are set to the output dimensions of the last layer.

Reimplemented from Layer< MatType >.

Reimplemented in ConcatType< MatType >, AddMergeType< MatType >, and AddMergeType< InputType, OutputType >.

Referenced by MultiLayer< InputType, OutputType >::Clone().

◆ CustomInitialize()

virtual void CustomInitialize ( MatType &  W,
const size_t  elements 
)
virtual

Initialize the weight matrix of the layer.

Parameters
WWeight matrix to initialize.
elementsNumber of elements.

Reimplemented from Layer< MatType >.

Referenced by MultiLayer< InputType, OutputType >::Clone().

◆ Forward() [1/2]

virtual void Forward ( const MatType &  input,
MatType &  output 
)
virtual

Perform a forward pass with the given input data.

output is expected to have the correct size (e.g. number of rows equal to OutputSize() of the last held layer; number of columns equal to input.n_cols).

Parameters
inputInput data to pass through the MultiLayer.
outputMatrix to store output in.

Reimplemented from Layer< MatType >.

Reimplemented in ConcatType< MatType >, AddMergeType< MatType >, and AddMergeType< InputType, OutputType >.

Referenced by MultiLayer< InputType, OutputType >::Clone().

◆ Forward() [2/2]

void Forward ( const MatType &  input,
MatType &  output,
const size_t  start,
const size_t  end 
)

Perform a forward pass with the given input data, but only on a subset of the layers in the MultiLayer.

output is expected to have the correct size (e.g. number of rows equal to OutputSize() of the last layer to be computed; number of columns equal to input.n_cols).

Parameters
inputInput data to pass through the MultiLayer.
outputMatrix to store output in.
startIndex of first layer to pass data through.
endIndex of last layer to pass data through.

◆ Gradient()

virtual void Gradient ( const MatType &  input,
const MatType &  error,
MatType &  gradient 
)
virtual

Compute the gradients of each layer.

This function is expected to be called for the same input data as Forward() and Backward() were just called for. That is, input here should be the same data as Forward() was called with.

gradient is expected to have the correct size already (e.g., number of rows equal to 1, and number of columns equal to WeightSize()).

Parameters
inputOriginal input data provided to Forward().
errorError as computed by Backward().
gradientMatrix to store the gradients in.

Reimplemented from Layer< MatType >.

Reimplemented in ConcatType< MatType >, AddMergeType< MatType >, and AddMergeType< InputType, OutputType >.

Referenced by MultiLayer< InputType, OutputType >::Clone().

◆ InitializeBackwardPassMemory()

void InitializeBackwardPassMemory ( const size_t  batchSize)
protected

Initialize memory that will be used by each layer for the backwards pass, assuming that the input will have the given batchSize.

When Backward() is called, each internally-held layer will output the results of its backwards pass into the memory allocated by this function (this is the internal member layerDeltaMatrix and its aliases layerDeltas).

Referenced by MultiLayer< InputType, OutputType >::Network().

◆ InitializeForwardPassMemory()

void InitializeForwardPassMemory ( const size_t  batchSize)
protected

Initialize memory that will be used by each layer for the forward pass, assuming that the input will have the given batchSize.

When Forward() is called, each internally-held layer will output its results into the memory allocated by this function (this is the internal member layerOutputMatrix and its aliases layerOutputs).

Referenced by MultiLayer< InputType, OutputType >::Network().

◆ InitializeGradientPassMemory()

void InitializeGradientPassMemory ( MatType &  gradient)
protected

Initialize memory for the gradient pass.

This sets the internal aliases layerGradients appropriately using the memory from the given gradient, such that each layer will output its gradient (via its Gradient() method) into the appropriate member of layerGradients.

Referenced by MultiLayer< InputType, OutputType >::Network().

◆ Loss()

virtual double Loss ( ) const
virtual

Compute the loss that should be added to the objective.

Referenced by MultiLayer< InputType, OutputType >::Clone().

◆ Network() [1/2]

const std::vector<Layer<MatType>*>& Network ( ) const
inline

Get the network (series of layers) held by this MultiLayer.

Definition at line 189 of file multi_layer.hpp.

◆ Network() [2/2]

std::vector<Layer<MatType>*>& Network ( )
inline

Modify the network (series of layers) held by this MultiLayer.

Be careful!

Definition at line 195 of file multi_layer.hpp.

◆ operator=() [1/2]

MultiLayer& operator= ( const MultiLayer< MatType > &  other)

Copy the given MultiLayer.

◆ operator=() [2/2]

MultiLayer& operator= ( MultiLayer< MatType > &&  other)

Take ownership of the given MultiLayer.

◆ serialize()

void serialize ( Archive &  ar,
const uint32_t   
)

◆ SetWeights()

virtual void SetWeights ( typename MatType::elem_type *  weightsPtr)
virtual

Set the weights of the layer to use the memory given as weightsPtr.

Reimplemented from Layer< MatType >.

Referenced by MultiLayer< InputType, OutputType >::Clone().

◆ WeightSize()

virtual size_t WeightSize ( ) const
virtual

Return the number of weights in the MultiLayer.

This is the sum of the number of weights in each layer.

Reimplemented from Layer< MatType >.

Reimplemented in HighwayType< InputType, OutputType >, and MultiplyMergeType< InputType, OutputType >.

Referenced by MultiLayer< InputType, OutputType >::Clone().

Member Data Documentation

◆ inSize

size_t inSize
protected

Definition at line 232 of file multi_layer.hpp.

◆ layerDeltaMatrix

MatType layerDeltaMatrix
protected

This matrix stores all of the backwards pass results of each layer when Backward() is called.

See InitializeBackwardPassMemory().

Definition at line 246 of file multi_layer.hpp.

◆ layerDeltas

std::vector<MatType> layerDeltas
protected

These are aliases of layerDeltaMatrix for each layer.

Definition at line 248 of file multi_layer.hpp.

Referenced by MultiLayer< InputType, OutputType >::Add().

◆ layerGradients

std::vector<MatType> layerGradients
protected

Gradient aliases for each layer.

Note that this is only valid in the context of Gradient()! We have it as a class member to avoid reallocating the MatTypes each call to Gradient().

Definition at line 253 of file multi_layer.hpp.

Referenced by MultiLayer< InputType, OutputType >::Add().

◆ layerOutputMatrix

MatType layerOutputMatrix
protected

This matrix stores all of the outputs of each layer when Forward() is called.

See InitializeForwardPassMemory().

Definition at line 240 of file multi_layer.hpp.

◆ layerOutputs

std::vector<MatType> layerOutputs
protected

These are aliases of layerOutputMatrix for each layer.

Definition at line 242 of file multi_layer.hpp.

Referenced by MultiLayer< InputType, OutputType >::Add().

◆ network

◆ totalInputSize

size_t totalInputSize
protected

Definition at line 234 of file multi_layer.hpp.

Referenced by ConcatType< MatType >::ComputeOutputDimensions().

◆ totalOutputSize

size_t totalOutputSize
protected

Definition at line 236 of file multi_layer.hpp.

Referenced by ConcatType< MatType >::ComputeOutputDimensions().


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/multi_layer.hpp