Highway< InputDataType, OutputDataType, CustomLayers > Class Template Reference

Implementation of the Highway layer. More...

Public Member Functions

 Highway ()
 Create the Highway object. More...

 
 Highway (const size_t inSize, const bool model=true)
 Create the Highway object. More...

 
 ~Highway ()
 Destroy the Highway object. More...

 
template<class LayerType , class... Args>
void Add (Args... args)
 Add a new module to the model. More...

 
void Add (LayerTypes< CustomLayers... > layer)
 Add a new module to the model. More...

 
template
<
typename
eT
>
void Backward (const arma::Mat< eT > &&, arma::Mat< eT > &&gy, arma::Mat< eT > &&g)
 Ordinary feed-backward pass of a neural network, calculating the function f(x) by propagating x backwards through f. More...

 
void DeleteModules ()
 Destroy all the modules added to the Highway object. More...

 
OutputDataType const & Delta () const
 Get the delta. More...

 
OutputDataType & Delta ()
 Modify the delta. More...

 
template
<
typename
eT
>
void Forward (arma::Mat< eT > &&input, arma::Mat< eT > &&output)
 Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

 
template
<
typename
eT
>
void Gradient (arma::Mat< eT > &&input, arma::Mat< eT > &&error, arma::Mat< eT > &&gradient)
 Calculate the gradient using the output delta and the input activation. More...

 
OutputDataType const & Gradient () const
 Get the gradient. More...

 
OutputDataType & Gradient ()
 Modify the gradient. More...

 
InputDataType const & InputParameter () const
 Get the input parameter. More...

 
InputDataType & InputParameter ()
 Modify the input parameter. More...

 
std::vector< LayerTypes< CustomLayers... > > & Model ()
 Return the modules of the model. More...

 
OutputDataType const & OutputParameter () const
 Get the output parameter. More...

 
OutputDataType & OutputParameter ()
 Modify the output parameter. More...

 
OutputDataType const & Parameters () const
 Get the parameters. More...

 
OutputDataType & Parameters ()
 Modify the parameters. More...

 
void Reset ()
 Reset the layer parameter. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const unsigned int)
 Serialize the layer. More...

 

Detailed Description


template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat, typename... CustomLayers>
class mlpack::ann::Highway< InputDataType, OutputDataType, CustomLayers >

Implementation of the Highway layer.

The Highway class can vary its behavior between that of feed-forward fully connected network container and that of a layer which simply passes its inputs through depending on the transform gate. Note that the size of the input and output matrices of this class should be equal.

For more information, refer the following paper.

@article{Srivastava2015,
author = {Rupesh Kumar Srivastava, Klaus Greff, Jurgen Schmidhuber},
title = {Training Very Deep Networks},
journal = {Advances in Neural Information Processing Systems},
year = {2015},
url = {https://arxiv.org/abs/1507.06228},
}
Template Parameters
InputDataTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 60 of file highway.hpp.

Constructor & Destructor Documentation

◆ Highway() [1/2]

Highway ( )

Create the Highway object.

◆ Highway() [2/2]

Highway ( const size_t  inSize,
const bool  model = true 
)

Create the Highway object.

Parameters
inSizeThe number of input units.
modelExpose all the network modules.

◆ ~Highway()

~Highway ( )

Destroy the Highway object.

Member Function Documentation

◆ Add() [1/2]

void Add ( Args...  args)
inline

Add a new module to the model.

Parameters
argsThe layer parameter.

Definition at line 129 of file highway.hpp.

◆ Add() [2/2]

void Add ( LayerTypes< CustomLayers... >  layer)
inline

Add a new module to the model.

Parameters
layerThe Layer to be added to the model.

Definition at line 136 of file highway.hpp.

◆ Backward()

void Backward ( const arma::Mat< eT > &&  ,
arma::Mat< eT > &&  gy,
arma::Mat< eT > &&  g 
)

Ordinary feed-backward pass of a neural network, calculating the function f(x) by propagating x backwards through f.

Using the results from the feed-forward pass.

Parameters
inputThe propagated input activation.
gyThe backpropagated error.
gThe calculated gradient.

◆ DeleteModules()

void DeleteModules ( )

Destroy all the modules added to the Highway object.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const
inline

Get the delta.

Definition at line 165 of file highway.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )
inline

Modify the delta.

Definition at line 167 of file highway.hpp.

◆ Forward()

void Forward ( arma::Mat< eT > &&  input,
arma::Mat< eT > &&  output 
)

Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.

◆ Gradient() [1/3]

void Gradient ( arma::Mat< eT > &&  input,
arma::Mat< eT > &&  error,
arma::Mat< eT > &&  gradient 
)

Calculate the gradient using the output delta and the input activation.

Parameters
inputThe input parameter used for calculating the gradient.
errorThe calculated error.
gradientThe calculated gradient.

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const
inline

Get the gradient.

Definition at line 170 of file highway.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )
inline

Modify the gradient.

Definition at line 172 of file highway.hpp.

References Highway< InputDataType, OutputDataType, CustomLayers >::serialize().

◆ InputParameter() [1/2]

InputDataType const& InputParameter ( ) const
inline

Get the input parameter.

Definition at line 155 of file highway.hpp.

◆ InputParameter() [2/2]

InputDataType& InputParameter ( )
inline

Modify the input parameter.

Definition at line 157 of file highway.hpp.

◆ Model()

std::vector<LayerTypes<CustomLayers...> >& Model ( )
inline

Return the modules of the model.

Definition at line 139 of file highway.hpp.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const
inline

Get the output parameter.

Definition at line 160 of file highway.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )
inline

Modify the output parameter.

Definition at line 162 of file highway.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const
inline

Get the parameters.

Definition at line 150 of file highway.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )
inline

Modify the parameters.

Definition at line 152 of file highway.hpp.

◆ Reset()

void Reset ( )

Reset the layer parameter.

◆ serialize()

void serialize ( Archive &  ar,
const unsigned  int 
)

The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-3.2.1/src/mlpack/methods/ann/layer/highway.hpp