DropConnect< InputDataType, OutputDataType > Class Template Reference

The DropConnect layer is a regularizer that randomly with probability ratio sets the connection values to zero and scales the remaining elements by factor 1 /(1 - ratio). More...

Public Member Functions

 DropConnect ()
 Create the DropConnect object. More...

 
 DropConnect (const size_t inSize, const size_t outSize, const double ratio=0.5)
 Creates the DropConnect Layer as a Linear Object that takes input size, output size and ratio as parameter. More...

 
 ~DropConnect ()
 
template
<
typename
eT
>
void Backward (arma::Mat< eT > &&input, arma::Mat< eT > &&gy, arma::Mat< eT > &&g)
 Ordinary feed backward pass of the DropConnect layer. More...

 
OutputDataType const & Delta () const
 Get the delta. More...

 
OutputDataType & Delta ()
 Modify the delta. More...

 
bool Deterministic () const
 The value of the deterministic parameter. More...

 
bool & Deterministic ()
 Modify the value of the deterministic parameter. More...

 
template
<
typename
eT
>
void Forward (arma::Mat< eT > &&input, arma::Mat< eT > &&output)
 Ordinary feed forward pass of the DropConnect layer. More...

 
template
<
typename
eT
>
void Gradient (arma::Mat< eT > &&input, arma::Mat< eT > &&error, arma::Mat< eT > &&)
 Calculate the gradient using the output delta and the input activation. More...

 
OutputDataType const & Gradient () const
 Get the gradient. More...

 
OutputDataType & Gradient ()
 Modify the gradient. More...

 
std::vector< LayerTypes<> > & Model ()
 Get the model modules. More...

 
OutputDataType const & OutputParameter () const
 Get the output parameter. More...

 
OutputDataType & OutputParameter ()
 Modify the output parameter. More...

 
OutputDataType const & Parameters () const
 Get the parameters. More...

 
OutputDataType & Parameters ()
 Modify the parameters. More...

 
double Ratio () const
 The probability of setting a value to zero. More...

 
void Ratio (const double r)
 Modify the probability of setting a value to zero. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const unsigned int)
 Serialize the layer. More...

 

Detailed Description


template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::DropConnect< InputDataType, OutputDataType >

The DropConnect layer is a regularizer that randomly with probability ratio sets the connection values to zero and scales the remaining elements by factor 1 /(1 - ratio).

The output is scaled with 1 / (1 - p) when deterministic is false. In the deterministic mode(during testing), the layer just computes the output. The output is computed according to the input layer. If no input layer is given, it will take a linear layer as default.

Note: During training you should set deterministic to false and during testing you should set deterministic to true.

For more information, see the following.

@inproceedings{WanICML2013,
title={Regularization of Neural Networks using DropConnect},
booktitle = {Proceedings of the 30th International Conference on Machine
Learning(ICML - 13)},
author = {Li Wan and Matthew Zeiler and Sixin Zhang and Yann L. Cun and
Rob Fergus},
year = {2013}
}
Template Parameters
InputDataTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 62 of file dropconnect.hpp.

Constructor & Destructor Documentation

◆ DropConnect() [1/2]

Create the DropConnect object.

◆ DropConnect() [2/2]

DropConnect ( const size_t  inSize,
const size_t  outSize,
const double  ratio = 0.5 
)

Creates the DropConnect Layer as a Linear Object that takes input size, output size and ratio as parameter.

Parameters
inSizeThe number of input units.
outSizeThe number of output units.
ratioThe probability of setting a value to zero.

◆ ~DropConnect()

Member Function Documentation

◆ Backward()

void Backward ( arma::Mat< eT > &&  input,
arma::Mat< eT > &&  gy,
arma::Mat< eT > &&  g 
)

Ordinary feed backward pass of the DropConnect layer.

Parameters
inputThe propagated input activation.
gyThe backpropagated error.
gThe calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const
inline

Get the delta.

Definition at line 129 of file dropconnect.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )
inline

Modify the delta.

Definition at line 131 of file dropconnect.hpp.

◆ Deterministic() [1/2]

bool Deterministic ( ) const
inline

The value of the deterministic parameter.

Definition at line 139 of file dropconnect.hpp.

◆ Deterministic() [2/2]

bool& Deterministic ( )
inline

Modify the value of the deterministic parameter.

Definition at line 142 of file dropconnect.hpp.

◆ Forward()

void Forward ( arma::Mat< eT > &&  input,
arma::Mat< eT > &&  output 
)

Ordinary feed forward pass of the DropConnect layer.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.

◆ Gradient() [1/3]

void Gradient ( arma::Mat< eT > &&  input,
arma::Mat< eT > &&  error,
arma::Mat< eT > &&   
)

Calculate the gradient using the output delta and the input activation.

Parameters
inputThe propagated input.
dThe calculated error.
gThe calculated gradient.

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const
inline

Get the gradient.

Definition at line 134 of file dropconnect.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )
inline

Modify the gradient.

Definition at line 136 of file dropconnect.hpp.

◆ Model()

std::vector<LayerTypes<> >& Model ( )
inline

Get the model modules.

Definition at line 116 of file dropconnect.hpp.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const
inline

Get the output parameter.

Definition at line 124 of file dropconnect.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )
inline

Modify the output parameter.

Definition at line 126 of file dropconnect.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const
inline

Get the parameters.

Definition at line 119 of file dropconnect.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )
inline

Modify the parameters.

Definition at line 121 of file dropconnect.hpp.

◆ Ratio() [1/2]

double Ratio ( ) const
inline

The probability of setting a value to zero.

Definition at line 145 of file dropconnect.hpp.

◆ Ratio() [2/2]

void Ratio ( const double  r)
inline

Modify the probability of setting a value to zero.

Definition at line 148 of file dropconnect.hpp.

References DropConnect< InputDataType, OutputDataType >::serialize().

◆ serialize()

void serialize ( Archive &  ar,
const unsigned  int 
)

Serialize the layer.

Referenced by DropConnect< InputDataType, OutputDataType >::Ratio().


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-3.2.1/src/mlpack/methods/ann/layer/dropconnect.hpp