AlphaDropoutType< MatType > Class Template Reference

The alpha - dropout layer is a regularizer that randomly with probability 'ratio' sets input values to alphaDash. More...

Inheritance diagram for AlphaDropoutType< MatType >:

Public Member Functions

 AlphaDropoutType (const double ratio=0.5, const double alphaDash=-alpha *lambda)
 Create the Alpha_Dropout object using the specified ratio. More...

 
 AlphaDropoutType (const AlphaDropoutType &other)
 Copy the given AlphaDropoutType layer. More...

 
 AlphaDropoutType (AlphaDropoutType &&other)
 Take ownership of the given AlphaDropoutType layer. More...

 
virtual ~AlphaDropoutType ()
 
double A () const
 Value to be multiplied with x for affine transformation. More...

 
double AlphaDash () const
 Value of alphaDash. More...

 
double B () const
 Value to be added to a*x for affine transformation. More...

 
void Backward (const MatType &, const MatType &gy, MatType &g)
 Ordinary feed backward pass of the alpha_dropout layer. More...

 
AlphaDropoutTypeClone () const
 Clone the AlphaDropoutType object. More...

 
void Forward (const MatType &input, MatType &output)
 Ordinary feed forward pass of the AlphaDropout layer. More...

 
const MatType & Mask () const
 Get the mask. More...

 
AlphaDropoutTypeoperator= (const AlphaDropoutType &other)
 Copy the given AlphaDropoutType layer. More...

 
AlphaDropoutTypeoperator= (AlphaDropoutType &&other)
 Take ownership of the given AlphaDropoutType layer. More...

 
double Ratio () const
 The probability of setting a value to alphaDash. More...

 
void Ratio (const double r)
 Modify the probability of setting a value to alphaDash. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the layer. More...

 
- Public Member Functions inherited from Layer< MatType >
 Layer ()
 Default constructor. More...

 
 Layer (const Layer &layer)
 Copy constructor. This is not responsible for copying weights! More...

 
 Layer (Layer &&layer)
 Move constructor. This is not responsible for moving weights! More...

 
virtual ~Layer ()
 Default deconstructor. More...

 
virtual void ComputeOutputDimensions ()
 Compute the output dimensions. More...

 
virtual void CustomInitialize (MatType &, const size_t)
 Override the weight matrix of the layer. More...

 
virtual void Forward (const MatType &, const MatType &)
 Takes an input and output object, and computes the corresponding loss of the layer. More...

 
virtual void Gradient (const MatType &, const MatType &, MatType &)
 Computing the gradient of the layer with respect to its own input. More...

 
const std::vector< size_t > & InputDimensions () const
 Get the input dimensions. More...

 
std::vector< size_t > & InputDimensions ()
 Modify the input dimensions. More...

 
virtual double Loss ()
 Get the layer loss. More...

 
virtual Layeroperator= (const Layer &layer)
 Copy assignment operator. This is not responsible for copying weights! More...

 
virtual Layeroperator= (Layer &&layer)
 Move assignment operator. This is not responsible for moving weights! More...

 
const std::vector< size_t > & OutputDimensions ()
 Get the output dimensions. More...

 
virtual size_t OutputSize () final
 Get the number of elements in the output from this layer. More...

 
virtual const MatType & Parameters () const
 Get the parameters. More...

 
virtual MatType & Parameters ()
 Set the parameters. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the layer. More...

 
virtual void SetWeights (typename MatType::elem_type *)
 Reset the layer parameter. More...

 
virtual bool const & Training () const
 Get whether the layer is currently in training mode. More...

 
virtual bool & Training ()
 Modify whether the layer is currently in training mode. More...

 
virtual size_t WeightSize () const
 Get the total number of trainable weights in the layer. More...

 

Additional Inherited Members

- Protected Attributes inherited from Layer< MatType >
std::vector< size_t > inputDimensions
 Logical input dimensions of each point. More...

 
std::vector< size_t > outputDimensions
 Logical output dimensions of each point. More...

 
bool training
 If true, the layer is in training mode; otherwise, it is in testing mode. More...

 
bool validOutputDimensions
 This is true if ComputeOutputDimensions() has been called, and outputDimensions can be considered to be up-to-date. More...

 

Detailed Description


template
<
typename
MatType
=
arma::mat
>

class mlpack::ann::AlphaDropoutType< MatType >

The alpha - dropout layer is a regularizer that randomly with probability 'ratio' sets input values to alphaDash.

The alpha - dropout layer is mostly used for SELU activation function where successive layers don't have same mean and variance.

For more information, see the following.

@article{Klambauer2017,
author = {Gunter Klambauer and Thomas Unterthiner and
Andreas Mayr},
title = {Self-Normalizing Neural Networks},
journal = {Advances in Neural Information Processing Systems},
year = {2017},
url = {https://arxiv.org/abs/1706.02515}
}
Template Parameters
MatTypeMatrix representation to accept as input and use for computation.

Definition at line 48 of file alpha_dropout.hpp.

Constructor & Destructor Documentation

◆ AlphaDropoutType() [1/3]

AlphaDropoutType ( const double  ratio = 0.5,
const double  alphaDash = -alpha *lambda 
)

Create the Alpha_Dropout object using the specified ratio.

Parameters
ratioThe probability of setting a value to alphaDash.
alphaDashThe dropout scaling parameter.

Referenced by AlphaDropoutType< MatType >::Clone(), and AlphaDropoutType< MatType >::~AlphaDropoutType().

◆ ~AlphaDropoutType()

◆ AlphaDropoutType() [2/3]

AlphaDropoutType ( const AlphaDropoutType< MatType > &  other)

Copy the given AlphaDropoutType layer.

◆ AlphaDropoutType() [3/3]

AlphaDropoutType ( AlphaDropoutType< MatType > &&  other)

Take ownership of the given AlphaDropoutType layer.

Member Function Documentation

◆ A()

double A ( ) const
inline

Value to be multiplied with x for affine transformation.

Definition at line 98 of file alpha_dropout.hpp.

◆ AlphaDash()

double AlphaDash ( ) const
inline

Value of alphaDash.

Definition at line 104 of file alpha_dropout.hpp.

◆ B()

double B ( ) const
inline

Value to be added to a*x for affine transformation.

Definition at line 101 of file alpha_dropout.hpp.

◆ Backward()

void Backward ( const MatType &  ,
const MatType &  gy,
MatType &  g 
)
virtual

Ordinary feed backward pass of the alpha_dropout layer.

Parameters
*(input) The propagated input activation.
gyThe backpropagated error.
gThe calculated gradient.

Reimplemented from Layer< MatType >.

Referenced by AlphaDropoutType< MatType >::~AlphaDropoutType().

◆ Clone()

AlphaDropoutType* Clone ( ) const
inlinevirtual

Clone the AlphaDropoutType object.

This handles polymorphism correctly.

Implements Layer< MatType >.

Definition at line 63 of file alpha_dropout.hpp.

References AlphaDropoutType< MatType >::AlphaDropoutType().

◆ Forward()

void Forward ( const MatType &  input,
MatType &  output 
)
virtual

Ordinary feed forward pass of the AlphaDropout layer.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.

Reimplemented from Layer< MatType >.

Referenced by AlphaDropoutType< MatType >::~AlphaDropoutType().

◆ Mask()

const MatType& Mask ( ) const
inline

Get the mask.

Definition at line 107 of file alpha_dropout.hpp.

◆ operator=() [1/2]

AlphaDropoutType& operator= ( const AlphaDropoutType< MatType > &  other)

◆ operator=() [2/2]

AlphaDropoutType& operator= ( AlphaDropoutType< MatType > &&  other)

Take ownership of the given AlphaDropoutType layer.

◆ Ratio() [1/2]

double Ratio ( ) const
inline

The probability of setting a value to alphaDash.

Definition at line 95 of file alpha_dropout.hpp.

◆ Ratio() [2/2]

void Ratio ( const double  r)
inline

Modify the probability of setting a value to alphaDash.

As 'a' and 'b' depend on 'ratio', modify them as well.

Definition at line 111 of file alpha_dropout.hpp.

References AlphaDropoutType< MatType >::serialize().

◆ serialize()

void serialize ( Archive &  ar,
const uint32_t   
)

Serialize the layer.

Referenced by AlphaDropoutType< MatType >::Ratio().


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/alpha_dropout.hpp