VirtualBatchNorm< InputDataType, OutputDataType > Class Template Reference

Declaration of the VirtualBatchNorm layer class. More...

Public Member Functions

 VirtualBatchNorm ()
 Create the VirtualBatchNorm object. More...

 
template
<
typename
eT
>
 VirtualBatchNorm (const arma::Mat< eT > &referenceBatch, const size_t size, const double eps=1e-8)
 Create the VirtualBatchNorm layer object for a specified number of input units. More...

 
template
<
typename
eT
>
void Backward (const arma::Mat< eT > &&, arma::Mat< eT > &&gy, arma::Mat< eT > &&g)
 Backward pass through the layer. More...

 
OutputDataType const & Delta () const
 Get the delta. More...

 
OutputDataType & Delta ()
 Modify the delta. More...

 
template
<
typename
eT
>
void Forward (const arma::Mat< eT > &&input, arma::Mat< eT > &&output)
 Forward pass of the Virtual Batch Normalization layer. More...

 
template
<
typename
eT
>
void Gradient (const arma::Mat< eT > &&, arma::Mat< eT > &&error, arma::Mat< eT > &&gradient)
 Calculate the gradient using the output delta and the input activations. More...

 
OutputDataType const & Gradient () const
 Get the gradient. More...

 
OutputDataType & Gradient ()
 Modify the gradient. More...

 
OutputDataType const & OutputParameter () const
 Get the output parameter. More...

 
OutputDataType & OutputParameter ()
 Modify the output parameter. More...

 
OutputDataType const & Parameters () const
 Get the parameters. More...

 
OutputDataType & Parameters ()
 Modify the parameters. More...

 
void Reset ()
 Reset the layer parameters. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const unsigned int)
 Serialize the layer. More...

 

Detailed Description


template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >

Declaration of the VirtualBatchNorm layer class.

Instead of using the batch statistics for normalizing on a mini-batch, it uses a reference subset of the data for calculating the normalization statistics.

For more information, refer to the following paper,

@article{Goodfellow2016,
author = {Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung,
Alec Radford, Xi Chen},
title = {Improved Techniques for Training GANs},
year = {2016},
url = {https://arxiv.org/abs/1606.03498},
}
Template Parameters
InputDataTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 83 of file layer_types.hpp.

Constructor & Destructor Documentation

◆ VirtualBatchNorm() [1/2]

Create the VirtualBatchNorm object.

◆ VirtualBatchNorm() [2/2]

VirtualBatchNorm ( const arma::Mat< eT > &  referenceBatch,
const size_t  size,
const double  eps = 1e-8 
)

Create the VirtualBatchNorm layer object for a specified number of input units.

Parameters
referenceBatchThe data from which the normalization statistics are computed.
sizeThe number of input units.
epsThe epsilon added to variance to ensure numerical stability.

Member Function Documentation

◆ Backward()

void Backward ( const arma::Mat< eT > &&  ,
arma::Mat< eT > &&  gy,
arma::Mat< eT > &&  g 
)

Backward pass through the layer.

Parameters
inputThe input activations.
gyThe backpropagated error.
gThe calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const
inline

Get the delta.

Definition at line 116 of file virtual_batch_norm.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )
inline

Modify the delta.

Definition at line 118 of file virtual_batch_norm.hpp.

◆ Forward()

void Forward ( const arma::Mat< eT > &&  input,
arma::Mat< eT > &&  output 
)

Forward pass of the Virtual Batch Normalization layer.

Transforms the input data into zero mean and unit variance, scales the data by a factor gamma and shifts it by beta.

Parameters
inputInput data for the layer.
outputResulting output activations.

◆ Gradient() [1/3]

void Gradient ( const arma::Mat< eT > &&  ,
arma::Mat< eT > &&  error,
arma::Mat< eT > &&  gradient 
)

Calculate the gradient using the output delta and the input activations.

Parameters
inputThe input activations.
errorThe calculated error.
gradientThe calculated gradient.

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const
inline

Get the gradient.

Definition at line 121 of file virtual_batch_norm.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )
inline

Modify the gradient.

Definition at line 123 of file virtual_batch_norm.hpp.

References VirtualBatchNorm< InputDataType, OutputDataType >::serialize().

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const
inline

Get the output parameter.

Definition at line 111 of file virtual_batch_norm.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )
inline

Modify the output parameter.

Definition at line 113 of file virtual_batch_norm.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const
inline

Get the parameters.

Definition at line 106 of file virtual_batch_norm.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )
inline

Modify the parameters.

Definition at line 108 of file virtual_batch_norm.hpp.

◆ Reset()

void Reset ( )

Reset the layer parameters.

◆ serialize()

void serialize ( Archive &  ar,
const unsigned  int 
)

The documentation for this class was generated from the following files:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-3.2.1/src/mlpack/methods/ann/layer/layer_types.hpp
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-3.2.1/src/mlpack/methods/ann/layer/virtual_batch_norm.hpp