VirtualBatchNorm< InputDataType, OutputDataType > Class Template Reference

Declaration of the VirtualBatchNorm layer class. More...

Public Member Functions

 VirtualBatchNorm ()
 Create the VirtualBatchNorm object. More...

 
template
<
typename
eT
>
 VirtualBatchNorm (const arma::Mat< eT > &referenceBatch, const size_t size, const double eps=1e-8)
 Create the VirtualBatchNorm layer object for a specified number of input units. More...

 
template
<
typename
eT
>
void Backward (const arma::Mat< eT > &, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
 Backward pass through the layer. More...

 
OutputDataType const & Delta () const
 Get the delta. More...

 
OutputDataType & Delta ()
 Modify the delta. More...

 
double Epsilon () const
 Get the epsilon value. More...

 
template
<
typename
eT
>
void Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
 Forward pass of the Virtual Batch Normalization layer. More...

 
template
<
typename
eT
>
void Gradient (const arma::Mat< eT > &, const arma::Mat< eT > &error, arma::Mat< eT > &gradient)
 Calculate the gradient using the output delta and the input activations. More...

 
OutputDataType const & Gradient () const
 Get the gradient. More...

 
OutputDataType & Gradient ()
 Modify the gradient. More...

 
size_t InSize () const
 Get the number of input units. More...

 
OutputDataType const & OutputParameter () const
 Get the output parameter. More...

 
OutputDataType & OutputParameter ()
 Modify the output parameter. More...

 
OutputDataType const & Parameters () const
 Get the parameters. More...

 
OutputDataType & Parameters ()
 Modify the parameters. More...

 
void Reset ()
 Reset the layer parameters. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the layer. More...

 

Detailed Description


template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >

Declaration of the VirtualBatchNorm layer class.

Instead of using the batch statistics for normalizing on a mini-batch, it uses a reference subset of the data for calculating the normalization statistics.

For more information, refer to the following paper,

@article{Goodfellow2016,
author = {Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung,
Alec Radford, Xi Chen},
title = {Improved Techniques for Training GANs},
year = {2016},
url = {https://arxiv.org/abs/1606.03498},
}
Template Parameters
InputDataTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 113 of file layer_types.hpp.

Constructor & Destructor Documentation

◆ VirtualBatchNorm() [1/2]

Create the VirtualBatchNorm object.

◆ VirtualBatchNorm() [2/2]

VirtualBatchNorm ( const arma::Mat< eT > &  referenceBatch,
const size_t  size,
const double  eps = 1e-8 
)

Create the VirtualBatchNorm layer object for a specified number of input units.

Parameters
referenceBatchThe data from which the normalization statistics are computed.
sizeThe number of input units / channels.
epsThe epsilon added to variance to ensure numerical stability.

Member Function Documentation

◆ Backward()

void Backward ( const arma::Mat< eT > &  ,
const arma::Mat< eT > &  gy,
arma::Mat< eT > &  g 
)

Backward pass through the layer.

Parameters
*(input) The input activations.
gyThe backpropagated error.
gThe calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const
inline

Get the delta.

Definition at line 116 of file virtual_batch_norm.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )
inline

Modify the delta.

Definition at line 118 of file virtual_batch_norm.hpp.

◆ Epsilon()

double Epsilon ( ) const
inline

Get the epsilon value.

Definition at line 129 of file virtual_batch_norm.hpp.

References VirtualBatchNorm< InputDataType, OutputDataType >::serialize().

◆ Forward()

void Forward ( const arma::Mat< eT > &  input,
arma::Mat< eT > &  output 
)

Forward pass of the Virtual Batch Normalization layer.

Transforms the input data into zero mean and unit variance, scales the data by a factor gamma and shifts it by beta.

Parameters
inputInput data for the layer.
outputResulting output activations.

◆ Gradient() [1/3]

void Gradient ( const arma::Mat< eT > &  ,
const arma::Mat< eT > &  error,
arma::Mat< eT > &  gradient 
)

Calculate the gradient using the output delta and the input activations.

Parameters
*(input) The input activations.
errorThe calculated error.
gradientThe calculated gradient.

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const
inline

Get the gradient.

Definition at line 121 of file virtual_batch_norm.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )
inline

Modify the gradient.

Definition at line 123 of file virtual_batch_norm.hpp.

◆ InSize()

size_t InSize ( ) const
inline

Get the number of input units.

Definition at line 126 of file virtual_batch_norm.hpp.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const
inline

Get the output parameter.

Definition at line 111 of file virtual_batch_norm.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )
inline

Modify the output parameter.

Definition at line 113 of file virtual_batch_norm.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const
inline

Get the parameters.

Definition at line 106 of file virtual_batch_norm.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )
inline

Modify the parameters.

Definition at line 108 of file virtual_batch_norm.hpp.

◆ Reset()

void Reset ( )

Reset the layer parameters.

◆ serialize()

void serialize ( Archive &  ar,
const uint32_t   
)

The documentation for this class was generated from the following files:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/layer_types.hpp
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/virtual_batch_norm.hpp