Declaration of the Batch Normalization layer class. More...

Public Member Functions
	BatchNorm ()
	Create the BatchNorm object. More...

	BatchNorm (const size_t size, const double eps=1e-8)
	Create the BatchNorm layer object for a specified number of input units. More...

template < typename eT >
void	Backward (const arma::Mat< eT > &&input, arma::Mat< eT > &&gy, arma::Mat< eT > &&g)
	Backward pass through the layer. More...

OutputDataType const &	Delta () const
	Get the delta. More...

OutputDataType &	Delta ()
	Modify the delta. More...

bool	Deterministic () const
	Get the value of deterministic parameter. More...

bool &	Deterministic ()
	Modify the value of deterministic parameter. More...

template < typename eT >
void	Forward (const arma::Mat< eT > &&input, arma::Mat< eT > &&output)
	Forward pass of the Batch Normalization layer. More...

template < typename eT >
void	Gradient (const arma::Mat< eT > &&input, arma::Mat< eT > &&error, arma::Mat< eT > &&gradient)
	Calculate the gradient using the output delta and the input activations. More...

OutputDataType const &	Gradient () const
	Get the gradient. More...

OutputDataType &	Gradient ()
	Modify the gradient. More...

OutputDataType const &	OutputParameter () const
	Get the output parameter. More...

OutputDataType &	OutputParameter ()
	Modify the output parameter. More...

OutputDataType const &	Parameters () const
	Get the parameters. More...

OutputDataType &	Parameters ()
	Modify the parameters. More...

void	Reset ()
	Reset the layer parameters. More...

template < typename Archive >
void	serialize (Archive &ar, const unsigned int)
	Serialize the layer. More...

OutputDataType	TrainingMean ()
	Get the mean over the training data. More...

OutputDataType	TrainingVariance ()
	Get the variance over the training data. More...

Detailed Description

template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::BatchNorm< InputDataType, OutputDataType >

Declaration of the Batch Normalization layer class.

The layer transforms the input data into zero mean and unit variance and then scales and shifts the data by parameters, gamma and beta respectively. These parameters are learnt by the network.

If deterministic is false (training), the mean and variance over the batch is calculated and the data is normalized. If it is set to true (testing) then the mean and variance accrued over the training set is used.

For more information, refer to the following paper,

@article{Ioffe15,
  author    = {Sergey Ioffe and
               Christian Szegedy},
  title     = {Batch Normalization: Accelerating Deep Network Training by
               Reducing Internal Covariate Shift},
  journal   = {CoRR},
  volume    = {abs/1502.03167},
  year      = {2015},
  url       = {http://arxiv.org/abs/1502.03167},
  eprint    = {1502.03167},
}

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 56 of file batch_norm.hpp.

Constructor & Destructor Documentation

◆ BatchNorm() [1/2]

BatchNorm ( )

Create the BatchNorm object.

◆ BatchNorm() [2/2]

BatchNorm	(	const size_t	size,
		const double	eps = `1e-8`
	)

Create the BatchNorm layer object for a specified number of input units.

Parameters

size	The number of input units.
eps	The epsilon added to variance to ensure numerical stability.

Member Function Documentation

◆ Backward()

void Backward	(	const arma::Mat< eT > &&	input,
		arma::Mat< eT > &&	gy,
		arma::Mat< eT > &&	g
	)

Backward pass through the layer.

Parameters

input	The input activations
gy	The backpropagated error.
g	The calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const

inline

Get the delta.

Definition at line 121 of file batch_norm.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )

inline

Modify the delta.

Definition at line 123 of file batch_norm.hpp.

◆ Deterministic() [1/2]

bool Deterministic ( ) const

inline

Get the value of deterministic parameter.

Definition at line 131 of file batch_norm.hpp.

◆ Deterministic() [2/2]

bool& Deterministic ( )

inline

Modify the value of deterministic parameter.

Definition at line 133 of file batch_norm.hpp.

◆ Forward()

void Forward	(	const arma::Mat< eT > &&	input,
		arma::Mat< eT > &&	output
	)

Forward pass of the Batch Normalization layer.

Transforms the input data into zero mean and unit variance, scales the data by a factor gamma and shifts it by beta.

Parameters

input	Input data for the layer
output	Resulting output activations.

◆ Gradient() [1/3]

void Gradient	(	const arma::Mat< eT > &&	input,
		arma::Mat< eT > &&	error,
		arma::Mat< eT > &&	gradient
	)

Calculate the gradient using the output delta and the input activations.

Parameters

input	The input activations
error	The calculated error
gradient	The calculated gradient.

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const

inline

Get the gradient.

Definition at line 126 of file batch_norm.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )

inline

Modify the gradient.

Definition at line 128 of file batch_norm.hpp.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 116 of file batch_norm.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )

inline

Modify the output parameter.

Definition at line 118 of file batch_norm.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const

inline

Get the parameters.

Definition at line 111 of file batch_norm.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )

inline

Modify the parameters.

Definition at line 113 of file batch_norm.hpp.

◆ Reset()

void Reset ( )

Reset the layer parameters.

◆ serialize()

void serialize	(	Archive &	ar,
		const unsigned	int
	)

Serialize the layer.

Referenced by BatchNorm< InputDataType, OutputDataType >::TrainingVariance().

◆ TrainingMean()

OutputDataType TrainingMean ( )

inline

Get the mean over the training data.

Definition at line 136 of file batch_norm.hpp.

◆ TrainingVariance()

OutputDataType TrainingVariance ( )

inline

Get the variance over the training data.

Definition at line 139 of file batch_norm.hpp.

References BatchNorm< InputDataType, OutputDataType >::serialize().

The documentation for this class was generated from the following file:

/home/jenkins-mlpack/mlpack.org/_src/mlpack-3.2.1/src/mlpack/methods/ann/layer/batch_norm.hpp

Public Member Functions

Detailed Description

template<typenameInputDataType=arma::mat,typenameOutputDataType=arma::mat> class mlpack::ann::BatchNorm< InputDataType, OutputDataType >

Constructor & Destructor Documentation

◆ BatchNorm() [1/2]

◆ BatchNorm() [2/2]

Member Function Documentation

◆ Backward()

◆ Delta() [1/2]

◆ Delta() [2/2]

◆ Deterministic() [1/2]

◆ Deterministic() [2/2]

◆ Forward()

◆ Gradient() [1/3]

◆ Gradient() [2/3]

◆ Gradient() [3/3]

◆ OutputParameter() [1/2]

◆ OutputParameter() [2/2]

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ Reset()

◆ serialize()

◆ TrainingMean()

◆ TrainingVariance()

template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::BatchNorm< InputDataType, OutputDataType >