InstanceNorm< InputDataType, OutputDataType > Class Template Reference

Declaration of the Instance Normalization layer class. More...

Public Member Functions

 InstanceNorm ()
 Create the InstanceNorm object. More...

 
 InstanceNorm (const size_t size, const size_t batchSize, const double eps=1e-5, const bool average=true, const double momentum=0.1)
 Create the InstanceNorm layer object with the specified parameters. More...

 
bool Average () const
 Get the average parameter. More...

 
bool Average ()
 Modify the average parameter. More...

 
template
<
typename
eT
>
void Backward (const arma::Mat< eT > &input, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
 Backward pass through the layer. More...

 
bool Batchsize () const
 Get the batchSize parameter. More...

 
bool Batchsize ()
 Modify the batchSize parameter. More...

 
OutputDataType const & Delta () const
 Get the delta. More...

 
OutputDataType & Delta ()
 Modify the delta. More...

 
bool Deterministic () const
 Get the value of deterministic parameter. More...

 
bool & Deterministic ()
 Modify the value of deterministic parameter. More...

 
double Epsilon () const
 Get the epsilon value. More...

 
double Epsilon ()
 Modify the epsilon value. More...

 
template
<
typename
eT
>
void Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
 Forward pass of the Instance Normalization layer. More...

 
template
<
typename
eT
>
void Gradient (const arma::Mat< eT > &input, const arma::Mat< eT > &error, arma::Mat< eT > &gradient)
 Calculate the gradient using the output delta and the input activations. More...

 
OutputDataType const & Gradient () const
 Get the gradient. More...

 
OutputDataType & Gradient ()
 Modify the gradient. More...

 
size_t InputSize () const
 Get the number of input units / channels. More...

 
size_t InputSize ()
 Modify the input units/ channels. More...

 
double Momentum () const
 Get the momentum value. More...

 
double Momentum ()
 Modify the momentum value. More...

 
OutputDataType const & OutputParameter () const
 Get the output parameter. More...

 
OutputDataType & OutputParameter ()
 Modify the output parameter. More...

 
OutputDataType const & Parameters () const
 Get the parameters. More...

 
OutputDataType & Parameters ()
 Modify the parameters. More...

 
template
<
typename
Archive
>
void serialize (Archive &ar, const uint32_t)
 Serialize the layer. More...

 
OutputDataType const & TrainingMean () const
 Get the mean over the training data. More...

 
OutputDataType & TrainingMean ()
 Modify the mean over the training data. More...

 
OutputDataType const & TrainingVariance () const
 Get the variance over the training data. More...

 
OutputDataType & TrainingVariance ()
 Modify the variance over the training data. More...

 

Detailed Description


template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::InstanceNorm< InputDataType, OutputDataType >

Declaration of the Instance Normalization layer class.

The layer transforms the input data into zero mean and unit variance and then scales and shifts the data by parameters, gamma and beta respectively. These parameters are learnt by the network. The mean and standard-deviation are calculated per-dimension separately for each object in a mini-batch.

If deterministic is false (training), the mean and variance are calculated and the data is normalized. If it is set to true (testing) then the mean and variance accrued over the training set is used.

For more information, refer to the following paper,

@article{Ulyanov17,
author = {Dmitry Ulyanov, Andrea Vedaldi and
Victor Lempitsky},
title = {Instance Normalization:
The Missing Ingredient for Fast Stylization},
year = {2017},
url = {https://arxiv.org/abs/1607.08022}
}
Template Parameters
InputDataTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 55 of file instance_norm.hpp.

Constructor & Destructor Documentation

◆ InstanceNorm() [1/2]

Create the InstanceNorm object.

◆ InstanceNorm() [2/2]

InstanceNorm ( const size_t  size,
const size_t  batchSize,
const double  eps = 1e-5,
const bool  average = true,
const double  momentum = 0.1 
)

Create the InstanceNorm layer object with the specified parameters.

Parameters
sizeThe number of input units / channels.
batchSizeSize of the minibatch.
epsThe epsilon added to variance to ensure numerical stability.
averageBoolean to determine whether cumulative average is used for updating the parameters or momentum.
momentumParameter used to update the running mean and variance.

Member Function Documentation

◆ Average() [1/2]

bool Average ( ) const
inline

Get the average parameter.

Definition at line 166 of file instance_norm.hpp.

◆ Average() [2/2]

bool Average ( )
inline

Modify the average parameter.

Definition at line 168 of file instance_norm.hpp.

◆ Backward()

void Backward ( const arma::Mat< eT > &  input,
const arma::Mat< eT > &  gy,
arma::Mat< eT > &  g 
)

Backward pass through the layer.

Parameters
inputThe input activations
gyThe backpropagated error.
gThe calculated gradient.

◆ Batchsize() [1/2]

bool Batchsize ( ) const
inline

Get the batchSize parameter.

Definition at line 171 of file instance_norm.hpp.

◆ Batchsize() [2/2]

bool Batchsize ( )
inline

Modify the batchSize parameter.

Definition at line 173 of file instance_norm.hpp.

References InstanceNorm< InputDataType, OutputDataType >::serialize().

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const
inline

Get the delta.

Definition at line 125 of file instance_norm.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )
inline

Modify the delta.

Definition at line 127 of file instance_norm.hpp.

◆ Deterministic() [1/2]

bool Deterministic ( ) const
inline

Get the value of deterministic parameter.

Definition at line 135 of file instance_norm.hpp.

◆ Deterministic() [2/2]

bool& Deterministic ( )
inline

Modify the value of deterministic parameter.

Definition at line 137 of file instance_norm.hpp.

◆ Epsilon() [1/2]

double Epsilon ( ) const
inline

Get the epsilon value.

Definition at line 155 of file instance_norm.hpp.

◆ Epsilon() [2/2]

double Epsilon ( )
inline

Modify the epsilon value.

Definition at line 157 of file instance_norm.hpp.

◆ Forward()

void Forward ( const arma::Mat< eT > &  input,
arma::Mat< eT > &  output 
)

Forward pass of the Instance Normalization layer.

Transforms the input data into zero mean and unit variance, scales the data by a factor gamma and shifts it by beta.

Parameters
inputInput data for the layer
outputResulting output activations.

◆ Gradient() [1/3]

void Gradient ( const arma::Mat< eT > &  input,
const arma::Mat< eT > &  error,
arma::Mat< eT > &  gradient 
)

Calculate the gradient using the output delta and the input activations.

Parameters
inputThe input activations
errorThe calculated error
gradientThe calculated gradient.

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const
inline

Get the gradient.

Definition at line 130 of file instance_norm.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )
inline

Modify the gradient.

Definition at line 132 of file instance_norm.hpp.

◆ InputSize() [1/2]

size_t InputSize ( ) const
inline

Get the number of input units / channels.

Definition at line 150 of file instance_norm.hpp.

◆ InputSize() [2/2]

size_t InputSize ( )
inline

Modify the input units/ channels.

Definition at line 152 of file instance_norm.hpp.

◆ Momentum() [1/2]

double Momentum ( ) const
inline

Get the momentum value.

Definition at line 161 of file instance_norm.hpp.

◆ Momentum() [2/2]

double Momentum ( )
inline

Modify the momentum value.

Definition at line 163 of file instance_norm.hpp.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const
inline

Get the output parameter.

Definition at line 118 of file instance_norm.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )
inline

Modify the output parameter.

Definition at line 122 of file instance_norm.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const
inline

Get the parameters.

Definition at line 113 of file instance_norm.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )
inline

Modify the parameters.

Definition at line 115 of file instance_norm.hpp.

◆ serialize()

void serialize ( Archive &  ar,
const uint32_t   
)

◆ TrainingMean() [1/2]

OutputDataType const& TrainingMean ( ) const
inline

Get the mean over the training data.

Definition at line 140 of file instance_norm.hpp.

◆ TrainingMean() [2/2]

OutputDataType& TrainingMean ( )
inline

Modify the mean over the training data.

Definition at line 142 of file instance_norm.hpp.

◆ TrainingVariance() [1/2]

OutputDataType const& TrainingVariance ( ) const
inline

Get the variance over the training data.

Definition at line 145 of file instance_norm.hpp.

◆ TrainingVariance() [2/2]

OutputDataType& TrainingVariance ( )
inline

Modify the variance over the training data.

Definition at line 147 of file instance_norm.hpp.


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/not_adapted/instance_norm.hpp