Declaration of the WeightNorm layer class. More...
Public Member Functions | |
WeightNorm (LayerTypes< CustomLayers... > layer=LayerTypes< CustomLayers... >()) | |
Create the WeightNorm layer object. More... | |
~WeightNorm () | |
Destructor to release allocated memory. More... | |
template < typename eT > | |
void | Backward (const arma::Mat< eT > &&input, arma::Mat< eT > &&gy, arma::Mat< eT > &&g) |
Backward pass through the layer. More... | |
OutputDataType const & | Delta () const |
Get the delta. More... | |
OutputDataType & | Delta () |
Modify the delta. More... | |
template < typename eT > | |
void | Forward (arma::Mat< eT > &&input, arma::Mat< eT > &&output) |
Forward pass of the WeightNorm layer. More... | |
template < typename eT > | |
void | Gradient (arma::Mat< eT > &&input, arma::Mat< eT > &&error, arma::Mat< eT > &&gradient) |
Calculate the gradient using the output delta, input activations and the weights of the wrapped layer. More... | |
OutputDataType const & | Gradient () const |
Get the gradient. More... | |
OutputDataType & | Gradient () |
Modify the gradient. More... | |
LayerTypes< CustomLayers... > const & | Layer () |
Get the wrapped layer. More... | |
OutputDataType const & | OutputParameter () const |
Get the output parameter. More... | |
OutputDataType & | OutputParameter () |
Modify the output parameter. More... | |
OutputDataType const & | Parameters () const |
Get the parameters. More... | |
OutputDataType & | Parameters () |
Modify the parameters. More... | |
void | Reset () |
Reset the layer parameters. More... | |
template < typename Archive > | |
void | serialize (Archive &ar, const unsigned int) |
Serialize the layer. More... | |
Declaration of the WeightNorm layer class.
The layer reparameterizes the weight vectors in a neural network, decoupling the length of those weight vectors from their direction. This reparameterization does not introduce any dependencies between the examples in a mini-batch.
This class will be a wrapper around existing layers. It will just modify the calculation and updation of weights of the layer.
For more information, refer to the following paper,
InputDataType | Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube). |
OutputDataType | Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube). |
CustomLayers | Additional custom layers that can be added. |
Definition at line 176 of file layer_types.hpp.
WeightNorm | ( | LayerTypes< CustomLayers... > | layer = LayerTypes< CustomLayers... >() | ) |
Create the WeightNorm layer object.
layer | The layer whose weights are needed to be normalized. |
~WeightNorm | ( | ) |
Destructor to release allocated memory.
void Backward | ( | const arma::Mat< eT > && | input, |
arma::Mat< eT > && | gy, | ||
arma::Mat< eT > && | g | ||
) |
Backward pass through the layer.
This function calls the Backward() function of the wrapped layer.
input | The input activations. |
gy | The backpropagated error. |
g | The calculated gradient. |
|
inline |
Get the delta.
Definition at line 117 of file weight_norm.hpp.
|
inline |
Modify the delta.
Definition at line 119 of file weight_norm.hpp.
void Forward | ( | arma::Mat< eT > && | input, |
arma::Mat< eT > && | output | ||
) |
Forward pass of the WeightNorm layer.
Calculates the weights of the wrapped layer from the parameter vector v and the scalar parameter g. It then calulates the output of the wrapped layer from the calculated weights.
input | Input data for the layer. |
output | Resulting output activations. |
void Gradient | ( | arma::Mat< eT > && | input, |
arma::Mat< eT > && | error, | ||
arma::Mat< eT > && | gradient | ||
) |
Calculate the gradient using the output delta, input activations and the weights of the wrapped layer.
input | The input activations. |
error | The calculated error. |
gradient | The calculated gradient. |
|
inline |
Get the gradient.
Definition at line 122 of file weight_norm.hpp.
|
inline |
Modify the gradient.
Definition at line 124 of file weight_norm.hpp.
|
inline |
Get the wrapped layer.
Definition at line 137 of file weight_norm.hpp.
References WeightNorm< InputDataType, OutputDataType, CustomLayers >::serialize().
|
inline |
Get the output parameter.
Definition at line 127 of file weight_norm.hpp.
|
inline |
Modify the output parameter.
Definition at line 129 of file weight_norm.hpp.
|
inline |
Get the parameters.
Definition at line 132 of file weight_norm.hpp.
|
inline |
Modify the parameters.
Definition at line 134 of file weight_norm.hpp.
void Reset | ( | ) |
Reset the layer parameters.
void serialize | ( | Archive & | ar, |
const unsigned | int | ||
) |
Serialize the layer.
Referenced by WeightNorm< InputDataType, OutputDataType, CustomLayers >::Layer().