Implementation of the Highway layer. More...
Public Member Functions | |
Highway () | |
Create the Highway object. More... | |
Highway (const size_t inSize, const bool model=true) | |
Create the Highway object. More... | |
~Highway () | |
Destroy the Highway object. More... | |
template<class LayerType , class... Args> | |
void | Add (Args... args) |
Add a new module to the model. More... | |
void | Add (LayerTypes< CustomLayers... > layer) |
Add a new module to the model. More... | |
template < typename eT > | |
void | Backward (const arma::Mat< eT > &&, arma::Mat< eT > &&gy, arma::Mat< eT > &&g) |
Ordinary feed-backward pass of a neural network, calculating the function f(x) by propagating x backwards through f. More... | |
void | DeleteModules () |
Destroy all the modules added to the Highway object. More... | |
OutputDataType const & | Delta () const |
Get the delta. More... | |
OutputDataType & | Delta () |
Modify the delta. More... | |
template < typename eT > | |
void | Forward (arma::Mat< eT > &&input, arma::Mat< eT > &&output) |
Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More... | |
template < typename eT > | |
void | Gradient (arma::Mat< eT > &&input, arma::Mat< eT > &&error, arma::Mat< eT > &&gradient) |
Calculate the gradient using the output delta and the input activation. More... | |
OutputDataType const & | Gradient () const |
Get the gradient. More... | |
OutputDataType & | Gradient () |
Modify the gradient. More... | |
InputDataType const & | InputParameter () const |
Get the input parameter. More... | |
InputDataType & | InputParameter () |
Modify the input parameter. More... | |
std::vector< LayerTypes< CustomLayers... > > & | Model () |
Return the modules of the model. More... | |
OutputDataType const & | OutputParameter () const |
Get the output parameter. More... | |
OutputDataType & | OutputParameter () |
Modify the output parameter. More... | |
OutputDataType const & | Parameters () const |
Get the parameters. More... | |
OutputDataType & | Parameters () |
Modify the parameters. More... | |
void | Reset () |
Reset the layer parameter. More... | |
template < typename Archive > | |
void | serialize (Archive &ar, const unsigned int) |
Serialize the layer. More... | |
Implementation of the Highway layer.
The Highway class can vary its behavior between that of feed-forward fully connected network container and that of a layer which simply passes its inputs through depending on the transform gate. Note that the size of the input and output matrices of this class should be equal.
For more information, refer the following paper.
InputDataType | Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube). |
OutputDataType | Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube). |
Definition at line 60 of file highway.hpp.
Highway | ( | const size_t | inSize, |
const bool | model = true |
||
) |
Create the Highway object.
inSize | The number of input units. |
model | Expose all the network modules. |
|
inline |
Add a new module to the model.
args | The layer parameter. |
Definition at line 129 of file highway.hpp.
|
inline |
Add a new module to the model.
layer | The Layer to be added to the model. |
Definition at line 136 of file highway.hpp.
void Backward | ( | const arma::Mat< eT > && | , |
arma::Mat< eT > && | gy, | ||
arma::Mat< eT > && | g | ||
) |
Ordinary feed-backward pass of a neural network, calculating the function f(x) by propagating x backwards through f.
Using the results from the feed-forward pass.
input | The propagated input activation. |
gy | The backpropagated error. |
g | The calculated gradient. |
void DeleteModules | ( | ) |
Destroy all the modules added to the Highway object.
|
inline |
Get the delta.
Definition at line 165 of file highway.hpp.
|
inline |
Modify the delta.
Definition at line 167 of file highway.hpp.
void Forward | ( | arma::Mat< eT > && | input, |
arma::Mat< eT > && | output | ||
) |
Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.
input | Input data used for evaluating the specified function. |
output | Resulting output activation. |
void Gradient | ( | arma::Mat< eT > && | input, |
arma::Mat< eT > && | error, | ||
arma::Mat< eT > && | gradient | ||
) |
Calculate the gradient using the output delta and the input activation.
input | The input parameter used for calculating the gradient. |
error | The calculated error. |
gradient | The calculated gradient. |
|
inline |
Get the gradient.
Definition at line 170 of file highway.hpp.
|
inline |
Modify the gradient.
Definition at line 172 of file highway.hpp.
References Highway< InputDataType, OutputDataType, CustomLayers >::serialize().
|
inline |
Get the input parameter.
Definition at line 155 of file highway.hpp.
|
inline |
Modify the input parameter.
Definition at line 157 of file highway.hpp.
|
inline |
Return the modules of the model.
Definition at line 139 of file highway.hpp.
|
inline |
Get the output parameter.
Definition at line 160 of file highway.hpp.
|
inline |
Modify the output parameter.
Definition at line 162 of file highway.hpp.
|
inline |
Get the parameters.
Definition at line 150 of file highway.hpp.
|
inline |
Modify the parameters.
Definition at line 152 of file highway.hpp.
void Reset | ( | ) |
Reset the layer parameter.
void serialize | ( | Archive & | ar, |
const unsigned | int | ||
) |
Serialize the layer.
Referenced by Highway< InputDataType, OutputDataType, CustomLayers >::Gradient().