CReLU< InputDataType, OutputDataType > Class Template Reference

A concatenated ReLU has two outputs, one ReLU and one negative ReLU, concatenated together. More...

Public Member Functions

 CReLU ()
 Create the CReLU object. More...

 
template
<
typename
DataType
>
void Backward (const DataType &input, const DataType &gy, DataType &g)
 Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f. More...

 
OutputDataType const & Delta () const
 Get the delta. More...

 
OutputDataType & Delta ()
 Modify the delta. More...

 
template
<
typename
InputType
,
typename
OutputType
>
void Forward (const InputType &input, OutputType &output)
 Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

 
OutputDataType const & OutputParameter () const
 Get the output parameter. More...

 
OutputDataType & OutputParameter ()
 Modify the output parameter. More...

 
template
<
typename
Archive
>
void serialize (Archive &, const uint32_t)
 Serialize the layer. More...

 
size_t WeightSize () const
 Get size of weights. More...

 

Detailed Description


template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::CReLU< InputDataType, OutputDataType >

A concatenated ReLU has two outputs, one ReLU and one negative ReLU, concatenated together.

In other words, for positive x it produces [x, 0], and for negative x it produces [0, x]. Because it has two outputs, CReLU doubles the output dimension.

Note: The CReLU doubles the output size.

For more information, see the following.

@inproceedings{ICML2016,
title = {Understanding and Improving Convolutional Neural Networks
via Concatenated Rectified Linear Units},
author = {LWenling Shang, Kihyuk Sohn, Diogo Almeida, Honglak Lee},
year = {2016},
url = {https://arxiv.org/abs/1603.05201}
}
Template Parameters
InputDataTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 50 of file c_relu.hpp.

Constructor & Destructor Documentation

◆ CReLU()

CReLU ( )

Create the CReLU object.

Member Function Documentation

◆ Backward()

void Backward ( const DataType &  input,
const DataType &  gy,
DataType &  g 
)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f.

Using the results from the feed forward pass.

Parameters
inputThe propagated input activation.
gyThe backpropagated error.
gThe calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const
inline

Get the delta.

Definition at line 87 of file c_relu.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )
inline

Modify the delta.

Definition at line 89 of file c_relu.hpp.

◆ Forward()

void Forward ( const InputType &  input,
OutputType &  output 
)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Works only for 2D Tensors.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const
inline

Get the output parameter.

Definition at line 82 of file c_relu.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )
inline

Modify the output parameter.

Definition at line 84 of file c_relu.hpp.

◆ serialize()

void serialize ( Archive &  ,
const uint32_t   
)

Serialize the layer.

Referenced by CReLU< InputDataType, OutputDataType >::WeightSize().

◆ WeightSize()

size_t WeightSize ( ) const
inline

Get size of weights.

Definition at line 92 of file c_relu.hpp.

References CReLU< InputDataType, OutputDataType >::serialize().


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/c_relu.hpp