The CELU activation function, defined by. More...

Inheritance diagram for CELUType< MatType >:

Public Member Functions
	CELUType (const double alpha=1.0)
	Create the CELU object using the specified parameter. More...

	CELUType (const CELUType &other)

	CELUType (CELUType &&other)

virtual	~CELUType ()

double const &	Alpha () const
	Get the non zero gradient. More...

double &	Alpha ()
	Modify the non zero gradient. More...

void	Backward (const MatType &input, const MatType &gy, MatType &g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f. More...

CELUType *	Clone () const
	Clone the CELUType object. This handles polymorphism correctly. More...

void	Forward (const MatType &input, MatType &output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

CELUType &	operator= (const CELUType &other)

CELUType &	operator= (CELUType &&other)

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

Public Member Functions inherited from Layer< MatType >
	Layer ()
	Default constructor. More...

	Layer (const Layer &layer)
	Copy constructor. This is not responsible for copying weights! More...

	Layer (Layer &&layer)
	Move constructor. This is not responsible for moving weights! More...

virtual	~Layer ()
	Default deconstructor. More...

virtual void	ComputeOutputDimensions ()
	Compute the output dimensions. More...

virtual void	CustomInitialize (MatType &, const size_t)
	Override the weight matrix of the layer. More...

virtual void	Forward (const MatType &, const MatType &)
	Takes an input and output object, and computes the corresponding loss of the layer. More...

virtual void	Gradient (const MatType &, const MatType &, MatType &)
	Computing the gradient of the layer with respect to its own input. More...

const std::vector< size_t > &	InputDimensions () const
	Get the input dimensions. More...

std::vector< size_t > &	InputDimensions ()
	Modify the input dimensions. More...

virtual double	Loss ()
	Get the layer loss. More...

virtual Layer &	operator= (const Layer &layer)
	Copy assignment operator. This is not responsible for copying weights! More...

virtual Layer &	operator= (Layer &&layer)
	Move assignment operator. This is not responsible for moving weights! More...

const std::vector< size_t > &	OutputDimensions ()
	Get the output dimensions. More...

virtual size_t	OutputSize () final
	Get the number of elements in the output from this layer. More...

virtual const MatType &	Parameters () const
	Get the parameters. More...

virtual MatType &	Parameters ()
	Set the parameters. More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

virtual void	SetWeights (typename MatType::elem_type *)
	Reset the layer parameter. More...

virtual bool const &	Training () const
	Get whether the layer is currently in training mode. More...

virtual bool &	Training ()
	Modify whether the layer is currently in training mode. More...

virtual size_t	WeightSize () const
	Get the total number of trainable weights in the layer. More...

Additional Inherited Members
Protected Attributes inherited from Layer< MatType >
std::vector< size_t >	inputDimensions
	Logical input dimensions of each point. More...

std::vector< size_t >	outputDimensions
	Logical output dimensions of each point. More...

bool	training
	If true, the layer is in training mode; otherwise, it is in testing mode. More...

bool	validOutputDimensions
	This is `true` if `ComputeOutputDimensions()` has been called, and `outputDimensions` can be considered to be up-to-date. More...

Detailed Description

template
<
typename
MatType
=
arma::mat
>

class mlpack::ann::CELUType< MatType >

The CELU activation function, defined by.

$\begin{eqnarray*} f(x) &=& \left\{ \begin{array}{lr} x & : x \ge 0 \\ \alpha(e^(\frac{x}{\alpha}) - 1) & : x < 0 \end{array} \right. \\ f'(x) &=& \left\{ \begin{array}{lr} 1 & : x \ge 0 \\ (\frac{f(x)}{\alpha}) + 1 & : x < 0 \end{array} \right. \end{eqnarray*}$

When not in training mode, there is no computation of the derivative.

Template Parameters

InputType	The type of the layer's inputs. The layer automatically cast inputs to this type (Default: arma::mat).
OutputType	The type of the computation which also causes the output to also be in this type. The type also allows the computation and weight type to differ from the input type (Default: arma::mat).

Definition at line 60 of file celu.hpp.

Constructor & Destructor Documentation

◆ CELUType() [1/3]

CELUType ( const double alpha = 1.0 )

Create the CELU object using the specified parameter.

The non zero gradient for negative inputs can be adjusted by specifying the CELU hyperparameter alpha (alpha > 0).

Parameters

alpha Scale parameter for the negative factor (default = 1.0).

Referenced by CELUType< MatType >::Clone(), and CELUType< MatType >::~CELUType().

◆ ~CELUType()

virtual ~CELUType ( )

inlinevirtual

Definition at line 77 of file celu.hpp.

References CELUType< MatType >::Backward(), CELUType< MatType >::CELUType(), CELUType< MatType >::Forward(), and CELUType< MatType >::operator=().

◆ CELUType() [2/3]

CELUType ( const CELUType< MatType > & other )

◆ CELUType() [3/3]

CELUType ( CELUType< MatType > && other )

Member Function Documentation

◆ Alpha() [1/2]

double const& Alpha ( ) const

inline

Get the non zero gradient.

Definition at line 112 of file celu.hpp.

◆ Alpha() [2/2]

double& Alpha ( )

inline

Modify the non zero gradient.

Definition at line 114 of file celu.hpp.

References CELUType< MatType >::serialize().

◆ Backward()

void Backward	(	const MatType &	input,
		const MatType &	gy,
		MatType &	g
	)

virtual

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f.

Using the results from the feed forward pass.

Parameters

input	The propagated input activation f(x).
gy	The backpropagated error.
g	The calculated gradient.

Reimplemented from Layer< MatType >.

Referenced by CELUType< MatType >::~CELUType().

◆ Clone()

CELUType* Clone ( ) const

inlinevirtual

Clone the CELUType object. This handles polymorphism correctly.

Implements Layer< MatType >.

Definition at line 73 of file celu.hpp.

References CELUType< MatType >::CELUType().

◆ Forward()

void Forward	(	const MatType &	input,
		MatType &	output
	)

virtual

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

Reimplemented from Layer< MatType >.

Referenced by CELUType< MatType >::~CELUType().

◆ operator=() [1/2]

CELUType& operator= ( const CELUType< MatType > & other )

Referenced by CELUType< MatType >::~CELUType().

◆ operator=() [2/2]

CELUType& operator= ( CELUType< MatType > && other )

◆ serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the layer.

Referenced by CELUType< MatType >::Alpha().

The documentation for this class was generated from the following file:

/home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/celu.hpp

Public Member Functions

Additional Inherited Members

Detailed Description

template<typenameMatType=arma::mat> class mlpack::ann::CELUType< MatType >

Constructor & Destructor Documentation

◆ CELUType() [1/3]

◆ ~CELUType()

◆ CELUType() [2/3]

◆ CELUType() [3/3]

Member Function Documentation

◆ Alpha() [1/2]

◆ Alpha() [2/2]

◆ Backward()

◆ Clone()

◆ Forward()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ serialize()

template
<
typename
MatType
=
arma::mat
>

class mlpack::ann::CELUType< MatType >