The DropConnect layer is a regularizer that randomly with probability ratio sets the connection values to zero and scales the remaining elements by factor 1 /(1 - ratio). More...

Inheritance diagram for DropConnectType< MatType >:

[legend]

Public Member Functions
	DropConnectType ()
	Create the DropConnect object. More...

	DropConnectType (const size_t outSize, const double ratio=0.5)
	Creates the DropConnect Layer as a Linear Object that takes the number of output units and a ratio as parameter. More...

	DropConnectType (const DropConnectType &other)
	Copy the given DropConnectType (except for weights). More...

	DropConnectType (DropConnectType &&other)
	Take ownership of the given DropConnectType (except for weights). More...

virtual	~DropConnectType ()

void	Backward (const MatType &input, const MatType &gy, MatType &g)
	Ordinary feed backward pass of the DropConnect layer. More...

DropConnectType *	Clone () const
	Clone the DropConnectType object. This handles polymorphism correctly. More...

void	ComputeOutputDimensions ()
	Compute the output dimensions of the layer based on `InputDimensions()`. More...

void	Forward (const MatType &input, MatType &output)
	Ordinary feed forward pass of the DropConnect layer. More...

void	Gradient (const MatType &input, const MatType &error, MatType &gradient)
	Calculate the gradient using the output delta and the input activation. More...

DropConnectType &	operator= (const DropConnectType &other)
	Copy the given DropConnectType (except for weights). More...

DropConnectType &	operator= (DropConnectType &&other)
	Take ownership of the given DropConnectType (except for weights). More...

double	Ratio () const
	The probability of setting a value to zero. More...

void	Ratio (const double r)
	Modify the probability of setting a value to zero. More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

void	SetWeights (typename MatType::elem_type *weightsPtr)
	Reset the layer parameter. More...

size_t	WeightSize () const
	Return the size of the weights. More...

Public Member Functions inherited from Layer< MatType >
	Layer ()
	Default constructor. More...

	Layer (const Layer &layer)
	Copy constructor. This is not responsible for copying weights! More...

	Layer (Layer &&layer)
	Move constructor. This is not responsible for moving weights! More...

virtual	~Layer ()
	Default deconstructor. More...

virtual void	Forward (const MatType &, const MatType &)
	Takes an input and output object, and computes the corresponding loss of the layer. More...

const std::vector< size_t > &	InputDimensions () const
	Get the input dimensions. More...

std::vector< size_t > &	InputDimensions ()
	Modify the input dimensions. More...

virtual double	Loss ()
	Get the layer loss. More...

virtual Layer &	operator= (const Layer &layer)
	Copy assignment operator. This is not responsible for copying weights! More...

virtual Layer &	operator= (Layer &&layer)
	Move assignment operator. This is not responsible for moving weights! More...

const std::vector< size_t > &	OutputDimensions ()
	Get the output dimensions. More...

virtual size_t	OutputSize () final
	Get the number of elements in the output from this layer. More...

virtual const MatType &	Parameters () const
	Get the parameters. More...

virtual MatType &	Parameters ()
	Set the parameters. More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

virtual bool const &	Training () const
	Get whether the layer is currently in training mode. More...

virtual bool &	Training ()
	Modify whether the layer is currently in training mode. More...

Additional Inherited Members
Protected Attributes inherited from Layer< MatType >
std::vector< size_t >	inputDimensions
	Logical input dimensions of each point. More...

std::vector< size_t >	outputDimensions
	Logical output dimensions of each point. More...

bool	training
	If true, the layer is in training mode; otherwise, it is in testing mode. More...

bool	validOutputDimensions
	This is `true` if `ComputeOutputDimensions()` has been called, and `outputDimensions` can be considered to be up-to-date. More...

Detailed Description

template
<
typename
MatType
=
arma::mat
>

class mlpack::ann::DropConnectType< MatType >

The DropConnect layer is a regularizer that randomly with probability ratio sets the connection values to zero and scales the remaining elements by factor 1 /(1 - ratio).

The output is scaled with 1 / (1 - p) when in training mode. During testing, the layer just computes the output. The output is computed according to the input layer. If no input layer is given, it will take a linear layer as default.

For more information, see the following.

@inproceedings{WanICML2013,
  title     = {Regularization of Neural Networks using DropConnect},
  booktitle = {Proceedings of the 30th International Conference on Machine
               Learning(ICML - 13)},
  author    = {Li Wan and Matthew Zeiler and Sixin Zhang and Yann L. Cun and
               Rob Fergus},
  year      = {2013},
  url       = {http://proceedings.mlr.press/v28/wan13.pdf}
}

Template Parameters

MatType Matrix representation to accept as input and use for computation.

Definition at line 50 of file dropconnect.hpp.

Constructor & Destructor Documentation

◆ DropConnectType() [1/4]

DropConnectType ( )

Create the DropConnect object.

Referenced by DropConnectType< MatType >::Clone().

◆ DropConnectType() [2/4]

DropConnectType	(	const size_t	outSize,
		const double	ratio = `0.5`
	)

Creates the DropConnect Layer as a Linear Object that takes the number of output units and a ratio as parameter.

Parameters

outSize	The number of output units.
ratio	The probability of setting a value to zero.

◆ ~DropConnectType()

virtual ~DropConnectType ( )

virtual

Referenced by DropConnectType< MatType >::Clone().

◆ DropConnectType() [3/4]

DropConnectType ( const DropConnectType< MatType > & other )

Copy the given DropConnectType (except for weights).

◆ DropConnectType() [4/4]

DropConnectType ( DropConnectType< MatType > && other )

Take ownership of the given DropConnectType (except for weights).

Member Function Documentation

◆ Backward()

void Backward	(	const MatType &	input,
		const MatType &	gy,
		MatType &	g
	)

virtual

Ordinary feed backward pass of the DropConnect layer.

Parameters

input	The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

Reimplemented from Layer< MatType >.

Referenced by DropConnectType< MatType >::Clone().

◆ Clone()

DropConnectType* Clone ( ) const

inlinevirtual

Clone the DropConnectType object. This handles polymorphism correctly.

Implements Layer< MatType >.

Definition at line 67 of file dropconnect.hpp.

References DropConnectType< MatType >::Backward(), DropConnectType< MatType >::DropConnectType(), DropConnectType< MatType >::Forward(), DropConnectType< MatType >::Gradient(), DropConnectType< MatType >::operator=(), and DropConnectType< MatType >::~DropConnectType().

◆ ComputeOutputDimensions()

void ComputeOutputDimensions ( )

virtual

Compute the output dimensions of the layer based on InputDimensions().

Reimplemented from Layer< MatType >.

Referenced by DropConnectType< MatType >::Ratio().

◆ Forward()

void Forward	(	const MatType &	input,
		MatType &	output
	)

virtual

Ordinary feed forward pass of the DropConnect layer.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

Reimplemented from Layer< MatType >.

Referenced by DropConnectType< MatType >::Clone().

◆ Gradient()

void Gradient	(	const MatType &	input,
		const MatType &	error,
		MatType &	gradient
	)

virtual

Calculate the gradient using the output delta and the input activation.

Parameters

input	The propagated input.
error	The calculated error.
*	(gradient) The calculated gradient.

Reimplemented from Layer< MatType >.

Referenced by DropConnectType< MatType >::Clone().

◆ operator=() [1/2]

DropConnectType& operator= ( const DropConnectType< MatType > & other )

Copy the given DropConnectType (except for weights).

Referenced by DropConnectType< MatType >::Clone().

◆ operator=() [2/2]

DropConnectType& operator= ( DropConnectType< MatType > && other )

Take ownership of the given DropConnectType (except for weights).

◆ Ratio() [1/2]

double Ratio ( ) const

inline

The probability of setting a value to zero.

Definition at line 108 of file dropconnect.hpp.

◆ Ratio() [2/2]

void Ratio ( const double r )

inline

Modify the probability of setting a value to zero.

Definition at line 111 of file dropconnect.hpp.

References DropConnectType< MatType >::ComputeOutputDimensions().

◆ serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the layer.

Referenced by DropConnectType< MatType >::WeightSize().

◆ SetWeights()

void SetWeights ( typename MatType::elem_type * )

virtual

Reset the layer parameter.

The method is called to assigned the allocated memory to the internal layer parameters like weights and biases. The method should be called before the first call of Forward(input, output). If you do not respect this rule, Forward(input, output) and Backward(input, gy, g) might compute incorrect results.

Parameters

weightsPtr This pointer should be used as the first element of the memory that is allocated for this layer. In general, SetWeights() implementations should use MakeAlias() with weightsPtr to wrap the weights of a layer.

Reimplemented from Layer< MatType >.

Referenced by DropConnectType< MatType >::WeightSize().

◆ WeightSize()

size_t WeightSize ( ) const

inlinevirtual

Return the size of the weights.

Reimplemented from Layer< MatType >.

Definition at line 121 of file dropconnect.hpp.

References DropConnectType< MatType >::serialize(), and DropConnectType< MatType >::SetWeights().

The documentation for this class was generated from the following file:

/home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/dropconnect.hpp

Public Member Functions

Additional Inherited Members

Detailed Description

template<typenameMatType=arma::mat> class mlpack::ann::DropConnectType< MatType >

Constructor & Destructor Documentation

◆ DropConnectType() [1/4]

◆ DropConnectType() [2/4]

◆ ~DropConnectType()

◆ DropConnectType() [3/4]

◆ DropConnectType() [4/4]

Member Function Documentation

◆ Backward()

◆ Clone()

◆ ComputeOutputDimensions()

◆ Forward()

◆ Gradient()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ Ratio() [1/2]

◆ Ratio() [2/2]

◆ serialize()

◆ SetWeights()

◆ WeightSize()

template
<
typename
MatType
=
arma::mat
>

class mlpack::ann::DropConnectType< MatType >