The Lookup class stores word embeddings and retrieves them using tokens. More...

Inheritance diagram for LookupType< InputType, OutputType >:

Public Member Functions
	LookupType (const size_t vocabSize=0, const size_t embeddingSize=0)
	Create the Lookup object using the specified vocabulary and embedding size. More...

void	Backward (const InputType &, const OutputType &gy, OutputType &g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...

LookupType *	Clone () const
	Clone the LookupType object. This handles polymorphism correctly. More...

size_t	EmbeddingSize () const
	Get the length of each embedding vector. More...

void	Forward (const InputType &input, OutputType &output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

void	Gradient (const InputType &input, const OutputType &error, OutputType &gradient)
	Calculate the gradient using the output delta and the input activation. More...

const std::vector< size_t > &	OutputDimensions () const
	Get the dimensions of the output. More...

OutputType const &	Parameters () const
	Get the parameters. More...

OutputType &	Parameters ()
	Modify the parameters. More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

size_t	VocabSize () const
	Get the size of the vocabulary. More...

const size_t	WeightSize () const
	Get the number of trainable parameters. More...

Public Member Functions inherited from Layer< InputType, OutputType >
	Layer ()
	Default constructor. More...

	Layer (const Layer &layer)
	Copy constructor. This is not responsible for copying weights! More...

	Layer (Layer &&layer)
	Move constructor. This is not responsible for moving weights! More...

virtual	~Layer ()
	Default deconstructor. More...

virtual void	Backward (const InputType &, const InputType &, InputType &)
	Performs a backpropagation step through the layer, with respect to the given input. More...

virtual void	ComputeOutputDimensions ()
	Compute the output dimensions. More...

virtual void	CustomInitialize (InputType &, const size_t)
	Override the weight matrix of the layer. More...

virtual void	Forward (const InputType &, InputType &)
	Takes an input object, and computes the corresponding output of the layer. More...

virtual void	Forward (const InputType &, const InputType &)
	Takes an input and output object, and computes the corresponding loss of the layer. More...

virtual void	Gradient (const InputType &, const InputType &, InputType &)
	Computing the gradient of the layer with respect to its own input. More...

const std::vector< size_t > &	InputDimensions () const
	Get the input dimensions. More...

std::vector< size_t > &	InputDimensions ()
	Modify the input dimensions. More...

virtual double	Loss ()
	Get the layer loss. More...

virtual Layer &	operator= (const Layer &layer)
	Copy assignment operator. This is not responsible for copying weights! More...

virtual Layer &	operator= (Layer &&layer)
	Move assignment operator. This is not responsible for moving weights! More...

const std::vector< size_t > &	OutputDimensions ()
	Get the output dimensions. More...

virtual size_t	OutputSize () final
	Get the number of elements in the output from this layer. More...

void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

virtual void	SetWeights (typename InputType ::elem_type *)
	Reset the layer parameter. More...

virtual bool const &	Training () const
	Get whether the layer is currently in training mode. More...

virtual bool &	Training ()
	Modify whether the layer is currently in training mode. More...

Additional Inherited Members
Protected Attributes inherited from Layer< InputType, OutputType >
std::vector< size_t >	inputDimensions
	Logical input dimensions of each point. More...

std::vector< size_t >	outputDimensions
	Logical output dimensions of each point. More...

bool	training
	If true, the layer is in training mode; otherwise, it is in testing mode. More...

bool	validOutputDimensions
	This is `true` if `ComputeOutputDimensions()` has been called, and `outputDimensions` can be considered to be up-to-date. More...

Detailed Description

template
<
typename
InputType
=
arma::mat
,
typename
OutputType
=
arma::mat
>

class mlpack::ann::LookupType< InputType, OutputType >

The Lookup class stores word embeddings and retrieves them using tokens.

The Lookup layer is always the first layer of the network. The input to the Lookup class is a matrix of shape (sequenceLength, batchSize). The matrix consists of tokens which are used to lookup the table (i.e. weights) to find the embeddings of those tokens.

The input shape : (sequenceLength, batchSize). The output shape : (embeddingSize, sequenceLength, batchSize).

Template Parameters

InputType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 41 of file lookup.hpp.

Constructor & Destructor Documentation

◆ LookupType()

LookupType	(	const size_t	vocabSize = `0`,
		const size_t	embeddingSize = `0`
	)

Create the Lookup object using the specified vocabulary and embedding size.

Parameters

vocabSize	The size of the vocabulary.
embeddingSize	The length of each embedding vector.

Referenced by LookupType< InputType, OutputType >::Clone().

Member Function Documentation

◆ Backward()

void Backward	(	const InputType &	,
		const OutputType &	gy,
		OutputType &	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters

*	(input) The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

Referenced by LookupType< InputType, OutputType >::Clone().

◆ Clone()

LookupType* Clone ( ) const

inlinevirtual

Clone the LookupType object. This handles polymorphism correctly.

Implements Layer< InputType, OutputType >.

Definition at line 53 of file lookup.hpp.

References LookupType< InputType, OutputType >::Backward(), LookupType< InputType, OutputType >::Forward(), LookupType< InputType, OutputType >::Gradient(), and LookupType< InputType, OutputType >::LookupType().

◆ EmbeddingSize()

size_t EmbeddingSize ( ) const

inline

Get the length of each embedding vector.

Definition at line 97 of file lookup.hpp.

◆ Forward()

void Forward	(	const InputType &	input,
		OutputType &	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

Referenced by LookupType< InputType, OutputType >::Clone().

◆ Gradient()

void Gradient	(	const InputType &	input,
		const OutputType &	error,
		OutputType &	gradient
	)

Calculate the gradient using the output delta and the input activation.

Parameters

input	The input parameter used for calculating the gradient.
error	The calculated error.
gradient	The calculated gradient.

Referenced by LookupType< InputType, OutputType >::Clone().

◆ OutputDimensions()

const std::vector<size_t>& OutputDimensions ( ) const

inline

Get the dimensions of the output.

This layer adds an extra dimension for the embedding.

Definition at line 104 of file lookup.hpp.

References Layer< InputType, OutputType >::inputDimensions, and LookupType< InputType, OutputType >::serialize().

◆ Parameters() [1/2]

OutputType const& Parameters ( ) const

inlinevirtual

Get the parameters.

Reimplemented from Layer< InputType, OutputType >.

Definition at line 89 of file lookup.hpp.

◆ Parameters() [2/2]

OutputType& Parameters ( )

inlinevirtual

Modify the parameters.

Reimplemented from Layer< InputType, OutputType >.

Definition at line 91 of file lookup.hpp.

◆ serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the layer.

Referenced by LookupType< InputType, OutputType >::OutputDimensions().

◆ VocabSize()

size_t VocabSize ( ) const

inline

Get the size of the vocabulary.

Definition at line 94 of file lookup.hpp.

◆ WeightSize()

const size_t WeightSize ( ) const

inlinevirtual

Get the number of trainable parameters.

Reimplemented from Layer< InputType, OutputType >.

Definition at line 100 of file lookup.hpp.

The documentation for this class was generated from the following file:

/home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/not_adapted/lookup.hpp

Public Member Functions

Additional Inherited Members

Detailed Description

template<typenameInputType=arma::mat,typenameOutputType=arma::mat> class mlpack::ann::LookupType< InputType, OutputType >

Constructor & Destructor Documentation

◆ LookupType()

Member Function Documentation

◆ Backward()

◆ Clone()

◆ EmbeddingSize()

◆ Forward()

◆ Gradient()

◆ OutputDimensions()

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ serialize()

◆ VocabSize()

◆ WeightSize()

template
<
typename
InputType
=
arma::mat
,
typename
OutputType
=
arma::mat
>

class mlpack::ann::LookupType< InputType, OutputType >