CategoricalDQN< OutputLayerType, InitType, NetworkType > Class Template Reference

Implementation of the Categorical Deep Q-Learning network. More...

Public Member Functions

 CategoricalDQN ()
 Default constructor. More...

 
 CategoricalDQN (const int inputDim, const int h1, const int h2, const int outputDim, TrainingConfig config, const bool isNoisy=false, InitType init=InitType(), OutputLayerType outputLayer=OutputLayerType())
 Construct an instance of CategoricalDQN class. More...

 
 CategoricalDQN (NetworkType &network, TrainingConfig config, const bool isNoisy=false)
 Construct an instance of CategoricalDQN class from a pre-constructed network. More...

 
void Backward (const arma::mat state, arma::mat &lossGradients, arma::mat &gradient)
 Perform the backward pass of the state in real batch mode. More...

 
void Forward (const arma::mat state, arma::mat &dist)
 Perform the forward pass of the states in real batch mode. More...

 
const arma::mat & Parameters () const
 Return the Parameters. More...

 
arma::mat & Parameters ()
 Modify the Parameters. More...

 
void Predict (const arma::mat state, arma::mat &actionValue)
 Predict the responses to a given set of predictors. More...

 
void ResetNoise ()
 Resets noise of the network, if the network is of type noisy. More...

 
void ResetParameters ()
 Resets the parameters of the network. More...

 

Detailed Description


template<typename OutputLayerType = ann::EmptyLoss<>, typename InitType = ann::GaussianInitialization, typename NetworkType = ann::FFN<OutputLayerType, InitType>>
class mlpack::rl::CategoricalDQN< OutputLayerType, InitType, NetworkType >

Implementation of the Categorical Deep Q-Learning network.

For more information, see the following.

@misc{bellemare2017distributional,
author = {Marc G. Bellemare, Will Dabney, RĂ©mi Munos},
title = {A Distributional Perspective on Reinforcement Learning},
year = {2017},
url = {http://arxiv.org/abs/1707.06887}
}
Template Parameters
OutputLayerTypeThe output layer type of the network.
InitTypeThe initialization type used for the network.
NetworkTypeThe type of network used for simple dqn.

Definition at line 48 of file categorical_dqn.hpp.

Constructor & Destructor Documentation

◆ CategoricalDQN() [1/3]

CategoricalDQN ( )
inline

Default constructor.

Definition at line 54 of file categorical_dqn.hpp.

◆ CategoricalDQN() [2/3]

CategoricalDQN ( const int  inputDim,
const int  h1,
const int  h2,
const int  outputDim,
TrainingConfig  config,
const bool  isNoisy = false,
InitType  init = InitType(),
OutputLayerType  outputLayer = OutputLayerType() 
)
inline

Construct an instance of CategoricalDQN class.

Parameters
inputDimNumber of inputs.
h1Number of neurons in hiddenlayer-1.
h2Number of neurons in hiddenlayer-2.
outputDimNumber of neurons in output layer.
configHyper-parameters for categorical dqn.
isNoisySpecifies whether the network needs to be of type noisy.
initSpecifies the initialization rule for the network.
outputLayerSpecifies the output layer type for network.

Definition at line 70 of file categorical_dqn.hpp.

◆ CategoricalDQN() [3/3]

CategoricalDQN ( NetworkType &  network,
TrainingConfig  config,
const bool  isNoisy = false 
)
inline

Construct an instance of CategoricalDQN class from a pre-constructed network.

Parameters
networkThe network to be used by CategoricalDQN class.
configHyper-parameters for categorical dqn.
isNoisySpecifies whether the network needs to be of type noisy.

Definition at line 110 of file categorical_dqn.hpp.

Member Function Documentation

◆ Backward()

void Backward ( const arma::mat  state,
arma::mat &  lossGradients,
arma::mat &  gradient 
)
inline

Perform the backward pass of the state in real batch mode.

Parameters
stateThe input state.
lossGradientsThe loss gradients.
gradientThe gradient.

Definition at line 201 of file categorical_dqn.hpp.

References SoftmaxType< InputType, OutputType >::Backward(), and core::v2::size().

◆ Forward()

void Forward ( const arma::mat  state,
arma::mat &  dist 
)
inline

Perform the forward pass of the states in real batch mode.

Parameters
stateThe input state.
distThe predicted distributions.

Definition at line 154 of file categorical_dqn.hpp.

References SoftmaxType< InputType, OutputType >::Forward().

◆ Parameters() [1/2]

const arma::mat& Parameters ( ) const
inline

Return the Parameters.

Definition at line 190 of file categorical_dqn.hpp.

◆ Parameters() [2/2]

arma::mat& Parameters ( )
inline

Modify the Parameters.

Definition at line 192 of file categorical_dqn.hpp.

◆ Predict()

void Predict ( const arma::mat  state,
arma::mat &  actionValue 
)
inline

Predict the responses to a given set of predictors.

The responses will reflect the output of the given output layer as returned by the output layer function.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Parameters
stateInput state.
actionValueMatrix to put output action values of states input.

Definition at line 131 of file categorical_dqn.hpp.

References SoftmaxType< InputType, OutputType >::Forward().

◆ ResetNoise()

void ResetNoise ( )
inline

Resets noise of the network, if the network is of type noisy.

Definition at line 180 of file categorical_dqn.hpp.

◆ ResetParameters()

void ResetParameters ( )
inline

Resets the parameters of the network.

Definition at line 172 of file categorical_dqn.hpp.


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/reinforcement_learning/q_networks/categorical_dqn.hpp