DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType > Class Template Reference

Implementation of the Dueling Deep Q-Learning network. More...

Public Member Functions

 DuelingDQN ()
 Default constructor. More...

 
 DuelingDQN (const int h1, const int h2, const int outputDim, const bool isNoisy=false, InitType init=InitType(), OutputLayerType outputLayer=OutputLayerType())
 Construct an instance of DuelingDQN class. More...

 
 DuelingDQN (FeatureNetworkType &featureNetwork, AdvantageNetworkType &advantageNetwork, ValueNetworkType &valueNetwork, const bool isNoisy=false)
 Construct an instance of DuelingDQN class from a pre-constructed network. More...

 
 DuelingDQN (const DuelingDQN &)
 Copy constructor. More...

 
void Backward (const arma::mat state, arma::mat &target, arma::mat &gradient)
 Perform the backward pass of the state in real batch mode. More...

 
void Forward (const arma::mat state, arma::mat &actionValue)
 Perform the forward pass of the states in real batch mode. More...

 
void operator= (const DuelingDQN &model)
 Copy assignment operator. More...

 
const arma::mat & Parameters () const
 Return the Parameters. More...

 
arma::mat & Parameters ()
 Modify the Parameters. More...

 
void Predict (const arma::mat state, arma::mat &actionValue)
 Predict the responses to a given set of predictors. More...

 
void Reset (const size_t inputDimensionality=0)
 Resets the parameters of the network. More...

 
void ResetNoise ()
 Resets noise of the network, if the network is of type noisy. More...

 

Detailed Description


template<typename OutputLayerType = ann::EmptyLoss, typename InitType = ann::GaussianInitialization, typename CompleteNetworkType = ann::FFN<OutputLayerType, InitType>, typename FeatureNetworkType = ann::MultiLayer<arma::mat>, typename AdvantageNetworkType = ann::MultiLayer<arma::mat>, typename ValueNetworkType = ann::MultiLayer<arma::mat>>
class mlpack::rl::DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType >

Implementation of the Dueling Deep Q-Learning network.

For more information, see the following.

@misc{wang2015dueling,
author = {Ziyu Wang, Tom Schaul, Matteo Hessel, Hado van Hasselt,
Marc Lanctot, Nando de Freitas},
title = {Dueling Network Architectures for Deep Reinforcement Learning},
year = {2015},
url = {https://arxiv.org/abs/1511.06581}
}
Template Parameters
OutputLayerTypeThe output layer type of the network.
InitTypeThe initialization type used for the network.
CompleteNetworkTypeThe type of network used for full dueling dqn.
FeatureNetworkTypeThe type of network used for feature network.
AdvantageNetworkTypeThe type of network used for advantage network.
ValueNetworkTypeThe type of network used for value network.

Definition at line 54 of file dueling_dqn.hpp.

Constructor & Destructor Documentation

◆ DuelingDQN() [1/4]

DuelingDQN ( )
inline

Default constructor.

Definition at line 58 of file dueling_dqn.hpp.

References MultiLayer< MatType >::Add().

◆ DuelingDQN() [2/4]

DuelingDQN ( const int  h1,
const int  h2,
const int  outputDim,
const bool  isNoisy = false,
InitType  init = InitType(),
OutputLayerType  outputLayer = OutputLayerType() 
)
inline

Construct an instance of DuelingDQN class.

Parameters
h1Number of neurons in hiddenlayer-1.
h2Number of neurons in hiddenlayer-2.
outputDimNumber of neurons in output layer.
isNoisySpecifies whether the network needs to be of type noisy.
initSpecifies the initialization rule for the network.
outputLayerSpecifies the output layer type for network.

Definition at line 83 of file dueling_dqn.hpp.

References MultiLayer< MatType >::Add().

◆ DuelingDQN() [3/4]

DuelingDQN ( FeatureNetworkType &  featureNetwork,
AdvantageNetworkType &  advantageNetwork,
ValueNetworkType &  valueNetwork,
const bool  isNoisy = false 
)
inline

Construct an instance of DuelingDQN class from a pre-constructed network.

Parameters
featureNetworkThe feature network to be used by DuelingDQN class.
advantageNetworkThe advantage network to be used by DuelingDQN class.
valueNetworkThe value network to be used by DuelingDQN class.
isNoisySpecifies whether the network needs to be of type noisy.

Definition at line 139 of file dueling_dqn.hpp.

References MultiLayer< MatType >::Add().

◆ DuelingDQN() [4/4]

DuelingDQN ( const DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType > &  )
inline

Copy constructor.

Definition at line 156 of file dueling_dqn.hpp.

Member Function Documentation

◆ Backward()

void Backward ( const arma::mat  state,
arma::mat &  target,
arma::mat &  gradient 
)
inline

Perform the backward pass of the state in real batch mode.

Parameters
stateThe input state.
targetThe training target.
gradientThe gradient.

Definition at line 213 of file dueling_dqn.hpp.

References MeanSquaredErrorType< MatType >::Backward().

◆ Forward()

void Forward ( const arma::mat  state,
arma::mat &  actionValue 
)
inline

Perform the forward pass of the states in real batch mode.

Parameters
stateThe input state.
actionValueMatrix to put output action values of states input.

Definition at line 195 of file dueling_dqn.hpp.

◆ operator=()

void operator= ( const DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType > &  model)
inline

Copy assignment operator.

Definition at line 160 of file dueling_dqn.hpp.

◆ Parameters() [1/2]

const arma::mat& Parameters ( ) const
inline

Return the Parameters.

Definition at line 248 of file dueling_dqn.hpp.

◆ Parameters() [2/2]

arma::mat& Parameters ( )
inline

Modify the Parameters.

Definition at line 250 of file dueling_dqn.hpp.

◆ Predict()

void Predict ( const arma::mat  state,
arma::mat &  actionValue 
)
inline

Predict the responses to a given set of predictors.

The responses will reflect the output of the given output layer as returned by the output layer function.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Parameters
stateInput state.
actionValueMatrix to put output action values of states input.

Definition at line 180 of file dueling_dqn.hpp.

◆ Reset()

void Reset ( const size_t  inputDimensionality = 0)
inline

Resets the parameters of the network.

Definition at line 228 of file dueling_dqn.hpp.

◆ ResetNoise()

void ResetNoise ( )
inline

Resets noise of the network, if the network is of type noisy.

Definition at line 236 of file dueling_dqn.hpp.

References NoisyLinearType< MatType >::ResetNoise().


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/reinforcement_learning/q_networks/dueling_dqn.hpp