QLearning< EnvironmentType, NetworkType, UpdaterType, PolicyType, ReplayType > Class Template Reference

Implementation of various Q-Learning algorithms, such as DQN, double DQN. More...

Public Types

using ActionType = typename EnvironmentType::Action
 Convenient typedef for action. More...

 
using StateType = typename EnvironmentType::State
 Convenient typedef for state. More...

 

Public Member Functions

 QLearning (TrainingConfig config, NetworkType network, PolicyType policy, ReplayType replayMethod, UpdaterType updater=UpdaterType(), EnvironmentType environment=EnvironmentType())
 Create the QLearning object with given settings. More...

 
 ~QLearning ()
 Clean memory. More...

 
bool & Deterministic ()
 Modify the training mode / test mode indicator. More...

 
const bool & Deterministic () const
 Get the indicator of training mode / test mode. More...

 
EnvironmentType & Environment ()
 Modify the environment in which the agent is. More...

 
const EnvironmentType & Environment () const
 Get the environment in which the agent is. More...

 
double Episode ()
 Execute an episode. More...

 
const NetworkType & Network () const
 Return the learning network. More...

 
NetworkType & Network ()
 Modify the learning network. More...

 
StateTypeState ()
 Modify the state of the agent. More...

 
const StateTypeState () const
 Get the state of the agent. More...

 
double Step ()
 Execute a step in an episode. More...

 
const size_t & TotalSteps () const
 

Detailed Description


template<typename EnvironmentType, typename NetworkType, typename UpdaterType, typename PolicyType, typename ReplayType = RandomReplay<EnvironmentType>>
class mlpack::rl::QLearning< EnvironmentType, NetworkType, UpdaterType, PolicyType, ReplayType >

Implementation of various Q-Learning algorithms, such as DQN, double DQN.

For more details, see the following:

@article{Mnih2013,
author = {Volodymyr Mnih and
Koray Kavukcuoglu and
David Silver and
Alex Graves and
Ioannis Antonoglou and
Daan Wierstra and
Martin A. Riedmiller},
title = {Playing Atari with Deep Reinforcement Learning},
journal = {CoRR},
year = {2013},
url = {http://arxiv.org/abs/1312.5602}
}
Template Parameters
EnvironmentTypeThe environment of the reinforcement learning task.
NetworkTypeThe network to compute action value.
UpdaterTypeHow to apply gradients when training.
PolicyTypeBehavior policy of the agent.
ReplayTypeExperience replay method.

Definition at line 58 of file q_learning.hpp.

Member Typedef Documentation

◆ ActionType

using ActionType = typename EnvironmentType::Action

Convenient typedef for action.

Definition at line 65 of file q_learning.hpp.

◆ StateType

using StateType = typename EnvironmentType::State

Convenient typedef for state.

Definition at line 62 of file q_learning.hpp.

Constructor & Destructor Documentation

◆ QLearning()

QLearning ( TrainingConfig  config,
NetworkType  network,
PolicyType  policy,
ReplayType  replayMethod,
UpdaterType  updater = UpdaterType(),
EnvironmentType  environment = EnvironmentType() 
)

Create the QLearning object with given settings.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Parameters
configHyper-parameters for training.
networkThe network to compute action value.
policyBehavior policy of the agent.
replayMethodExperience replay method.
updaterHow to apply gradients when training.
environmentReinforcement learning task.

◆ ~QLearning()

~QLearning ( )

Clean memory.

Member Function Documentation

◆ Deterministic() [1/2]

bool& Deterministic ( )
inline

Modify the training mode / test mode indicator.

Definition at line 120 of file q_learning.hpp.

◆ Deterministic() [2/2]

const bool& Deterministic ( ) const
inline

Get the indicator of training mode / test mode.

Definition at line 122 of file q_learning.hpp.

◆ Environment() [1/2]

EnvironmentType& Environment ( )
inline

Modify the environment in which the agent is.

Definition at line 115 of file q_learning.hpp.

◆ Environment() [2/2]

const EnvironmentType& Environment ( ) const
inline

Get the environment in which the agent is.

Definition at line 117 of file q_learning.hpp.

◆ Episode()

double Episode ( )

Execute an episode.

Returns
Return of the episode.

◆ Network() [1/2]

const NetworkType& Network ( ) const
inline

Return the learning network.

Definition at line 125 of file q_learning.hpp.

◆ Network() [2/2]

NetworkType& Network ( )
inline

Modify the learning network.

Definition at line 127 of file q_learning.hpp.

◆ State() [1/2]

StateType& State ( )
inline

Modify the state of the agent.

Definition at line 110 of file q_learning.hpp.

◆ State() [2/2]

const StateType& State ( ) const
inline

Get the state of the agent.

Definition at line 112 of file q_learning.hpp.

◆ Step()

double Step ( )

Execute a step in an episode.

Returns
Reward for the step.

◆ TotalSteps()

const size_t& TotalSteps ( ) const
inline
Returns
Total steps from beginning.

Definition at line 107 of file q_learning.hpp.


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-3.2.1/src/mlpack/methods/reinforcement_learning/q_learning.hpp