DoublePoleCart Class Reference

Implementation of Double Pole Cart Balancing task. More...

Classes

class  State
 Implementation of the state of Double Pole Cart. More...

 

Public Types

enum  Action
{
  backward
,
  forward
,
  size

}
 Implementation of action of Double Pole Cart. More...

 

Public Member Functions

 DoublePoleCart (const double m1=0.1, const double m2=0.01, const double l1=0.5, const double l2=0.05, const double gravity=9.8, const double massCart=1.0, const double forceMag=10.0, const double tau=0.02, const double thetaThresholdRadians=36 *2 *3.1416/360, const double xThreshold=2.4, const double doneReward=0.0, const size_t maxSteps=0)
 Construct a Double Pole Cart instance using the given constants. More...

 
void Dsdt (const State &state, const Action &action, arma::vec &dydx)
 This is the ordinary differential equations required for estimation of next state through RK4 method. More...

 
State InitialSample ()
 Initial state representation is randomly generated within [-0.05, 0.05]. More...

 
bool IsTerminal (const State &state) const
 This function checks if the car has reached the terminal state. More...

 
size_t MaxSteps () const
 Get the maximum number of steps allowed. More...

 
size_t & MaxSteps ()
 Set the maximum number of steps allowed. More...

 
void RK4 (const State &state, const Action &action, arma::vec &dydx, State &nextState)
 This function calls the RK4 iterative method to estimate the next state based on given ordinary differential equation. More...

 
double Sample (const State &state, const Action &action, State &nextState)
 Dynamics of Double Pole Cart instance. More...

 
double Sample (const State &state, const Action &action)
 Dynamics of Double Pole Cart. More...

 
size_t StepsPerformed () const
 Get the number of steps performed. More...

 

Detailed Description

Implementation of Double Pole Cart Balancing task.

This is an extension of the existing CartPole environment. The environment comprises of a cart of a cart with two poles of different lengths and masses. The agent is meant to balance the poles by applying force on the cart.

Definition at line 27 of file double_pole_cart.hpp.

Member Enumeration Documentation

◆ Action

enum Action

Implementation of action of Double Pole Cart.

Enumerator
backward 
forward 
size 

Definition at line 91 of file double_pole_cart.hpp.

Constructor & Destructor Documentation

◆ DoublePoleCart()

DoublePoleCart ( const double  m1 = 0.1,
const double  m2 = 0.01,
const double  l1 = 0.5,
const double  l2 = 0.05,
const double  gravity = 9.8,
const double  massCart = 1.0,
const double  forceMag = 10.0,
const double  tau = 0.02,
const double  thetaThresholdRadians = 36 * 2 * 3.1416 / 360,
const double  xThreshold = 2.4,
const double  doneReward = 0.0,
const size_t  maxSteps = 0 
)
inline

Construct a Double Pole Cart instance using the given constants.

Parameters
m1The mass of the first pole.
m2The mass of the second pole.
l1The length of the first pole.
l2The length of the second pole.
gravityThe gravity constant.
massCartThe mass of the cart.
lengthThe length of the pole.
forceMagThe magnitude of the applied force.
tauThe time interval.
thetaThresholdRadiansThe maximum angle.
xThresholdThe maximum position.
doneRewardReward recieved by agent on success.
maxStepsThe number of steps after which the episode terminates. If the value is 0, there is no limit.

Definition at line 118 of file double_pole_cart.hpp.

Member Function Documentation

◆ Dsdt()

void Dsdt ( const State state,
const Action action,
arma::vec &  dydx 
)
inline

This is the ordinary differential equations required for estimation of next state through RK4 method.

Parameters
stateThe current state.
actionThe action taken.
dydxThe differential.

Definition at line 192 of file double_pole_cart.hpp.

References DoublePoleCart::State::Angle(), and DoublePoleCart::State::AngularVelocity().

Referenced by DoublePoleCart::RK4(), and DoublePoleCart::Sample().

◆ InitialSample()

State InitialSample ( )
inline

Initial state representation is randomly generated within [-0.05, 0.05].

Returns
Initial state for each episode.

Definition at line 284 of file double_pole_cart.hpp.

References DoublePoleCart::State::State().

◆ IsTerminal()

bool IsTerminal ( const State state) const
inline

This function checks if the car has reached the terminal state.

Parameters
stateThe desired state.
Returns
true if state is a terminal state, otherwise false.

Definition at line 296 of file double_pole_cart.hpp.

References DoublePoleCart::State::Angle(), Log::Info, and DoublePoleCart::State::Position().

Referenced by DoublePoleCart::Sample().

◆ MaxSteps() [1/2]

size_t MaxSteps ( ) const
inline

Get the maximum number of steps allowed.

Definition at line 322 of file double_pole_cart.hpp.

◆ MaxSteps() [2/2]

size_t& MaxSteps ( )
inline

Set the maximum number of steps allowed.

Definition at line 324 of file double_pole_cart.hpp.

◆ RK4()

void RK4 ( const State state,
const Action action,
arma::vec &  dydx,
State nextState 
)
inline

This function calls the RK4 iterative method to estimate the next state based on given ordinary differential equation.

Parameters
stateThe current state.
actionThe action to be applied.
dydxThe differential.
nextStateThe next state.

Definition at line 233 of file double_pole_cart.hpp.

References DoublePoleCart::State::Data(), DoublePoleCart::Dsdt(), and DoublePoleCart::State::State().

Referenced by DoublePoleCart::Sample().

◆ Sample() [1/2]

double Sample ( const State state,
const Action action,
State nextState 
)
inline

Dynamics of Double Pole Cart instance.

Get reward and next state based on current state and current action.

Parameters
stateThe current state.
actionThe current action.
nextStateThe next state.
Returns
reward, it's always 1.0.

When done is false, it means that the cartpole has fallen down. For this case the reward is 1.0.

Definition at line 154 of file double_pole_cart.hpp.

References DoublePoleCart::State::AngularVelocity(), DoublePoleCart::Dsdt(), DoublePoleCart::IsTerminal(), DoublePoleCart::RK4(), and DoublePoleCart::State::Velocity().

Referenced by DoublePoleCart::Sample().

◆ Sample() [2/2]

double Sample ( const State state,
const Action action 
)
inline

Dynamics of Double Pole Cart.

Get reward based on current state and current action.

Parameters
stateThe current state.
actionThe current action.
Returns
reward, it's always 1.0.

Definition at line 273 of file double_pole_cart.hpp.

References DoublePoleCart::Sample().

◆ StepsPerformed()

size_t StepsPerformed ( ) const
inline

Get the number of steps performed.

Definition at line 319 of file double_pole_cart.hpp.


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-3.2.1/src/mlpack/methods/reinforcement_learning/environment/double_pole_cart.hpp