Implementation of Continuous Double Pole Cart Balancing task. More...
Classes  
struct  Action 
Implementation of action of Continuous Double Pole Cart. More...  
class  State 
Implementation of the state of Continuous Double Pole Cart. More...  
Public Member Functions  
ContinuousDoublePoleCart (const double m1=0.1, const double m2=0.01, const double l1=0.5, const double l2=0.05, const double gravity=9.8, const double massCart=1.0, const double forceMag=10.0, const double tau=0.02, const double thetaThresholdRadians=36 *2 *3.1416/360, const double xThreshold=2.4, const double doneReward=0.0, const size_t maxSteps=0)  
Construct a Double Pole Cart instance using the given constants. More...  
void  Dsdt (const State &state, const Action &action, arma::vec &dydx) 
This is the ordinary differential equations required for estimation of next state through RK4 method. More...  
State  InitialSample () 
Initial state representation is randomly generated within [0.05, 0.05]. More...  
bool  IsTerminal (const State &state) const 
This function checks if the car has reached the terminal state. More...  
size_t  MaxSteps () const 
Get the maximum number of steps allowed. More...  
size_t &  MaxSteps () 
Set the maximum number of steps allowed. More...  
void  RK4 (const State &state, const Action &action, arma::vec &dydx, State &nextState) 
This function calls the RK4 iterative method to estimate the next state based on given ordinary differential equation. More...  
double  Sample (const State &state, const Action &action, State &nextState) 
Dynamics of Continuous Double Pole Cart instance. More...  
double  Sample (const State &state, const Action &action) 
Dynamics of Continuous Double Pole Cart. More...  
size_t  StepsPerformed () const 
Get the number of steps performed. More...  
Implementation of Continuous Double Pole Cart Balancing task.
This is an extension of the existing CartPole environment. The environment comprises of a cart with two upright poles of different lengths and masses. The agent is meant to balance the poles by applying force on the cart.
Definition at line 28 of file continuous_double_pole_cart.hpp.

inline 
Construct a Double Pole Cart instance using the given constants.
m1  The mass of the first pole. 
m2  The mass of the second pole. 
l1  The length of the first pole. 
l2  The length of the second pole. 
gravity  The gravity constant. 
massCart  The mass of the cart. 
length  The length of the pole. 
forceMag  The magnitude of the applied force. 
tau  The time interval. 
thetaThresholdRadians  The maximum angle. 
xThreshold  The maximum position. 
doneReward  Reward recieved by agent on success. 
maxSteps  The number of steps after which the episode terminates. If the value is 0, there is no limit. 
Definition at line 117 of file continuous_double_pole_cart.hpp.
This is the ordinary differential equations required for estimation of next state through RK4 method.
state  The current state. 
action  The action taken. 
dydx  The differential. 
Definition at line 192 of file continuous_double_pole_cart.hpp.
References ContinuousDoublePoleCart::Action::action, ContinuousDoublePoleCart::State::Angle(), and ContinuousDoublePoleCart::State::AngularVelocity().
Referenced by ContinuousDoublePoleCart::RK4(), and ContinuousDoublePoleCart::Sample().

inline 
Initial state representation is randomly generated within [0.05, 0.05].
Definition at line 284 of file continuous_double_pole_cart.hpp.
References ContinuousDoublePoleCart::State::State().

inline 
This function checks if the car has reached the terminal state.
state  The desired state. 
Definition at line 296 of file continuous_double_pole_cart.hpp.
References ContinuousDoublePoleCart::State::Angle(), Log::Info, and ContinuousDoublePoleCart::State::Position().
Referenced by ContinuousDoublePoleCart::Sample().

inline 
Get the maximum number of steps allowed.
Definition at line 322 of file continuous_double_pole_cart.hpp.

inline 
Set the maximum number of steps allowed.
Definition at line 324 of file continuous_double_pole_cart.hpp.
This function calls the RK4 iterative method to estimate the next state based on given ordinary differential equation.
state  The current state. 
action  The action to be applied. 
dydx  The differential. 
nextState  The next state. 
Definition at line 233 of file continuous_double_pole_cart.hpp.
References ContinuousDoublePoleCart::State::Data(), ContinuousDoublePoleCart::Dsdt(), and ContinuousDoublePoleCart::State::State().
Referenced by ContinuousDoublePoleCart::Sample().
Dynamics of Continuous Double Pole Cart instance.
Get reward and next state based on current state and current action.
state  The current state. 
action  The current action. 
nextState  The next state. 
When done is false, it means that the cartpole has fallen down. For this case the reward is 1.0.
Definition at line 154 of file continuous_double_pole_cart.hpp.
References ContinuousDoublePoleCart::State::AngularVelocity(), ContinuousDoublePoleCart::Dsdt(), ContinuousDoublePoleCart::IsTerminal(), ContinuousDoublePoleCart::RK4(), and ContinuousDoublePoleCart::State::Velocity().
Referenced by ContinuousDoublePoleCart::Sample().
Dynamics of Continuous Double Pole Cart.
Get reward based on current state and current action.
state  The current state. 
action  The current action. 
Definition at line 273 of file continuous_double_pole_cart.hpp.
References ContinuousDoublePoleCart::Sample().

inline 
Get the number of steps performed.
Definition at line 319 of file continuous_double_pole_cart.hpp.