Acrobot Class Reference

Implementation of Acrobot game. More...

class  Action

class  State

## Public Member Functions

Acrobot (const size_t maxSteps=500, const double gravity=9.81, const double linkLength1=1.0, const double linkLength2=1.0, const double linkMass1=1.0, const double linkMass2=1.0, const double linkCom1=0.5, const double linkCom2=0.5, const double linkMoi=1.0, const double maxVel1=4 *M_PI, const double maxVel2=9 *M_PI, const double dt=0.2, const double doneReward=0)
Construct a Acrobot instance using the given constants. More...

arma::colvec Dsdt (arma::colvec state, const double torque) const
This is the ordinary differential equations required for estimation of nextState through RK4 method. More...

State InitialSample ()
This function does random initialization of state space. More...

bool IsTerminal (const State &state) const
This function checks if the acrobot has reached the terminal state. More...

size_t MaxSteps () const
Get the maximum number of steps allowed. More...

size_t & MaxSteps ()
Set the maximum number of steps allowed. More...

arma::colvec Rk4 (const arma::colvec state, const double torque) const
This function calls the RK4 iterative method to estimate the next state based on given ordinary differential equation. More...

double Sample (const State &state, const Action &action, State &nextState)
Dynamics of the Acrobot System. More...

double Sample (const State &state, const Action &action)
Dynamics of the Acrobot System. More...

size_t StepsPerformed () const
Get the number of steps performed. More...

double Torque (const Action &action) const
This function calculates the torque for a particular action. More...

double Wrap (double value, const double minimum, const double maximum) const
Wrap funtion is required to truncate the angle value from -180 to 180. More...

## Detailed Description

Implementation of Acrobot game.

Acrobot is a 2-link pendulum with only the second joint actuated. Intitially, both links point downwards. The goal is to swing the end-effector at a height at least the length of one link above the base. Both links can swing freely and can pass by each other, i.e., they don't collide when they have the same angle.

Definition at line 28 of file acrobot.hpp.

## ◆ Acrobot()

 Acrobot ( const size_t maxSteps = `500`, const double gravity = `9.81`, const double linkLength1 = `1.0`, const double linkLength2 = `1.0`, const double linkMass1 = `1.0`, const double linkMass2 = `1.0`, const double linkCom1 = `0.5`, const double linkCom2 = `0.5`, const double linkMoi = `1.0`, const double maxVel1 = `4 * M_PI`, const double maxVel2 = `9 * M_PI`, const double dt = `0.2`, const double doneReward = `0` )
inline

Construct a Acrobot instance using the given constants.

Parameters

Definition at line 122 of file acrobot.hpp.

## ◆ Dsdt()

 arma::colvec Dsdt ( arma::colvec state, const double torque ) const
inline

This is the ordinary differential equations required for estimation of nextState through RK4 method.

Parameters
 state Current State. torque The torque Applied.

Definition at line 249 of file acrobot.hpp.

References M_PI.

Referenced by Acrobot::Rk4().

## ◆ InitialSample()

 State InitialSample ( )
inline

This function does random initialization of state space.

Definition at line 213 of file acrobot.hpp.

References Acrobot::State::State().

## ◆ IsTerminal()

 bool IsTerminal ( const State & state ) const
inline

This function checks if the acrobot has reached the terminal state.

Parameters
 state The current State.
Returns
true if state is a terminal state, otherwise false.

Definition at line 225 of file acrobot.hpp.

References Log::Info, Acrobot::State::Theta1(), and Acrobot::State::Theta2().

Referenced by Acrobot::Sample().

## ◆ MaxSteps() [1/2]

 size_t MaxSteps ( ) const
inline

Get the maximum number of steps allowed.

Definition at line 350 of file acrobot.hpp.

## ◆ MaxSteps() [2/2]

 size_t& MaxSteps ( )
inline

Set the maximum number of steps allowed.

Definition at line 352 of file acrobot.hpp.

## ◆ Rk4()

 arma::colvec Rk4 ( const arma::colvec state, const double torque ) const
inline

This function calls the RK4 iterative method to estimate the next state based on given ordinary differential equation.

Parameters
 state The current State. torque The torque applied.

Definition at line 335 of file acrobot.hpp.

References Acrobot::Dsdt().

Referenced by Acrobot::Sample().

## ◆ Sample() [1/2]

 double Sample ( const State & state, const Action & action, State & nextState )
inline

Dynamics of the Acrobot System.

To get reward and next state based on current state and current action. Always return -1 reward.

Parameters
 state The current State. action The action taken. nextState The next state.
Returns
reward, it's always -1.0.

The value of angular velocity is bounded in min and max value.

Definition at line 160 of file acrobot.hpp.

Referenced by Acrobot::Sample().

## ◆ Sample() [2/2]

 double Sample ( const State & state, const Action & action )
inline

Dynamics of the Acrobot System.

To get reward and next state based on current state and current action. This function calls the Sample function to estimate the next state return reward for taking a particular action.

Parameters
 state The current State. action The action taken.
Returns
nextState The next state.

Definition at line 204 of file acrobot.hpp.

References Acrobot::Sample().

## ◆ StepsPerformed()

 size_t StepsPerformed ( ) const
inline

Get the number of steps performed.

Definition at line 347 of file acrobot.hpp.

## ◆ Torque()

 double Torque ( const Action & action ) const
inline

This function calculates the torque for a particular action.

0 : negative torque, 1 : zero torque, 2 : positive torque.

Parameters
 action Action taken.

Definition at line 322 of file acrobot.hpp.

References Acrobot::Action::action, and mlpack::math::Random().

Referenced by Acrobot::Sample().

## ◆ Wrap()

 double Wrap ( double value, const double minimum, const double maximum ) const
inline

Wrap funtion is required to truncate the angle value from -180 to 180.

This function will make sure that value will always be between minimum to maximum.

Parameters
 value Scalar value to wrap. minimum Minimum range of wrap. maximum Maximum range of wrap.

Definition at line 298 of file acrobot.hpp.

Referenced by Acrobot::Sample().

The documentation for this class was generated from the following file:
• /home/jenkins-mlpack/mlpack.org/_src/mlpack-git/src/mlpack/methods/reinforcement_learning/environment/acrobot.hpp