mlpack  git-master
ExponentialBackoff Class Reference

Exponential backoff stepsize reduction policy for parallel SGD. More...

Public Member Functions

 ExponentialBackoff (const size_t firstBackoffEpoch, const double step, const double beta)
 Member initializer constructor to construct the exponential backoff policy with the required parameters. More...

 
double StepSize (const size_t numEpoch)
 Get the step size for the current gradient update. More...

 

Detailed Description

Exponential backoff stepsize reduction policy for parallel SGD.

For more information, see the following.

{1106.5730, Author = {Feng Niu and Benjamin Recht and Christopher Re and Stephen J. Wright}, Title = {HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent}, Year = {2011}, Eprint = {arXiv:1106.5730}, }

This stepsize update scheme gives robust 1/k convergence rates to the implementation of parallel SGD.

Definition at line 38 of file exponential_backoff.hpp.

Constructor & Destructor Documentation

◆ ExponentialBackoff()

ExponentialBackoff ( const size_t  firstBackoffEpoch,
const double  step,
const double  beta 
)
inline

Member initializer constructor to construct the exponential backoff policy with the required parameters.

Parameters
firstBackoffEpochThe number of updates to run before the first stepsize backoff.
stepThe initial stepsize(gamma).
betaThe reduction factor. This should be a value in range (0, 1).

Definition at line 50 of file exponential_backoff.hpp.

Member Function Documentation

◆ StepSize()

double StepSize ( const size_t  numEpoch)
inline

Get the step size for the current gradient update.

Parameters
numEpochThe iteration number of the current update.
Returns
The stepsize for the current iteration.

Definition at line 65 of file exponential_backoff.hpp.


The documentation for this class was generated from the following file: