mlpack  git-master
CyclicalDecay Class Reference

Simulate a new warm-started run/restart once a number of epochs are performed. More...

Public Member Functions

 CyclicalDecay (const size_t epochRestart, const double multFactor, const double stepSize)
 Construct the CyclicalDecay technique a restart method, where the step size decays after each batch and peridically resets to its initial value. More...

 
double EpochBatches () const
 Get the restart fraction. More...

 
double & EpochBatches ()
 Modify the restart fraction. More...

 
double StepSize () const
 Get the step size. More...

 
double & StepSize ()
 Modify the step size. More...

 
void Update (arma::mat &, double &stepSize, const arma::mat &)
 This function is called in each iteration after the policy update. More...

 

Detailed Description

Simulate a new warm-started run/restart once a number of epochs are performed.

Importantly, the restarts are not performed from scratch but emulated by increasing the step size while the old step size value of as an initial parameter.

For more information, please refer to:

@article{Loshchilov2016,
title = {Learning representations by back-propagating errors},
author = {Ilya Loshchilov and Frank Hutter},
title = {{SGDR:} Stochastic Gradient Descent with Restarts},
journal = {CoRR},
year = {2016},
url = {https://arxiv.org/abs/1608.03983}
}

Definition at line 40 of file cyclical_decay.hpp.

Constructor & Destructor Documentation

◆ CyclicalDecay()

CyclicalDecay ( const size_t  epochRestart,
const double  multFactor,
const double  stepSize 
)
inline

Construct the CyclicalDecay technique a restart method, where the step size decays after each batch and peridically resets to its initial value.

Parameters
epochRestartInitial epoch where decay is applied.
multFactorFactor to increase the number of epochs before a restart.
stepSizeInitial step size for each restart.
batchSizeSize of each mini-batch.
numFunctionsThe number of separable functions (the number of predictor points).

Definition at line 55 of file cyclical_decay.hpp.

Member Function Documentation

◆ EpochBatches() [1/2]

double EpochBatches ( ) const
inline

Get the restart fraction.

Definition at line 110 of file cyclical_decay.hpp.

◆ EpochBatches() [2/2]

double& EpochBatches ( )
inline

Modify the restart fraction.

Definition at line 112 of file cyclical_decay.hpp.

◆ StepSize() [1/2]

double StepSize ( ) const
inline

Get the step size.

Definition at line 105 of file cyclical_decay.hpp.

◆ StepSize() [2/2]

double& StepSize ( )
inline

Modify the step size.

Definition at line 107 of file cyclical_decay.hpp.

◆ Update()

void Update ( arma::mat &  ,
double &  stepSize,
const arma::mat &   
)
inline

This function is called in each iteration after the policy update.

Parameters
iterateParameters that minimize the function.
stepSizeStep size to be used for the given iteration.
gradientThe gradient matrix.

Definition at line 74 of file cyclical_decay.hpp.

References M_PI.


The documentation for this class was generated from the following file: