mlpack  git-master
BarzilaiBorweinDecay Class Reference

Barzilai-Borwein decay policy for Stochastic variance reduced gradient (SVRG). More...

Public Member Functions

 BarzilaiBorweinDecay (const double maxStepSize=DBL_MAX, const double eps=1e-7)
 
void Initialize (const size_t, const size_t)
 The Initialize method is called by SVRG Optimizer method before the start of the iteration update process. More...

 
void Update (const arma::mat &iterate, const arma::mat &iterate0, const arma::mat &, const arma::mat &fullGradient, const size_t numBatches, double &stepSize)
 Barzilai-Borwein update step for SVRG. More...

 

Detailed Description

Barzilai-Borwein decay policy for Stochastic variance reduced gradient (SVRG).

For more information, please refer to:

@incollection{Tan2016,
title = {Barzilai-Borwein Step Size for Stochastic Gradient Descent},
author = {Tan, Conghui and Ma, Shiqian and Dai, Yu-Hong
and Qian, Yuqiu},
booktitle = {Advances in Neural Information Processing Systems 29},
editor = {D. D. Lee and M. Sugiyama and U. V. Luxburg and I. Guyon
and R. Garnett},
pages = {685--693},
year = {2016},
publisher = {Curran Associates, Inc.}
}

Definition at line 40 of file barzilai_borwein_decay.hpp.

Constructor & Destructor Documentation

◆ BarzilaiBorweinDecay()

BarzilaiBorweinDecay ( const double  maxStepSize = DBL_MAX,
const double  eps = 1e-7 
)
inline

Definition at line 50 of file barzilai_borwein_decay.hpp.

Member Function Documentation

◆ Initialize()

void Initialize ( const size_t  ,
const size_t   
)
inline

The Initialize method is called by SVRG Optimizer method before the start of the iteration update process.

Parameters
rowsNumber of rows in the gradient matrix.
colsNumber of columns in the gradient matrix.

Definition at line 63 of file barzilai_borwein_decay.hpp.

◆ Update()

void Update ( const arma::mat &  iterate,
const arma::mat &  iterate0,
const arma::mat &  ,
const arma::mat &  fullGradient,
const size_t  numBatches,
double &  stepSize 
)
inline

Barzilai-Borwein update step for SVRG.

Parameters
iterateThe current function parameter at time t.
iterate0The last function parameters at time t - 1.
gradientThe current gradient matrix at time t.
fullGradientThe computed full gradient.
numBatchesThe number of batches.
stepSizeStep size to be used for the given iteration.

Definition at line 76 of file barzilai_borwein_decay.hpp.


The documentation for this class was generated from the following file: