mlpack  git-master
AdaptiveStepsize Class Reference

Definition of the adaptive stepize technique, a non-monotonic stepsize scheme that uses curvature estimates to propose new stepsize choices. More...

Public Member Functions

 AdaptiveStepsize (const double backtrackStepSize=0.1, const double searchParameter=0.1)
 Construct the AdaptiveStepsize object with the given function and parameters. More...

 
double BacktrackStepSize () const
 Get the backtracking step size. More...

 
double & BacktrackStepSize ()
 Modify the backtracking step size. More...

 
double SearchParameter () const
 Get the search parameter. More...

 
double & SearchParameter ()
 Modify the search parameter. More...

 
template
<
typename
DecomposableFunctionType
>
void Update (DecomposableFunctionType &function, double &stepSize, arma::mat &iterate, const arma::mat &gradient, const double gradientNorm, const double sampleVariance, const size_t offset, const size_t batchSize, const size_t backtrackingBatchSize, const bool)
 This function is called in each iteration. More...

 

Detailed Description

Definition of the adaptive stepize technique, a non-monotonic stepsize scheme that uses curvature estimates to propose new stepsize choices.

direction.

For more information, please refer to:

@article{De2017,
title = {Big Batch {SGD:} Automated Inference using Adaptive Batch
Sizes},
author = {Soham De and Abhay Kumar Yadav and David W. Jacobs and
Tom Goldstein},
journal = {CoRR},
year = {2017},
url = {http://arxiv.org/abs/1610.05792},
}

Definition at line 40 of file adaptive_stepsize.hpp.

Constructor & Destructor Documentation

◆ AdaptiveStepsize()

AdaptiveStepsize ( const double  backtrackStepSize = 0.1,
const double  searchParameter = 0.1 
)
inline

Construct the AdaptiveStepsize object with the given function and parameters.

The defaults here are not necessarily good for the given problem, so it is suggested that the values used be tailored to the task at hand.

Parameters
backtrackStepSizeThe backtracking step size for each iteration.
searchParameterThe backtracking search parameter for each iteration.

Definition at line 53 of file adaptive_stepsize.hpp.

Member Function Documentation

◆ BacktrackStepSize() [1/2]

double BacktrackStepSize ( ) const
inline

Get the backtracking step size.

Definition at line 117 of file adaptive_stepsize.hpp.

◆ BacktrackStepSize() [2/2]

double& BacktrackStepSize ( )
inline

Modify the backtracking step size.

Definition at line 119 of file adaptive_stepsize.hpp.

◆ SearchParameter() [1/2]

double SearchParameter ( ) const
inline

Get the search parameter.

Definition at line 122 of file adaptive_stepsize.hpp.

◆ SearchParameter() [2/2]

double& SearchParameter ( )
inline

Modify the search parameter.

Definition at line 124 of file adaptive_stepsize.hpp.

◆ Update()

void Update ( DecomposableFunctionType &  function,
double &  stepSize,
arma::mat &  iterate,
const arma::mat &  gradient,
const double  gradientNorm,
const double  sampleVariance,
const size_t  offset,
const size_t  batchSize,
const size_t  backtrackingBatchSize,
const bool   
)
inline

This function is called in each iteration.

Template Parameters
DecomposableFunctionTypeType of the function to be optimized.
Parameters
functionFunction to be optimized (minimized).
stepSizeStep size to be used for the given iteration.
iterateParameters that minimize the function.
gradientThe gradient matrix.
gradientNormThe gradient norm to be used for the given iteration.
offsetThe batch offset to be used for the given iteration.
batchSizeBatch size to be used for the given iteration.
backtrackingBatchSizeBacktracking batch size to be used for the given iteration.
resetReset the step size decay parameter.

Definition at line 75 of file adaptive_stepsize.hpp.


The documentation for this class was generated from the following file: