mlpack  git-master
SMORMS3Update Class Reference

SMORMS3 is an optimizer that estimates a safe and optimal distance based on curvature and normalizing the stepsize in the parameter space. More...

Public Member Functions

 SMORMS3Update (const double epsilon=1e-16)
 Construct the SMORMS3 update policy with given epsilon parameter. More...

 
double Epsilon () const
 Get the value used to initialise the mean squared gradient parameter. More...

 
double & Epsilon ()
 Modify the value used to initialise the mean squared gradient parameter. More...

 
void Initialize (const size_t rows, const size_t cols)
 The Initialize method is called by SGD::Optimize method with UpdatePolicy SMORMS3Update before the start of the iteration update process. More...

 
void Update (arma::mat &iterate, const double stepSize, const arma::mat &gradient)
 Update step for SMORMS3. More...

 

Detailed Description

SMORMS3 is an optimizer that estimates a safe and optimal distance based on curvature and normalizing the stepsize in the parameter space.

It is a hybrid of RMSprop and Yann LeCun’s method in "No more pesky learning rates".

For more information, see the following.

@misc{Funk2015,
author = {Simon Funk},
title = {RMSprop loses to SMORMS3 - Beware the Epsilon!},
year = {2015}
url = {http://sifter.org/~simon/journal/20150420.html}
}

Definition at line 37 of file smorms3_update.hpp.

Constructor & Destructor Documentation

◆ SMORMS3Update()

SMORMS3Update ( const double  epsilon = 1e-16)
inline

Construct the SMORMS3 update policy with given epsilon parameter.

Parameters
epsilonValue used to initialise the mean squared gradient parameter.

Definition at line 46 of file smorms3_update.hpp.

Member Function Documentation

◆ Epsilon() [1/2]

double Epsilon ( ) const
inline

Get the value used to initialise the mean squared gradient parameter.

Definition at line 95 of file smorms3_update.hpp.

◆ Epsilon() [2/2]

double& Epsilon ( )
inline

Modify the value used to initialise the mean squared gradient parameter.

Definition at line 97 of file smorms3_update.hpp.

◆ Initialize()

void Initialize ( const size_t  rows,
const size_t  cols 
)
inline

The Initialize method is called by SGD::Optimize method with UpdatePolicy SMORMS3Update before the start of the iteration update process.

Parameters
rowsNumber of rows in the gradient matrix.
colsNumber of columns in the gradient matrix.

Definition at line 56 of file smorms3_update.hpp.

◆ Update()

void Update ( arma::mat &  iterate,
const double  stepSize,
const arma::mat &  gradient 
)
inline

Update step for SMORMS3.

Parameters
iterateParameter that minimizes the function.
stepSizeStep size to be used for the given iteration.
gradientThe gradient matrix.

Definition at line 71 of file smorms3_update.hpp.


The documentation for this class was generated from the following file:
  • /var/www/www.mlpack.org/mlpack-git/src/mlpack/core/optimizers/smorms3/smorms3_update.hpp