mlpack  git-master
L_BFGS Class Reference

The generic L-BFGS optimizer, which uses a back-tracking line search algorithm to minimize a function. More...

Public Member Functions

 L_BFGS (const size_t numBasis=10, const size_t maxIterations=10000, const double armijoConstant=1e-4, const double wolfe=0.9, const double minGradientNorm=1e-6, const double factr=1e-15, const size_t maxLineSearchTrials=50, const double minStep=1e-20, const double maxStep=1e20)
 Initialize the L-BFGS object. More...

 
double ArmijoConstant () const
 Get the Armijo condition constant. More...

 
double & ArmijoConstant ()
 Modify the Armijo condition constant. More...

 
double Factr () const
 Get the factr value. More...

 
double & Factr ()
 Modify the factr value. More...

 
size_t MaxIterations () const
 Get the maximum number of iterations. More...

 
size_t & MaxIterations ()
 Modify the maximum number of iterations. More...

 
size_t MaxLineSearchTrials () const
 Get the maximum number of line search trials. More...

 
size_t & MaxLineSearchTrials ()
 Modify the maximum number of line search trials. More...

 
double MaxStep () const
 Return the maximum line search step size. More...

 
double & MaxStep ()
 Modify the maximum line search step size. More...

 
double MinGradientNorm () const
 Get the minimum gradient norm. More...

 
double & MinGradientNorm ()
 Modify the minimum gradient norm. More...

 
const std::pair< arma::mat, double > & MinPointIterate () const
 Return the point where the lowest function value has been found. More...

 
double MinStep () const
 Return the minimum line search step size. More...

 
double & MinStep ()
 Modify the minimum line search step size. More...

 
size_t NumBasis () const
 Get the memory size. More...

 
size_t & NumBasis ()
 Modify the memory size. More...

 
template
<
typename
FunctionType
>
double Optimize (FunctionType &function, arma::mat &iterate)
 Use L-BFGS to optimize the given function, starting at the given iterate point and finding the minimum. More...

 
double Wolfe () const
 Get the Wolfe parameter. More...

 
double & Wolfe ()
 Modify the Wolfe parameter. More...

 

Detailed Description

The generic L-BFGS optimizer, which uses a back-tracking line search algorithm to minimize a function.

The parameters for the algorithm (number of memory points, maximum step size, and so forth) are all configurable via either the constructor or standalone modifier functions. A function which can be optimized by this class's Optimize() method must implement the following methods:

  • a default constructor
  • double Evaluate(const arma::mat& coordinates);
  • void Gradient(const arma::mat& coordinates, arma::mat& gradient);
  • arma::mat& GetInitialPoint();

Definition at line 35 of file lbfgs.hpp.

Constructor & Destructor Documentation

◆ L_BFGS()

L_BFGS ( const size_t  numBasis = 10,
const size_t  maxIterations = 10000,
const double  armijoConstant = 1e-4,
const double  wolfe = 0.9,
const double  minGradientNorm = 1e-6,
const double  factr = 1e-15,
const size_t  maxLineSearchTrials = 50,
const double  minStep = 1e-20,
const double  maxStep = 1e20 
)

Initialize the L-BFGS object.

There are many parameters that can be set for the optimization, but default values are given for each of them.

Parameters
numBasisNumber of memory points to be stored (default 5).
maxIterationsMaximum number of iterations for the optimization (0 means no limit and may run indefinitely).
armijoConstantControls the accuracy of the line search routine for determining the Armijo condition.
wolfeParameter for detecting the Wolfe condition.
minGradientNormMinimum gradient norm required to continue the optimization.
maxLineSearchTrialsThe maximum number of trials for the line search (before giving up).
minStepThe minimum step of the line search.
maxStepThe maximum step of the line search.

Member Function Documentation

◆ ArmijoConstant() [1/2]

double ArmijoConstant ( ) const
inline

Get the Armijo condition constant.

Definition at line 100 of file lbfgs.hpp.

◆ ArmijoConstant() [2/2]

double& ArmijoConstant ( )
inline

Modify the Armijo condition constant.

Definition at line 102 of file lbfgs.hpp.

◆ Factr() [1/2]

double Factr ( ) const
inline

Get the factr value.

Definition at line 115 of file lbfgs.hpp.

◆ Factr() [2/2]

double& Factr ( )
inline

Modify the factr value.

Definition at line 117 of file lbfgs.hpp.

◆ MaxIterations() [1/2]

size_t MaxIterations ( ) const
inline

Get the maximum number of iterations.

Definition at line 95 of file lbfgs.hpp.

◆ MaxIterations() [2/2]

size_t& MaxIterations ( )
inline

Modify the maximum number of iterations.

Definition at line 97 of file lbfgs.hpp.

◆ MaxLineSearchTrials() [1/2]

size_t MaxLineSearchTrials ( ) const
inline

Get the maximum number of line search trials.

Definition at line 120 of file lbfgs.hpp.

◆ MaxLineSearchTrials() [2/2]

size_t& MaxLineSearchTrials ( )
inline

Modify the maximum number of line search trials.

Definition at line 122 of file lbfgs.hpp.

◆ MaxStep() [1/2]

double MaxStep ( ) const
inline

Return the maximum line search step size.

Definition at line 130 of file lbfgs.hpp.

◆ MaxStep() [2/2]

double& MaxStep ( )
inline

Modify the maximum line search step size.

Definition at line 132 of file lbfgs.hpp.

◆ MinGradientNorm() [1/2]

double MinGradientNorm ( ) const
inline

Get the minimum gradient norm.

Definition at line 110 of file lbfgs.hpp.

◆ MinGradientNorm() [2/2]

double& MinGradientNorm ( )
inline

Modify the minimum gradient norm.

Definition at line 112 of file lbfgs.hpp.

◆ MinPointIterate()

const std::pair<arma::mat, double>& MinPointIterate ( ) const

Return the point where the lowest function value has been found.

Returns
arma::vec representing the point and a double with the function value at that point.

◆ MinStep() [1/2]

double MinStep ( ) const
inline

Return the minimum line search step size.

Definition at line 125 of file lbfgs.hpp.

◆ MinStep() [2/2]

double& MinStep ( )
inline

Modify the minimum line search step size.

Definition at line 127 of file lbfgs.hpp.

◆ NumBasis() [1/2]

size_t NumBasis ( ) const
inline

Get the memory size.

Definition at line 90 of file lbfgs.hpp.

◆ NumBasis() [2/2]

size_t& NumBasis ( )
inline

Modify the memory size.

Definition at line 92 of file lbfgs.hpp.

◆ Optimize()

double Optimize ( FunctionType &  function,
arma::mat &  iterate 
)

Use L-BFGS to optimize the given function, starting at the given iterate point and finding the minimum.

The maximum number of iterations is set in the constructor (or with MaxIterations()). Alternately, another overload is provided which takes a maximum number of iterations as a parameter. The given starting point will be modified to store the finishing point of the algorithm, and the final objective value is returned.

Template Parameters
FunctionTypeType of the function to be optimized.
Parameters
functionFunction to optimize; must have Evaluate() and Gradient().
iterateStarting point (will be modified).
Returns
Objective value of the final point.

◆ Wolfe() [1/2]

double Wolfe ( ) const
inline

Get the Wolfe parameter.

Definition at line 105 of file lbfgs.hpp.

◆ Wolfe() [2/2]

double& Wolfe ( )
inline

Modify the Wolfe parameter.

Definition at line 107 of file lbfgs.hpp.


The documentation for this class was generated from the following file:
  • /var/www/www.mlpack.org/mlpack-git/src/mlpack/core/optimizers/lbfgs/lbfgs.hpp