mlpack  git-master
SnapshotEnsembles Class Reference

Simulate a new warm-started run/restart once a number of epochs are performed. More...

Public Member Functions

 SnapshotEnsembles (const size_t epochRestart, const double multFactor, const double stepSize, const size_t maxIterations, const size_t snapshots)
 Construct the CyclicalDecay technique a restart method, where the step size decays after each batch and peridically resets to its initial value. More...

 
double EpochBatches () const
 Get the restart fraction. More...

 
double & EpochBatches ()
 Modify the restart fraction. More...

 
std::vector< arma::mat > Snapshots () const
 Get the snapshots. More...

 
std::vector< arma::mat > & Snapshots ()
 Modify the snapshots. More...

 
double StepSize () const
 Get the step size. More...

 
double & StepSize ()
 Modify the step size. More...

 
void Update (arma::mat &iterate, double &stepSize, const arma::mat &)
 This function is called in each iteration after the policy update. More...

 

Detailed Description

Simulate a new warm-started run/restart once a number of epochs are performed.

Importantly, the restarts are not performed from scratch but emulated by increasing the step size while the old step size value of as an initial parameter.

For more information, please refer to:

@inproceedings{Huang2017,
title = {Snapshot ensembles: Train 1, get m for free},
author = {Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu,
John E. Hopcroft, and Kilian Q. Weinberger},
booktitle = {Proceedings of the International Conference on Learning
Representations (ICLR)},
year = {2017},
url = {https://arxiv.org/abs/1704.00109}
}

Definition at line 40 of file snapshot_ensembles.hpp.

Constructor & Destructor Documentation

◆ SnapshotEnsembles()

SnapshotEnsembles ( const size_t  epochRestart,
const double  multFactor,
const double  stepSize,
const size_t  maxIterations,
const size_t  snapshots 
)
inline

Construct the CyclicalDecay technique a restart method, where the step size decays after each batch and peridically resets to its initial value.

Parameters
epochRestartInitial epoch where decay is applied.
multFactorFactor to increase the number of epochs before a restart.
stepSizeInitial step size for each restart.
batchSizeSize of each mini-batch.
numFunctionsThe number of separable functions (the number of predictor points).
maxIterationsMaximum number of iterations allowed (0 means no limit).
snapshotsMaximum number of snapshots.

Definition at line 58 of file snapshot_ensembles.hpp.

Member Function Documentation

◆ EpochBatches() [1/2]

double EpochBatches ( ) const
inline

Get the restart fraction.

Definition at line 135 of file snapshot_ensembles.hpp.

◆ EpochBatches() [2/2]

double& EpochBatches ( )
inline

Modify the restart fraction.

Definition at line 137 of file snapshot_ensembles.hpp.

◆ Snapshots() [1/2]

std::vector<arma::mat> Snapshots ( ) const
inline

Get the snapshots.

Definition at line 140 of file snapshot_ensembles.hpp.

Referenced by SnapshotSGDR< UpdatePolicyType >::Snapshots().

◆ Snapshots() [2/2]

std::vector<arma::mat>& Snapshots ( )
inline

Modify the snapshots.

Definition at line 142 of file snapshot_ensembles.hpp.

◆ StepSize() [1/2]

double StepSize ( ) const
inline

Get the step size.

Definition at line 130 of file snapshot_ensembles.hpp.

◆ StepSize() [2/2]

double& StepSize ( )
inline

Modify the step size.

Definition at line 132 of file snapshot_ensembles.hpp.

◆ Update()

void Update ( arma::mat &  iterate,
double &  stepSize,
const arma::mat &   
)
inline

This function is called in each iteration after the policy update.

Parameters
iterateParameters that minimize the function.
stepSizeStep size to be used for the given iteration.
gradientThe gradient matrix.

Definition at line 93 of file snapshot_ensembles.hpp.

References M_PI.


The documentation for this class was generated from the following file: