[mlpack] [GSOC '16] : Student interested in Low rank/sparse optimization using Frank-Wolfe

[Ryan Curtin]

On Sat, Mar 12, 2016 at 10:50:49PM +0000, Kwon, Jin Kyoung wrote:
> > Ah, you should probably look at this documentation instead:
> > http://mlpack.org/docs/mlpack-git/doxygen.php
> > The other documentation was very old and I've removed it from the
> > website.
> > If you would like to document the OptimizerType and FunctionType policy
> > classes in a document kind of like
> > http://mlpack.org/docs/mlpack-git/doxygen.php?doc=metrics.html
> > or http://mlpack.org/docs/mlpack-git/doxygen.php?doc=trees.html I'd be
> > happy to take a look over a PR and work with you to get it into the main
> > codebase.
> 
> I think this task is definitely something I'd be interested in! I'd
> love to learn Doxygen and writing annotated C++ files. To confirm,
> would the task be adding "optimizers.hpp" and "functions.hpp" under
> "doc/policies"? I generated local files by running "doxygen" command,
> and was wondering what our config file that doxygen uses to generate
> files. Is it the "Doxyfile" in the top directory? I see that it parses
> all *.hpp files, so I wouldn't need to change the config file, right?

Yes, the Doxyfile is the configuration for doxygen.  You can generate
the documentation either by running doxygen like you did, or by running
'make doc' in a build directory.

Making some documentation in optimizers.hpp and functions.hpp would be
appreciated.

> > The neural network code can be found in src/mlpack/methods/ann/.  Looks
> > like the changes happened here:
> Thank you so much for pointing this out to me! The downloaded copy
> does not include ann module. Was it left out for a particular purpose? 

It seems like you downloaded a release (like 2.0.0 or 2.0.1) instead of
the current git master branch.  The releases don't contain the ANN code
since it isn't quite ready for release yet.

> I also had a question regarding aug_lagrangian module. The optimizer
> was wrapped as a function (aug_lagrangian_function) to be usable by
> simple optimizers like L-BFGS (referring to comments on line 23-37 in
> aug_lagrangian_function.hpp). But aug_lagrangian.hpp uses L-BFGS
> optimizer (referring to commends on line 5-7 in aug_lagrangian.hpp),
> and there doesn't seem to be wrapper functions for lbfgs.hpp... Will
> you help me understand how do they work in conjunction with another?
> Also, do you see Frank-Wolfe optimizer being usable by other
> optimizers? If so, should wrapper classes for it be created?

The augmented Lagrangian optimizer is a bit different: it optimizes
functions with constraints using L-BFGS, which requires a reformulation
of the function to be optimized that includes the constrains (this is
what the AugLagrangianFunction is).  Not all optimizers do this.  If you
are interested in working on the Frank-Wolfe optimizer, you should focus
on reading about the non-constrained optimizers, like L-BFGS, SGD, SA,
and RMSprop, and providing an interface that looks like those.

Please let me know if I can clarify anything.

Thanks,

Ryan

-- 
Ryan Curtin    | "This is how Number One works!"
ryan at ratml.org |   - Number One