[mlpack] Query regarding constrained and unconstrained methods

[Ryan Curtin]

On Wed, Jan 17, 2018 at 01:39:50PM +0100, Marcus Edel wrote:
> > I apologize if I misunderstood what constrained problems are, but can't we apply
> > constraints to the methods already present in "src/mlpack/methods/*" directory?
> > Or, are these unrelated? In the latter case, are there some specialized methods
> > for constrained problems that need to be implemented for this project?
>
> Currently, mlpack does not implement an optimizer that can handle constrained
> problems. So for example, if you like to solve the constrained (cube, line)
> Rosenbrock function:
> 
> f(x, y) = (1 - x)^2 + 100(y - x^2)^2
> 
> with constraints (x - 1)^3 - y +1 < 0 and x + y - 2 < 0
> 
> Currently, there is no structure to represent the problem and there is no
> optimizer that can solve the constrained problem. Comming up with a structure is
> one part of the project implementing an optimizer (PSO) that can handle
> constrained problems is the other part. But as pointed out in the project idea,
> it's recommended to start with a PSO implementation for unconstrained problems
> and to extend the work later on.

There are a couple of one-off implementations that support various types
of constraints, but Marcus is right that there is no good structure to
represent the problem and no unified abstraction to solve it.

The Frank-Wolfe optimizer written by Chenzhe Diao as a part of GSoC 2017
does support some constrained problems: src/mlpack/core/optimizers/fw/

And the SDP (semidefinite program) solvers have support for constraints,
although that is in the context of SDPs: src/mlpack/core/optimizers/sdp/

I suspect it would be worth taking a look at those and then thinking
about a nice abstraction that could work for both the general problems
suggested in the GSoC idea and also the existing optimizers that we
already have. :)

-- 
Ryan Curtin    | "Do they hurt?"
ryan at ratml.org |   - Jessica 6