[mlpack] Layer with a regularizer: should it work?

[Marcus Edel]

Hello Ryan,

the network code is currently based on boost::variant, which requires to
pass the types of the layers at compile-time, we are about to change that
see https://github.com/mlpack/mlpack/pull/2777 for more details. That said
the code below works with the latest GitHub version.

 #include <mlpack/core.hpp>
#include <mlpack/methods/ann/layer/layer.hpp>
#include <mlpack/methods/ann/loss_functions/mean_squared_error.hpp>
#include <mlpack/methods/ann/ffn.hpp>
     #include <mlpack/methods/ann/regularizer/lregularizer.hpp>

FFN<NegativeLogLikelihood<>, RandomInitialization, Linear<arma::mat,
arma::mat, LRegularizer<2>>> model;
model.Add<Linear<arma::mat, arma::mat, LRegularizer<2>>>(24 * 2 * 3, 4,
LRegularizer<2>(0.1));

Once #2777 is ready, it will make adding new layers a lot easier, but
until than you have to pass a linear layer with regularization as another
template parameter.

Thanks,
Marcus

Hi all,
>
> I have tried adding a L2Regularizer to a Linear ANN layer instead of the
> default NoRegularizer.
> Making the default NoRegularizer explicit works:
> FFN<NegativeLogLikelihood<>, RandomInitialization> model;
> model.Add<Linear<arma::mat, arma::mat, NoRegularizer>>(24 * uInRows *
> uInCols, uNumCategories, NoRegularizer());
>
> However, the following produces a slew of incomprehensible compiler errors:
> model.Add<Linear<arma::mat, arma::mat, L2Regularizer>>(24 * uInRows *
> uInCols, uNumCategories, L2Regularizer(0.1));
>
> The error messages end with:
>
> mlpack::ann::TransposedConvolution<mlpack::ann::NaiveConvolution<mlpack::ann::ValidConvolution>,mlpack::ann::NaiveConvolution<mlpack::ann::ValidConvolution>,mlpack::ann::NaiveConvolution<mlpack::ann::ValidConvolution>,arma::mat,arma::mat>
> *,mlpack::ann::WeightNorm<arma::mat,arma::mat>
> *,mlpack::ann::MoreTypes>>>::push_back(const _Ty &)': cannot convert
> argument 1 from 'LayerType *' to 'const _Ty &'
> mlpack-3.4.2\src\mlpack\methods\ann\ffn.hpp    290
>
> Should I expect this to work? I don't see any use of regularizers in the
> examples. However, the LRegularizer code is in the library so I imagine
> I could be using this incorrectly?
>
> Thanks for any help.
> Best regards,
> Ryan
>
>
> Disclaimer: http://www2.peralex.com/disclaimer.html
>
>
> _______________________________________________
> mlpack mailing list
> mlpack at lists.mlpack.org
> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20210331/ed08e365/attachment.htm>