[mlpack] Ready to use Models in mlpack (GSOC '21)

[Aakash kaushik]

Hey everyone,

My name is Aakash Kaushik <https://github.com/Aakash-kaushik> and I
have been contributing for some time specifically on the ANN codebase in
mlpack.

And the project idea that is ready to use Models in mlpack peaks my
interest. So initially i would like to propose a data loader and 2 models
for semantic segmentation because i see that the data loaders for image
classification and object detection are already there and including a
couple of models and a data loader in GSOC for semantic segmentation will
open the gates for further contribution of models in all three fields as
they would only need to worry about the model and not loading the data and
also will have some reference models in that field

So the data loader would be capable of taking up image segmentation data
that is the real image, segmentation map, segmentation map to class
mapping, and for the models i am a bit confused as if we want some basic
nets such as U-nets or a combination of both a basic net and state of the
art model, or two state of the art model. Pytorch supports couple of models
in the semantic segmentation fields which are:

1. FCN ResNet50, ResNet101
2. DeepLabV3 ResNet50, ResNet101, MobileNetV3-Large
3. LR-ASPP MobileNetV3-Large

And so i should be able to convert their weights from pytorch to mlpack by
modifying the utility created by kartik dutt which is
mlpack-PyTorch-Weight-Translator
<https://github.com/kartikdutt18/mlpack-PyTorch-Weight-Translator>

I am trying to keep the deliverables to just three which is a data loader
and 2 models as the GSOC period is reduced to just 1.5 months and for these
three things i would have to write tests, documentation and example usage
in the example repository.

And before this work as we are in the process of removing boost visitors
from the ANN codebase and had couple of major changes to the mlpack
codebase the models repo wasn't able to keep up with it so my main goal
before GSOC starts would be to work on the PR that is to  Swap
boost::variant with vtable <https://github.com/mlpack/mlpack/pull/2777> and
then make changes to the code in models repo to adjust the change in boost
visitors, serialization and porting tests to catch2.

I wanted to hear from you if this is the right path and if the number of
deliverables are right for this and help in choosing the exact models that
i should pick that would be the most helpful or beneficial to the library.

Best,
Aakash
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20210316/9585e298/attachment.htm>