[mlpack] Ready to use Models in mlpack (GSOC '21)

[Marcus Edel]

Hello Aakash,

thanks for the interest in the project and all the contributions; what you proposed
looks quite useful to me and as you already pointed out would integrate really well
with some of the existing functionalities.

I guess for loading segmentation datasets we will stick with a common format e.g.
COCO, and add support for the data loader and potentially add support for other
formats later?

One remark about the scope, you might want to remove one model from the list, and
add a note to the proposal something along the lines of, if there is time left at the end
of the summer, I propose to work on z, but the focus is on x and y.

I hope what I said was useful; please don't hesitate to ask if anything needs clarification.

Thanks,
Marcus

> On 16. Mar 2021, at 00:16, Aakash kaushik <kaushikaakash7539 at gmail.com> wrote:
> 
> Hey everyone,
> 
> My name is Aakash Kaushik <https://github.com/Aakash-kaushik> and I have been contributing for some time specifically on the ANN codebase in mlpack.
> 
> And the project idea that is ready to use Models in mlpack peaks my interest. So initially i would like to propose a data loader and 2 models for semantic segmentation because i see that the data loaders for image classification and object detection are already there and including a couple of models and a data loader in GSOC for semantic segmentation will open the gates for further contribution of models in all three fields as they would only need to worry about the model and not loading the data and also will have some reference models in that field
> 
> So the data loader would be capable of taking up image segmentation data that is the real image, segmentation map, segmentation map to class mapping, and for the models i am a bit confused as if we want some basic nets such as U-nets or a combination of both a basic net and state of the art model, or two state of the art model. Pytorch supports couple of models in the semantic segmentation fields which are:
> 
> 1. FCN ResNet50, ResNet101
> 2. DeepLabV3 ResNet50, ResNet101, MobileNetV3-Large
> 3. LR-ASPP MobileNetV3-Large
> 
> And so i should be able to convert their weights from pytorch to mlpack by modifying the utility created by kartik dutt which is mlpack-PyTorch-Weight-Translator <https://github.com/kartikdutt18/mlpack-PyTorch-Weight-Translator>
> 
> I am trying to keep the deliverables to just three which is a data loader and 2 models as the GSOC period is reduced to just 1.5 months and for these three things i would have to write tests, documentation and example usage in the example repository. 
> 
> And before this work as we are in the process of removing boost visitors from the ANN codebase and had couple of major changes to the mlpack codebase the models repo wasn't able to keep up with it so my main goal before GSOC starts would be to work on the PR that is to  Swap boost::variant with vtable <https://github.com/mlpack/mlpack/pull/2777> and then make changes to the code in models repo to adjust the change in boost visitors, serialization and porting tests to catch2. 
> 
> I wanted to hear from you if this is the right path and if the number of deliverables are right for this and help in choosing the exact models that i should pick that would be the most helpful or beneficial to the library. 
> 
> Best, 
> Aakash  
> _______________________________________________
> mlpack mailing list
> mlpack at lists.mlpack.org
> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20210317/74247daa/attachment.htm>