[mlpack] Updates on Improve Ensemble Support GSoC Project

[RISHABH GARG]

Hey everyone!

Till now, I am having a great summer. Great thanks to my mentors Ryan and
German for their continuous support and help. A huge thanks to the mlpack
community too. Hoping my GSOCer (Is this even a valid word?🤔) friends are
also having a great time too.

I have recently achieved the first big milestone of my project i.e. the
DecisionTreeRegressor. #2905 is complete and under the review process.
Hopefully, in the next week, we may finally get it merged. It's been three
months since I have been working on that PR and now I am feeling very great
and proud of myself that I am able to contribute such a nice method that
almost every mlpack user will use.

It would be awesome if more people could take a look at it and give their
valuable feedback. NG Sai, I remember you earlier mentioned some changes
and I told you that it was not the right time. I think now is the best time
for those suggestions. :)

Some of the highlights of the work done are:-
1. A separate class DecisionTreeRegressor for regression trees.
2. An efficient prefix sum binary split finding algorithm that runs in O(n).
3. With the above optimization, our DecisionTreeRegressor implementation is
giving good competition to sklearn in terms of both performance and
accuracy.
4. Improved memory consumption for the Tree object by removing the use of
`arma::vec` from the splitInfo for regression. (There are some thoughts on
doing some changes to try to improve the same for the existing Decision
Tree Classifier. But this is gonna be a future endeavour.)

Now, the next part is going to be even more exciting. The next milestone is
the XGBoost Regressor. There are going to be some crazy optimizations
involved in this one and It will be fun to implement it. The architecture
and API have been finalised and from tomorrow onwards, I will begin working
on it.

Thanks,
Rishabh Garg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20210627/faf3624a/attachment.htm>