Neural Evolution Algorithms for NES games - Week 6 and 7 Progress

these 2 weeks on writing some more test cases for the CMAES algorithm. I faced a lot of difficulties along the way also and had to spend a lot of time on errors. i implemented logistic regression using CMAES which is working well and is able to classify accurately. the rosenbrock function which was used as a test case only converges for 2 dimensions giving accurate value and very close to zero optimization. The test case fails for functions having higher dimension. I tried a lot of different alterations and spent a lot of time in fixing this but i mostly ended up with flat fitness due to which convergence stops in between.

the next was implementing the CMAES optimizer on neural networks. At very high number of iterations the CMAES is able to optimize the vanilla network but fails at small number of evaluations which it is supposed to do because it will be used in almost real time. I tried to fix this using tuning the parameters of the function but nothing happened. Im still working on this and trying to get it converge fast.

If that does not happen in the next 1 days. I will try to rewind the whole commit to the stage where armadillo library was not implemented and carefully change the things to correct the error, because before when it was implemented in purely C language, I was able to converge rosenbrock for higher dimensions as well. this looks promising and if needed i will spend the time it will take to do so.

Moreover, i was also in busy in my campus placement last week for software engineering job and due to which I was not able to give a lot of time in last week. Finally I am placed now. I am 2.5 weeks behind schedule and will have to pace if selected in second evaluation to complete my gsoc project before time.

more...

Neural Evolution Algorithms for NES games - Week 4 and 5 Progress

All this time me and Marcus were improving the CMAES code using armadillo functions. We optimised everything to armadillo library, alot of changes are made to the library and now its written like other mlpack optimizers. The algorithm is a bit complex and I struggled with errors a lot of time. This also includes more than 150 style checks corrected and making the test case pass for the algorithm. finally everything looks correct for now and all builds are passing.

During this time ive also made a blog post on CMAES algorithm on medium which u can see here medium blog post

the next week will be spend adding more test cases like logistic regression and rosenbrock function and also testing the code on a neural network.

more...

Neural Evolution Algorithms for NES games - Week 3 Progress

My last week totally went on fixing the CMAES code. It took me a lot of time as the code was a bit long. But fortunately it works now. Also certain changes are made for it to support armadillo mat and vec that the previous memory pointer passing.

this PR is not yet merged but will be very soon (fingers crossed).

BOOST test code is also made referring the test code of stochastic gradient descent..

The next work is to implement the neural network code to do the real evoultion on games. it will use the ann code in the MLPack library in conjuction with the cmaes optimizer.

more...

Neural Evolution Algorithms for NES games - Week 2 Progress

A new optimizer is made called CMAES which works like the rest of the optimizer that MLPack implements

it takes as parameters -

1) number of dimensions we are working on

2) Initial start position of optimization

3) Initial standard Deviations

the rest of the parameters are computed automatically which is convinient as the algorithm is too sensitive to those parameters and default parameters are fine tuned and found according to an implementation and differs from code to code.

the user can if want set some parameters for which appropriate getters and setters are made

moreover the most important parameters that user finds to change according to application is the algorithms stop criteria for which appropirate getters and setters are made.

the above mentioned things are all coded and two important small changes remains that are -

1) making an optimize() function according to mlpack way of optimization

2) evaluate() function that is used to input the function to optimize

moreover Boost test remains of which i am not aware much

the above 2 things mentioned will take me one more day to complete.. Also then code clean and passing all 3 builds remains with changes to be made specified by my mentor.

My next weeks goal is to get familiarize with the ann code base and to be able to implement this new optimizer successfully with good results. Or maybe if my mentor suggests to write some tests i'll be happy to start with that.. :)

more...

Neural Evolution Algorithms for NES games - Week 1 Progress

My work is to implement neural evolution algorithms that can be furthur extended to NES games and other artificial neural network optimization. Later on benchmarking and comparing various algorithms like CNE , NEAT, Hyper-NEAT and CMAES.

I have gone through CMA-ES algorithm for neural evolution learning and have decided to implement it as an optimizer after discussing it with my mentor.

Went through the research paper by Mr. Hansen for CMAES understanding this algorithm deeper which took a lot of time.

Went through doxygen documentation and implementation.

Read the existing optimizer codes for eg. stochastic gradient descend.

Read the ann code as well.

Started coding for CMAES.

and the task for next week is to complete CMAES along with tests.

more...