Conventional Neural Evolution (CNE) is a class of evolutionary algorithms focused on dealing with fixed topology networks. The CNE class implements this algorithm as an optimization technique to converge a given function to minima.
The algorithm works by creating a fixed number of candidates, with random weights. Each candidate is tested upon the training set, and a fitness score is assigned to it. Given the selection percentage of best candidates by the user, for a single generation that many percentage of candidates are selected for the next generation and the rest are removed. The selected candidates for a particular generation then become the parents for the next generation and evolution takes place.
A list of all the sections this tutorial contains.
The CNE class is a simple implementation of the CNE optimizer to converge a given neural network.
Using the CNE class is very simple and can be divided into 3 simple steps:
1) The CNE object is made in which the constructor requires 7 input parameters. The default values and detailed explaination have been discussed in a separate section below.
2) Making a neural network model and giving CNE as an optimizer to train the model. For our test, we will be using a feed forward network or vanilla network from the artificial neural network class.
3) The trained model can then be used by calling:
Given the data to predict in armadillo matrix format. Matrix result is modified and the output of prediction is stored in it.
All the parameters are optional. The default values provided over here are not necessarily suitable for a given function. Therefore it is highly recommended to adjust the parameters according to the problem.
The constructor parameters are as follows -
1) populationSize: The number of candidates in the population. Default value is 500 candidates.
populationSize should be at least greator than or equal to 4.
2) maxGenerations: The maximum number of generations allowed for CNE. Default value is 5000.
Note: the algorithm may terminate in between if the termination conditions specified by the user are met.
3) mutationProb: Probability that a weight will get mutated. The more the the value between [0, 1] the more chances of mutation in link weights. Default value is 0.1.
4) mutationSize: The range of mutation noise to be added. This range is between 0 and mutationSize. Default value is 0.02.
Note: This is not a constant but a range from which the mutation noise will be chosen.
5) selectPercent: The percentage of candidates to select to become the the next generation. Value between 0 and 1. Where 1 represents 100%. Default value is 0.2.
6) tolerance: The final value of the objective function for termination. Not considered if not provided by the user. Default value is 1e-5.
Note: If set to negative value, tolerance will not be taken into consideration.
7) objectiveChange: Minimum change in best fitness values between two consecutive generations should be greater than objectiveChange value. Default value is 1e-5.
Note: If set to negative value, objectiveChange will not be taken into consideration.
Creating a model using mlpack's ANN class is simple and straightforward. Below is an example of a feedforward neural network.
First an object is created with the name
network of type
FFN (feedforward network). Layers can be added by calling the
Add() method and specifying the type of layer and the arguments necessary to construct the layer.
In this example we will be using 2 input nodes, 2 hidden nodes, and 2 output layer nodes. To train the network, we can use the following code:
Train() method takes the following three parameters:
train: The armadillo training data matrix.
Note: Data points are arranged columnwise, where each column represents one data point. Therefore the number of training data provided is the number of columns in the dataset.
labels: The output of the training data in armadillo format.
Note: This is also columnwise as the training dataset matrix.
opt: The type of optimizer. We will be using CNE in this tutorial.
Predict() method can be called after training to obtain the result:
The parameter definitions for
test: armadillo test set matrix in the above test set specified format.
predictors: Will be modified by the model and output based on the test case prediction will be added in this matrix.
In this example we will have two input nodes and the output should be the XOR of the two values. As mentioned before, our network structure is 2 input, 2 hidden and 2 output nodes.
Though CNE stands for Conventional "Neural" Evolution, we have implemented it as a generic optimizer. Therefore, it is able to converge for logistic regression function also.
The code below uses mlpack's
LogisticRegression class, optimizing with CNE (a separate tutorial exists for LogisticRegression).
For further documentation on the CNE class, consult the complete API documentation.