Activity 16 - Neural Networks

In the previous activities, we have used Euclidean distance and LDA as classifiers for pattern recognition. In this activity, we tried the neural networks. A neural network is a computational model of how neurons in the brain works. Basically, just as the brain does pattern recognition, the neural network is fed with features of known patterns; in which case, it "learns" the rules for classification from the input it receives.
Just as the neurons pass electrical signals to other neurons, this model (McCoulloch, Pitts 1943) essentially works the same way. With input signals x_i, (see Fig.1) a neuron receives weighted inputs(w_i*x_i) and lets the sum of all the signal it receives act on an activation function g. The resulting signal z is then passed to other neurons[1].

Figure 1. Artificial neuron

The same process takes place as the signal is passed from neuron to neuron. This network of neurons is called the neural network.

Figure 2. Neural Network

The neural network is composed of an input layer, a hidden layer, and the output layer. The input layer receives the signal (features) which is passed onto the hidden layer. The hidden layer then passes it to the output layer[1].

Results

Using a code originally written by J. Tugaff [2], we use neural network to classify objects as 1-peso coin, 25-cents coin, long leaf, short leaf or flower using r-g chromaticity and dimension ratio (length/width) as features. (Code needs modification depending on the number of classes and features used.)

Learning process
As training , we used the features of 5 samples/class as inputs to the neural network. It is trained such that it outputs: [1 0 0 0 0] for 1-peso coin, [0 1 0 0 0] for 25-cents coin and so on.

Figure 3. Plot of the distinguishing capability of the neural network with (left) learning rate and (right) training cycles T.

Before the classification, we checked the distinguishing capability (the ability to properly separate classification from other classes)of the neural network as a function of the learning rate and training cycles T. We checked it by using the training set as our test set. We took the average value of the "supposedly 1" outputs. Fig.3(left) shows the average against learning rate. An increasing trend is observed. As learning rate increases, the output approaches to 1 thus the network is able to classify the objects properly. We also observed an increasing trend when T increases. High T improves learning process resulting to better classification.
Based on these results, it is quite a good idea to choose learning rate and T as high as possible. However, this is very time-consuming to do. This will slow down the learning process which makes it impractical.¹ For our case, we use T = 5000 and learning rate = 1 for training.

After training, we tested the training set to verify that the network has "learned". The result of classification is summarized on the table below.

Table 1. Classification of the training set used in the learning process of the neural network.

Here we see that the network is able to identify properly identified the training set hence the network has already learned, we can now use it to classify our test objects.

Classification of test objects
In the table below, we show the result of classification of the test objects. Here, we have achieved a 96% correct classification.

Table 2. Classification of test objects using the neural network

¹ If one, on the other hand, is willing to wait for better results, he might as well choose higher learning rate and T.

I thank Master, Kaye and Thirdy and Irene for very useful help.

I give myself a grade of 10 for doing a good job.

References:
[1] Applied Physics 186 Activity 16 manual
[2] http://cole-ap186.blogspot.com/