WORK OF BACKPROPAGATION AND LEVENBERG-MARKARDT ALGORITHMS FOR RECOGNITION ON BIOMETRICS DATA

In this work is presented a study of two algorithms for patterns recognition and classification to solving a specific application task. Three criteria are studied and compared: performance (execution time), the number of errors in classification and the influence of the selection and structuring of data. AMS Subject Classification: 62H30, 68T10


Introduction
The development of methods for solving recognition and data analysis tasks includes many software tools and a variety of methods.This provides the option to select the algorithm with possible exact solution.The situation in which we have several alternative algorithms available is standard.Therefore, it is necessary to examine the performance of the algorithms on real data base, formed to solve a specific task.In the recognition tasks a notion of similarity or difference for deciding to belong to a particular class is used.The necessary information is contained both in the individual features that the objects possess and in their various combinations [1].In this work three criteria are studied and compared: performance (execution time), the number of errors in classification and the influence of the selection and structuring of data.
To achieve the objectives of the study the following tasks are solved: 1. Create a database; 2. Conduct experiments with Backpropagation (BP) algorithms and Levenberg-Markardt (LM), realized with the Toolbox of Matlab and PRTools.

The Toolbox
For the realization of the practical examination it is used PRTools-Matlab based pattern recognition toolkit that supports around 200 custom tool for tasks from the field of pattern recognition.PRTools include procedures for data generation, classifiers, training functions for the selection, evaluation, cluster analysis, visualization [2], [3] [4].

Algorithm with Back Propagation of the Error (Backpropagation)
The algorithm is implemented using a neural classifier BPXNC (Back-Propagationtrained Neural net feed forward Classifier).The underlying algorithm used works with radiating signals ahead (feed forward) and performing an error correction in the opposite direction (back propagation of errors).It is used for simple dual layer or multilayer neural networks.The neural networks with back propagation of errors are made up of several layers of neurons: an input layer, a few hidden layer and one output layer.Each neuron receives signals from all neurons of the preceding layer and sends the output signals (processed) to all neurons of the next layer.The resulting (blank) image of a neuron is the sum of the products of the network and the output signals of the preceding layer and it is processed further by smoothing (squashing) function to obtain the source image in the range [0.00-1.00].[10], [11].
The number of i/o neurons matches the dimensionality of the i/o vector.
In order the images to be classified correctly, it is necessary for the neural net to be trained with the appropriate volume and representatively of the sample images, called the learning sample (learning set).The input vectors (patterns) are omitted through the network, as the resulting output vectors are compared with the target output vectors.

Levenberg-Markardt Algorithm (LM)
The algorithm is a modification of the Backpropagation algorithm upgrade and is implemented using the classification function LMNC (Levenberg-Markardt Neural Classifier).What is used is a neural network with N hidden layers with N(I) units in the layer I, computed for the dataset.Training is stopped after a number of iterations (at least 50) or if the iteration number exceeds twice that of the best classification result.The Levenberg-Markardt (LM) algorithm is an iterative technique that locates the minimum of a multivariate function that is expressed as the sum of squares of non-linear real-valued functions.It has become a standard technique for non-linear least-squares problems, widely adopted in a broad spectrum of disciplines.LM can be thought of as a combination of steepest descent and the Gauss-Newton method.When the current solution is far from the correct one, the algorithm behaves like a gradient descent method: slow, but guaranteed to 1 converge.When the current solution is close to the correct solution, it becomes a Gauss-Newton method [7], [8], [9], [10].

The Database Generating
The database is created on MS Excel with real data obtained from the physical measured children indicators.Database is converted to Matlab mat files and contained information for children physical development from the age of 3 to 6 data regarding weight, height, head circumference, waist and a tour of the right hand.The birthplace is related to three major groups: Sofia, towns, villages.A fragment of a base, established for the purpose of the study with the help of MS Excel is presented in Fig. 1.
Training and classification functions are built into the PR Tools [10], [11].Input and output parameters are by default.The algorithms are compared on the same input database.
Procedures used in the program realization: • Procedure GENDATB-Generation of a 2-dimensional 2-class dataset A of N objects with a banana shaped distribution.The data is uniformly • Procedure GENDATC -Generation of a K-dimensional 2-class dataset A of N objects.Both classes are spherically Gaussian distributed.Class 1 has the identity matrix as a covariance matrix and mean U.If U is a scalar then [U,0,0,..] is used as class mean.Class 2 has also the identity matrix as a covariance matrix, except for a variance of 4 for the first two features.Its mean is 0. Class priors are P(1) = P(2) = 0.5.
• Procedure GENDATS -Generation of a K-dimensional 2-class dataset A of N objects.Both classes are Gaussian distributed with identity matrix as covariance matrix.Their means are on a distance D. Class priors are P(1) = P(2) = 0.5.
• Procedure GENDATD -Generation of a K-dimensional 2-class dataset A of N objects.Class variances are very different for the first two dimensions.Separation is thereby for small sample sizes, 'difficult'.D1 is the difference between the means for the first feature, D2 is the difference between the means for the second feature.In all other directions the means are equal.
The two covariance matrices are equal with a variance of 1 in all directions except for the second feature, which has a variance of 40.The first two feature are rotated over 45 degrees to construct a strong correlation.Class priors are P(1) = P(2) = 0.5.

Study and Comparison of the Performance of Algorithms
The algorithms can be trained on the same database and are com-pared by: • classifiers; • mean squared error (MSE); • classification error -number of incorrectly recognized patterns for each class [12] ; • algorithm operation time (in seconds).
The research results are presented in graphical and tabular form.The work of procedures Gendatc, Gendatb, Gendatd and Gendatb are illustrated with the relevant figures.The circle, which surrounds the image in each figure, is a marker showing the number of the selected object and its affiliation to a particular class.
From Table 1 we can see that the optimal criteria for the Backpropagation algorithm have been achieved in the implementation of the procedure Gendatd with the parameter MSE-0.0619 and a minimum number of incorrectly classified patterns -5 for class men and 7 for women class and the parameter time -2.9253 on the Gendats.
From Table 2 we can see that "best" criteria are achieved in terms of LM algorithm on the Gendatb procedure with the criteria MSE-0.0567 and a minimum number of incorrectly classified under Gendatc, respectively 5 class men and 7 women class.In terms of criteria time-3.9263seconds is reached when using the Gendatc procedure.

Conclusions
The aim of the practical part is a comparison and study of pattern recognition algorithms Backpropagation and Levenberg-Markardt and their behaviour on the same database.
Aiming at better presentation of the survey results they are presented in comparative tables with the studied parameters defined according to different procedures.
It can be seen that the Backpropagation algorithm (Table 3, procedure Gendatad) with a minimal number of incorrect classified patterns and minimal error (MSE) works slower then Levenberg-Markardt (Table 4).The Backpropagation algorithm works most quickly with the Gendats procedure, but the number of incorrect classified patterns and MSE are greatest.The number of incorrect classified patterns and MSE are smallest for Levenberg-Markardt algorithm (Table 3, procedure Gendatab), but it runs approximately 1.3 times slower in comparison a minimum time of Backpropagation algorithm (Table 4, Procedure Gendats).
Table 3 shows that the smallest one is the mean squared error (MSE) in the LM algorithm and the Gendatb procedure.From the detailed analysis of the number of errors in recognition, it appears that the best result is achieved with the Levenberg-Markardt algorithm.The Backpropagation algorithm is faster in the procedure Gendats.Algorithm, Levenberg-Markardt works more slowly, but in particular for the purposes of the study it is also the preferred algorithm in Gendatb procedure -the least number of incorrectly recognized patterns and After the study it can be concluded that the Levenberg-Markardt algorithm is better suited for the purpose of the task.

Figure 1 :
Figure 1: Fragment from the database

Figure 2 :
Figure 2: A graphical representation of the work of the Gendatd procedure and BP algorithm.

Figure 3 :
Figure 3: A graphical representation of the work of the Gendatc procedure and BP algorithm.

Figure 4 :
Figure 4: A graphical representation of the work of the Gendats procedure and BP algorithm.

Figure 5 :
Figure 5: A graphical representation of the work of the Gendatb procedure and BP algorithm.

Figure 6 :
Figure 6: A graphical representation of the work of the Gendatd procedure and LM algorithm.

Figure 7 :
Figure 7: A graphical representation of the work of the Gendats procedure and BP algorithm.

Figure 8 :
Figure 8: A graphical representation of the work of the Gendatc procedure and LM algorithm.

Figure 9 :
Figure 9: A graphical representation of the work of the Gendatb procedure and BP algorithm.

Table 1 :
Comparison criteria of Backpropagation algorithm

Table 4 :
Comparison of the results in fast response (Time/s)