Euro Exchange Rate Forecasting with Differential Neural Networks with an Extended Tracking Procedure

This paper is aimed at developing a new kind of non-parametrical artificial neural network useful to forecast exchange rates. To do this, we departure from the so-called Differential or Dynamic neural Networks (DNN) and extend the tracking procedure. Under this approach, we examine the daily closing values of the exchange rates of the Euro against the US dollar, the Japanese yen and the British pound. With our proposal, Extended DNN or EDNN, we perform the tracking procedure from February 15, 1999, to August 31, 2013, and, subsequently, the forecasting procedure from September 2 to September 13, 2013. The accuracy of the obtained results is remarkable, since the percentage of the error in the predicted values is within the range from 0.001% to 0.69% in the forecasting period. JEL Classification: F31, C45, C53.


Introduction
Many financial phenomena are difficult to be modeled because they deal with many uncertainty sources and internal and external disturbances.The modeling of these phenomena, usually, depends strongly on the required inputs and the fragile assumptions of linearity and normality, among others.One possibility to deal with these phenomena is to consider them as a black box, with some humility, considering that only input and output information is supposed to be measurable avoiding unrealistic assumptions.Under this situation, Artificial Neural Networks (ANNs) provide appropriate identification properties in the presence of uncertainty or external disturbances.Two types of ANN are known in the specialized literature: one called passive neural network that uses the back-propagation technique as the learning law, and the other called differential neural network (DNN) that uses Lyapunovs stability theorem to develop the learning law (Poznyak et al. 2001).The first one deals with the class of global optimization problems trying to adjust the weights of the ANN to minimize the identification error.The second approach exploits the feedback properties of the DNN that avoids many of the problems related to global extreme search, converting the learning (training) process into an adequate feedback design.If the mathematical model of a considered phenomenon is incomplete or only partially known, the DNN approach provides an effective instrument to deal with a wide spectrum of problems such as identification, state estimation, and trajectory tracking (Poznyak et al., 1999).Hence, this seems to be an appropriate solution for the control design in financial analysis.The main reason to use the DNN approach developed by Poznyak et al. (2001) is related with their continuous representation (as a nonlinear differential equation), which appears as a more realistic tool in order to obtain a complete dynamical description of the complex financial systems.According to Cabrera and Ortiz (2012), the use of ANN as a technique to simulate and describe the behavior of nonlinear dynamical systems has provided suitable results, with the advantage that its implementation and operation is relatively easy.It has also been observed that ANNs provide suitable results for complex input values and to identify patterns of behavior, which with other statistical or econometric techniques would not be possible to obtain.The use of artificial neural networks as a technique to estimate the behavior of nonlinear systems describing financial phenomena was initially proposed by Haykin (1999) and McNellis (2005).They exhibit several applications in using artificial neural networks in problems of financial nature.Other papers dealing with applications of ANNs in financial issues are, for instance: Chen  In their work, they modeled the trading of the EUR-USD exchange rate at the ECB.To do this, they used four types of neural networks to model the exchange rate between the Euro and the US Dollar.Their found that the best results obtained were those from Higher Order Neural Networks (HONN); however, the disadvantage of this technique is that it is quite complex from the mathematical and computational points of view.Finally, Yu et al. (2007) presented several alternatives derived from ANN to carry out forecasting of exchange rates of several currencies. 1 Another important work to mention is that from Hu and Zhang (1997) who examined the effect of the number of input and hidden nodes, the size of the training sample, and the enhanced output sample in an ANN model.They used the daily exchange rate of the British Pound against the US Dollar, and found that neural networks outperform linear models, particularly when the forecast horizon is short.They also showed that the number of input nodes have a greater impact on the accuracy of the results than that of the number of hidden nodes; however, even using a large number of observations they fail to reduce the prediction errors.In a more recent paper, Singh and Kumar (2010) analyzed the accuracy in the forecast on the calculation of stock prices using ANN.They compared the results with several econometric techniques.Surprisingly, the results were better using the ANN.Later, Kishore and Prasad (2011) proposed an ANN model that allows them to obtain appropriate results in forecasting stock prices for one day and also for two months.Finally, Bilbrey and Riley (2011) developed an ANN model to simulate trading strategies in the stock market.It is also important to bring up the work of Zhang and Berardi (2001) that used neural networks to obtain predictions on the exchange rate between the British pound and the US dollar.They established a systematic partition for constructing neural network ensembles.They found that the models formed by sets of different neural structures can consistently improve the predictions obtained with the use of a single network.
They also showed that the networks based on different sets of data partitions are more efficient than those developed with the training data set to carry out the prediction of values outside the sample.A prominent feature in the papers mentioned above is that all of them use discrete artificial neural networks as a prediction technique.These ANNs have some limitations in their use either as monitoring in the dynamic system behavior or as forecasting mechanisms.Another common limitation of these technics is the amount of data required to establish a period of learning for the neural network, usually a discrete neural network requires at least 500 historical data (two years) to establish a pattern of appropriate learning (McNellis 2005).Furthermore, the speed of response to changes in the trend behavior of the system is relatively low, which may cause a delay in the perception of the system behavior and, consequently, on the forecasted values.To offset these limitations of discrete ANNs in the late 90s a class of continuous ANN called Differential or Dynamic Neural Networks (DNN) were introduced by Poznyak et al. (2001).In this type of networks learning rules are based on the use of Lyapunov stability theory, which among other things allows obtaining the error convergence more accelerated with a less number of iterations than those from ANN.This has allowed the learning process in the network to be reduced significantly; as we mentioned before, a discrete ANN requires at least 500 data to establish one appropriate learning pattern.In the case of DNN, this number is fifteen times smaller; in such a way that the learning period is shorter and then it can be exploited further the potential of networks monitoring and forecasting (Ortiz et al. 2012a).The DNNs have been also applied in biotechnology, especially on fermentation processes as in Cabrera (2007), or in the assessment for drugs dose in cancer treatment in Aguilar (2006).Also in the case of financial applications, we can find very Due to the inherent nonlinearity and the incomplete description of the financial phenomena, the DNN may provide an interesting possibility to be explored.However, the control function design could be a very inefficient process, especially if the performance index optimization is used for tracking the error minimization (Isidori, 1999).In order to amend this limitation, this paper develops a new non-parametric dynamic neural network, based on an identification scheme, where the error convergence between the states of the mathematical model and the actual states (financial time series) is the main objective.In this paper, we provided a new procedure for identification (tracking) extending the DNN approach (from now on called Extended DNN or EDNN).The proposal is useful to analyze, describe and predict values of daily closing prices of the Euro exchange rate versus the US dollar, the Japanese yen and the British pound.The time series for monitoring or description are considered from February 15, 1999, to August 31, 2013, and for prediction are carried out from September 2 to September 13, 2013.The results have shown remarkable accuracy for tracking and predicting the daily closing values over two weeks out of sample time.
This paper is organized as follows: Section 2, we, briefly, describe the methodology of Differential Neural Networks and the concept of the identifier of states for one-dimensional financial time series; in Section 3, we show the results and provide a short discussion of them; through Section 4, we present the conclusions remarking the benefits of our proposal extending the framework of DNN.

Methodology Description
In this section, the theoretical foundations of differential neural networks and the functional form of the identifier system of states are presented.The learning laws for one dimension neural network are established through Theorem 2.1.

Differential Neural Network (DNN)
In this subsection we describe the DNN as a coupled system consisting of a neural identifier and a controlled tracker.Through this structure is possible to register the different states of a dynamic system and then to reconstruct all them through a dynamic estimation model; see Figure 1.The estimated states are then used by the controlled tracker to introduce them into a feedback function to produce a new input in order to track the reference model trajectories.
In general, the DNN structure corresponds to a structure of an ANN with a Hopfields multilayer form, whose dynamics can be described by the following continuous nonlinear differential equation (Poznyak et al. 1999 and2001): with x 0 known, and where xt ∈ R n is the estimated state vector, A ∈ R n×n is a Hurwitz matrix, y t is the output vector at any time t, C is the transformation matrix from the state x t to the output state y t , and ξ 1, t and ξ 2, t are both white noise that cannot be measured (but bounded) on the dynamics of states and the output, respectively.Hence, the white noise processes are characterized by ξ j, t ≤ Υ, j = 1, 2, for some value Υ, and are associated with the errors of measuring of available values of variables, like days off, Sundays and Saturdays (Ortiz et al., 2012).
The matrix W 1,t ∈ R n×k contains the feedback weights of a nonlinear state, and W 2,t ∈ R n×r is the input weight matrix.Both matrices are the external weight matrices of the DNN.Function σ(•) : R n → R k is a vector field which is constructed with classical sigmoid activation functions.In this way, it will have only elements with a behavior that grows monotonically as a sigmoid function (Ortiz et al., 2012).That is, In the case of the transformation function φ(•) : R n → R r×s it is constructed a sigmoidal activation function in each element of this diagonal matrix, thus its We require that each component of activation function satisfies the following conditions: The function γ(u t ) : R s → R q is the control action or external power; its values come from the data base.It is assumed to be bounded in such a way that γ (u t ) 2 ≤ ū .In this research, we consider γ (u t ) = u t as the identity function.In Figure 2, it is shown a typical structure of a Differential Neural Network.

Identifier of DNN
The identifier design requires a positive-definite Hurwitz matrix Q, such that the Riccatis differential equation has a positive solution: For the identifier design, we use the system class described by This equation represents the nonlinear system class that we will use to describe the dynamic evolution for the foreign exchange rates.Also, we suppose that there is a differential neural network to match exactly the system described before, given by where W * 1,t and W * 2,t are unknown matrices with constant entries.The differential neural network to estimate the nonlinear system is given by The error between the nonlinear system values and the estimate states given by the differential neural network estimator is denoted by: The dynamics for this error must be stable, but we need to prove this statement to ensure the stability for the error function.In this case, the Lyapunov function is assumed to be where the matrices W 1,t and W 2,t are defined by In this paper the proof will be based on a nonlinear system and a neural network both for the one-dimensional case.Therefore, we state the following: Theorem 1.Consider the following non-linear system ẋt = f (x t , u t , t) (12) and the equation of the identifier of the neural whose weights are adjusted by the following learning laws: where K 1 and K 2 are positive values, p > 0 is the solution for Riccattis equation given by ( 6) for the one-dimensional given by ap + pa + prp + q = 0 2ap + p 2 r + q = 0 (15) We also assume that the mappings σ(•) and φ(• ) satisfy the following sector conditions: , where λ σ , λ ϕ , D σ , D ϕ , Z σ , Z ϕ , C σ , C ϕ are known positive constants.Moreover, we assume that whereω 1 andω 2 are positive constants and the control input function γ Then the weights dynamics are bounded and converge lim Proof.Suppose that the system (12) has a complete description given by ẋt and selected the Lyapunov function by After computing the derivative of this function, we obtain and the error function derivative is Therefore, we obtain Finally, where Thus, in equation ( 17) the term ∆t may be replaced for the first term in equation ( 18): Equivalently, If we now use the inequality in the term 2p∆ t ω * 1 ⌢ φ t for the scalar case, then Similarly, And given the assumptions and If we now consider in the expression where r = ω1 + ω2 andq = D σ + D φ γ 2 (u t ) + q 0 , then equation (24) takes the form: Vt ≤ 2ap + p 2 r + q − q 0 ∆ 2 t , with q 0 > 0. Finally, by using the solution for the Riccatis equation in (15), we obtain then the system is asymptotical stable.QED

Extended DNN, Data Periods and Intervals for Estimation
We use data from the European Central Bank of daily market closing exchange rates: Euro-US dollar, Euro-Yen and Euro-Pound from January 4, 1999 to September 13, 2013.Upon this data, we establish three different periods to work with the Extended DNN (EDNN) for each exchange rate.The first period is the learning or training period, this period includes 30 values, near 1.5 months of data, from January 4, 1999, to February 12, 1999.The second period runs from February 15, 1999, to August 31, 2013.This is the period in which the EDNN describes all daily closing values of the exchange rate of Euro with respect to the three currencies (the identification process).The range of third and final period is from September 2 to September 13, 2013.This is the period where the forecasting takes place of daily closing values of Euro with respect to the three currencies.

Discussion of Empirical Results
In this section, we show the obtained results from the extended DNN on the basis of equations ( 1) to (4).In order to program the algorithm, we used the MatLab platform and the Simulink Toolbox.For the numerical calculations of the Dormand-Prince algorithm, we solve ordinary differential equations with a variable step (Dormand, 1980).The estimated states and the actual data are displayed together in order to compare the evolution, these are plotted in the figures below.

Identification Process
As mentioned before, the network was trained from the dynamics of a 30-day period of the series of daily market closing values of the exchange rate of Euro-US Dollar, covering the period from January 4, 1999, to February 12, 1999.We compute the initial weight matrices in a process of trial and error.Then, we selected the best choice for setting the initial weight matrix, generating the set of error values described in equation (10) of the learning law (14), and repeating this in each iteration.Subsequently, we follow the same process for the other two exchange rates, Euro-Yen and Euro-Pound, respectively.The identification of the evolution of actual daily closing values of the foreign exchange rate Euro versus US dollar, yen and pound and the identification of the calculated EDNN daily closing values of foreign exchange rates are shown in Figures 3 to 11.In these Figures, it is possible to distinguish the precision with which the EDNN conducts the identification of each series.We will differentiate through zooms how the network narrows its values to converge quickly to the actual values of the series for each case.The first zoom focuses on hundreds of days, the second zoom considers less days, and the third one takes into account only a few days.This allows us clearly to see how the network values approach to the actual behavior of the examined exchange rates.

Forecasting Process
In this section, we show how the EDNN produces forecasts of daily closing values of the foreign exchange rates for a period of two weeks with a remarkable    In Figure 13, we observe the relative error between the daily market closing We observe the remarkable precision in the predicted values.In the worst case, the percentage error of the forecasting is less than 0.13 Figure 15 shows the percentage error between the actual daily market closing values of the exchange rate Euro-Japanese Yen versus those predicted by the EDNN.We observe that percentage error ranges from 0.035% to 0.192% .The Figure 16 shows the Actual Foreign Exchange Rate Euro-Japanese Yen versus forecasting with EDNN.
In case of the British Pound series case, Figure 17 shows the percentage error between the actual daily market closing values versus those forecasted by the EDNN.We can see the precision range is remarkable, near 0.001% in the best forecasting, and 0.071% in the worst case of the predicted values.

Conclusions
The main objective of this paper was to highlight the power of the EDNN in carrying out the processes of identification or tracking (monitoring) of financial variables.In this work, we used proposed an EDNN to reproduce past values of financial variables and to predict their future values.The EDNN provide a    In carrying out the forecast with EDNN for the daily closing values of these exchange rates for 2 weeks, the results were remarkable because the relative error of forecasts during the first week of prediction (September 2 to 6, 2013) ranges from 0.002 Finally, we also wish to highlight, on the basis of the obtained results, that the description of the EDNN reached acceptable estimates, including sudden changes of the values in a short period of time; see Figures 7 and 10.We believe that this methodology is appropriate to perform this kind of predictions, since the obtained results in the description and forecast were remarkable, since the error rates are very close to zero.
et al. (2001) worked on inflation forecasting and Hanias et al. (2007) carried out predictions of the Athens Stock Exchange Price Indicator.Also, ANNs have been used to perform predictions of currencies parities in Dunis et al. (2010).
recent work in Ortiz et al. (2012) considering the use of the DNNs to carry out monitoring and forecasting of the daily values of stock indices and prices of stock in Spain and Mexico.Finally, in Ortiz et al. (2012a) a model for monitoring the exchange rate values of the US dollar against the Mexican peso is developed by using a model based on DNN.

Figure 2 :c kl x l − 1 i = 1 ,
Figure 2: Typical structure of a DNN.Source: Author's own elaboration

Figure 3 :
Figure 3: Observed daily closing values of the exchange rate Euro-US Dollar versus the estimated values with EDNN (first zoom).Source: Author's own elaboration

Figure 4 :
Figure 4: Observed daily closing values of the exchange rate Euro-US Dollar versus the estimated values with EDNN (second zoom).Source: Author's own elaboration

Figure 5 :
Figure 5: Observed daily closing values of the exchange rate Euro-US Dollar versus the estimated values with EDNN (third zoom).Source: Author's own elaboration

Figure 6 :
Figure 6: Observed daily closing values of the exchange rate Euro-Japanese Yen versus the estimated values with EDNN (first zoom).Source: Author's own elaboration

Figure 7 :
Figure 7: Observed daily closing values of the exchange rate Euro-Japanese Yen versus the estimated values with EDNN (second zoom).Source: Author's own elaboration

Figure 8 :
Figure 8: Observed daily closing values of the exchange rate Euro-Japanese Yen versus the estimated values with EDNN (third zoom).Source: Author's own elaboration

Figure 9 :
Figure 9: Observed daily closing values of the exchange rate Euro-British Pound versus the estimated values with EDNN (first zoom).Source: Author's own elaboration

Figure 10 :
Figure 10: Observed daily closing values of the exchange rate Euro-British Pound versus the estimated values with EDNN (second zoom).Source: Author's own elaboration

Figure 11 :
Figure 11: Observed daily closing values of the exchange rate Euro-British Pound versus the estimated values with EDNN (third zoom).Source: Author's own elaboration

Figure 12 :
Figure 12: Actual daily closing values of the exchange rate Euro-US Dollar versus the forecasted values with EDNN.Source: Author's own elaboration

Figure 13 :
Figure 13: Percentage error between the actual daily market closing values versus those forecasted with EDNN.Source: Author's own elaboration

Figure 14 :
Figure 14: Actual Foreign Exchange Rate Euro-Japanese Yen versus the forecasted values with EDNN.Source: Author's own elaboration

Figure 15 :
Figure 15: Percentage error between the actual daily market closing values of the exchange rate Euro-Japanese Yen versus those forecasted by using the EDNN approach.Source: Author's own elaboration

Figure 16 :
Figure 16: Actual Foreign Exchange Rate Euro-British Pound vs. forecasting with EDNN.Source: Author's own elaboration

Figure 17 :
Figure 17: Percentage error between the actual daily market closing values of the foreign exchange rate Euro-British Pound vs. those forecasting by using the EDNN approach.Source: Author's own elaboration