ON THE STABILITY OF A NEURAL NETWORK WITH LINKS BASED ON THE WATTS-STROGATZ MODEL

We study the stability problem for a model, similar to the WattsStrogatz model, with some parameter p, ranging from 0 to 1. When p = 0, the model is a deterministic model of a ring delayed neural network in which each neuron is connected to several neighbors in the ring. When p = 1, links are random with the preservation of their density. Under certain intermediate values of p we obtain the small world neural network. We find out whether the stability of the model in this process is improved. Our numerical experiments give a double response: if the forces of interaction between the different network nodes are the same, then the transition from deterministic to random network contributes to the loss of stability; if the forces are substantially different, then the stability region increases. This response refines the previously known results on the stability of small world type networks.


Introduction
The Watts-Strogatz model [1] contains a parameter p, such that for p = 0 the Received: September 18, 2015 c 2015 Academic Publications, Ltd. url: www.acadpubl.eu§ Correspondence author model is a deterministic graph, for p = 1 it is a graph with random connections.In certain intermediate range of p-values the Watts-Strogatz model represents the links of small world type.Several neural structures within living organisms are organized by the type of small world ( [2] and [3]).It is natural to compare the stability of the steady states of small world type of neural networks with the stability of the two extreme models within the Watts-Strogatz model: deterministic (p = 0) and random (p = 1) at the same density of connections in models.This is the main purpose of our study.
In [2] for a model of neural populations, rather far from the Watts-Strogatz model, the following conclusion is drawn: the stability region in the parameter space expands in process of increase randomness of connections within the network.Our neural network model is closer to the model of Watts-Strogatz.Our answer is not as definite.In our model the interaction of neurons is asymmetrical: if the force of action of one neuron on another is equal, for example, a, it isn't necessarily equal to the force of the reverse action, equal, let's suppose, b.It was found that in our model if |a − b| ≫ 0 then the stability region increases in the process of increasing the parameter p, while at |a − b| ≈ 0 the stability region weakly depends on p, but a few decreases with increasing p.
The paper is organized as follows.In Section 2, the model of a neural network is formulated.In Section 3 we demonstrate results of the numerical experiments, based on theoretical considerations outlined in the previous works [4] and [9] and [7].In Section 4, we formulate the conclusions about the dynamics of the network stability when changing p.

The General Equation of a Neural Network
In this subsection we describe a general model of neural networks.The model is discrete.It includes delayed interactions, and it contains a matrix whose elements define a system of connections in the network.In the next subsection we will indicate how to form connections in our model.We suggest that the neural network contains n neurons with numbers 1, 2, . . ., n.The neuron j at moment s (s = 1, 2, . ..) is characterized by a signal x j s .The neural network at moment s is characterized by a vector x s = (x 1 s , . . ., x n s ) T .If each neuron is isolated, we believe that the dynamics of a neural network is governed by the equation x s = αx s−r , (−1 < α < 1), r ∈ Z + .Therefore, in the absence of connections between neurons zero steady state is stable.Here α denotes the damping coefficient, while r denotes the delay of neuron in response to its own signal.In the matrix of neurons interactions B = (β ij ) n i,j=1 the value of β ij is the force of action of the neuron j on the neuron i.A graph of the neural network is made up of the set of vertices V = {1, 2, . . ., n} and the set of directed edges E, such that (i, j) ∈ E if and only if β ij = 0.
We suppose that the interaction between the different neurons takes place with a delay m (m > r).As a result, we obtain the following equation of neural network dynamics (see also [4] and [5]):

System of Connections in the Neural Network and its Changes
In this subsection, we describe the system of connections in the neural network with the parameters p (0 p 1) and k (k ≪ n).The value of p characterizes a degree of randomness of connections within the network, the quantity k/n characterizes the density of connections.CASE 1: p = 0.In the beginning we will dispose all neurons 1, 2, . . ., n on a ring.Then we will conduct the directed edges from every neuron to k nearby neurons clockwise.For each of such edges we will provide a weight b.Then we supplement each edge, in a clockwise direction, by the edge in the opposite direction with a weight a.Thus we form the matrix B in equation (1).CASE 2: p = 0.The first step is the same as in the previous case: we draw edges from each neuron to nearby k neurons clockwise.Each of such edges is provided by weight b.Then we perform the switching process.Each directed edge (i, j) (1 i, j n) with probability p is replaced by the edge (i, v) (1 v n).By doing this, we follow the rules: i) previously constructed edges should not be repeated; ii) all vertices v, which allowed for selection, are equally probable; iii) we attach weight b, to all the edges.
After the switching process completed, each edge (i, j) is complemented by the edge (j, i) with the weight a.Thus, the matrix B will be formed.
The communication system of the network is shown in Figure 1.Equation ( 1) is the equation of the network's dynamics with the matrix B.
If you change the value of p from 0 to 1, the network varies from a strictly deterministic to a completely random one.For some values of p between 0 to 1 the network presents a small world network [2].

Stability of the Neural Network: From Deterministic to Random
The linear matrix equation ( 1) is stable if all its solutions are bounded, and it is asymptotically stable if all its solutions tend to 0 when s → ∞.The equation det( of the order n × m is the characteristic equation for (1).In (3), I is a unit n × n matrix.The equation ( 1) is asymptotically stable if and only if all roots of it's characteristic equation ( 3) lie strictly inside the unit circle of the complex plane.The algorithms for diagnostics of equation stability (1) by means of the stability cones are expounded in [8] and [9].
In the models of [2] and [3] the stability region in the parameter space is expanding with increasing p and preservation of other parameters.From here the authors of [2] conclude that if the main desired characteristic of the network is the stability, it is necessary to increase the random factor when creating neural network connections.Our model differs from those of [2] and [3].In [2] the interaction force of any two neurons is the same, so the stability analysis is based only on the analysis of the graph connections in the network.In [3] interaction forces are random variables with a Gaussian distribution, whereas in our model, the half of links has an interaction force a, the remaining half has a force b.In addition, there is no delay in [2] and [3].
Therefore, it is important to figure out how the stability depends on p in our model.
Using the method of the stability cones we performed numerical experiments to determine the regions of network stability in the space of parameters a, b, considering that the number of neurons is unlimited.Typical stability regions at different values of p are shown in Figure 2.

Discusson
Let us explain why we consider Figure 2 typical.First, when the number of neurons is large, a further increase in their number has a little influence on the change of stability region.When the network is a deterministic ring, this conclusion follows from the formulas that define the spectrum of the matrix B ( [4] and [6]).For its randomized versions we tested this conclusion in numerous experiments.Second, the change of the damping coefficient α and the delays r, m does not change the pattern qualitatively, which is also confirmed by our numerical experiments.
Let us proceed to the commentaries to Figure 2. At first, let's pay attention to the diagonal a = b.In case a = b, our model is closer to the Watts-Strogatz model, as in this case, all the interaction forces are equal and neural network properties depend on the geometry of connections only.Figure 2 shows that for a = b the region of stability is weakly dependent on p, slightly decreasing with increasing p.The same is true for evaluation of stability regions at |a − b| ≈ 0.
Additionally, Figure 2 shows that if |a−b| ≫ 0 then the opposite monotonic dependancy of stability on p occurs: the larger p, the greater the region of network stability.The only exception is the network's behavior when p ≈ 1 and |a + b| ≈ 0.

Conclusion
In our model, there are three special types of networks: deterministic (p = 0), small world networks (some values of p ∈ (0, 1)), and random (p = 1).There may be two extreme points of view on the comparison of the stability of these three types of networks at a fixed total density of connections.First, the stability can decrease when moving from one network to another in order of their mention.Second, the stability may increase in this same process.Our results demonstrate that both of these extreme positions incorrect in some networks with a variety of forces of interaction between neurons.
Our conclusions are also true for continuous neural network model described by a differential equation instead of the difference equation (1).This is not surprising in light of how the stability theory of difference equations of type (1) is similar to the theory for corresponding differential equations ( [9] and [10]).