A SUBGRADIENT METHODS FOR NON-SMOOTH VECTOR OPTIMIZATION PROBLEMS

Vector optimization problems are a significant extension of scalar optimization and have wide range of application in various fields of economics, decision theory, game theory, information theory and optimal control theory. In this paper, unlike general subgradient methods, bundle methods and gradient sampling methods used to solve nonsmooth vector optimization problems which include scalarization approach, a subgradient method without usual scalarization approached is proposed for minimizing a non-differentiable convex function which works directly with vector-valued function. A general sub-gradient method for non-smooth convex optimization that includes regularization and interior point variants of Newton’s Method are proposed. This algorithm builds a sequence of efficient points at the interior of epigraph of objective function which satisfies KKT conditions. In this paper, under the suitable conditions it is proved that the sequence generated by algorithm converges to ǫ-efficient point. AMS Subject Classification: 90C29, 90C30


Introduction
Vector optimization problems are a significant extension of scalar optimization and have many real life applications. Which also referred multiobjective optimization problems, deals with the problem where several objective functions have to be simultaneously minimized or maximized. For nontrivial problems, no single point minimizes or maximizes all given objective at a time, and so the concept of maximum point or minimum point is to be replaced by the concept of Pareto Optima or efficient solution. Multiobjective optimization problem are traditionally solved by two different approaches: scalarization and ordering techniques [22,29]. The scalarization approach enables the computation of efficient (Pareto) or weakly efficient solutions by formulating single objective optimization problems (with or without constraints) and choosing good parameters in advance. Other scalarization techniques that are free of establishing parameters can compute a discrete approximation of the optimal set. The solution strategies that use ordering techniques require a prior ranking specification for the objectives. For solving smooth type multiobjective optimization problem, descent, steepest descent and Newton methods, which neither require parameter nor ordering information, are proposed by several authors ( [11,12,25]). Unlike to smooth vector optimization problem, non-smooth vector optimization deals with the problems where the functions involved are not continuously differentiable. The gradient does not exist implying that the function may have kinks or corner points and therefore the functions cannot be maximized or minimized by general proposed analytical process. The Feasible Direction Interior Point Algorithm and the projected subgradient methods for non-smooth vector optimization are proposed by several authors [16,17,20,21]. But most of the existing numerical methods for non-smooth vector optimization problems are either based on subgradient using scalarization approach and space dilation type algorithms [20,22,29]. In this paper we deal with subgradient method.
Subgradient method was developed in 1962 by N.Z. Shor [36] and used by him for solving large-scale transportation problem of Linear Programming Problems. His work extensively expanded both in theory and in practice by several authors [8,12,13,14,15,16,18,23,26]. Held and Karp [19], unaware of the work of Shor, develop a method for the travelling salesman problem that uses subgradient optimization to compute a bound in a Lagrangean Relaxation scheme. This seminal contribution also led to others, see Fisher [10]. Research work in this direction led by Allen et al. [2], Ermoliev [8], Ermoliev and Shor [9], Goffine [13,14], Goffine and Kiwiel [15] and Polyak [31] gave a precise statement of the method and its convergence theorems which may be taken in to account as the culmination of the first stage in developing subgradient techniques. All of their work in this field are well reported in their book, Refer Shor [37,38,39] and Polyak [34] as well as in the second book by Shor [40]. Also refer the book by Nesterov [27].
The outline of this article is as follows. In section 2 first we briefly describe our notations and basic definitions. In section 3 we presented subgradient method for vector optimization and state some basic preliminaries materials. The convergence of standard derivative free methods like Powel's method and genetic algorithms has been proved only for smooth functions in section 4 we describe the convergence analysis of the algorithm for non-smooth function.

Preliminaries and Basic Definitions
Let F be a non-empty, closed and convex subset of R n . The inner product in R n is denoted by ·, · and the norm generated by the inner product by · . Given a proper, closed, convex and pointed cone K of R m , we consider the Partial Order '≤' defined as x ≤ y if and only if y − x ∈ K (y − x ∈ int(K)). In similar way, we define the Partial Order '≥'. Throughout in this article, the zero vector (0, 0, 0, 0, . . .) in R n is denoted by 0. The non-negative orthant in R n is denoted by R + n = {x ∈ R n ; x ≥ 0} . Consider a function f : R n → R m . In this paper we are interested in the following problem Solution set. We denote the solution set of our problem by F * and define as When m = 1 and K = R + , the problem will be called a scalar valued optimization problem.
Definition 2.3. The positive Polar cone of K, denoted by K * is given by At differentiable points we have unique subgradient and that is gradient of given function at that point but at non-differentiable point there are infinite number of subgradients i.e. there exists infinite number of supporting hyper plane at that point.
where f 1 and f 2 are defined below: , f can be represented as follows: In the open interval (1,4), the subgradient g = −1 and it is unique. Similarly if x < 1 or x > 4, then g = −2(x − 2) is also a unique subgradient of f . But at the point x = 1 and x = 4, the subgradients are not unique because many supporting hyperplanes exist. At x = 1 any g in the interval [−1, 2] is a subgradient of f . Similarly any g in the interval [−4, −1] is a subgradient of f at x = 4.
It follows that there exists U ∈ R m×n such that g = U T z, and therefore For every x,x ∈ R n . Therefore we consider ∂f : R n → R m×m defined as In analog with the scalar case we will call this set the subdifferential of vector valued function f and its elements subgradients of f at x. If we consider K = R m + and all the components of f i.e. f i are convex in the usual sense then any matrix U ∈ R m×n with the property that its ith line is a subgradient of f i at x for i = 1, 2, 3, . . . , m, belongs to ∂f (x).

A Sub-gradient Method for Vector Optimization
Our goal is to minimize f : R n → R m which is component wise K-convex. To do this, the subgradient method uses the simple iteration to update x i+1 as Where x i is the ith iterate, g i is any subgradient of f i and t i is the step size which generate a sequence that satisfy This rule has been considered by several authors for similar methods [1,6,34,35].

Iterative
Step: Given x i ∈ F , take U i ∈ ∂f i and compute with and t i satisfy (3.2) and Stopping Criterion. If g i = 0 or x i ∈ F * then stop, otherwise compute The proposed algorithm generates a Quasi-Fejer convergent bounded sequence.
Definition 3.1. Let S be nonempty subset of R n . A sequence {x i } is said to Quasi-Fejerand convergence to S if and only if for all x ∈ S, there existī and a summable sequence {δ i } ⊂ R + such that (3.7) Lemma 3.2. Let {x i } be a bounded sequence and f a K-convex function. Then, any sequence {U i } ∈ ∂f (x i ) is bounded.

Remark.
A very popular strategy for solving vector optimization problem is the scalarization approach where vector valued function is converted into scalar valued function by different techniques. Most popular techniques are "parametric" and "weight" method. Disadvantage of the parametric approach is that the choice of the parameters is not known in advance, leaving the modeler and the decision maker with the burden of choosing them [6]. Another method is "weight" method. Basically in this technique one minimize a linear combination of the objectives where the vector of "weights" is not known a priori. This procedure may lead to unbounded numerical problems, which therefore may lack minimizers, see [7,17,22]. In this proposed method we do not used scalarization. Therefore we neither required parameter to scalarize nor weight parameter. We have explored the structure of the vectors problem.

Convergence Result
There are many results given by numerous researcher [2,5,6,7] on convergence of the subgradient method for nonsmooth convex optimization problem, here functions are scalar functions. Even the convergence of subgradient methods for vector optimization by using scalarization technique has been also studied by several authors [14,15]. In this case of content step size (or step length) the subgradient algorithm is guaranteed to converge to within some rang of the optimal value, i.e.
Here f * is optimal value. In this section we attempt to establish convergence of the sequence generated by the algorithm by vector valued function without using scalarization, to a point in the solution set. From now on we denote by {x k }, the sequence generated by our algorithm starting from x 0 ∈ F . To establish the convergence we also propose and prove followings: Proof. By Induction x 0 ∈ F . Assume x i ∈ F . Since g i ∈ F i = F − x i and g i = x i+1 −x i . Implies that x i+1 ∈ F . Therefore we conclude that {x i } ∈ F . Proof. Let x i / ∈ F * . Then there exist z ∈ F such that f z ≺ f (x i ). Let us take g = z − x i . It is true that g = 0. By K-convexity of f , f (z) f (x i ) + U i g. Implies that U i g ∈ − int(K), and therefore max Here we define and make the following assumption Under the above assumption set T is non-empty. Next we will prove that the sequence generated by the algorithm is a Quasi-Fejer convergent sequence and hence bounded.
Proof. First we prove the following inequality In order to prove this inequality note that To prove We have to prove that According to definition of g i i.e.
and by the triangular inequality of norm Therefore, we get that Now by simple algebraic manipulation, for all x ∈ F Now by formula for the subdifferetial of a maximum of convex function and definition of g i (as in (4.6)) there exists a positive integer I(i) and y ∈ K * and λ i k > 0 with 1 ≤ k ≤ I(i), Whereȳ i is max of y. Therefore From (4.5b), we have By using (2.2) for K-convex function we can say (4.14) Therefore from (4.13) and (4.14), we get Implies that

Boundedness
Since t i satisfy (3.2) and T = Φ, therefore the sequence {x i } generated by the algorithm is quasi-Fejer convergent in T and hence by property of quasi-Fejer, sequence {x i } bounded for all i =ī [6].
Theorem 4.6. The sequence {x i } generated by algorithm converge to some cluster x * ∈ T .
Proof. Since µ i ≤ ρ, where ρ is radius of closed ball centered at 0. Therefore for x ∈ F we obtain For sake of a simpler notation let us define the sequence and rewrite above inequality (4.16) as Taking summation on both sides from i = 0 to i = m, we get Taking m → ∞, we get Therefore from (4.18) we conclude that lim inf γ i = 0. Since By definition of γ i , we obtain that Let x * be cluster point of {x i j } such that x * = lim i j l then The following theorem is our main result. Proof. From Theorem 4.1, we have that {x i } converge to cluster point x * ∈ T . Also x * is a cluster point of {x i } associated to the subsequence {x i j }. Follows from Preposition 4.1 that x * ∈ F . Now we have to prove that x * ∈ F * . Let us suppose that x * / ∈ F * . Therefore there existsx and an index j such that f k (x) ≤ f k (x * ) − ǫ, k = 1, 2, 3, 4, . . . , m and f j (x) < f j (x * ) − ǫ.

Conclusion
In this paper our algorithms do not use scalarization method, a very popular strategy for solving multi-objective problem. Our methods strongly explored the vector structure of the problem. But as we observe during journey of discussion, the vector subgradient method out performs the simpler method in many instances. But if method work good (under suitable initialization and step size like Newton's method) it's work very good and give suitable optimal solution but if it is bad(as in case of ill-conditioning or in case of gully-shaped functions) it is very bad and always far away from efficient solution. Therefore with suitable step-size and good initialization the idea discussed in this paper may be explored to deal with more efficient method to solve vector optimization problem like bundle method, cutting plane method etc.