A STUDY ON THE MIXTURE OF EXPONENTIATED-WEIBULL DISTRIBUTION PART I (THE METHOD OF MAXIMUM LIKELIHOOD ESTIMATION)

Mixtures of measures or distributions occur frequently in the theory and applications of probability and statistics. In the simplest case it may, for example, be reasonable to assume that one is dealing with the mixture in given proportions of a finite number of normal populations with different means or variances. The mixture parameter may also be denumerable infinite, as in the theory of sums of a random number of random variables, or continuous, as in the compound Poisson distribution. The use of finite mixture distributions, to control for unobserved heterogeneity, has become increasingly popular among those estimating dynamic discrete choice models. One of the barriers to using mixture models is that parameters that could previously be estimated in stages must now be estimated jointly: using mixture distributions destroys any additive reparability of the log likelihood function. In this thesis, the maximum likelihood estimators have been obtained for the parameters of the mixture of exponentiated Weibull distribution when sample is available from censoring scheme.The maximum likelihood estimators of the parameters and the asymptotic variance covariance matrix have been obtained. A numerical illustration for these new results is given.


Introduction
In probability and statistics, a mixture distribution is the probability distribution of a random variable whose values can be interpreted as being derived in a simple way from an underlying set of other random variables. In particular, the final outcome value is selected at random from among the underlying values, with a certain probability of selection being associated with each. Here the underlying random variables may be random vectors, each having the same dimension, in which case the mixture distribution is a multivariate distribution.
In cases where each of the underlying random variables is continuous, the outcome variable will also be continuous and its probability density function is sometimes referred to as a mixture density. The c.d.f. of a mixture is convex combination of the c.d.f's of its components. Similarly, the p.d.f. (pdf) of the mixture can also express as a convex combination of the p.d.f's of its components. The number of components in mixture distribution is often restricted to being finite, although in some cases the components may be countable. More general cases (i.e., an uncountable set of component distributions), as well as the countable case, are treated under the title of compound distributions.
A mixture is a weighted average of probability distribution with positive weights that add up to one. The distributions thus mixed are called the components of the mixture. The weights themselves comprise a probability distribution called the mixing distribution. Because of these weights, a mixture is in particular again a probability distribution. Probability distributions of this type arise when observed phenomena can be the consequence of two or more related, but usually unobserved phenomena, each of which leads to a different probability distribution. Mixtures and related structures often arise in the construction of probabilistic models. Pearson (1894) was the first researcher in the field of mixture distributions who considered the mixture of two normal distributions. After the study of Pearson (1894) there was long gap in the field of mixture distributions. Decay (1964) has improved the results of Pearson (1894), Hasselbled (1968) studied in greater detail about the finite mixture of distributions.
Life testing is an important method for evaluating component's reliability by assuming a suitable lifetime distribution. Once the test is carried out by subjecting a sample of items of interest to stresses and environmental conditions that typify the intended operating conditions, the lifetimes of the failed items are recorded. Due to time and cost constraints, often the test is stopped at a predetermined time (Type I censoring) or at a predetermined number of failures (Type II censoring). If each item in the tested sample has the same chance of being selected, then the equal probability sampling scheme is appropriate, and this has lead theoretically to the use of standard distributions to fit the obtained data. If the proper sampling frame is absent and items are sampled according to certain measurements such as their length, size, age or any other characteristic (for example, observing in a given sample of lifetimes that large values are more likely to be observed than small ones). In such a case the standard distributions cannot be used due to the presence of certain bias (toward large value in our example), and must be corrected using weighted distributions.
In life testing reliability and quality control problems, mixed failure populations are sometimes encountered. Mixture distributions comprise a finite or infinite number of components, possibly of different distributional types, that can describe different features of data. Some of the most important references that discussed different types of mixtures of distributions are Jaheen (2005b) and AL-Hussaini and Hussien (2011).
Finite mixture models have been used for more years, but have seen a real boost in popularity over the last decade due to the tremendous increase in available computing power. The areas of application of mixture models range from biology and medicine to physics, economics and marketing. On the one hand these models can be applied to data where observations originate from various groups and the group affiliations are not known, and on the other hand to provide approximations for multi-modal distributions [see Everitt and Hand (1981) We shall consider the exponentiated Weibull model, which includes as special case the Weibull and exponential models. The Exponentiated Weibull family EW [introduced by Mudholkar and Srivastava (1993) as extention of the Weibull family] contains distributions with bathtub shaped and unimodal failure rates besides a broader class of monoton failure rates. Applications of the exponentiated models have been carried out by some authors as Bain (1974); Gore et al. (1986); and Mudholkar and Hutson (1996).
Some statistical properties of this distribution (EW) are discussed by Singh et al. (2002) obtained Bayes estimators for the distribution parameters, reliability function and hazard function with type II censored sample under squared error loss function as well as under LINEX loss function. Nassar and Eissa (2004) obtained Bayes estimators of the two parameters EW distribution, reliability and failure rate functions using Bayes approximation form due to Lindley (1980) under the squared error loss and LINEX loss functions. Elshahat (2006), derived Bayes estimators for the two unknown shape parameters of the EW based on progressive type I interval censored sample. Salem and Abo-Kasem (2011) derived Bayes estimators for the two unknown shape parameters of the EW based on progressive hybrid censored sample. Approximate Bayes estimators for the two unknown shape parameters are drived by Elshahat (2008) based on Lindley (1980) and tierny and kadane (1986) and approximate credible intervals for the unknown parameters are obtained with progressive interval censoring. Ashour and afifiy (2008) derived maximum likelihood estimators of the parameters for EW with type II progressive interval censoring with random removals and their asymptotic variances.
Elshahat and Mahmoud (2016) obtained maximum likelihood estimators of the parameters of the mixture of exponentiated Weibull distribution, reliability and hazard functions from type II censored samples.

1.
Obtain Bayesian and approximate Bayesian (Lindley's-approximation) estimators of the parameters with two different loss function, squared error loss function and LINEX loss function.

2.
Monte Carlo simulation study will be done to compare between these estimators and the maximum likelihood, Bayesian and approximate Bayesian ones.

3.
Applications of mixed models will be presented. In additional to above introduction, the research contains four sections. Section (2) is devoted to some important definitions and notation which will be used in the present research. In Section (3) the estimation of the Mixture of the exponentiated Weibull distribution parameters has been drived via Bayesian method. In Section (4) estimation of the Mixture of the exponentiated Weibull distribution parameters has been drived via approximate Bayesian method and in Section (5) Numerical illustration using real data and simulation technique has been used to study the behaviour of the estimators using the Mathcad (2011) packages.

Definitions and Notation
This section is devoted to some important definitions and notation which are used in the present research.

One Stage Single Censord Samples
In such experiment, one of the two main types of censoring schemes (type -I or type -II) is used. Suppose we put-n items on a test and terminate the experiment at pre-assigned time (T), the samples obtained from such an experiment are called "time-censored" samples. The number of failures r and all the failure times are random variables. If the experiment is terminated when pre-assigned fixed number of items, say n r  have failed, the samples obtained from such an experiment are called "failurecensored" samples. The likelihood function of type-I (time censored) and type-II (failure censored) censored can be given as follow: where f (.) and F (.) are the density and distribution functions, respectively.

When
T   , then (1) reduces to the likelihood function of type-I censored, and when ) (r t   then (1) reduces to the likelihood function of type-II censored. Type-I and type-II censoring corresponding to complete sampling when r n  .

Mixture Model
Mixtures of life distributions occur when two different causes of failure are present, each with the same parametric form of life distributions. In recent years, the finite mixtures of life distributions have proved to be of considerable interest both in terms of their methodological development and practical applications [see Titterington et al. (1985), Mclachlan and Basford (1988), Lindsay (1995), Mclachlan and Peel (2000) and Demidenko (2004)].
Mixture model is a model in which independent variables are fractions of a total. One of the types of mixture of the distribution functions which has its practical uses in a variety of disciplines.
Finite mixture distributions go back to end of the last century when Everitt and Hand (1981) published a paper on estimating the five parameters in a mixture of two normal distributions. Finite mixtures involve a finite number of components. It results from the fact that different causes of failure of a system could lead to different failure distributions, this means that the population under study is non-homogenous.
Suppose that T is a continuous random variable having a probability density function of the form: where k is the number of components, the parameters are called mixing parameters, where represent the probability that a given observation comes from population "i" with density ( ), and ( ) ( ),…, ( ) are the component densities of the mixture. When the number of components k=2, a two component mixture and can be written as: When the mixing proportion 'p' is closed to zero, the two component mixture is said to be not well separated.

Definition (1):
Suppose that T and Y be two random variables. Let ( | ) be the distribution function of T given Y and G(y) be the distribution function of Y. The marginal distribution function ( ), defined by: is called a mixture of the distribution function ( | ) and ( ) where ( | ) is known as the kernel of the integral and ( ) as the mixing distribution .
A special case from definition (1) when the random variable Y is a discrete number of points * + and G is discrete and assigns positive probabilities to only those values of Y; the integral (4) can be replaced by a sum to give a countable mixture: By differentiating (5) with respect to T, the finite mixture of probability density functions can be obtained as follows where, In (6), the masses called the mixing proportions, they satisfy the conditions: The parameters in number of expressions (5) or (6) can be divided into three types. The first consists solely of k, the components of the finite mixture. The second consists of the mixing proportions w . The third consists of the component parameters (the parameters of ( ) or ( )) There is a number of papers dealing with 2-fold mixture models for times to failure modeling. For example, Jiang and Murthy (1995) characterized the 2-fold Weibull mixture models in terms of the Weibull probability plotting, and examined the graphical plotting approach to determine if a given data set can be modeled by such models. Ling and pan (1998) proposed the method to estimate the parameters for the sum of two threeparameter Weibull distributions. Based on these findings, a new procedure for the selection of population distribution and parameter estimation was presented. The reliability of the mixture distributions is given by:

Exponentiated Weibull Distribution (EW)
Salem and Abo-Kasem (2011) derived EW distribution in the following details; the "exponentiated Weibull family" introduced by Mudholkar and Srivastava (1993) as extension of the Weibull family, contains distribution with bathtub shaped and unimodale failure rates besides a broader class of monotone failure rates. The applications of the exponentiated Weibull (EW) distribution in reliability and survival studies were illustrated by Mudholkar et al. (1995). Its properties have been studied in more detail by Mudholkar and Hutson (1996) and Nassar and Eissa (2003). The probability density function (p.d.f.), the cumulative distribution function (c.d.f.) and the reliability function of the exponentiated Weibull are given respectively by; Where α and θ are the shape parameters of the model (8). The distinguished feature of EW distribution from other life time distribution is that it accommodates nearly all types of failure rates both monotone and non-monotone (unimodal and bathtub). The EW distribution includes a number of distributions as particular cases: if the shape parameter θ = 1, then the p.d.f is that Weibull distribution, when α = 1 then the p.d.f is that Exponentiated Exponential distribution, if α = 1 and θ = 1 then the pdf is that Exponential distribution and if α = 2 then the p.d.f is that one parameters Burr-X distribution. Mudholkar and Hutson (1996) showed that the density of A random variable T is said to be followed a finite mixture distribution with k components, if the p.d.f, c.d.f and R(t) of T can be written as in the forms (2), (3) and (7) respectively [see Tittrerington et al. (1985)].
The hazard function (HF) of the mixture is given by; function of the EW distribution is decreasing when αθ ≤ 1 and unimodal when αθ ≥ 1.
The natural logarithm of the likelihood function (1) is given by;

The Mixture of Two Exponentiated Weibull Distribution (MTEW)
In this chapter, we shall consider the mixture of twocomponent Exponentiated Weibull (MTEW) distribution. Some properties of the model with some graphs of the density and hazard functions are discussed. Elshahat and Mahmoud (2016) obtained the following maximum likelihood estimation under type II censored samples.
The failure of an item or a system can be caused by one or more than one cause of failure; it results that the density of time to failure can have one mode or multimodal shape and in that case, finite mixtures represent a good tool to model such phenomena. Suppose that two populations of the exponentiated Weibull (EW) distribution with two shapes parameters α and θ [see Mudholkar and Hutson (1996)] mixed in unknown proportions p and (1-p) respectively.
A random variable T is said to follow a finite mixture distribution with k components, if the p.d.f. of T can be written in the form (2) [see Titterington et al. (1985)]. Where , f j (t) the j th p.d.f. component (8) and the mixing proportions, p j , satisfy the is the j th c.d.f., component (9) , the reliability function (RF) of the mixture is given by (7), where R j (t) is the j th reliability component (10) . The hazard function (HF) of the mixture is given by (11), where ) (t f and ) (t R are defined in (2) and (7) respectively. (8) and (9) in (2) and (3), the p.d.f and c.d.f. of MTEW components are given respectively, by:

Mixture of K EW components: Substituting
By observing that R(t) = 1-F(t) and    k j j p 1 1, the RF of MTEW distribution components can be obtained from (7) and (10) as: 3), we obtain the HF of MTEW distribution as: If k = 2, the p.d.f., c.d.f. RF and HF of MTEW distribution are then given, respectively and,

Maximum likelihood Estimation for the Unknown Parameters of MTEW under Type II Censored Sample
Suppose a type-II censored sample ) ,..., , (  (1) is given by equation (12). Assuming that the parameters, and are unknown, we differentiate the natural logarithm of the likelihood function (12) with respect to so the likelihood equations are given by where is the first derivatives of the natural logarithm of the likelihood function (12) with  (18) and (19) in (17), we obtain where is the first derivatives of the natural logarithm of the likelihood function (12) with respect to for j = 1,2, and i = 1,2, ..., r and ( ) ( ) are given by (20) and (21) respectively. Assuming that the parameters, and are unknown, we differentiate the natural logarithm of the likelihood function (12) with respect to so the likelihood equations are given by: ::  (12) with respect to , from (13) and (15) respectively, we have where p 1 = p, p 2 = 1p, Substituting (25) and (26) in (24), we obtain  is the first derivatives of the natural logarithm of the likelihood function (12) with respect to for j = 1,2, and i = 1,2,…, r,  ( )  ( ) ( ) and ( ) are given respectively by (23), (27) and (28).
The solution of the four nonlinear likelihood equations (22) and (29) yields the maximum likelihood estimate (MLE): The MLE's of R(t) and H(t) are given, respectively, by (15) and (16)  Since the equations (22) and (29) are clearly transcendental equations in ̂ and ̂ that is, no closed form solutions are known they must be solved by iterative numerical techniques to provide solutions (estimators), ̂ and ̂ , in the desired degree of accuracy.
To study the variation of the MLE's ̂ and ̂ , the asymptotic variance of these estimators are obtained.
The asymptotic variance covariance matrix of ̂ and ̂ is obtained by inverting the information matrix with elements that are negative expected values of the second order derivatives of natural logarithm of the likelihood function, for sufficiently large samples, a reasonable approximation to the asymptotic variance covariance matrix of the estimators can be obtained as; ( ) The appropriate (30) is used to derive the 100 (1-) % confidence intervals of the parameters as in following forms : where, is the upper percentile of the standard normal distribution.
The asymptotic variancecovariance matrix will be obtained by inverting the information matrix with the elements that are negative of the observed values of the second order derivate of the logarithm of the likelihood functions .using the logarithm of the likelihood functions (12), the elements of the information matrix are given by: where is the first derivatives of the natural logarithm of the likelihood function (12) with respect to and is the first derivatives of the natural logarithm of the likelihood function (12) with respect to . and for i = 1, 2, …, r where Where is the second derivatives of the natural logarithm of the likelihood function (12) with respect to , for the functions (.) and (.) are given by (20) and (21),  (.) and  (.) by (23).

LINEX Loss Function
Linear-Exponential loss function (LINEX) was proposed by Varian (1975) in the context of real-estate valuations. Klebanov (1976a) derived this loss function in developing his theory of loos function satisfying a Rao-Blackwell condition. The name LINEX is justified by the fact that this loss function rises approximately linearly on one side of zero and approximately exponentially on the other side, Zellner (1986) provided a detailed study of LINEX loss function and initiated a good deal of interest in estimation under this loss function. The LINEX loss function may be expressed as: where .      the sign and magnitude of the shape parameter c reflects the direction and degree of asymmetry respectively. (If c > 0, the overestimation is mor serious than underestimation, and viceversa). For c closed to zero, the LINEX loss is approximately squared error loss and therefore almost symmetric.
The posterior expectation of the LINEX loss function is: where E θ (.) denoting posterior expectation with respect to the posterior density of θ. By a result of Zellner (1986)

Approximate Bayesian Methods
When the posterior distribution takes a ratio form that involves integration in the denominator and cannot be reduced to a closed form, the evaluation of the posterior expectation for obtaining the Bayes estimators will be tedious. Among the various methods suggested to approximate the ratio of integrals in this case, perhaps the simplest one is Lindley's (1980) approximation method Lindley's procedure was developed by Lindley (1980) to evaluate the posterior expectation of where is the prior density of  .
This procedure has been used by many authors to obtain Bayes estimators of the parameters of some distributions. See for example Soliman (2000), Jaheen (2005b).
The ratio of the integrals (48) may thus be approximated by using a form due to Lindley (1980) which reduces, in the case of two parameters, to the form: where and All functions in equation (29)

Bayesian Estimators for the Unknown parameters of MTEW under Type II Censored Sample
In this section, Bayesian method is used to obtain the estimators and posterior variance of the unknown parameters of finite mixture of two Exponentiated Weibull (MTEW) distribution. Bayesian risks are also obtained using the symmetric squared error loss. Moreover, for illustration, numerical examples are given.

Estimation under squared error loss function: (when and are known)
The likelihood function of the total life times t 1;n , t 2;n , …, t r;n , where t i;n is the i th component to fail. Considering the type II censored case this sample of failure times are obtained and recorded from a life test of n items independent and identically distributed, the Bayes estimators for using the likelihood function given by (1).
Assumed that the parameters have independent prior distribution and let the non informative prior (NIP) for are respectively given by.

( ) and ( )
Consequently, the joint (NIP) will be as follows: Multiplying (1) by (50), the joint posterior density of given the data will be where, is normalized constant equal to Now, the marginal posterior of can be obtained as.
and finally, the marginal posterior of can be obtained as.
It is well known that under a squared error loss function, the Bayes estimator of a parameter will be its posterior expectation. To obtain the posterior mean and posterior variance of the unknown parameters, nontractable integrals will be confronted. So, in this problem numerical integration is required. Then, both the posterior mean and posterior variance of the unknown parameters ( ) are expressed as follow. and and Equations (57) to (60) are very complicated for solving. An iterative procedure is applied to solve these equations numerically using mathcad (2011).

Estimation under LINEX Loss Function: (when and are known)
The Bayes estimators for the parameters of MTEW distribution can be expressed as: where, for j, s= 1, 2 and j s, For j, s = 1,2 and j and ( ) (75)

Approximate Bayes (Lindley's Procedure) Estimation Under Squared Error Loss Function: (when are known)
Now, we shall study the case when the parameters are known, Suppose that the mixing proportion, and are known.

Approximate Bayes Estimation (Lindley's Procedure) Under LINEX Loss Function
On the basis of the LINEX loss function (47), the Bayes estimate of a function w = w( ) of the unknown parameters is given by where is the region in the and plane on which the posterior density ( | ) is positive.

Real Data Set
We obtained in the above chapters [3 and 4], Bayesian [Bayes and approximate Bayes] and non-Bayesian (MLE's) estimators of the vector parameters , of MTEW distribution. We adopted the squared error loss and LINEX loss function. In order to asses the statistical performances of these estimators, a real data study is conducted. The data set is from Cancho et al. (2007).
To illustrate the approaches developed in the previous chapters [3 and 4], we consider the data set presented in Aarset (1987)  Considering the data in Aarset (1987), table (A) we fit (MTEW) distribution to the data set and summarized it in tables (1)and (2) by using MATHCAD package (2011). For comparison purpose we compute Bayes and approximate for the parameters of the mixture of two exponentiated Weibull distribution when .

Simulation Study
In the above sections [3and 4], Bayesian estimators of the vector parameter of MTEW distribution are presented. Approximate Bayes and Bayes estimates with squared error loss function are also obtained.
In order to assess the statistical performances of these estimates, a simulation study is conducted. The computations are carried out for censoring percentages of 60% for each sample size (n =10, 15, 20, 25, 30, 40 and 50). The mean square errors (MSE's) using generated random samples of different sizes are computed for each estimator.

Simulation Study for (Bayesian and approximate Bayesian) methods
Also, MATHCAD package was used to evaluate Bayes and approximate Bayes estimators under censored type-II using equations [(57) and (59)] for Bayes and [(48)] for approximate Bayes and for parameters values (θ 1 = 2.8, θ 2 = 4). The performance of the resulting estimators of the parameters has been considered in terms of the mean square error (MSE). The Simulation procedures will be described below: Step 1: 1000 random samples of sizes 10,15,20,25,30,40, and 50 were generated from MTEW model. If U has a uniform (0, 1) random number, then Step 2: Choose the number of failure r, we choose r to be less than the sample size n.
Step 3: Numerical integration method was used for solving the equations (58) and (59), to obtain Bayes estimators under squared error loss and LINEX loss function or the posterior mean and the mean square error (MSE).
Step 4: The approximate Bayes (Lindley's) estimators relative to squared error loss, ̃ computed, using (49) after considering the appropriate changes according to section (4.1). Also, the approximate Bayes (Lindley's) estimators relative to LINEX loss ̃ computed, using (49) after considering the appropriate changes according to sectionsr (4.2 and 4.3).  The following proofs for some equations which were included in this research. (67)