Finite mixture models is a pattern recognition technique that is used for fitting complex data distributions. Parameters of this mixture models are usually determined via the Expectation Maximization (EM) algorithm. A modified version of the EM algorithm is proposed earlier to handle data sets with missing values. This algorithm is affected by the occurrence of outliers in the data, the overlap among classes in the data space and the bias in generating the data from its classes. In addition, it only works well when the missing value rate is low. In this paper, a new algorithm is proposed to overcome these problems. A comparison study shows the superiority of the new algorithm over the modified EM algorithm and other algorithms commonly used in the literature.