Universität Bielefeld - "Graduiertenkolleg Aufgabenorientierte Kommunikation"

Resolution-Based Complexity Control for Gaussian Mixture Models

Peter Meinicke and Helge Ritter

Abstract

In the domain of unsupervised learning, mixtures of Gaussians have become a popular tool for statistical modelling. For this class of generative models we present a complexity control scheme, which provides an effective means for avoiding the problem of over-fitting usually encountered with unconstrained (mixtures of) Gaussians in high dimensions. According to some prespecified level of resolution as implied by a fixed variance noise model the scheme provides an automatic selection of the dimensionalities of some local signal subspaces by maximum likelihood estimation. Together with a resolution-based control scheme for adjusting the number of mixture components, we arrive at an incremental model refinement procedure within a common deterministic annealing framework, which enables an efficient exploration of the model space. The advantages of the resolution-based framework are illustrated by experimental results on synthetic and high-dimensional real world data.

Postscript-File (~ 136 k)

Anke Weinberger, 1999-03-16, 1999-09-30