Training Hidden Markov Models with Multiple Observations -- A Combinatorial Method

Xiaolin Li, Marc Parizeau and Réjean Plamondon

Abstract - Hidden Markov models (HMMs) are stochastic models capable of statistical learning and classification. They have been applied in speech recognition and handwriting recognition because of their great adaptability and versatility in handling sequential signals. On the other hand, as these models have a complex structure, and also because the involved data sets usually contain uncertainty, it is difficult to analyze the multiple observation training problem without certain assumptions. For many years researchers have used Levinson's training equations in speech and handwriting applications simply assuming that all observations are independent of each other. This paper present a formal treatment of HMM multiple observation training without imposing the above assumption. In this treatment, the multiple observation probability is expressed as a combination of individual observation probabilities without losing generality. This combinatorial method gives one more freedom in making different dependence-independence assumptions. By generalizing Baum's auxiliary function into this framework and building up an associated objective function using Lagrange multiplier method, it is proved that the derived training equations guarantee the maximization of the objective function. Furthermore, we show that Levinson's training equations can be easily derived as a special case in this treatment.

download document


    author    = { Xiaolin Li and Marc Parizeau and Réjean Plamondon },
    title     = { Training Hidden Markov Models with Multiple Observations -- A Combinatorial Method },
    volume    = { 22 },
    number    = { 4 },
    pages     = { 371-377 },
    year      = { 2000 },
    month     = { April },
    journal   = { IEEE transactions on Pattern Analysis and Machine Intelligence }

Last modification: Jun 14 2002 4:24PM by parizeau


©2002-. Computer Vision and Systems Laboratory. All rights reserved