Rocky Mountain College * Department of Computer Science * 406 208 3193 * turn on javascript to see my email

Learning and Initialisation for Visual Tracking

Andrew Wildenberg, PhD thesis, University of Oxford Dept. Engineering Science, Oxford, UK, 1997.



The research described in this thesis describes work towards making Active Contours fully autonomous mechanisms capable of tracking biological targets in real-world situations. The problem of tracking the apparent motion of cauliflower plants photographed from a tractor in a field was of special interest. In order to reach this simple goal it was necessary to complete research in a number of different areas.

Active Contours use a stochastic model of motion to accurately predict the likely location of a target given previous observations. The standard method to acquire the parameters of this model is to learn them from training sequences. These learnt models track more accurately and robustly than hand-tuned models. Implicit in previous learning algorithms is the assumption that the same object is tracked in all the training sequences, and that after training the same object will be tracked when the new model is used. Many times it is useful to track a class of objects instead of a particular object. When tracking classes of objects, there are two sources of variation of the shape as observed in an image: change due to the motion of a particular target, and change due to different targets having different shapes. When these two sources of variation are modelled as one, the model cannot adequately model both, and the robustness of the tracking is reduced.

To allow the robust tracking of classes of objects, a new, augmented, model for tracking is introduced. A novel learning algorithm is presented that can learn the parameters of the augmented model through maximum likelihood estimation. The learning algorithm also correctly learns with short training sequences, a problem with the previous learning algorithm. The new model with learnt dynamics is shown to track more robustly than the previous model with learnt dynamics.

An area of research important to making Active Contour trackers autonomous is the initialisation problem. In the past, the initialisation problem has been solved by the operator manually positioning the curve, and then starting the tracker with a large initial uncertainty. To initialise the contour automatically, three steps are necessary:

Firstly, it is necessary to locate features of the target in the image. One way to do this is to segment the image into the target and background. A variety of pixel-based and region-based classification schemes are investigated for this. While the data provided by the segmentation in general is not sufficient for good tracking, it is possible to use this segmentation to recover the initial location of the target.

Once the target has been identified, it is necessary to recover the pose of the target. It is shown that there is no general solution to this problem. However, when the motion models have certain forms, it is possible to completely recover the pose. Methods are presented for recovering affine transformations and transformations contained within the affine space. For more general models, such as those based on PCA, methods are presented for partial pose recovery. Both are evaluated on both real and synthetic data.

The last step of Active Contour initialisation is determining the accuracy/uncertainty of the pose recovery. Previously known and novel methods to do this are analysed from a probabilistic standpoint. One novel method is presented which allows the initial uncertainty to be learned from training sequences in a similar manner to the way the motion model is learned. The problem of simultaneously estimating the initial pose and uncertainty is also discussed briefly.

Throughout the thesis, both theoretical arguments and experimental results are presented to support the work completed.

You know we're constantly taking. We don't make most of the food we eat, we don't grow it, anyway. We wear clothes other people make, we speak a language other people developed, we use a mathematics other people evolved and spent their lives building. I mean we're constantly taking things. It's a wonderful ecstatic feeling to create something and put it into the pool of human experience and knowledge. -- Steve Jobs, Rolling Stone, November 1983.