Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features

Citation

Abel A, Gao C, Smith L, Watt R & Hussain A (2018) Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features. In: 2018 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE Symposium Series on Computational Intelligence, SSCI 2018, Bangalore, India, 18.11.2018-21.11.2018. Piscataway, NJ, USA: IEEE, pp. 1033-1040. https://doi.org/10.1109/SSCI.2018.8628931

Abstract
The extraction of relevant lip features is of continuing interest in the speech domain. Using end-to-end feature extraction can produce good results, but at the cost of the results being difficult for humans to comprehend and relate to. We present a new, lightweight feature extraction approach, motivated by glimpse based psychological research into racial barcodes. This allows for 3D geometric features to be produced using Gabor based image patches. This new approach can successfully extract lip features with a minimum of processing, with parameters that can be quickly adapted and used for detailed analysis, and with preliminary results showing successful feature extraction from a range of different speakers. These features can be generated online without the need for trained models, and are also robust and can recover from errors, making them suitable for real world speech analysis.

Keywords
Feature extraction; Lips; Mouth; Psychology; Adaptation models; Shape; Training