Multi-modal Speech Processing Methods: An Overview and Future Research Directions Using a MATLAB Based Audio-Visual Toolbox

Citation

Abel A & Hussain A (2009) Multi-modal Speech Processing Methods: An Overview and Future Research Directions Using a MATLAB Based Audio-Visual Toolbox. In: Esposito A, Hussain A, Marinaro M, Martone & R (eds.) Multimodal Signals: Cognitive and Algorithmic Issues: COST Action 2102 and euCognition International School Vietri sul Mare, Italy, April 21-26, 2008: Revised, Selected and Invited Papers. Lecture Notes in Computer Science, 5398. Berlin, Germany: Springer-Verlag, pp. 121-129. http://link.springer.com/chapter/10.1007%2F978-3-642-00525-1_12; https://doi.org/10.1007/978-3-642-00525-1_12

Abstract
This paper presents an overview of the main multi-modal speech enhancement methods reported to date. In particular, a new MATLAB based Toolbox developed by Barbosa et al (2007) for processing audio-visual data is reviewed and its performance potential evaluated. It is shown that the tool does not represent a complete and comprehensive speech processing solution, but rather serves as a standardised, yet versatile base to build upon with further research. To demonstrate this versatility, preliminary examples that make use of these computational procedures with an audiovisual corpus are demonstrated. Finally, some future research directions in the area of multi-modal speech processing are outlined, including future research that the authors aim to carry out with the aid of this newly developed audio-visual MATLAB toolbox, including toolbox customisation, and processing noisy speech in real world environments.

Keywords
C; DIRECTION; Future; method; methods; Research; Speech