Deep learning driven multimodal fusion for automated deception detection

Citation

Gogate M, Adeel A & Hussain A (2018) Deep learning driven multimodal fusion for automated deception detection. In: 2017 IEEE Symposium Series on Computational Intelligence, SSCI 2017, Honolulu, HI, USA, November 27 - Dec. 1, 2017. 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, HI, USA, 27.11.2017-01.12.2017. Piscataway, NJ, USA: IEEE. https://doi.org/10.1109/ssci.2017.8285382

Abstract
Humans ability to detect lies is no more accurate than chance according to the American Psychological Association. The state-of-the-art deception detection methods, such as deception detection stem from early theories and polygraph have proven to be unreliable. Recent advancement in deception detection includes the application of advanced data analysis and machine learning algorithms. This paper presents a novel deep learning driven multimodal fusion for automated deception detection, incorporating audio cues for the first time along with the visual and textual cues. The critical analysis and comparison of the proposed deep convolutional neural network (CNN) based approach with the state-of-the-art multimodal fusion methods have revealed significant performance improvement up to 96% as compared to the 82% prediction accuracy reported in the recent literature.