A novel approach combining recurrent neural network and support vector machines for time series classification

Citation

Alalshekmubarak A & Smith L (2013) A novel approach combining recurrent neural network and support vector machines for time series classification. In: 2013 9th International Conference on Innovations in Information Technology, IIT 2013. 9th International Conference on Innovations in Information Technology (IIT), 2013, Abu Dhabi, UAE, 17.03.2013-19.03.2013. Piscataway, NJ, USA: IEEE, pp. 42-47. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=6544391&abstractAccess=no&userType=inst; https://doi.org/10.1109/Innovations.2013.6544391

Abstract
Echo state network (ESN) is a relatively recent type of recurrent neural network that has proved to achieve state-of-the-art performance in a variety of machine-learning tasks. This robust performance that incorporates the simplicity of ESN implementation has led to wide adoption in the machine-learning community. ESN's simplicity stems from the weights of the recurrent nodes being assigned randomly, known as the reservoir, and weights are only learnt in the output layer using a linear read-out function. In this paper, we present a novel approach that combines ESN with support vector machines (SVMs) for time series classification by replacing the linear read-out function in the output layer with SVMs with the radial basis function kernel. The proposed model has been evaluated with an Arabic digits speech recognition task. The well-known Spoken Arabic Digits Dataset, which contains 8800 instances of Arabic digits 0-9 spoken by 88 different speakers (44 males and 44 females) was used to develop and validate the suggested approach. The result of our system can be compared to the state-of-the-art models introduced by Hammami et al. (2011) and P. R. Cavalin et al. (2012) , which are the best reported results found in the literature that used the same dataset. The result shows that ESN and ESNSVMs can both provide superior performance at a 96.91% and 97.45% recognition accuracy, respectively, compared with 95.99% and 94.04% for other models. The result also shows that when using a smaller reservoir size significant differences exist in the performance of ESN and ESNSVMs, as the latter approach achieves higher accuracy by more than 15% in extreme cases.