Chaotic Time Series Recognition: A Deep Learning Model Inspired by Complex Systems Characteristics

Document Type : Original Article

Authors

1 Faculty of Electrical Engineering, K. N. Toosi University of Technology, Tehran, Iran

2 School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran

Abstract

A deep learning method is developed for chaotic time series classification. We investigate the chaotic state of a dynamical system, based on the output of the system. One of the main obstacles in time series classification is mapping a high-dimensional vector into a scalar value. To reduce the dimensions, it is common to use an average pooling layer block after feature extraction block. This blind process results in models with high computational complexity and potent to overfitting. One alternative is to extract the features manually, then apply shallow learning models to classify the time series. In fact, since complexity lies between the chaos and order, it is a sound idea to refer to complex systems characteristics to explore the chaotic region entrance. Therefore, chaotic state of a dynamical system can be recognized solely based on these characteristics. Inspired by this concept, we conclude that there is a feature space in which the output vector can be sparsified. Thus, we propose a deep learning method which the feature space dimensions successively are reduced in the feature extraction process. Specifically, we employ a fully convolutional network and add on two maximum pooling layers to the relevant feature extraction block. To validate the proposed model, the Lorenz system is employed which exhibits chaotic/non-chaotic states. We generate a labeled dataset containing 10000 samples each with 20000 features of the output of Lorenz system. The proposed model achieves 99.45 percent accuracy over 2000 unseen samples, higher than all the other competitor methods.

Keywords

Main Subjects

References

  1. Lorenz, E.N. and Haman, K., "The essence of chaos", Pure and Applied Geophysics, Vol. 147, No. 3, (1996), 598-599.
  2. Modaresahmadi, S., Ghazavi, M. and Sheikhzad Saravani, M., "Dynamic analysis of a rotor supported on ball bearings with waviness and centralizing springs and squeeze film dampers", International Journal of Engineering, Transactions C: Aspects, Vol. 28, No. 9, (2015), 1351-1358. doi: 10.5829/idosi.ije.2015.28.09c.13
  3. Al-ani, B. and Erkan, E., "A study of load demand forecasting models in electricity using artificial neural networks and fuzzy logic model", International Journal of Engineering, Transactions C: Aspects, Vol. 35, No. 6, (2022), 1111-1118. doi: 10.5829/ije.2022.35.06c.02
  4. Sparrow, C., "The lorenz equations: Bifurcations, chaos, and strange attractors, Springer Science & Business Media, Vol. 41,  (2012). doi:  10.1002/zamm.19840640122
  5. Chinforoush, N. and Latif Shabgahi, G., "A novel method for forecasting surface wind speed using wind-direction based on hierarchical markov model", International Journal of Engineering, Transactions B: Applications, Vol. 34, No. 2, (2021), 414-426. doi: 10.5829/ije.2021.34.02b.13
  6. Derbentsev, V., Babenko, V., Khrustalev, K., Obruch, H. and Khrustalova, S., "Comparative performance of machine learning ensemble algorithms for forecasting cryptocurrency prices", International Journal of Engineering, Transactions A: Basics, Vol. 34, No. 1, (2021), 140-148. doi: 10.5829/ije.2021.34.01a.16
  7. Babaei, A., Jafari, H., Banihashemi, S. and Ahmadi, M., "Mathematical analysis of a stochastic model for spread of coronavirus", Chaos, Solitons & Fractals, Vol. 145, (2021), 110788. doi: 10.1016/j.chaos.2021.110788
  8. Fereidunian, A., Lesani, H., Zamani, M.A., Kolarijani, M.A.S., Hassanpour, N. and Mansouri, S.S., "A complex adaptive system of systems approach to human–automation interaction in smart grid", Contemporary Issues in Systems Science and Engineering, (2015), 425-500. doi: 10.1002/9781119036821.ch12
  9. Barahona, M. and Poon, C.-S., "Detection of nonlinear dynamics in short, noisy time series", Nature, Vol. 381, No. 6579, (1996), 215-217. doi: 10.1038/381215a0
  10. Kodba, S., Perc, M. and Marhl, M., "Detecting chaos from a time series", European Journal of Physics, Vol. 26, No. 1, (2004), 205. doi: 10.1088/0143-0807/26/1/021
  11. Wernecke, H., Sándor, B. and Gros, C., "How to test for partially predictable chaos", Scientific reports, Vol. 7, No. 1, (2017), 1-12. doi: 10.1038/s41598-017-01083-x
  12. Gottwald, G.A. and Melbourne, I., "On the implementation of the 0–1 test for chaos", SIAM Journal on Applied Dynamical Systems, Vol. 8, No. 1, (2009), 129-145. doi: 10.1137/080718851
  13. Tempelman, J.R. and Khasawneh, F.A., "A look into chaos detection through topological data analysis", Physica D: Nonlinear Phenomena, Vol. 406, (2020), 132446. doi: 10.1016/j.physd.2020.132446
  14. Bhattacharya, C. and Ray, A., "Data-driven detection and classification of regimes in chaotic systems via hidden markov modeling", ASME Letters in Dynamic Systems and Control, Vol. 1, No. 2, (2021). doi: 10.1115/1.4047817
  15. Khosravi, A. and Gholipour, R., "Parameter estimation of loranz chaotic dynamic system using bees algorithm", International Journal of Engineering, Transactions C: Aspects, Vol. 26, No. 3, (2013), 257-262. doi: 10.5829/idosi.ije.2013.26.03c.05
  16. Kirichenko, L., Radivilova, T. and Bulakh, V., "Binary classification of fractal time series by machine learning methods", in International Scientific Conference “Intellectual Systems of Decision Making and Problem of Computational Intelligence”, Springer. (2019), 701-711. doi: 10.1007/978-3-030-26474-1_49
  17. Pourafzal, A. and Fereidunian, A., "A complex systems approach to feature extraction for chaotic behavior recognition", in 2020 6th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS), IEEE. (2020), 1-6. doi: 10.1109/ICSPIS51611.2020.9349551
  18. Safarihamid, K., Pourafzal, A. and Fereidunian, A., "A joint-entropy approach to time-series classification", in 2021 7th International Conference on Signal Processing and Intelligent Systems (ICSPIS), IEEE. (2021), 1-7. doi: 10.1109/ICSPIS54653.2021.9729371
  19. Hayles, N.K., "Chaos bound: Orderly disorder in contemporary literature and science, Cornell University Press, (2018).
  20. Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L. and Muller, P.-A., "Deep learning for time series classification: A review", Data Mining and Knowledge Discovery, Vol. 33, No. 4, (2019), 917-963. doi: 10.1007/s10618-019-00619-1
  21. Zhou, T., Chu, C., Xu, C., Liu, W. and Yu, H., "Detecting predictable segments of chaotic financial time series via neural network", Electronics, Vol. 9, No. 5, (2020), 823. doi: 10.3390/electronics9050823
  22. Wang, Z., Yan, W. and Oates, T., "Time series classification from scratch with deep neural networks: A strong baseline", in 2017 International joint conference on neural networks (IJCNN), IEEE. (2017), 1578-1585. doi: 10.1109/IJCNN.2017.7966039
  23. Pei, S.-C. and Tseng, C.-C., "Complex adaptive iir notch filter algorithm and its applications", IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, Vol. 41, No. 2, (1994), 158-163. doi: 10.1109/82.281849
  24. Cui, Z., Chen, W. and Chen, Y., "Multi-scale convolutional neural networks for time series classification", arXiv preprint arXiv:1603.06995, (2016). doi: 10.48550/arXiv.1603.06995
  25. Boullé, N., Dallas, V., Nakatsukasa, Y. and Samaddar, D., "Classification of chaotic time series with deep learning", Physica D: Nonlinear Phenomena, Vol. 403, (2020), 132261. doi: 10.1016/j.physd.2019.132261
  26. Lorenz, E.N., "Simplified dynamic equations applied to the rotating-basin experiments", Journal of the Atmospheric Sciences, Vol. 19, No. 1, (1962), 39-51. doi: 10.1175/1520-0469(1962)019<0039:SDEATT>2.0.CO;2
  27. Marsden, J.E. and McCracken, M., "The hopf bifurcation and its applications, Springer Science & Business Media, Vol. 19,  (2012). doi: 10.1007/978-1-4612-6374-6
  28. Grus, L., Crompvoets, J. and Bregt, A.K., "Spatial data infrastructures as complex adaptive systems", International Journal of Geographical Information Science, Vol. 24, No. 3, (2010), 439-463. doi: 10.1080/13658810802687319
  29. Diebold, F.X. and Kilian, L., "Measuring predictability: Theory and macroeconomic applications", Journal of Applied Econometrics, Vol. 16, No. 6, (2001), 657-669. doi: 10.1002/jae.619
  30. Hurst, H.E., "Long-term storage capacity of reservoirs", Transactions of the American Society of Civil Engineers, Vol. 116, No. 1, (1951), 770-799. doi: 10.1061/TACEAT.0006518
  31. Anderson, P.W., "More is different: Broken symmetry and the nature of the hierarchical structure of science", Science, Vol. 177, No. 4047, (1972), 393-396. doi: 10.1126/science.177.4047.393
  32. Gershenson, C. and Fernández, N., "Complexity and information: Measuring emergence, self‐organization, and homeostasis at multiple scales", Complexity, Vol. 18, No. 2, (2012), 29-44. doi: 10.1002/cplx.21424
  33. Chollet, F., "Deep learning with python, Simon and Schuster, (2021).
  34. Ioffe, S. and Szegedy, C., "Batch normalization: Accelerating deep network training by reducing internal covariate shift", in International conference on machine learning, PMLR., (2015), 448-456.
  35. Lucas, C., Lesani, H. and Fereidunian, A., "Distribution systems reconfiguration using pattern recognizer neural networks", International Journal of Engineering, Vol. 15, No. 2, (2002), 135-144.
  36. Torkkola, K., "Feature extraction by non-parametric mutual information maximization", Journal of Machine Learning Research, Vol. 3, No. Mar, (2003), 1415-1438.
  37. Kingma, D.P. and Ba, J., "Adam: A method for stochastic optimization", arXiv preprint arXiv:1412.6980, (2014). doi: 10.48550/arXiv.1412.6980
International Journal of Engineering
Volume 36, Issue 1
TRANSACTIONS A: Basics
January 2023
Pages 1-9

Files

Cited by

Share

How to cite

Statistics