Using Sliding Mode Controller and Eligibility Traces for Controlling the Blood Glucose in Diabetic Patients at the Presence of Fault

Document Type : Original Article


1 Faculty of Electrical and Robotics Engineering, Shahrood University of Technology, Shahrood, Iran

2 Department of Electrical Engineering, Faculty of Engineering, Ferdowsi University of Mashhad, Mashhad, Iran


Some people suffering from diabetes use insulin injection pumps to control the blood glucose level. Sometimes, the fault may occur in the sensor or actuator of these pumps. The main objective of this paper is controlling the blood glucose level at the desired level and fault-tolerant control of these injection pumps. To this end, the eligibility traces algorithm is combined with the sliding mode control. The eligibility traces algorithm is one of the newest solving methods of the Reinforcement Learning approach. The major disadvantage of the sliding mode control method is the chattering phenomenon. In this paper, the novel idea is the combination of these methods to remove the chattering phenomena in simulation results. To demonstrate the superiority of the proposed method, it is compared with another combinatory method that is the sliding mode control and Artificial Neural Networks. Simulation results reveal that the combination of the eligibility traces algorithm and the sliding mode control can control the blood glucose level and insulin with a higher speed and bring them to the desired level, even in the case the sensor and actuator faults are present in the system. When the proposed hybrid method is used, the injected dosage of the drug is lower, which will result in reduced side effects. Finally, the noise, as well as the uncertainty in system parameters and initial conditions are applied to the system to investigate the performance of the proposed controller, under the faulty condition.


1. Wild, S., Roglic, G., Green, A., Sicree, R. and King, H., "Global
prevalence of diabetes: Estimates for the year 2000 and
projections for 2030", Diabetes Care,  Vol. 27, No. 5, (2004),
2. Oroojeni Mohammad Javad, M., Agboola, S., Jethwani, K.,
Zeid, I. and Kamarthi, S., "Reinforcement learning algorithm for
blood glucose control in diabetic patients", in ASME 2015
International Mechanical Engineering Congress and Exposition,
American Society of Mechanical Engineers Digital Collection.,
3. Daskalaki, E., Diem, P. and Mougiakakou, S.G., "Personalized
tuning of a reinforcement learning control algorithm for glucose
regulation", in 2013 35th Annual international conference of the
IEEE engineering in medicine and biology society (EMBC),
IEEE., (2013), 3487-3490. 
4. Hochberg, I., Feraru, G., Kozdoba, M., Mannor, S.,
Tennenholtz, M. and Yom-Tov, E., "A reinforcement learning
system to encourage physical activity in diabetes patients",
arXiv preprint arXiv:1605.04070,  (2016). 
5. Yasini, S., Naghibi-Sistani, M. and Karimpour, A., "Agentbased
Journal of Applied Science, Engineering and
Technology,  Vol. 5, No. 1, (2009), 40-49. 
6. Weng, W.-H., Gao, M., He, Z., Yan, S. and Szolovits, P.,
"Representation and reinforcement learning for personalized
glycemic control in septic patients", arXiv preprint
arXiv:1712.00654,  (2017). 
7. Grant, P., "A new approach to diabetic control: Fuzzy logic and
insulin pump technology", Medical Engineering & Physics, 
Vol. 29, No. 7, (2007), 824-827. 
8. Campos-Delgado, D.U., Hernández-Ordoñez, M., Femat, R. and
Gordillo-Moscoso, A., "Fuzzy-based controller for glucose
regulation in type-1 diabetic patients by subcutaneous route",
IEEE Transactions on Biomedical Engineering,  Vol. 53, No.
11, (2006), 2201-2210. 
9. Kavakiotis, I., Tsave, O., Salifoglou, A., Maglaveras, N.,
Vlahavas, I. and Chouvarda, I., "Machine learning and data
mining methods in diabetes research", Computational and
Structural Biotechnology Journal,  Vol. 15, (2017), 104-116. 
10. Fernandes, F., Vicente, H., Abelha, A., Machado, J., Novais, P.
and Neves, J., "Artificial neural networks in diabetes control", in
2015 Science and Information Conference (SAI), IEEE., (2015),
11. Yavari, K., "Anti-angiogenesis therapy of cancer cells using
153sm-bevasesomab", Emerging Science Journal,  Vol. 2, No.
3, (2018), 130-139. 
12. Skach, J., Punčochář, I. and Lewis, F.L., "Temporal-difference
q-learning in active fault diagnosis", in 2016 3rd Conference on
Control and Fault-Tolerant Systems (SysTol), IEEE., (2016),
13. Hsu, Y.Y. and Yu, C.C., "A self-learning fault diagnosis system
based on reinforcement learning", Industrial & Engineering
Chemistry Research,  Vol. 31, No. 8, (1992), 1937-1946. 
14. He, Q. and Shayman, M.A., "Using reinforcement learning for
pro-active network fault management", in WCC 2000-ICCT
2000. International Conference on Communication Technology
Proceedings (Cat. No. 00EX420), IEEE. Vol. 1, (2000), 515521.
15. Frank, P.M., "Residual evaluation for fault diagnosis based on
adaptive fuzzy thresholds",  (1995). doi: 10.1049/ic:19950512 
16. Tagina, M., "A novel fault detection approach combining
adaptive thresholding and fuzzy reasoning", arXiv preprint
arXiv:1203.5454, (2012). 
17. Berdnikov, V. and Lokhin, V., "Synthesis of guaranteed stability
regions of a nonstationary nonlinear system with a fuzzy
controller", Civil Engineering Journal,  Vol. 5, No. 1, (2019),
18. Cao, J., "Using reinforcement learning for agent-based network
fault diagnosis system", in 2011 IEEE International Conference
on Information and Automation, IEEE., (2011), 750-754. 
19. Mahmoudi, Z., Boiroux, D., Hagdrup, M., Nørgaard, K.,
Poulsen, N.K., Madsen, H. and Jørgensen, J.B., "Application of
the continuous-discrete extended kalman filter for fault detection
in continuous glucose monitors for type 1 diabetes", in 2016
European Control Conference (ECC), IEEE., (2016), 714-719. 
20. Song, G., Zhao, C. and Sun, Y., "A classification-based fault
detection method for continuous glucose monitoring (CGM)", in
2016 12th World Congress on Intelligent Control and
Automation (WCICA), IEEE., (2016), 956-961. 
21. Zhao, C. and Fu, Y., "Automatic and online fault detection of
sensor problems using continuous glucose monitoring data for
type 1 diabetes", in Proceedings of the 33rd Chinese Control
Conference, IEEE., (2014), 3181-3186. 
22. Škach, J. and Punčochář, I., "Input design for fault detection
using extended kalman filter and reinforcement learning",
IFAC-PapersOnLine,  Vol. 50, No. 1, (2017), 7302-7307. 
23. Farivar, F. and Ahmadabadi, M.N., "Continuous reinforcement
learning to robust fault tolerant control for a class of unknown
nonlinear systems", Applied Soft Computing,  Vol. 37, (2015),
24. Han, K.-Z., Feng, J. and Cui, X., "Fault-tolerant optimised
tracking control for unknown discrete-time linear systems using
a combined reinforcement learning and residual compensation
methodology", International Journal of Systems Science,  Vol.
48, No. 13, (2017), 2811-2825. 
25. Zhang, D., Lin, Z. and Gao, Z., "Reinforcement-learning based
fault-tolerant control", in 2017 IEEE 15th International
Conference on Industrial Informatics (INDIN)., (2017), 671-676. 
26. Jiang, H., Zhang, H., Liu, Y. and Han, J., "Neural-networkbased
control scheme for a class of nonlinear systems with
actuator faults via data-driven reinforcement learning method",
Neurocomputing,  Vol. 239, (2017), 1-8. 
27. Liu, L., Wang, Z. and Zhang, H., "Adaptive fault-tolerant
tracking control for mimo discrete-time systems via
reinforcement learning algorithm with less learning parameters",
IEEE Transactions on Automation Science and Engineering, 
Vol. 14, No. 1, (2016), 299-313. 
28. Afshari, H.H., Al-Ani, D. and Habibi, S., "Fault prognosis of
roller bearings using the adaptive auto-step reinforcement
learning technique", in ASME 2014 Dynamic Systems and
Control Conference, American Society of Mechanical Engineers
Digital Collection., (2014). 
29. Ahmadzadeh, S.R., Kormushev, P. and Caldwell, D.G., "Multiobjective
reinforcement learning for auv thruster failure
recovery", in 2014 IEEE Symposium on Adaptive Dynamic
Programming and Reinforcement Learning (ADPRL), IEEE.,
(2014), 1-8. 
30. Maleki, M.R., "Online monitoring and fault diagnosis of
multivariate-attribute process mean using neural networks and
discriminant analysis technique", International Journal of
Engineering, Transactions B: Applications, Vol. 28, No. 11,
(2015), 1634-1643. 
31. Heidari, M., "Fault detection of bearings using a rule-based
classifier ensemble and genetic algorithm", International 
Journal of Engineering, Transactions A: Basics, Vol. 30, No.
4, (2017), 604-609. 
32. Hashemi, F. and Mohammadi, M., "Combination continuous
action reinforcement learning automata & pso for design of pid
controller for avr system", International Journal of
Engineering-Transactions A: Basics,  Vol. 28, No. 1, (2014),
33. Golzarfar, A., Sedighi, A.R. and Asadi, A., "Optimal placement
and sizing of fault current limiter in a real network: A case
study", International Journal of Engineering, Transactions C:
Aspects,  Vol. 28, No. 3, (2015), 402-409. 
34. Roy, A. and Parker, R.S., Dynamic modeling of exercise effects
on plasma glucose and insulin levels. 2007, SAGE Publications. 
35. Magni, P. and Bellazzi, R., "A stochastic model to assess the
variability of blood glucose time series in diabetic patients selfmonitoring",
IEEE Transactions on Biomedical Engineering, 
Vol. 53, No. 6, (2006), 977-985. 
36. Makroglou, A., Li, J. and Kuang, Y., "Mathematical models and
software tools for the glucose-insulin regulatory system and
diabetes: An overview", Applied Numerical Mathematics,  Vol.
56, No. 3-4, (2006), 559-573. 
37. Kueh, Y.C., Morris, T., Borkoles, E. and Shee, H., "Modelling
of diabetes knowledge, attitudes, self-management, and quality
of life: A cross-sectional study with an australian sample",
Health and Quality of Life Outcomes,  Vol. 13, No. 1, (2015),
38. Nani, F. and Jin, M., "Mathematical modeling and simulations
of the pathophysiology of type-2 diabetes mellitus", in 2015 8th
International Conference on Biomedical Engineering and
Informatics (BMEI), IEEE., (2015), 296-300.  

39. Moore, J.R. and Adler, F., "Mathematical modeling of type 1
diabetes in the nod mouse: Separating incidence and age of
onset", arXiv preprint arXiv:1412.6566,  (2014). 
40. Mahata, A., Mondal, S.P., Alam, S. and Roy, B., "Mathematical
model of glucose-insulin regulatory system on diabetes mellitus
in fuzzy and crisp environment", Ecological Genetics and
Genomics,  Vol. 2, (2017), 25-34. 
41. Lehmann, E. and Deutsch, T., "A physiological model of
glucose-insulin interaction in type 1 diabetes mellitus", Journal
of Biomedical Engineering,  Vol. 14, No. 3, (1992), 235-242. 
42. Onvlee, A., Blauw, H., Middelhuis, N. and Zwart, H., "In silico
modeling of patients with type 1 diabetes mellitus", Master,
University of Twente,  (2016). 
43. Sharief, A.A. and Sheta, A., "Developing a mathematical model
to detect diabetes using multigene genetic programming",
International Journal of Advanced Research in Artificial
Intelligence,  Vol. 3, No. 10, (2014). doi:
44. Sandhya danKumar, D., "Mathematical model for
glucoseinsulinregulatorysystem of diabetes mellitus", Advances
in Applied Mathematical Biosciences,  Vol. 2, No. 1, (2011),
45. Sutton, R.S. and Barto, A.G., "Reinforcement learning: An
introduction",  Vol., No., (2011). 
46. Gholizade-Narm, H. and Noori, A., "Control the population of
free viruses in nonlinear uncertain hiv system using q-learning",
International Journal of Machine Learning and Cybernetics, 
Vol. 9, No. 7, (2018), 1169-1179. 
47. Slotine, J.-J.E. and Li, W., "Applied nonlinear control, Prentice
hall Englewood Cliffs, NJ,  Vol. 199,  (1991). 
48. Hagan, M.T., Demuth, H.B. and Beale, M.H., "Neural network
design (electrical engineering)", Thomson Learning, (1995).