Empowering Face Recognition Methods using a GAN-based Single Image Super-Resolution Network

Document Type : Original Article

Authors

Computer Engineering and IT Department, Shahrood University of Technology, Shahrood, Iran

Abstract

Face recognition is one of the most common authentication techniques widely used due to its easy access. In many face recognition applications, captured images are often of low resolution. Face recognition methods perform poorly on low resolution images because they are trained on high resolution face images. Although existing face hallucination methods may generate visually pleasing images, they cannot improve the performance of face recognition methods at low resolution as the structure of the face image and high-frequency details are not sufficiently preserved. Recent advances in deep learning have been used in this paper to propose a new face super-resolution approach to empower face recognition methods. In this paper, a Generative Adversarial Network is used to empower face recognition in low-resolution images. This network considers image edges and reconstructs high-frequency details to preserve the face structure. The proposed technique to generate super-resolved features is usable in any face recognition method. We have used some state-of-the-art face recognition methods to evaluate the proposed method. The results showed a significant impact of the proposed method on the accuracy of face recognition of low resolution images.

Keywords

Main Subjects

References

  1. Kortli, Y., Jridi, M., Al Falou, A. and Atri, M., "Face recognition systems: A survey", Sensors, Vol. 20, No. 2, (2020), 342. doi: 10.3390/s20020342.
  2. Annalakshmi, M., Roomi, S. and Naveedh, A.S., "A hybrid technique for gender classification with slbp and hog features", Cluster Computing, Vol. 22, No. 1, (2019), 11-20. doi: 10.1007/s10586-017-1585-x.
  3. Yang, W., Gao, H., Jiang, Y., Yu, J., Sun, J., Liu, J. and Ju, Z., "A cascaded feature pyramid network with non-backward propagation for facial expression recognition", IEEE Sensors Journal, Vol. 21, No. 10, (2020), 11382-11392. doi: 10.1109/JSEN.2020.2997182.
  4. Taigman, Y., Yang, M., Ranzato, M.A. and Wolf, L., "Deepface: Closing the gap to human-level performance in face verification", in Proceedings of the IEEE conference on computer vision and pattern recognition. (2014), 1701-1708.
  5. Schroff, F., Kalenichenko, D. and Philbin, J., "Facenet: A unified embedding for face recognition and clustering", in Proceedings of the IEEE conference on computer vision and pattern recognition, (2015), 815-823.
  6. Deng, J., Guo, J., Xue, N. and Zafeiriou, S., "Arcface: Additive angular margin loss for deep face recognition", in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (2019), 4690-4699.
  7. Parkhi, O.M., Vedaldi, A. and Zisserman, A., "Deep face recognition", (2015).
  8. Huang, G.B., Mattar, M., Berg, T. and Learned-Miller, E., "Labeled faces in the wild: A database forstudying face recognition in unconstrained environments", in Workshop on faces in'Real-Life'Images: detection, alignment, and recognition, (2008).
  9. He, K., Zhang, X., Ren, S. and Sun, J., "Deep residual learning for image recognition", in Proceedings of the IEEE conference on computer vision and pattern recognition, (2016), 770-778.
  10. Taigman, Y., Yang, M., Ranzato, M.A. and Wolf, L., "Web-scale training for face identification", in Proceedings of the IEEE conference on computer vision and pattern recognition, (2015), 2746-2754.
  11. Cao, Q., Shen, L., Xie, W., Parkhi, O.M. and Zisserman, A., "Vggface2: A dataset for recognising faces across pose and age", in 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), IEEE. (2018), 67-74. doi: 10.1109/FG.2018.00020.
  12. Guo, Y., Zhang, L., Hu, Y., He, X. and Gao, J., "Ms-celeb-1m: A dataset and benchmark for large-scale face recognition", in European conference on computer vision, Springer. (2016), 87-102. doi: 10.1007/978-3-319-46487-9_6.
  13. Yi, D., Lei, Z., Liao, S. and Li, S.Z., "Learning face representation from scratch", arXiv preprint arXiv:1411.7923, (2014). doi: 10.48550/arXiv.1411.7923.
  14. Lui, Y.M., Bolme, D., Draper, B.A., Beveridge, J.R., Givens, G. and Phillips, P.J., "A meta-analysis of face recognition covariates", in 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems, IEEE. (2009), 1-8. doi: 10.1109/BTAS.2009.5339025.
  15. Zou, W.W. and Yuen, P.C., "Very low resolution face recognition problem", IEEE Transactions on Image Processing, Vol. 21, No. 1, (2011), 327-340. doi: 10.1109/TIP.2011.2162423.
  16. Van Ouwerkerk, J., "Image super-resolution survey", Image and Vision Computing, Vol. 24, No. 10, (2006), 1039-1052. doi: 10.1016/j.imavis.2006.02.026.
  17. Wu, J., Ding, S., Xu, W. and Chao, H., "Deep joint face hallucination and recognition", arXiv preprint arXiv:1611.08091, (2016). doi: 10.48550/arXiv.1611.08091.
  18. Lu, Z., Jiang, X. and Kot, A., "Deep coupled resnet for low-resolution face recognition", IEEE Signal Processing Letters, Vol. 25, No. 4, (2018), 526-530. doi: 10.1109/LSP.2018.2810121.
  19. Luevano, L.S., Chang, L., Méndez-Vázquez, H., Martínez-Díaz, Y. and González-Mendoza, M., "A study on the performance of unconstrained very low resolution face recognition: Analyzing current trends and new research directions", IEEE Access, Vol. 9, \ (2021), 75470-75493. doi: 10.1109/ACCESS.2021.3080712.
  20. Grm, K., Scheirer, W.J. and Štruc, V., "Face hallucination using cascaded super-resolution and identity priors", IEEE Transactions on Image Processing, Vol. 29, (2019), 2150-2165. doi: 10.1109/TIP.2019.2945835.
  21. Seyyedyazdi, S. and Hassanpour, H., "Super-resolution of defocus blurred images", International Journal of Engineering, Transactions A: Basics, Vol. 33, No. 4, (2020), 539-545. doi: 10.5829/ije.2020.33.04a.04.
  22. Banerjee, S. and Das, S., "Lr-gan for degraded face recognition", Pattern Recognition Letters, Vol. 116,  (2018), 246-253. doi: 10.1016/j.patrec.2018.10.034.
  23. Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B. and Bharath, A.A., "Generative adversarial networks: An overview", IEEE Signal Processing Magazine, Vol. 35, No. 1, (2018), 53-65. doi: 10.1109/MSP.2017.2765202.
  24. Zangeneh, E., Rahmati, M. and Mohsenzadeh, Y., "Low resolution face recognition using a two-branch deep convolutional neural network architecture", Expert Systems with Applications, Vol. 139, (2020), 112854. doi: 10.1016/j.eswa.2019.112854.
  25. Tan, W., Yan, B. and Bare, B., "Feature super-resolution: Make machine see more clearly", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. (2018), 3994-4002.
  26. Abbaspoor, N. and Hassanpour, H., "Face recognition in a large dataset using a hierarchical classifier", Multimedia Tools and Applications, (2022), 1-19. doi: 10.1007/s11042-022-12382-5
  27. Hassanpour, H. and Ghasemi, M., "A three-stage filtering approach for face recognition", International Journal of Engineering, Transactions B: Applications Vol. 34, No. 8, (2021). doi: 10.5829/ije.2021.34.08b.06.
  28. Dey, A. and Dasgupta, K., "Emotion recognition using deep learning in pandemic with real-time email alert", in Proceedings of Third International Conference on Communication, Computing and Electronics Systems, Springer. (2022), 175-190. doi: 10.1007/978-981-16-8862-1_13.
  29. Chaabane, S.B., Hijji, M., Harrabi, R. and Seddik, H., "Face recognition based on statistical features and svm classifier", Multimedia Tools and Applications, Vol. 81, No. 6, (2022), 8767-8784. doi: 10.1007/s11042-021-11816-w.
  30. Shahbakhsh, M.B. and Hassanpour, H., "Enhancing face super-resolution via improving the edge and identity preserving network", in 2021 7th International Conference on Signal Processing and Intelligent Systems (ICSPIS), IEEE, (2021), 1-4. doi: 10.1109/ICSPIS54653.2021.9729372.
  31. Dong, C., Loy, C.C., He, K. and Tang, X., "Learning a deep convolutional network for image super-resolution", in European conference on computer vision, Springer, (2014), 184-199. doi: 10.1007/978-3-319-10593-2_13.
  32. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J. and Wang, Z., "Photo-realistic single image super-resolution using a generative adversarial network", in Proceedings of the IEEE conference on computer vision and pattern recognition, (2017), 4681-4690.
  33. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D. and Wang, Z., "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network", in Proceedings of the IEEE conference on computer vision and pattern recognition, (2016), 1874-1883.
  34. Wang, Z., Chen, J. and Hoi, S.C., "Deep learning for image super-resolution: A survey", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, No. 10, (2020), 3365-3387. doi: 10.1109/TPAMI.2020.2982166.
  35. Montufar, G.F., Pascanu, R., Cho, K. and Bengio, Y., "On the number of linear regions of deep neural networks", Advances in Neural Information Processing Systems, Vol. 27, (2014).
  36. Kim, J., Lee, J.K. and Lee, K.M., "Accurate image super-resolution using very deep convolutional networks", in Proceedings of the IEEE conference on computer vision and pattern recognition, (2016), 1646-1654.
  37. Kim, J., Lee, J.K. and Lee, K.M., "Deeply-recursive convolutional network for image super-resolution", in Proceedings of the IEEE conference on computer vision and pattern recognition, (2016), 1637-1645.
  38. Lim, B., Son, S., Kim, H., Nah, S. and Mu Lee, K., "Enhanced deep residual networks for single image super-resolution", in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, (2017), 136-144.
  39. Chen, Y., Tai, Y., Liu, X., Shen, C. and Yang, J., "Fsrnet: End-to-end learning face super-resolution with facial priors", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2018), 2492-2501.
  40. Newell, A., Yang, K. and Deng, J., "Stacked hourglass networks for human pose estimation", in European conference on computer vision, Springer, (2016), 483-499. doi: 10.1007/978-3-319-46484-8_29.
  41. Kim, D., Kim, M., Kwon, G. and Kim, D.-S., "Progressive face super-resolution via attention to facial landmark", arXiv preprint arXiv:1908.08239, (2019). doi: 10.48550/arXiv.1908.08239.
  42. Pavithra, L. and Sharmila, T.S., "An efficient framework for image retrieval using color, texture and edge features", Computers & Electrical Engineering, Vol. 70, (2018), 580-593. doi: 10.1016/j.compeleceng.2017.08.030
  43. Mortezaie, Z., Hassanpour, H. and Asadi Amiri, S., "An adaptive block based un-sharp masking for image quality enhancement", Multimedia Tools and Applications, Vol. 78, No. 16, (2019), 23521-23534. doi: 10.1007/s11042-019-7594-4
  44. Liu, H., Zheng, X., Han, J., Chu, Y. and Tao, T., "Survey on gan-based face hallucination with its model development", IET Image Processing, Vol. 13, No. 14, (2019), 2662-2672. doi: 10.1049/iet-ipr.2018.6545.
  45. Podpora, M., Korbas, G.P. and Kawala-Janik, A., "Yuv vs rgb-choosing a color space for human-machine interaction", in FedCSIS (Position Papers), (2014), 29-34. doi: 10.15439/2014F206.
  46. Phillips, P.J., Moon, H., Rizvi, S.A. and Rauss, P.J., "The feret evaluation methodology for face-recognition algorithms", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 10, (2000), 1090-1104. doi: 10.1109/34.879790.
  47. Milborrow, S., Morkel, J. and Nicolls, F., "The muct landmarked face database", Pattern Recognition Association of South Africa, Vol. 201,  (2010).
  48. Thomaz, C.E. and Giraldi, G.A., "A new ranking method for principal components analysis and its application to face image analysis", Image and Vision Computing, Vol. 28, No. 6, (2010), 902-913. doi: 10.1016/j.imavis.2009.11.005
  49. Liu, Z., Luo, P., Wang, X. and Tang, X., "Large-scale celebfaces attributes (celeba) dataset", Retrieved August, Vol. 15, No. 2018, (2018), 11.
  50. Kim, J., Li, G., Yun, I., Jung, C. and Kim, J., "Edge and identity preserving network for face super-resolution", Neurocomputing, Vol. 446, (2021), 11-22. doi: 10.1016/j.neucom.2021.03.048
  51. Wang, X., Li, Y., Zhang, H. and Shan, Y., "Towards real-world blind face restoration with generative facial prior", in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2021), 9168-9178.
  52. Nikan, F. and Hassanpour, H., "Face recognition using non-negative matrix factorization with a single sample per person in a large database", Multimedia Tools and Applications, Vol. 79, No. 37, (2020), 28265-28276. doi: 10.1007/s11042-020-09394-4.
  53. Jose, E., Greeshma, M., Haridas, M.T. and Supriya, M., "Face recognition based surveillance system using facenet and mtcnn on jetson tx2", in 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS), IEEE, (2019), 608-613. doi: 10.1109/ICACCS.2019.8728466.
  54. Prasad, P.S., Pathak, R., Gunjan, V.K. and Ramana Rao, H., Deep learning based representation for face recognition, in Iccce 2019. 2020, Springer.419-424. doi: 10.1007/978-981-13-8715-9_50.
  55. Li, X., Chen, C., Zhou, S., Lin, X., Zuo, W. and Zhang, L., "Blind face restoration via deep multi-scale component dictionaries", in European Conference on Computer Vision, Springer, (2020), 399-415. doi: 10.1007/978-3-030-58545-7_23.
International Journal of Engineering
Volume 35, Issue 10
TRANSACTIONS A: Basics
October 2022
Pages 1858-1866

Files

Cited by

Share

How to cite

Statistics