Semantic Segmentation of Lesions from Dermoscopic Images using Yolo-DeepLab Networks

Document Type : Original Article

Authors

1 Department of Industrial Engineering, K. N. Toosi University of Technology, Pardis Street, Molla Sadra Ave, Tehran, Iran

2 Department of Electrical Engineering, Faculty of Engineering, Golestan University, Gorgan, Iran

Abstract

Accurate segmentation of lesions from dermoscopic images is very important for timely diagnosis and treatment of skin cancers. Due to the variety of shapes, sizes, colors, and locations of lesions in dermoscopic images, automatic segmentation of skin lesions remains a challenge. In this study, a two-stage method for the segmentation of skin lesions based on deep learning is presented. In the first stage, convolutional neural networks (CNNs) estimate the approximate size and location of the lesion. A sub-image around the estimated bounding box is cropped from the original image. The sub-image is resized to an image of a predefined size. In order to segment the exact area of the lesion from the normal image, other CNNs are used in the DeepLab structure. The accuracy of the normalization stage has a significant impact on the final performance. In order to increase the normalization accuracy, a combination of four networks in the structure of Yolov3 is used. Two approaches are proposed to combine Yolov3 structures. The segmentation results of two networks in the DeepLab v3+ structure are also combined to improve the performance of the second stage. Another challenge is the small number of training images. To overcome this problem, the data augmentation is used, as well as using different modes of an image in each stage. In order to evaluate the proposed method, experiments are performed on the well-known ISBI 2017 dataset. Experimental results show that the proposed lesion segmentation method outperforms the state-of-the-art methods.

Keywords

References

1.     National Cancer Institute, "SEER Cancer Stat Facts: Melanoma of the Skin", (2017). Retrieved from Bethesda, MD: https://seer. cancer. gov/statfacts/html/melan. html.
2.     Balch, C. M., Gershenwald, J. E., Soong, S.-J., Thompson, J. F., Atkins, M. B., Byrd, D. R., Buzaid, A. C., and Al., E., "Final version of 2009 AJCC melanoma staging and classification", Journal of Clinical Oncology, Vol. 27, No. 36, (2009), 6199–6206. doi:10.1200/JCO.2009.23.4799
3.     Vestergaard, M. E., Macaskill, P. H. P. M., Holt, P. E., and Menzies, S. W., "Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta-analysis of studies performed in a clinical setting", British Journal of Dermatology, Vol. 159 No. (3), (2008), 669–676. doi 10.1111/j.1365-2133.2008.08713.x
4.     Binder, M., Schwarz, M., Winkler, A., Steiner, A., Kaider, A., and Wolff, K. H., "a useful tool for the diagnosis of pigmented skin lesions for formally trained dermatologists", Arch Dermatol, Vol. 131, No. 3, (1995), 286–291. doi: 10.1001/archderm.1995.01690150050011
5.     Celebi, M. E., Iyatomi, H., Schaefer, G., and Stoecker, W. V., "Lesion border detection in dermoscopy images", Computerized Medical Imaging and Graphics, Vol. 33, No. 2, (2009), 148–153. doi:10.1016/j.compmedimag.2008.11.002
6.     Ganster, H., Pinz, P., Rohrer, R., Wildling, E., Binder, M., and Kittler, H., "Automated melanoma recognition", IEEE Transactions on Medical Imaging, Vol. 20, No. 3, (2001), 233–239. doi:10.1109/42.918473
7.     Celebi, M. E., Wen, Q., Iyatomi, H., Shimizu, K., Zhou, H., and Schaefer, G., "A state-of-the-art survey on lesion border detection in dermoscopy images", Dermoscopy Image Analysis, Vol. 10, (2015), 97–129. doi:10.1201/B19107-5
8.     Al-masni, M. A., Al-antari, M. A., Choi, M., Han, S., and Kim, T., "Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks", Computer Methods and Programs in Biomedicine, Vol. 162, (2018), 221–231. doi:10.1016/j.cmpb.2018.05.027
9.     Hassanpour, H., and Yousefian, H., "An improved pixon-based approach for image segmentation", International Journal of Engineering, Transactions A: Basics, Vol. 24, No. 1, (2011), 25–35. Retrieved from http://www.ije.ir/article_71880_c9bb4bcef741659a548a0979992905c5.pdf
10.   Nikbakhsh, N., Baleghi Damavandi, Y., and Agahi, H., "Plant classification in images of natural scenes using segmentations fusion", International Journal of Engineering Transactions C: Aspects, Vol. 33, No. 9, (2020), 1743–1750. doi:10.5829/ije.2020.33.09c.07
11.   Liu, X., Deng, Z., and Yang, Y., "Recent progress in semantic image segmentation", Artificial Intelligence Review, Vol. 52, No. 2, (2019), 1089–1106. doi:10.1007/s10462-018-9641-3
12.   LeCun, Y., Bengio, Y., and Hinton, G., "Deep learning", Nature, Vol. 521, (2015), 436–444. doi: 10.1038/nature14539
13.   Krizhevsky, A., Sutskever, I., and Hinton, G. E., "Imagenet classification with deep convolutional neural networks", Advances in Neural Information Processing Systems, Vol. 25, No. 2, (2012), 1097–1105. doi:10.1145/3065386
14.   Al-Masni, M. A., Al-Antari, M. A., Park, J. M., Gi, G., Kim, T. Y., Rivera, P., Valarezo, E., Han, S.-M., and Kim, T.-S., "Detection and classification of the breast abnormalities in digital mammograms via regional Convolutional Neural Network", Proceedings of 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (2017), 1230–1233. doi:10.1109/EMBC.2017.8037053
15.   Ciregan, D., Meier, U., and Schmidhuber, J., "Multi-column deep neural networks for image classification", 2012 IEEE Conference on Computer Vision and Pattern Recognition, (2012), 3642–3649. doi:10.1109/CVPR.2012.6248110
16.   Cernazanu-Glavan, C., and Holban, S., "Segmentation of bone structure in X-ray images using convolutional neural network", Advances in Electrical and Computer Engineering, Vol. 13, No. 1, (2013), 87–94. doi:10.4316/AECE.2013.01015
17.   ISIC: ISBI, Skin lesion analysis towards melanoma detec- tion. Vol. 3, (2016). [Online]. Retrieved from Https://Goo.Gl/2A1913, Accessed 2016.
18.   Burdick, J., Marques, O., Weinthal, J., and Furht, B., "Rethinking Skin Lesion Segmentation in a Convolutional Classifier", Journal of Digital Imaging, Vol. 31, No. 4, (2017), 435–440. doi:10.1007/s10278-017-0026-y
19.   Yu, L., Chen, H., Dou, Q., Qin, J., and Heng, P.-A., "Automated melanoma recognition in dermoscopy images via very deep residual networks", IEEE Transactions on Medical Imaging, Vol. 36, No. 4, (2017), 994–1004. doi:10.1109/TMI.2016.2642839
20.   Yuan, Y., Chao, M., and Lo, Y.-C., "Automatic skin lesion segmentation using deep fully convolutional networks with Jaccard distance", IEEE Transactions on Medical Imaging, Vol. 36, No. 9, (2017), 1876–1886. doi:10.1109/tmi.2017.2695227
21.   Lin, B. S., Michael, K., Kalra, S., and Tizhoosh, H. R., "Skin lesion segmentation: UNets versus clustering", 2017 IEEE Symposium Series on Computational Intelligence (SSCI), (2017). doi:10.1109/SSCI.2017.8280804
22.   Li, Y., and Shen, L., "Skin lesion analysis towards melanoma detection using deep learning network", Sensors, Vol. 18, No. 2 (556), (2018), 1–16. doi:10.3390/s18020556
23.   Bi, L., Kim, J., Ahn, E., and Feng, D., "Automatic Skin Lesion Analysis using Large-scale Dermoscopy Images and Deep Residual Networks", (2017), 6–9. Http://Arxiv.Org/Abs/1703.04197
24.   Yuan, Y., and Lo, Y.-C., "Improving Dermoscopic Image Segmentation With Enhanced Convolutional-Deconvolutional Networks", IEEE Journal of Biomedical and Health Informatics, Vol. 23, No. 2, (2019), 519–526. doi:10.1109/JBHI.2017.2787487
25.   Baghersalimi, S., Bozorgtabar, B., Schmid-saugeon, P., Ekenel, H. K., and Thiran, J., "DermoNet : densely linked convolutional neural network for efficient skin lesion segmentation", EURASIP Journal on Image and Video Processing, Vol. 71, (2019), 1–10. doi:https://doi.org/10.1186/s13640-019-0467-y
26.   Hasan, M. K., Dahal, L., Samarakoon, P. N., Tushar, F. I., and Martí, R., "DSNet: Automatic dermoscopic skin lesion segmentation", Computers in Biology and Medicine, Vol. 120, No. April, (2020), 103738. doi:10.1016/j.compbiomed.2020.103738
27.   Tang, P., Liang, Q., Yan, X., Xiang, S., Sun, W., Zhang, D., and Coppola, G., "Efficient skin lesion segmentation using separable-Unet with stochastic weight averaging", Computer Methods and Programs in Biomedicine, Vol. 178, (2019), 289–301. doi:10.1016/j.cmpb.2019.07.005
28.   Litjens, G., Kooi, T., Bejnordi, B. E., Arindra, A., Setio, A., Ciompi, F., Ghafoorian, M., Laak, J. A. W. M. Van Der, Ginneken, B. Van, and Sánchez, C. I., "A survey on deep learning in medical image analysis", Medical Image Analysis, Vol. 42, No. December 2012, (2017), 60–88. doi:10.1016/j.media.2017.07.005
29.   Shen, W., Yang, F., Mu, W., Yang, C., Yang, X., and Tian, J., "Automatic localization of vertebrae based on convolutional neural networks", Proceedings of the SPIE on Medical Imaging, Vol. 9413, (2015), 94132E. doi:10.1117/12.2081941
30.   Kawahara, J., and Hamarneh, G., "Multi-resolution-tract CNN with hybrid pretrained and skin-lesion trained layers", Machine Learning in Medical Imaging. Cham: Springer International Publishing, Vol. 10019, (2016), 164–171. doi: 10.1007/978-3-319-47157-0_20
31.   Codella, N. C. F., Gutman, D., Celebi, M. E., Helba, B., Marchetti, M. A., Dusza, S. W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., and Halpern, A., "Skin Lesion Analysis Toward Melanoma Detection: a Challenge at The 2017 International Symposium on Biomedical Imaging (ISBI), Hosted By The International Skin Imaging Collaboration (ISIC)", Retrieved from ArXiv:1710.05006v3, (2017).
32.   ISIC, Skin lesion analysis towards melanoma detection. (2017). Available: [Accessed 08 02 2020], Retrieved from Https://Challenge.Isic-Archive.Com/Landing/2017
33.   Vaidya, B., and Paunwala, C., "Deep Learning Architectures for Object Detection and Classification (Chapter 4)", Springer International Publishing, (2019), 53–79. doi:10.1007/978-3-030-03131-2
34.   Girshick, R., Donahue, J., Darrell, T., and Malik, J., "Rich feature hierarchies for accurate object detection and semantic segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2014), 580–587.
35.   Girshick, R., "Fast R-CNN", 2015 IEEE International Conference on Computer Vision (ICCV), (2015). doi:10.1109/ICCV.2015.169
36.   Ren, S., He, K., Girshick, R., and Sun, J., "Faster R-CNN : Towards Real-Time Object Detection with Region Proposal Networks", NIPS’15: Proceedings of the 28th International Conference on Neural Information Processing Systems, Vol. 1, (2015), 91–99.
37.   Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C., and Berg, A. C., "SSD : Single Shot MultiBox Detector", Computer Vision – ECCV 2016. Lecture Notes in Computer Science, Vol 9905. Springer, Cham, (2016). doi:10.1007/978-3-319-46448-0_2
38.   Redmon, J., Divvala, S., Girshick, R., and Farhadi, A., "You Only Look Once: Unified, Real-Time Object Detection", 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016). doi:10.1109/CVPR.2016.91
39.   Zou, Z., Shi, Z., Guo, Y., and Ye, J., "Object Detection in 20 Years: A Survey", (2019), 1–39. Retrieved from Http://Arxiv.Org/Abs/1905.05055
40.   He, Y., Zeng, H., Fan, Y., Ji, S., and Wu, J., "Application of Deep Learning in Integrated Pest Management: A Real-Time System for Detection and Diagnosis of Oilseed Rape Pests", Mobile Information Systems, Vol. 2019, (2019), 1–14. doi:10.1155/2019/4570808
41.   Redmon, J., and Farhadi, A., "YOLO9000: Better, faster, stronger", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017). doi:10.1109/CVPR.2017.690
42.   Redmon, J., and Farhadi, A., "YOLOv3: An Incremental Improvement", (2018). Retrieved from ArXiv:1804.02767v1
43.   Haridas, R., and R L, J., "Convolutional neural networks: A comprehensive survey", International Journal of Applied Engineering Research, Vol. 14, No. 3, (2019), 780–789. doi:10.37622/IJAER/14.3.2019.780-789.
44.   Simonyan, K., and Zisserman, A., "Very Deep Convolutional Networks For Large-Scale Image Recognition", International Conference on Learning Representations, (2015), 1–14.
45.   He, K., Zhang, X., Ren, S., and Sun, J., "Deep Residual Learning for Image Recognition", 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016). doi:10.1109/CVPR.2016.90
46.   Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A., "Going Deeper with Convolutions", 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2015). doi:10.1109/CVPR.2015.7298594
47.   Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A., "Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning", AAAI’17: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, (2017), 4278–4284.
48.   Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. L., "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", International Conference on Learning Representations, (2015). Retrieved from https://arxiv.org/abs/1412.7062
49.   Chen, L., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H., "Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation", Proceedings of the European Conference on Computer Vision (ECCV), (2018), 801–818.
50.   Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. L., "DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, No. 4, (2017), 834–848. doi:10.1109/TPAMI.2017.2699184
51.   Chen, L.-C., Papandreou, G., Schroff, F., and Adam, H., "Rethinking Atrous Convolution for Semantic Image Segmentation", (2017). ArXiv Preprint Http://Arxiv.Org/Abs/1706.05587
52.   Al-antari, M. A., Al-masni, M. A., Park, S. U., Park, J. H., Metwally, M. K., Kadah, Y. M., Han, S. M., and Kim, T.-S., "An automatic computer-aided diagnosis system for breast cancer in digital mammograms via deep belief network", Journal of Medical and Biological Engineering, Vol. 38, (2018), 443–456. doi:10.1007/S40846-017-0321-6
53.   Powers, D. M., "Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation", Journal of Machine Learning Technologies, Vol. 2, No. 1, (2011), 37–63. Retrieved from https://arxiv.org/abs/2010.16061
54.   Pereira, S., Pinto, A., Alves, V., and Silva, C. A., "Brain tumor segmentation using convolutional neural networks in MRI images", IEEE Transactions on Medical Imaging, Vol. 35, No. 5, (2016), 1240–1251. doi:10.1109/TMI.2016.2538465
 
 
International Journal of Engineering
Volume 34, Issue 2
TRANSACTIONS B: Applications
February 2021
Pages 458-469

Files

Cited by

Share

How to cite

Statistics