A Radon-based Convolutional Neural Network for Medical Image Retrieval


1 Institute for Intelligent System Research and Innovation, Deakin University, Australia

2 KIMIA Lab., University of Waterloo, Ontario, Canada

3 School of Information Technology, Deakin University, Australia


Image classification and retrieval systems have gained more attention because of easier access to high-tech medical imaging. However, the lack of availability of large-scaled balanced labelled data in medicine is still a challenge. Simplicity, practicality, efficiency, and effectiveness are the main targets in medical domain. To achieve these goals, Radon transformation, which is a well-known technology in medical field, is utilized along with a deep network to propose a retrieval system for a highly imbalanced medical benchmark. The main contribution of this study is to propose a deep model which is trained on the Radon-based transformed input data. The experimental results show that applying this transformation as input to feed into a convolutional neural network, significantly increases the performance, compared with other retrieval systems. The proposed scheme clearly increases the retrieval performance, compared with almost all models which use Radon transformation to retrieve medical images.


1.     Müller, H., Michoux, N., Bandon, D. and Geissbuhler, A., "A review of content-based image retrieval systems in medical applications—clinical benefits and future directions", International Journal of Medical Informatics,  Vol. 73, No. 1, (2004), 1-23.
2.     Müller, H., Deselaers, T., Deserno, T., Clough, P., Kim, E. and Hersh, W., "Overview of the imageclefmed 2006 medical retrieval and medical annotation tasks", in Workshop of the Cross-Language Evaluation Forum for European Languages, Springer., (2006), 595-608.
3.     Ojala, T., Pietikainen, M. and Maenpaa, T., "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns", IEEE Transactions on Pattern Analysis and Machine Intelligence,  Vol. 24, No. 7, (2002), 971-987.
4.     Ahonen, T., Hadid, A. and Pietikainen, M., "Face description with local binary patterns: Application to face recognition", IEEE Transactions on Pattern Analysis and Machine Intelligence,  Vol. 28, No. 12, (2006), 2037-2041.
5.     Dalal, N. and Triggs, B., "Histograms of oriented gradients for human detection", in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, IEEE. Vol. 1, (2005), 886-893.
6.     Khatami, A., Mirghasemi, S., Khosravi, A., Lim, C.P. and Nahavandi, S., "A new pso-based approach to fire flame detection using k-medoids clustering", Expert Systems with Applications,  Vol. 68, No., (2017), 69-80.
7.     Khatami, A., Mirghasemi, S., Khosravi, A. and Nahavandi, S., "A new color space based on k-medoids clustering for fire detection", in Systems, Man, and Cybernetics (SMC), 2015 IEEE International Conference on, IEEE., (2015), 2755-2760.
8.     Khatami, A., Mirghasemi, S., Khosravi, A. and Nahavandi, S., "An efficient hybrid algorithm for fire flame detection", in Neural Networks (IJCNN), 2015 International Joint Conference on, IEEE., (2015), 1-6.
9.     Keyvanpour, M., Tavoli, R. and Mozafari, S., "Document image retrieval based on keyword spotting using relevance feedback", International Journal of Engineering-Transactions A: Basics,  Vol. 27, No. 1, (2013), 7-14.
10.   Ezoji, M. and Iravani, S., "A general framework for 1-d histogram-baesd image contrast enhancement", International Journal of Engineering-Transactions A: Basics,  Vol. 29, No. 10, (2016), 1384-1392.
11.   Kashif, M., Deserno, T.M., Haak, D. and Jonas, S., "Feature description with sift, surf, brief, brisk, or freak? A general question answered for bone age assessment", Computers in Biology and Medicine,  Vol. 68, (2016), 67-75.
12.   Sargent, D., Chen, C.-I., Tsai, C.-M., Wang, Y.-F. and Koppel, D., "Feature detector and descriptor for medical images", in SPIE Medical Imaging, International Society for Optics and Photonics., (2009), 72592Z-72592Z-72598.
13.   Lehmann, T.M., Gold, M., Thies, C., Fischer, B., Spitzer, K., Keysers, D., Ney, H., Kohnen, M., Schubert, H. and Wein, B.B., "Content-based image retrieval in medical applications", Methods of Information in Medicine,  Vol. 43, No. 4, (2004), 354-361.
14.   Lehmann, T.M., Güld, M.O., Deselaers, T., Keysers, D., Schubert, H., Spitzer, K., Ney, H. and Wein, B.B., "Automatic categorization of medical images for content-based retrieval and data mining", Computerized Medical Imaging and Graphics,  Vol. 29, No. 2, (2005), 143-155.
15.   Weisi, L., Tao, D., Kacprzyk, J., Li, Z., Izquierdo, E. and Wang, H. (Eds.), "Multimedia analysis, processing and communications", Springer Science & Business Media,  Vol. 346,  (2011).
16.   Metz, C.E. and Pan, X., "A unified analysis of exact methods of inverting the 2-d exponential radon transform, with implications for noise control in spect", IEEE Transactions on Medical Imaging,  Vol. 14, No. 4, (1995), 643-658.
17.   Defrise, M. and Clack, R., "A cone-beam reconstruction algorithm using shift-variant filtering and cone-beam backprojection", IEEE Transactions on Medical Imaging,  Vol. 13, No. 1, (1994), 186-195.
18.   Kuchment, P., "The radon transform and medical imaging, SIAM",  (2013), doi: org/10.1137/1.9781611973297.
19.   Tizhoosh, H.R., "Barcode annotations for medical image retrieval: A preliminary investigation", in Image Processing (ICIP), 2015 IEEE International Conference on, IEEE., (2015), 818-822.
20.   Tizhoosh, H.R. and Rahnamayan, S., "Evolutionary projection selection for radon barcodes", in Evolutionary Computation (CEC), 2016 IEEE Congress on, IEEE., (2016), 1-8.
21.   Babaie, M., Tizhoosh, H., Zhu, S. and Shiri, M., "Retrieving similar x-ray images from big image data using radon barcodes with single projections", arXiv preprint arXiv:1701.00449,  (2017), https://arxiv.org/abs/1701.00449.
22.   Krizhevsky, A., Sutskever, I. and Hinton, G.E., "Imagenet classification with deep convolutional neural networks", in Advances in neural information processing systems., (2012), 1097-1105.
23.   Khatami, A., Babaie, M., Khosravi, A., Tizhoosh, H., Salaken, S.M. and Nahavandi, S., "A deep-structural medical image classification for a radon-based image retrieval", in Electrical and Computer Engineering (CCECE), 2017 IEEE 30th Canadian Conference on, IEEE., (2017), 1-4.
24.   LeCun, Y., Bottou, L., Bengio, Y. and Haffner, P., "Gradient-based learning applied to document recognition", Proceedings of the IEEE,  Vol. 86, No. 11, (1998), 2278-2324.
25.   Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bouchard, N., Warde-Farley, D. and Bengio, Y., "Theano: New features and speed improvements", arXiv Preprint arXiv:1211.5590,  (2012), https://arxiv.org/abs/1211.5590.
26.   Khatami, A., Babaie, M., Khosravi, A., Tizhoosh, H. and Nahavandi, S., "Parallel deep solutions for image retrieval from imbalanced medical imaging archives", Applied Soft Computing,  Vol. 63, (2018), 197-205.
27.   Liu, X., Tizhoosh, H.R. and Kofman, J., "Generating binary tags for fast medical image retrieval based on convolutional nets and radon transform", in Neural Networks (IJCNN), 2016 International Joint Conference on, IEEE., (2016), 2872-2878.
28.   Zhu, S. and Tizhoosh, H.R., "Radon features and barcodes for medical image retrieval via svm", in Neural Networks (IJCNN), 2016 International Joint Conference on, IEEE., (2016), 5065-5071.
29.   Sze-To, A., Tizhoosh, H.R. and Wong, A.K., "Binary codes for tagging x-ray images via deep de-noising autoencoders", in Neural Networks (IJCNN), 2016 International Joint Conference on, IEEE., (2016), 2864-2871.
30.   Tizhoosh, H.R., Mitcheltree, C., Zhu, S. and Dutta, S., "Barcodes for medical image retrieval using autoencoded radon transform", arXiv preprint arXiv:1609.05112,  (2016), https://arxiv.org/abs/1211.5590.