Improved Object Matching in Multi-Objects Tracking Based On Zernike Moments and Combination of Multiple Similarity Metrics

Document Type : Original Article

Authors

Department of Electrical & Computer Engineering, Babol Noshirvani University of Technology, Babol, Iran

Abstract

In video surveillance, multiple objects tracking (MOT) is a challenging task due to object matching problem in consecutive frames. The present paper aims to propose an improved object matching approach in MOT based on Zernike Moments and combination of multiple similarity distance metrics. In this work, the object is primarily detected using background subtraction method while the Gaussian Mixture Model (GMM) is applied for object extraction in the next frames. Subsequently, the color histogram and the magnitude of Zernike moments of the objects are calculated. In the next step, the objects are matched in the current and the previous frames based on the Hausdorff distance between objects, Earth Mover's (EMD) distance between their color histograms, and Chi-square distance between their Zernike moments. Then, a voting mechanism is designed to find the best consensus object matching from the aforementioned metrics. Eventually, the location of each object is predicted by the Kalman filter to continue tracking in subsequent frames. The results show that the object tracking and matching performance is improved using the proposed method in the video sequences of the multi-camera pedestrian "EPFL" video dataset. Specifically, errors caused by the merging of targets are reduced in the proposed tracking process.

Keywords

References

1.     Asvadi, A., Mahdavinataj, H., Karami, M. and Baleghi, Y., "Incremental discriminative color object tracking", in Artificial Intelligence and Signal Processing, Cham, Springer International Publishing. (2014), 71-81.
2.     Ardeshir, G. and Khakpour, F., "Using a novel concept of potential pixel energy for object tracking", International Journal of Engineering,  Vol. 27, No. 7, (2014), 1023-1032.
3.     Abbass, M.Y., Kwon, K.-C., Kim, N., Abdelwahab, S.A., El-Samie, F.E.A. and Khalaf, A.A.M., "A survey on online learning for visual tracking", The Visual Computer, (2020), doi: 10.1007/s00371-020-01848-y.
4.     Manafifard, M., Ebadi, H. and Abrishami Moghaddam, H., "A survey on player tracking in soccer videos", Computer Vision and Image Understanding,  Vol. 159, (2017), 19-46, doi: 10.1016/j.cviu.2017.02.002.
5.     Asvadi, A., Karami, M. and Baleghi, Y., "Efficient object tracking using optimized k-means segmentation and radial basis function neural networks", Itrcjrn,  Vol. 4, No. 1, (2012), 29-39.
6.     Asvadi, A. and Karami-Mollaie, M., "Object tracking using adaptive object color modeling. (2013).
7.     Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W. and Kim, T.-K., "Multiple object tracking: A literature review", Artificial Intelligence,  Vol. 293, (2021), 103448, doi: https://doi.org/10.1016/j.artint.2020.103448.
8.     Xing, J., Ai, H., Liu, L. and Lao, S., "Multiple player tracking in sports video: A dual-mode two-way bayesian inference approach with progressive observation modeling", IEEE Transactions on Image Processing,  Vol. 20, No. 6, (2011), 1652-1667, doi: 10.1109/TIP.2010.2102045.
9.     Asvadi, A., Karami-Mollaie, M., Baleghi, Y. and Seyyedi-Andi, H., "Improved object tracking using radial basis function neural networks", in 2011 7th Iranian Conference on Machine Vision and Image Processing. (2011), 1-5, doi: 10.1109/IranianMVIP.2011.6121604.
10.   Khare, M., Srivastava, R.K. and Khare, A., "Object tracking using combination of daubechies complex wavelet transform and zernike moment", Multimedia Tools and Applications,  Vol. 76, No. 1, (2017), 1247-1290, doi: 10.1007/s11042-015-3068-5.
11.   Fadaei, S. and Rashno, A., "Content-based image retrieval speedup based on optimized combination of wavelet and zernike features using particle swarm optimization algorithm", International Journal of Engineering,  Vol. 33, No. 5, (2020), 1000-1009, doi: 10.5829/ije.2020.33.05b.34.
12.   Di Gesù, V. and Starovoitov, V., "Distance-based functions for image comparison", Pattern Recognition Letters,  Vol. 20, No. 2, (1999), 207-214, doi: 10.1016/S0167-8655(98)00115-9.
13.   Taha, A.A. and Hanbury, A., "An efficient algorithm for calculating the exact hausdorff distance", IEEE Transactions on Pattern Analysis and Machine Intelligence,  Vol. 37, No. 11, (2015), 2153-2163, doi: 10.1109/TPAMI.2015.2408351.
14.   Rubner, Y., Tomasi, C. and Guibas, L.J., "The earth mover's distance as a metric for image retrieval", International Journal of Computer Vision,  Vol. 40, No. 2, (2000), 99-121, doi: 10.1023/A:1026543900054.
15.   Asvadi, A., Mahdavinataj, H., Karami, M.R. and Baleghi, Y., "Online visual object tracking using incremental discriminative color learning", The Csi Journal On Computer Science and Engineering,  Vol. 12, No. 2-4 (B), (2014), 16-28.
16.   Dendorfer, P., Ošep, A., Milan, A., Schindler, K., Cremers, D., Reid, I., Roth, S. and Leal-Taixé, L., "Motchallenge: A benchmark for single-camera multiple target tracking", arXiv preprint arXiv:2010.07548, (2020).
17.   Ciaparrone, G., Sánchez, F.L., Tabik, S., Troiano, L., Tagliaferri, R. and Herrera, F., "Deep learning in video multi-object tracking: A survey", Neurocomputing,  Vol. 381, (2020), 61-88.
18.   Li, X., Wang, K., Wang, W. and Li, Y., "A multiple object tracking method using kalman filter", in The 2010 IEEE International Conference on Information and Automation. (2010), 1862-1866, doi: 10.1109/ICINFA.2010.5512258.
19.   Xi, Z., Xu, D., Song, W. and Zheng, Y., "A* algorithm with dynamic weights for multiple object tracking in video sequence", Optik,  Vol. 126, No. 20, (2015), 2500-2507, doi: https://doi.org/10.1016/j.ijleo.2015.06.020.
20.   Wu, Z., Thangali, A., Sclaroff, S. and Betke, M., "Coupling detection and data association for multiple object tracking,  (2012),  1948-1955, doi: 10.1109/CVPR.2012.6247896.
21.   Arun Kumar, N.P., Laxmanan, R., Ram Kumar, S., Srinidh, V. and Ramanathan, R., "Performance study of multi-target tracking using kalman filter and hungarian algorithm", in Security in Computing and Communications, Singapore, Springer Singapore. (2021), 213-227.
22.   Li, H., Liu, Y., Lin, W., Xu, L. and Wang, J., "Data association methods via video signal processing in imperfect tracking scenarios: A review and evaluation", Mathematical Problems in Engineering,  Vol. 2020, (2020), 1-26.
23.   Habtemariam, B.K., Tharmarasa, R., Kirubarajan, T., Grimmett, D. and Wakayama, C., "Multiple detection probabilistic data association filter for multistatic target tracking", in 14th International Conference on Information Fusion, (2011), 1-6.
24.   Motro, M. and Ghosh, J., "Scaling data association for hypothesis-oriented mht", in 2019 22th International Conference on Information Fusion (FUSION), IEEE. (2019), 1-8.
25.   Kim, C., Li, F., Ciptadi, A. and Rehg, J.M., "Multiple hypothesis tracking revisited", in 2015 IEEE International Conference on Computer Vision (ICCV). (2015), 4696-4704, doi: 10.1109/ICCV.2015.533.
26.   Songhwai, O., Russell, S. and Sastry, S., "Markov chain monte carlo data association for general multiple-target tracking problems", in 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601). (2004), 735-742 Vol.731, doi: 10.1109/CDC.2004.1428740.
27.   Zhao, Z.-Q., Zheng, P., Xu, S.-t. and Wu, X., "Object detection with deep learning: A review", IEEE Transactions on Neural Networks and Learning Systems,  Vol. 30, No. 11, (2019), 3212-3232.
28.   Bansal, M., Kumar, M. and Kumar, M., "2d object recognition techniques: State-of-the-art work", Archives of Computational Methods in Engineering,  (2020), doi: 10.1007/s11831-020-09409-1.
29.   Ma, J., Jiang, X., Fan, A., Jiang, J. and Yan, J., "Image matching from handcrafted to deep features: A survey", International Journal of Computer Vision,  Vol. 129, No. 1, (2021), 23-79.
30.   Lowe, D.G., "Distinctive image features from scale-invariant keypoints", International Journal of Computer Vision,  Vol. 60, No. 2, (2004), 91-110.
31.   Mikolajczyk, K. and Schmid, C., "Scale & affine invariant interest point detectors", International Journal of Computer Vision,  Vol. 60, No. 1, (2004), 63-86.
32.   Bay, H., Tuytelaars, T. and Van Gool, L., "Surf: Speeded up robust features", in European conference on computer vision, Springer. (2006), 404-417.
33.   Kaushal, M., Khehra, B.S. and Sharma, A., "Soft computing based object detection and tracking approaches: State-of-the-art survey", Applied Soft Computing,  Vol. 70, (2018), 423-464.
34.   Uçar, A., Demir, Y. and Güzeliş, C., "Object recognition and detection with deep learning for autonomous driving applications", Simulation,  Vol. 93, No. 9, (2017), 759-769.
35.   Zhou, X., Gong, W., Fu, W. and Du, F., "Application of deep learning in object detection", in 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS), IEEE. (2017), 631-634.
36.   Cheng, S., Luo, X. and Bhandarkar, S.M., "A multiscale parametric background model for stationary foreground object detection", in 2007 IEEE Workshop on Motion and Video Computing (WMVC'07). (2007), 18-18, doi: 10.1109/WMVC.2007.1.
37.   Binh, N., "Human object tracking in nonsubsampled contourlet domain", International Journal of Advanced Computer Science and Applications,  Vol. 7, (2016).
38.   Khare, M., Binh, N.T. and Srivastava, R.K., Human object classification using dual tree complex wavelet transform and zernike moment, in Transactions on large-scale data-and knowledge-centered systems xvi. 2014, Springer.87-101.
39.   Górniak, A. and Skubalska-Rafajłowicz, E., "Object classification using sequences of zernike moments", in IFIP International Conference on Computer Information Systems and Industrial Management, Springer. (2017), 99-109.
40.   Farahi, F. and Yazdi, H.S., "Probabilistic kalman filter for moving object tracking", Signal Processing: Image Communication,  Vol. 82, (2020), 115751, doi: 10.1016/j.image.2019.115751.
41.   Bernardin, K. and Stiefelhagen, R., "Evaluating multiple object tracking performance: The clear mot metrics", EURASIP Journal on Image and Video Processing,  Vol. 2008, No. 1, (2008), 246309, doi: 10.1155/2008/246309.
42.   Lu, X., Izumi, T., Teng, L. and Wang, L., "Particle filter vehicle tracking based on surf feature matching", IEEJ Journal of Industry ApplicationS,  Vol. 3, (2014), 182-191.
43.   Wei, H., Takayoshi, Y., Hongtao, L. and Shihong, L., "Surf tracking", in 2009 IEEE 12th International Conference on Computer Vision. (2009), 1586-1592, doi: 10.1109/ICCV.2009.5459360.
44.   Qi, Z., Ting, R., Husheng, F. and Jinlin, Z., "Particle filter object tracking based on harris-sift feature matching", Procedia Engineering,  Vol. 29, (2012), 924-929, doi: 10.1016/j.proeng.2012.01.065.
45.   Salmane, H., Ruichek, Y. and Khoudour, L., "Object tracking using harris corner points based optical flow propagation and kalman filter", in 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC). (2011), 67-73, doi: 10.1109/ITSC.2011.6083031.
46.   Soleh, M., Jati, G. and Hilman, M., "Multi object detection and tracking using optical flow density-hungarian kalman filter (ofd-hkf) algorithm for vehicle counting. (2018).
47.   Sahbani, B. and Adiprawita, W., "Kalman filter and iterative-hungarian algorithm implementation for low complexity point tracking as part of fast multiple object tracking system", in 20166th International Conference on System Engineering and Technology (ICSET). (2016), 109-115, doi: 10.1109/ICSEngT.2016.7849633.
48.   Kim, B., Yuvaraj, N., Sri Preethaa, K., Santhosh, R. and Sabari, A., "Enhanced pedestrian detection using optimized deep convolution neural network for smart building surveillance", Soft Computing,  Vol. 24, (2020), 17081-17092.
49.   Ren, S., He, K., Girshick, R. and Sun, J., "Faster r-cnn: Towards real-time object detection with region proposal networks", IEEE Transactions on Pattern Analysis and Machine Intelligence,  Vol. 39, No. 6, (2016), 1137-1149.
50.   Acharya, D., Khoshelham, K. and Winter, S., "Real-time detection and tracking of pedestrians in cctv images using a deep convolutional neural network", in Proceedings of the 4th annual conference of research@ locate. Vol. 1913, (2017), 31-36.
 
 
 
International Journal of Engineering
Volume 34, Issue 6
TRANSACTIONS C: Aspects
June 2021
Pages 1445-1454

Files

Cited by

Share

How to cite

Statistics