Super-resolution of License-plates Using Weighted Interpolation of Neighboring Pixels from Video Frames

Document Type : Original Article

Authors

Faculty of Electrical Engineering and Robotics, Shahrood University of Technology, Shahrood, Iran

Abstract

Recognizing the license plate from a set of low-resolution video frames using an Optical Character Recognition system (OCR) is a very challenging task. OCR systems fail to properly work in this condition. The use of high-quality cameras is a costly solution to this situation. To overcome this problem, we propose a weighted interpolation method that enhances the resolution of the license plate, using consecutive frames of a video. For this purpose, first, we register the low-resolution video frames of the license plate to the reference license plate in two steps. In the first step, a coarse registration is performed by matching the SURF features. Then a fine registration on the license plate region is performed using the phase correlation technique. After registration, the reference image of the license plate is up-sampled to the desired scale. We propose a method for estimating the intensity of pixels in the up-sampled image with an unknown value. In this method, we use a weighted averaging strategy to estimate the intensity of unknown pixels using the neighboring pixels from video frames.  The obtained super-resolution is suitable for OCR. Experimental results show that applying the proposed method on low-resolution frames of the license plate, improves the quality of the license plate significantly.

Keywords


1. Mjolsness, E., “Fingerprint hallucination”, Ph.D. dissertation,
California Institute of Technology, (1985). 
2. Tsai, R., “Multiframe image restoration and registration”,
Advance Computer Visual and Image Processing, Vol. 1,
(1984), 317–339.  
3. Nasrollahi, K. and Moeslund, T. B., “Super-resolution: a
comprehensive survey”, Machine Vision and Applications, Vol.
25, No. 6, (2014), 1423–1468.  
4. Irani, M. and Peleg, S., “Super resolution from image
sequences”, In Proceedings of 10
th
 International Conference on
Pattern Recognition (Vol. 2), IEEE, (1990), 115–120.  
5. Stark, H. and Oskoui, P., “High-resolution image recovery from
image-plane arrays, using convex projections”, Journal of the
Optical Society of America A, Vol. 6, No. 11, (1989), 1715–
1726.  
6. Amiri, M., Ahmadyfard, A., and Abolghasemi, V., “A fast video
super resolution for facial image”, Signal Processing: Image
Communication, Vol. 70, (2019), 259–270.  
7. Cheeseman, P., Kanefsky, B., Kraft, R., Stutz, J. and Hanson, R.,
“Super-resolved surface reconstruction from multiple images”,
In Maximum entropy and bayesian methods, Springer,
Dordrecht, (1996), 293–308.  
8. Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B. and Salesin,
D. H., “Image analogies”, In Proceedings of the 28
th
 Annual
Conference on Computer Graphics and Interactive Techniques,
(2001), 327–340.  
9. Seibel, H., Goldenstein, S. and Rocha, A., “Eyes on the target:
Super-resolution and license-plate recognition in low-quality
surveillance videos”, IEEE Access, Vol. 5, (2017), 20020–
20035.  
10. Mallikarachchi, L. and Dharmaratne, A., “Super Resolution to
Identify License Plate Numbers in Low Resolution Videos”, In
Proceedings of the 7
th
 International Symposium on Visual
Information Communication and Interaction, (2014), 220–223.  
11. Lee, Y., Jun, J., Hong, Y. and Jeon, M., “Practical License Plate
Recognition in Unconstrained Surveillance Systems with
Adversarial Super-Resolution”, (2019). arXiv preprint
arXiv:1910.04324. 
12. Vasek, V., Franc, V. and Urban, M., “License Plate Recognition
and Super-resolution from Low-Resolution Videos by
Convolutional Neural Networks”, In Proceeding of British
Machine Vision Conference, (2018), 1–12.  
13. Zou, Y., Wang, Y., Guan, W. and Wang, W., “Semantic Superresolution for Extremely Low-resolution Vehicle license plate”,In ICASSP 2019 IEEE International Conference on Acoustics,
Speech and Signal Processing, IEEE, (2019), 3772–3776.  
14. Ojansivu, V. and Heikkila, J., “Image registration using blurinvariant
phase correlation”, IEEE Signal Processing Letters,
Vol. 14, No. 7, (2007), 449–452.  
15. Seibel, H., Goldenstein, S. and Rocha, A., “Fast and effective
geometric k-nearest neighbors multi-frame super-resolution”, In
2015 28th  SIBGRAPI Conference on Graphics, Patterns and
Images, IEEE, (2015), 103–110.  
16. SHAHAAB, Image Processing and Deep Learning Solutions,
[Online] https://shahaab-co.ir/license-plate-recognition-library/.