基于YOLO的无约束场景中文车牌检测与识别

doi:10.16180/j.cnki.issn1007-7820.2023.10.001

Abstract

Abstract:

In view of the problems of traditional Chinese license plate recognition methods, such as the requirement of scenes, poor real-time performance, and inability to deploy on edge devices, this study proposes a Chinese license plate detection and recognition method based on YOLO(You Only Look Once) in unconstrained scenes. This method is divided into two modules: license plate detection and license plate character recognition. In the license plate detection part, the improved YOLOv5 model is used to predict four groups of key points for license plate correction based on the prediction of target candidate regions, and the pre-training model trained on the COCO data set is used for training, which reduces the error detection problem caused by the complex environment and has high real-time performance. In the license plate character recognition part, the CRNN(Convolutional Recurrent Neural Network) model is improved, which greatly reduces the parameters and computation of the algorithm, so that it can be successfully deployed in various edge devices. Experimental results show that the proposed method can efficiently detect and recognize license plates in complex environments. The map value of the proposed license plate detection model is 3.0% higher than that of Retina-face in the license plate detection data set. Compared with LPR-Net, the accuracy of license plate character recognition model in license plate recognition data set is improved by 4.2%.

Key words: license plate detection, license plate recognition, neural network, deep learning, character recognition, object detection, data set, wing loss

CLC Number:

TP183

CHEN Ziang,LIU Na,YUAN Ye,LI Qingdu,WAN Lihong. Chinese License Plate Detection and Recognition in Unconstrained Scenarios Based on YOLO[J].Electronic Science and Technology, 2023, 36(10): 1-8.

Figures/Tables 11

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Table 1.

Table 2.

Table 3.

Figure 8.

References 23

[1]	张长青, 杨楠. 基于混合神经网络的车牌字符识别技术[J]. 电子科技, 2019, 32(9):51-54.
	Zhang Changqing, Yang Nan. License plate character r-ecognition technology based on hybrid neural network[J]. Electronic Science and Technology, 2019, 32(9):51-54.
[2]	马艳娥, 宋金沙, 张甜. 车牌图像定位识别的研究与应用[J]. 中国新通信, 2019, 21(10):127-128.
	Ma Yan'e, Song Jinsha, Zhang Tian, et al. Research and application of license plate image localization and recognition[J]. China New Telecommunications, 2019, 21(10):127-128.
[3]	王能, 赵红立, 琚生根. 一种改进的Canny边缘检测自适应算法[J]. 四川大学学报(自然科学版), 2014, 51(3):479-482.
	Wang Neng, Zhao Hongli, Ju Shenggen, et al. An improved adaptive Canny edge detection algorithm[J]. Journal of Sichuan University(Natural Scinece Edition), 2014, 51(3):479-482.
[4]	谭司庭, 胡志坤. 基于HSV色空间的车牌定位综合方法[J]. 计算机与应用化学, 2011, 28(7):903-906.
	Tan Siting, Hu Zhikun. An effective integration method for license plate location based on HSV color space[J]. Computers and Applied Chemistry, 2011, 28(7):903-906.
[5]	朱晓红, 张永来, 刘冬林. 非正交 haar 小波变换的车牌定位技术研究[J]. 计算机工程与应用, 2009, 45(13):175-175. doi: 10.3778/j.issn.1002-8331.2009.13.051
	Zhu Xiaohong, Zhang Yonglai, Liu Donglin, et al. License plate location technology based on non-orthogonal haar wavelet transformation[J]. Computer Engineering and Applications, 2009, 45(13):175-175 doi: 10.3778/j.issn.1002-8331.2009.13.051
[6]	陈墨林. 基于卷积神经网络的免字符分割车牌识别算法[J]. 智能计算机与应用, 2021, 11(11):125-127,130.
	Chen Molin. License plate recognition algorithm without character segmentation based on convolutional neural network[J]. Intelligent Computer and Applications, 2021, 11(11) :125-127,130.
[7]	林哲聪. 基于卷积神经网络的车牌识别系统设计和算法实现[D]. 杭州: 浙江工业大学, 2018:2-4.
	Lin Zhecong. Design and algorithm implementation of license plate recognition system based on convolutional neural network[D]. Hangzhou: Zhejiang University of Technology, 2018:2-4.
[8]	Xu Z, Yang W, Meng A, et al. Towards end-to-end lice-nse plate detection and recognition:A large dataset and baseline[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:255-271.
[9]	Redmon J, Divvala S, Girshick R, et al. You only look once:Unified,real-time object detection[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:779-788.
[10]	Redmon J, Farhadi A. YOLO9000:Better,faster,stronger[C]. Holunono: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:7263-7271.
[11]	Zhao Z Q, Zheng P, Xu S, et al. Object detection with deep learning:A review[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(11):3212-3232. doi: 10.1109/TNNLS.5962385
[12]	Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]. Holunono: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:2117-2125.
[13]	Xiao Y, Tian Z, Yu J, et al. A review of object detection based on deep learning[J]. Multimedia Tools and Applications, 2020, 79(33):23729-23791. doi: 10.1007/s11042-020-08976-6
[14]	赵崇, 迟蒙蒙, 储聪. 导盲犬行走机构运动仿真及其视觉识别算法研究[J]. 电子科技, 2021, 34(9):66-72.
	Zhao Chong, Chi Mengmeng, Chu Cong, et al. Research on motion simulation and visual recognition algorithm of guide dog walking mechanism[J]. Electronic Scienceand Technology, 2021, 34(9):66-72.
[15]	Feng Z H, Kittler J, Awais M, et al. Wing loss for robust facial landmark localisation with convolutional neural networks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:2235-2245.
[16]	Shi B, Bai X, Yao C. An end-to-end trainable neural n-etwork for image-based sequence recognition and its application to scene text recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(11):2298-2304. doi: 10.1109/TPAMI.2016.2646371
[17]	Pak M, Kim S. A review of deep learning in image recognition[C]. Bali Island: The Fourth International Conference on Computer Applications and Information Processing Technology, 2017:1-3.
[18]	Howard A, Sandler M, Chu G, et al. Searching for mobilenetv3[C]. Seoul: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019:1314-1324.
[19]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:7132-7141.
[20]	Zhang S, Zheng D, Hu X, et al. Bidirectional long short-term memory networks for relation classification[C]. Shanghai: Proceedings of the Twenty-ninth Pacific Asia Conference on Language,Information and Computation, 2015:73-78.
[21]	Graves A, Fernández S, Gomez F, et al. Connectionist temporal classification:Labelling unsegmented sequence data with recurrent neural networks[C]. Pittsburgh: Proceedings of the Twenty-third International Conferenceon Machine Learning, 2006:369-376.
[22]	Deng J, Guo J, Ververas E, et al. Retinaface:Single-shot multi-level face localisation in the wild[C]. Seattle: Pr-oceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020:5203-5212.
[23]	Bian L, Wang H, Zhu C, et al. Image-free multi-character recognition[J]. Optics Letters, 2022, 47(6):1343-1346. doi: 10.1364/OL.451777 pmid: 35290309

省份	普通真实车牌数/张	普通生成车牌数/张	新能源真实车牌数/张	新能源生成车牌数/张	共计
京	711	4 000	143	1 000	5 854
沪	5 060	0	1 330	0	6 390
津	256	4 000	24	1 000	5 280
渝	225	4 000	27	1 000	5 252
冀	502	4 000	8	1 000	5 510
晋	261	4 000	0	1 000	5 261
蒙	62	1 000	0	0	1 062
辽	175	4 000	13	1 000	5 188
吉	22	4 000	5	1 000	5 027
黑	95	4 000	16	1 000	5 111
苏	5 119	0	839	700	6 658
浙	2 455	2 500	961	600	6 516
皖	108 045	0	9 042	0	117 087
闽	7 560	0	502	1 500	9 562
赣	678	4 000	78	1 000	5 756
鲁	752	4 000	55	1 000	5 807
豫	969	10 000	149	3 000	14 118
鄂	726	4 000	106	1 000	5 832
湘	406	4 000	53	1 000	5 459
粤	835	4 000	151	1 000	5 986
桂	28	4 000	8	1 000	5 036
琼	12	4 000	7	1 000	5 019
川	264	4 000	36	1 000	5 300
贵	39	4 000	14	1 000	5 053
云	90	4 000	7	1 000	5 097
藏	3	1 000	0	0	1 003
陕	345	4 000	30	1 000	5 375
甘	56	1 000	0	0	1 056
青	25	1 000	0	0	1 025
宁	124	900	0	0	1 024
新	39	1 000	0	0	1 039

模型	mAP/%	Recall/%	FPS/ frame·s^-1
Retina-face	96.8	96.4	32
本文	99.8	99.8	45

模型	Acuracy/%	Parameter/MB
LPR-Net	94.4	4.8
CRNN	96.8	138.0
本文	98.6	6.2

Chinese License Plate Detection and Recognition in Unconstrained Scenarios Based on YOLO

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 23

Related Articles 15

Metrics

Comments

Recommended 0

[1]	WEN Yanfei,WANG Wanxiong. Short-Term Power Load Forecasting Based on FA-SVR-LSTM Combined Model [J]. Electronic Science and Technology, 2023, 36(9): 1-7.
[2]	Bin ,WANG Sen. Visual Detection of Structural Cracks Using Depth Deformable Contour ModelLAI [J]. Electronic Science and Technology, 2023, 36(9): 35-40.
[3]	DANG Xiaofang,CAI Xingyu. Transformer-Based Maneuvering Target Tracking [J]. Electronic Science and Technology, 2023, 36(9): 86-92.
[4]	WANG Kejie,TONG Dongbing. Fixed-Time Synchronization for Fractional-Order Memristive Neural Networks with Time-Delays [J]. Electronic Science and Technology, 2023, 36(8): 81-87.
[5]	YUE Shengyao,XU Baiqiang,XU Guidong,XU Chenguang,ZHANG Sai. Super-Resolution Imaging of Laminate Debonding Defects via Deconvolutional Neural Network and Ultrasound Guided Waves [J]. Electronic Science and Technology, 2023, 36(8): 7-13.
[6]	HU Yongyang,LI Miao,MENG Fankai,ZHANG Feng,MENG Yiwei,SONG Yukun. Structured Compression and Acceleration of Network Based on Tiny-YOLOv3 [J]. Electronic Science and Technology, 2023, 36(8): 43-48.
[7]	SUN Hong,ZHAO Yingzhi. Lightweight Generative Adversarial Networks Based on Multi-Scale Gradient [J]. Electronic Science and Technology, 2023, 36(7): 32-38.
[8]	OU Jingyi,TIAN Ying,XIANG Xin,SONG Qizhe. Fault Diagnosis of Few Shot Industrial Process Based on Transfer BN-CNN Framework [J]. Electronic Science and Technology, 2023, 36(7): 49-55.
[9]	ZENG Xinxin,ZHANG Hongyan. A Farmland Parcel Extraction Network Based on Multi-Scale Semantic Information Enhancement [J]. Electronic Science and Technology, 2023, 36(7): 70-74.
[10]	ZHA Junwei,ZHANG Hongyan. Dynamic Receptive Field Feature Selection Dehazing Network [J]. Electronic Science and Technology, 2023, 36(7): 56-63.
[11]	LI Keran,CHEN Sheng,KE Panpan. A Method of Facial Mask Segmentation Based on CA-Net [J]. Electronic Science and Technology, 2023, 36(6): 64-71.
[12]	WANG Huimin,BI Jiazhen,SHEN Tuo,ZHANG Xuanxiong. A Research on the Spacing Measurement Between Two Trains Located at A Curved Track by Machine Vision Technology [J]. Electronic Science and Technology, 2023, 36(6): 34-40.
[13]	SHI Jianke,QIAO Meiying,LI Bingfeng,ZHAO Yan. Underwater Occlusion Target Detection Algorithm Based on Attention Mechanism [J]. Electronic Science and Technology, 2023, 36(5): 62-70.
[14]	LIU Yuwei,CAO Min,FENG Haojia. CNAS Recognition Criteria Automatic Benchmarking System Based on Natural Language Processing [J]. Electronic Science and Technology, 2023, 36(5): 28-33.
[15]	ZHENG Yuheng,FU Dongxiang. UAV Detection Based on Slim-YOLOv4 with Embedded Device [J]. Electronic Science and Technology, 2023, 36(5): 55-61.