基于超像素分割的孪生网络双目立体匹配方法研究

doi:10.16180/j.cnki.issn1007-7820.2021.09.013

摘要/Abstract

摘要：

图像若存在纹理缺乏、遮挡、边缘深度不连续等情况,其双目匹配精度会降低。针对这一问题,文中将经过均值滤波预处理和SLIC超像素分割形成的超像素区域作为Siamese孪生深度学习网络模型的输入,进行了图像局部信息匹配代价计算。该方法有效提升了视差计算时边缘区域的辨识度,避免了视差图边缘膨胀、模糊、不连续等问题,提高了立体匹配的准确度。实验部分基于Middlebury 立体视觉数据集测试平台,并与SGM半全局匹配、自适应权重AD-Census等方法得出的视差图进行比较。结果显示,该算法在深度不连续区域和缺乏纹理区域的匹配效果改善较为明显,且平均视差误差和平均错误率均有所降低,具有更好的鲁棒性。

关键词: 双目视觉, 超像素分割, SLIC算法, 立体匹配, 孪生网络, SGM算法, AD-Census算法, Middlebury数据集

Abstract:

In view of the low accuracy problem in binocular stereo matching caused by image texture lacking, occlusion and depth discontinuous, this study proposes the matching cost calculation of local image features based on deep learning Siamese neural network model which takes mean filter preprocessing and SLIC super-pixel segmentation as input. The method effectively improves the accuracy of edge regional identification when disparity computation happens, and avoids the problems of edge expansion, blur and discontinuity, and thus improving the accuracy of stereo matching. The experiments are based on the Middlebury stereovision data set test platform, and compared with the disparity maps obtained by SGM, adaptive weight AD-Census and other possible methods. The results show that the algorithm has a significant improvement in the matching effect of the depth discontinuous area and the lack of texture area, and the average parallax error and average error rate are reduced, indicating that the proposed method has better robustness.

Key words: binocular vision, super pixel segmentation, SLIC algorithm, stereo matching, siamese neural network, SGM algorithm, AD-Census algorithm, Middlebury dataset

中图分类号:

TP181

陆玮,刘翔,薛冕. 基于超像素分割的孪生网络双目立体匹配方法研究[J]. 电子科技, 2021, 34(9): 73-78.

LU Wei,LIU Xiang,XUE Mian. Siamese Network Binocular Stereo Matching Based on Super-Pixel Segmentation[J]. Electronic Science and Technology, 2021, 34(9): 73-78.

图/表 12

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

表1

参考文献 19

[1]	刘国锐. 基于双目立体视觉的三维重建系统研究[D]. 长春:长春工业大学, 2016.
	Liu Guorui. Research on three dimensional reconstruction system based on binocular vision[D]. Changchun:Changchun University of Technology, 2016.
[2]	黄经州. 高分辨率近似纹理图像立体匹配技术研究[D]. 杭州:浙江大学, 2015.
	Huang Jingzhou. Stereo matching of high- resolution images with similar texture[D]. Hangzhou:Zhejiang University, 2015.
[3]	庞星. 双目立体匹配的理论研究及算法优化[D]. 南京:南京理工大学, 2015.
	Pang Xing. Theoretical research and algorithm optimization of binocular stereo matching[D]. Nanjing:Nanjing University of Science and Technology, 2015.
[4]	李小林, 李文国, 李浩. 基于视差与灰度双层支持窗的立体匹配算法[J]. 电子科技, 2019, 32(11):12-27.
	Li Xiaolin, Li Wenguo, Li Hao. A stereo matching using double layer support windows based on RGB map and disparity map[J]. Electronic Science and Technology, 2019, 32(11):12-27.
[5]	郭龙源, 罗百通, 欧先锋, 等. 自适应窗口和半全局立体匹配算法研究[J]. 成都工业学院学报, 2017, 20(1):7-15.
	Guo Longyuan, Luo Baitong, Ou Xianfeng, et al. Adaptive windows and semi-global stereo matching algorithm[J]. Journal of Chengdu Technological University, 2017, 20(1):7-15.
[6]	Zbontar J, Lecun Y. Stereo matching by training a convolutional neural network to compare image patches[J]. The Journal of Machine Learning Research, 2016, 17(1):2287-2318.
[7]	Hirschmüller H. Semi-global matching-motivation, developments and applications[C]. Stuttgart:Photogrammetric Week, 2011.
[8]	王云峰, 吴炜, 余小亮, 等. 基于自适应权重AD-Census变换的双目立体匹配[J]. 工程科学与技术, 2018, 50(4):153-160.
	Wang Yunfeng, Wu Wei, Yu Xiaoliang, et al. A stereo matching system with the adaptive weight AD-census[J]. Advanced Engineering Sciences, 2018, 50(4):153-160.
[9]	吕鹏程, 厉小润. 基于AD-census和多权值的自适应窗口的立体匹配算法[J]. 工业控制计算机, 2018, 31(3):49-52.
	Lü Pengcheng, Li Xiaorun. Stereo matching algorithm based on AD-census and multi- weight adaptive window[J]. Industrial Control Computer, 2018, 31(3):49-52.
[10]	Luo W J, Schwing A G, Urtasun R. Efficient deep learning for stereo matching[C]. San Francisco:The IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[11]	Achanta R, Susstrunk S. Superpixels and polygons using simple non-iterative clustering[C]. San Francisco:The IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[12]	王春瑶, 陈俊周, 李炜. 超像素分割算法研究综述[J]. 计算机应用研究, 2014, 31(1):6-12.
	Wang Chunyao, Chen Junzhou, Li Wei. Review on superpixel segmentation algorithms[J]. Application Research of Computers, 2014, 31(1):6-12.
[13]	Kim S, Min D B, Kim S, et al. Feature augmentation for learning confidence measure in stereo matching[J]. IEEE Transactions on Image Processing, 2017, 26(12):6019-6033. doi: 10.1109/TIP.2017.2750404
[14]	郭倩, 张福杨, 孙农亮. 融合多特征表示和超像素优化的双目立体匹配[J]. 计算机工程与应用, 2020, 56(1):216-223.
	Guo Qian, Zhang Fuyang, Sun Nongliang. Binocular stereo matching with multi-feature representation and super-pixel optimization[J]. Computer Engineering and Applications, 2020, 56(1):216-223.
[15]	王旭初, 刘辉煌, 牛彦敏. 融合多尺度局部特征与深度特征的双目立体匹配[J]. 光学学报, 2020, 40(2):119-131.
	Wang Xuchu, Liu Huihuang, Niu Yanmin. Binocular stereo matching by combining multiscale local and deep features[J]. Acta Optica Sinica, 2020, 40(2):119-131.
[16]	Scharstein D, Szeliski R, Hirschmüuer H. Middlebury stereo benchmark[EB/OL].(2017-05-17)[2020-01-15]http://vision.middlebury.edu/stereo.
[17]	Vieira G D S, Soares F A A, De Lima J C, et al. A disparity computation framework[C]. Milwaukee:IEEE the Forty-third Annual Computer Software and Applications Conference, 2019.
[18]	Yu L D, Wang Y C, Wu Y W, et al. Deep stereo matching with explicit cost aggregation sub-architecture[C]. New Orleans:The Thirty-second AAAI Conference on Artificial Intelligence, 2018.
[19]	钱超, 张晓林. 基于稳定特竹点和SLIC超像素分割的快速立体匹配[J]. 电子设计工程, 2016, 24(23):146-148.
	Qian Chao, Zhang Xiaolin. Fast stereo matching algorithm based on steady features and SLIC superpixels[J]. Electronic Design Engineering, 2016, 24(23):146-148.

算法	avgErr	avgdispErr
SGM	42.591	37.212
AD-Census	31.641	26.903
本文算法	26.505	16.338