Abstract:

The inpainting algorithm for the generative adversarial networks often has the errors when filling arbitrary masked areas,because the convolution operation treats all input pixels as valid pixels.In order to solve this problem,an image inpainting algorithm based on gated convolution is proposed,which uses gated convolution to replace the traditional convolution in the residual block of the network to effectively learn the relationship between the known areas and the masked areas.The algorithm uses a two-stage generation of an adversarial inpainting network with edge repair and texture repair.First,we use the edge detection algorithm to detect the structure of the known area in the damaged image.Next,we combine the edge in the mask area with the color and texture information on the known area to repair the structure.Then,the complete structure is combined with the damaged image and sent to the texture repair network for texture repair.Finally,the complete image is output.In the process of network training,the Spectral Normalized Markovian Discriminator is used to improve the problem of slow weight change in the iterative process,thereby speeding up the convergence speed and improving the accuracy of the model.Experimental results on the Places2 dataset show that the proposed algorithm is different in peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) compared with the previous two-stage inpainting algorithm when repairing images with different shapes and sizes in damaged areas.The peak signal-to-noise ratio and structural similarity are improved by 3.8% and 3% respectively,and the subjective visual effect is significantly improved.

Key words: image inpainting, gated convolution, deep learning, generative adversarial network