Abstract

We can see people clicking picture and putting efforts to store them. But with time pictures get damaged. To recover pictures from those damages like scratches, graphics image inpainting technique can be used. Image inpainting technique either completes or removes the missing region in images. It’s one among the highly challenging topic in image processing area. Image inpainting techniques are divided into traditional techniques and deep learning techniques.  It starts with introduction, types of image inpainting, literature review, discussion and the conclusion.

Keywords

  • image inpainting
  • deep learning

References

  1. S. Shivaranjani and R. Priyandharsini, “A Survey on Inpainting Techniques,” IEEE International Conference on Electrical, Electronics and Optimization Techniques (ICEEOT), PP. 2934-2937, 2016.
  2. Mr. Mahesh Mahajan and Mr. Praveen Bhanodia, “Image Inpainting Techniques for Removal of Object,” IEEE ICICES, 2014.
  3. Guillemot, Christine and Olivier Le Meur, “Image inpainting: Overview and recent advances,” IEEE Signal Processing Magazine, vol. 31, Iss. 1, 2014.
  4. Zhen Qin, Qingliang Zeng, YixinZong and Fan Xu, “Image Inpainting based on Deep Learning: A review,” Elsevier, Displays 69 (2021) 102028, pp. 1-14, 2021.
  5. M. Bertalmio, G. Sapiro, V. Caselles and C. Ballester, “Image inpainting,” In Proc. ACM Conf. Compo Graphics (SIGGRAPH), New Orleans, LU, pp. 417-424, 2000.
  6. David Josue Barrientos Rojas, Bruno Jose Torres Fernandes and Sergio MuriloMaciel Fernandes, “A Review on Image Inpainting Techniques and Datasets,” IEEE 33rd SIBGRAPI Conference on Graaphics, Patterns and Images, pp. 240-247, 2020.
  7. A. Efros and T. K. Leung, “Texture synthesis by non-parametric sampling,’’ in Proc. 7th IEEE ICCv, vol. 2, pp. 10331038, Sep. 1999.
  8. Awati, Anupama Sanjay, “Digital Image Inpainting: A Review,” International Research Journal of Engineering and Technology (IRJET), vol. 3, Iss. 1, 2016.
  9. Ogawa, Takahiro, Miki Haseyama, “Image inpainting based on sparse representation with a perceptual metric,” EURASIP J. Adv. Signal Process, 2013.
  10. O. Lemeur, M. Ebdeli, C. Guillemot, “Heirarchical superresolution-based inpainting,” IEEE Trans. Image Process., vol. 22, no. 10, pp. 3779-3790, Oct 2013.
  11. M. Daisy, D. Tschumperl, O. Lzoray, “A fast spatial patch blending algorithm for artefact reduction in pattern-based image inpainting,” in Proc. SIGGRAPH Asia Tech. Briefs, Art. ID 8, 2013.
  12. Kangshun Li, Y. Wei, Z. Yang and W. Wei, “Image inpainting algorithm based on tv model and evolutionary algorithm,” Soft Computing 20, 2016.
  13. Sung Ha Kang, Chan T. Chan and S. Soatto, “Inpainting from multiple views,” First Interntional Symposium on 3D Data Processing Visualization and Transmission, pp. 622-625, 2002.
  14. A. Levin, A. Zomet aand Y. Weiss, “Learning how to inpaint from global image statistics,” Ninth IEEE International Conference on Computer Vision, vol. 1, pp. 305-312, 2003.
  15. Casaca Wallace, Maurilio Boaventura, Marcos Proenca de Almeida and Luis Gustavo Nonato, “Combining anisotropic diffusion, transport equation and texture synthesis for image inpainting textured images,” Pattern Recognition Letters, 2014.
  16. Wong, Alexander and Jeff Orchard, “A Nonlocal-means approach to exemplar-based inpainting,” Proceeding of 15th IEEE International conference on Image processing, 2008.
  17. J. Lin, S. Yang, Y. Fang and Z. Guo, “Structure-guided image inpainting using homography transformation,” IEEE Transaction on Multimedia, 2018.
  18. A. Criminisi, P. Perez and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,” IEEE Transactions on Image Processing 13(9), pp. 1200-1212, 2004.
  19. T. Ruzic and A. Pizurica, “Context aware patch based image inpainting using markov random field modeling,” IEEE Trans-actions on Image Processing, col. 24, pp. 444-456, 2015.
  20. Minquin Wang a, “A novel image inpainting method based on image decomposition,” Elsevier, Procedia Engineering 15, pp. 3733-3738, 2011.
  21. Pathak Deepak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell and Alexei A. Efros, “Context Encoder feature learning by inpainting,” The IEEE Conference on computer vision and pattern recognition (CVPR), 2016.
  22. Satoshi Lizuka, Edgar Simo-Serra and Hiroshi Ishikawa, “Globally and locally consistent image completion,” ACM Transactions on Graphics (Proc. SIGGRAPH), vol. 36, no. 4, pp. 107:1-107:14, 2017.
  23. G. Liu, F. A. Reda, K. J. Shih, T-C. Wang, A. Tao and B. Catanzaro, “Image inpainting for irregular holes using partial convolutions,” in Proceedings of the European Conference on Computer Vision (ECCV), pp. 89-105, 2018.
  24. Y. Zeng, J. Fu, H. Chao and B. Guo, “Learning pyramid-context encoder network for high-quality image inpainting,” in Proceeding of the IEEE conference on computer Vision and Pattern Recognition (CVPR), pp. 1486-1494, 2019.
  25. Jingyuan Li, F. He, L. Zhang, B. Du and D. Tao, “Progressive reconstruction of visual structure for image inpainting,” IEEE/CVF International Conference on Computer Vision (ICCV) pp. 5961-5970, 2019.
  26. J. Li, N. Wang, L. Zhang, B. Du and D. Tao, “Recurrent feature reasoning for image inpainting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7757-7765, 2020.
  27. J. Yu, Z. Lin, J. Yan, X. Shen, X. Lu and T. S. Huang, “Generative image inpainting with contextual attention,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5505-5514, 2018.
  28. Z. Yan, X. Li, M. Li, W. Zuo and S. Shan, “Shift-net: image inpainting via deep feature rearrangement,” in Proceedings of the European Conference on Computer Vision (ECCV), pp. 1-17, 2018.
  29. H. Liu, B. Jiang, Y. Xiao and C. Yang, “Coherent semantic attention for image inpainting,” in Proceeding of IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4169-4178, 2019.
  30. M.-C. Sagong, Y.-G. Shin, S.–W. Kim, S. Park and S.-J. Ko, “Pepsi: fast image inpainting with parallel decoding network,” in Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11360-11368, 2019.
  31. Z. Yi, Q. Tang, S. Azizi, D. Jang and Z. Xu, “Contextual residual aggregation for ultra high-resolution image inpainting,” in Proceeding of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.7505-7514, 2020.
  32. K. Nazeri, E. Ng, T. Joseph, F. Z. Qureshi and M. Ebrahimi, “EdgeConnect: structure guided image inpainting using edge prediction,” in Proceeding of the IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3265-3274, 2019.
  33. A. Lahiri, A. K. Jain, S. Agrawal, P. Mitra and P. K. Biswas, “Prior guided GAN based semantic inpainting,” in Proceeding of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13693-13702, 2020.
  34. R. Kohler, C. Schuler, B. Scholkopf and S. Harmeling, “Mask-specific inpainting with deep neural networks,” in German Conference on Pattern Recognition. Springer, pp. 523-534, 2014.
  35. R. A. Yeh, C. Chen, T. Y. Lim, A. G. Schwing, M. H.-Johnson and M. N. Do, “Semantic image inpainting with deep generative models,” in Proceeding of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6882-6890, 2017.
  36. P. Isola, J-Y. Zhu, T. Zhou and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in Proceeding of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967-5976, 2017.
  37. N. M. Salem, H. M.K. Mahdi and H. M. Abbas, “A novel face inpainting approach based on guided deep learning,” IEEE International Conference on Communications, Signal Processing and their Application (ICCSPA), pp. 1-6, 2020.
  38. H. Liu, G. Lu, X. Bi, J. Yan and W. Wang, “Image inpainting based on generative adversarial networks,” IEEE 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), pp. 373-378, 2018.
  39. H. Xu, X. Li, K. Zhang, Y. He, H. Fan, S. Liu and C. Hao, “SR-inpaint: a general deep learning framework for high resolution image inpainting,” Algorithm 14, 236. https://doi.org/10.3390/a14080236, pp.1-13, 2021.
  40. L. Zhao and R. Zhao, “Research on image inpainting based on generative adversarial network,” in Proceeding of IEEE International Conference on Computer Network, Electronic and Automation (ICCNEA), pp. 259-263, 2020.
  41. Z. Ruixia and Z. Li, “Image inpainting research based on deep learning,” International Journal of Advanced Network, Monitoring and Controls, vol. 05, no. 02, pp. 23-30, 2020.
  42. H. Li, G. Li, L. Lin, H. Yu and Y. Yu, “Context-aware semantic inpainting,” IEEE Transactions on Cybernetics, pp. 1-14, 2018.
  43. C. Yang, X. Lu, Z. Lin, E. Shechtman, O. Wang and H. Li, “High-resolution image inpainting using multi-scale neural patch synthesis,” in Proceeding of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4076-40.84, 2017.
  44. Z. Liu, P. Luo, X. Wang and X. Tang, “Deep learning face attributes in the wild,” in Proceeding of IEEE International Conference on Computer Vision, pp. 3730-3738, 2015.
  45. R. A. Yeh, T. Y. Lim, C. Chen, A. G. Schwing, M. H-Johnson and M. N. Do, “Image restoration with deep generative models,” IEEE, ICASSP, pp. 6772-6776, 2018.
  46. J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu and T. Huang, “Free-form image inpainting with gated convolution,” in Proceeding of IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4470-4479, 2019.
  47. J. Xie, L. Xu and E. Chen, “Image denoising and inpainting with deep neural networks,” ICTACT, pp. 1-9, 2022.