基于生成对抗网络的图像艺术风格迁移
PDF下载 (515)董 伟,赵杰煜.基于生成对抗网络的图像艺术风格迁移[J].宁波大学学报(理工版),2019,32(5):30-35.DOI:
DONG Wei,ZHAO Jieyu.Image stylization transfer using generative adversarial neural networks[J].Journal of Ningbo University(Natural Science & Engineering Edition),2019,32(5):30-35.DOI:
| Title: | Image stylization transfer using generative adversarial neural networks |
| 作者: | 董 伟, 赵杰煜 |
| Author(s): | DONG Wei, ZHAO Jieyu |
| 关键词: | 结构相似性指数; 最小二乘生成对抗网络; 图像艺术风格化; 非真实感绘制 |
| Keywords: | structural similarity index measurement; least squares generative adversarial nets; image art stylization; non-photorealistic rendering |
| 分类号: | TP301.6 |
| 文献标识码: | A |
| 摘要: | 提出了一种新颖的图像艺术风格化算法, 利用结构相似性指数和最小二乘生成对抗网络, 搭建图像艺术风格化模型. 通过对模型生成器和判别器的对抗训练以及重建约束, 该模型可以生成一幅逼真的风格化作品. 根据在人脸肖像素描sketch-photo数据集和中国水墨画风格beihong-photo数据集实验表明, 与目前流行的DualGAN算法、CycleGAN算法、Pix2Pix算法和GAN算法相比, 本文提出的方法具有更好的风格化效果. |
| Abstract: | This paper proposes a novel neural style transfer algorithm. The Structural Similarity Index Measurement (SSIM) with the Least Squares Generate Adversarial Nets (LSGAN) is adopted to build a model for image repainting with neural styles. Through the adversarial training and reconstruction constraints, the model can generate a more realistic and reasonable stylized picture. Two groups of experiments are performed with CHUK sketch-photo dataset and a Chinese ink-painted style dataset (named beihong-photo which is constructed by the authors’ lab). Experimental results show that the presented method achieves a more acceptable state-of-art effect compared with those from the most popular algorithms, including DualGAN, CycleGAN, Pix2Pix and the classic GAN. |
| 参考文献 /References: | [1] Lecun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444. [2] Bengio Y. Learning deep architectures for AI[J]. Foundations and Trends in Machine Learning, 2009, 2(1): 1-55. [3] 曹毅. 基于图像的水墨画绘制方法的研究[D]. 长春: 吉林大学, 2012. [4] Gatys L A, Ecker A S, Bethge M. Image style transfer using convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Communication Society, 2016:2414-2423. [5] Johnson J, Alahi A, Li F F. Perceptual losses for real-time style transfer and super-resolution[C]//European Conference on Computer Vision. Berlin: Springer, 2016:694-711. [6] Ulyanov D, Lebedev V, Vedaldi A, et al. Texture networks: Feed-forward synthesis of textures and stylized images [C]//Proceedings of 33rd International Conference on Machine Learning. New York: ACM, 2016:1349-1357. [7] Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Computer Society, 2017: 5967-5976. [8] Zhu J Y, Park T, Isola P, et al. Unpaired Image-to-image translation using cycle-consistent adversarial networks[C]// Proceedings of the IEEE International Conference on Computer Vision. New York: IEEE Computer Society, 2017:2242-2251. [9] Yi Z, Zhang H, Tan P, et al. DualGAN: Unsupervised dual learning for image-to-image translation[C]//IEEE International Conference on Computer Vision. New York: IEEE Computer Society, 2017:2868-2876. [10] 谢志峰, 叶冠桦, 闫淑萁, 等. 基于生成对抗网络的HDR图像风格迁移技术[J]. 上海大学学报(自然科学版), 2018, 24(4):524-534. [11] 林嘉骏, 诸葛晶晶, 张晴. 基于像素级生成对抗网络的复杂场景灰度图彩色化[J]. 计算机辅助设计与图形学学报, 2019, 31(3):439-446. [12] Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: From error visibility to structural similarity [J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. [13] Mao X, Li Q, Xie H, et al. Least squares generative adversarial networks[C]//IEEE International Conference on Computer Vision. New York: IEEE Computer Society, 2017:2813-2821. [14] Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Massachusetts: MIT Press, 2014: 2672-2680. [15] Nguyen A, Clune J, Bengio Y, et al. Plug & play generative networks: Conditional iterative generation of images in latent space[C]//IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Computer Society, 2017:3510-3520. [16] Li C, Wand M. Precomputed real-time texture synthesis with markovian generative adversarial networks[C]// European Conference on Computer Vision. Berlin: Springer, 2016:702-716. [17] Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Computer Society, 2017:105-114. [18] Zhang H, Xu T, Li H. StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks[C]//IEEE International Conference on Computer Vision. New York: IEEE Computer Society, 2017:5908-5916. [19] 王坤峰, 苟超, 段艳杰, 等. 生成式对抗网络GAN的研究进展与展望[J]. 自动化学报, 2017, 43(3):321-332. [20] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Computer Society, 2016:770-778. [21] Gretton A, Fukumizu K H, Teo C, et al. A kernel method for the two-sample-problem[C]//Advances in Neural Information Processing Systems 19 Proceedings of the 2006 Conference. Massachusetts: MIT Press, 2007:1672- 1678. [22] Heusel M, Ramsauer H, Unterthiner T, et al. GANs trained by a two time-scale update rule converge to a local nash equilibrium[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Massachusetts: MIT Press, 2017: 6626-6637. [23] Salimans T, Goodfellow I J, Zaremba W, et al. Improved techniques for training GANs[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Massachusetts: MIT Press, 2016: 2232-2242. |
| 备注/Memo: | 收稿日期: 2018－08－03. 宁波大学学报（理工版）网址: http://journallg.nbu.edu.cn/ 基金项目: 国家自然科学基金（61571247）; 浙江省自然科学基金（LZ16F030001）; 浙江省国际合作项目（2013C24027）. 宁波大学学报（理工版）网址：http://journallg.nbu.edu.cn/第一作者: 董伟（1993－）, 男, 安徽宣城人, 在读硕士研究生, 主要研究方向: 机器学习与深度学习. E-mail: 1511082629@nbu.edu.cn *通信作者: 赵杰煜（1965－）, 男, 浙江宁波人, 教授, 主要研究方向: 计算机图像处理、机器学习与神经网络. E-mail: zhao_jieyu@nbu.edu.cn |
