Details of Research Outputs

TitleAsymmetric CNN for Image Superresolution
Author (Name in English or Pinyin)
Tian, C.1; Xu, Y.2; Zuo, W.3; Lin, C.4; Zhang, D.5
Date Issued2021
Source PublicationIEEE Transactions on Systems, Man, and Cybernetics: Systems
ISSN21682216
EISSN2168-2232
Volume52Issue:6Pages:3718-3730
AbstractDeep convolutional neural networks (CNNs) have been widely applied for low-level vision over the past five years. According to the nature of different applications, designing appropriate CNN architectures is developed. However, customized architectures gather different features via treating all pixel points as equal to improve the performance of given application, which ignores the effects of local power pixel points and results in low training efficiency. In this article, we propose an asymmetric CNN (ACNet) comprising an asymmetric block (AB), a memory enhancement block (MEB), and a high-frequency feature enhancement block (HFFEB) for image superresolution (SR). The AB utilizes one-dimensional (1-D) asymmetric convolutions to intensify the square convolution kernels in horizontal and vertical directions for promoting the influences of local salient features for single image SR (SISR). The MEB fuses all hierarchical low-frequency features from AB via a residual learning technique to resolve the long-term dependency problem and transforms obtained low-frequency features into high-frequency features. The HFFEB exploits low- and high-frequency features to obtain more robust SR features and address the excessive feature enhancement problem. Additionally, it also takes charge of reconstructing a high-resolution image. Extensive experiments show that our ACNet can effectively address SISR, blind SISR, and blind SISR of blind noise problems. The code of the ACNet is shown at https://github.com/hellloxiaotian/ACNet.
KeywordTask analysis Convolution Training Superresolution Feature extraction Kernel Degradation Asymmetric architecture blind single image superresolution (SISR) convolutional neural network (CNN) image superresolution (SR) multilevel feature fusion multiple degradation task
DOI10.1109/TSMC.2021.3069265
Indexed BySCIE
language英语
Funding ProjectNational Nature Science Foundation of China [61876051]; Shenzhen Municipal Science and Technology Innovation Council [JCYJ20180306172101694]; Ministry of Science and Technology, Taiwan [110-2634-F-007-015-]
WOS Research AreaAutomation & Control Systems ; Computer Science
WOS SubjectAutomation & Control Systems ; Computer Science, Cybernetics
WOS IDWOS:000732169100001
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Original Document TypeArticle
Firstlevel Discipline计算机科学技术
Education discipline科技类
Published range国外学术期刊
References
[1] X. Liang, D. Zhang, G. Lu, Z. Guo, and N. Luo, "A novel multicamera system for high-speed touchless palm recognition," IEEE Trans. Syst., Man, Cybern., Syst., vol. 51, no. 3, pp. 1534-1548, Mar. 2021
[2] W. Shi et al., "Cardiac image super-resolution with global correspondence using multi-atlas patchmatch," in Proc. Int. Conf. Med. Image Comput. Comput. Assist. Intervent., 2013, pp. 9-16
[3] B. Du, Q. Wei, and R. Liu, "An improved quantum-behaved particle swarm optimization for endmember extraction," IEEE Trans. Geosci. Remote Sens., vol. 57, no. 8, pp. 6003-6017, Aug. 2019
[4] M. Irani and S. Peleg, "Improving resolution by image registration," CVGIP Graph. Models Image Process., vol. 53, no. 3, pp. 231-239, 1991
[5] J. Sun, Z. Xu, and H.-Y. Shum, "Image super-resolution using gradient profile prior," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2008, pp. 1-8
[6] Z. Xiong, X. Sun, and F. Wu, "Robust Web image/video superresolution," IEEE Trans. Image Process., vol. 19, no. 8, pp. 2017-2028, Aug. 2010
[7] H. Chang, D.-Y. Yeung, and Y. Xiong, "Super-resolution through neighbor embedding," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., vol. 1, 2004, pp. 275-282
[8] G. Gao and J. Yang, "A novel sparse representation based framework for face image super-resolution," Neurocomputing, vol. 134, pp. 92-99, Jun. 2014
[9] V. Alvarez, V. Ponomaryov, and S. Sadovnychiy, "Image superresolution via wavelet feature extraction and sparse representation," Radioengineering, vol. 27, no. 2, pp. 602-609, 2018
[10] V. Alvarez, V. Ponomaryov, S. Sadovnychiy, and R. Reyes, "Image super-resolution via two coupled dictionaries and sparse representation," Multimedia Tools Appl., vol. 77, no. 11, pp. 13487-13511, 2018
[11] W. Yang, T. Yuan, W. Wang, F. Zhou, and Q. Liao, "Single-image superresolution by subdictionary coding and kernel regression," IEEE Trans. Syst., Man, Cybern., Syst., vol. 47, no. 9, pp. 2478-2488, Sep. 2017
[12] S. Schulter, C. Leistner, and H. Bischof, "Fast and accurate image upscaling with super-resolution forests," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2015, pp. 3791-3799
[13] C. Tian, Y. Xu, and W. Zuo, "Image denoising using deep CNN with batch renormalization," Neural Netw., vol. 121, pp. 461-473 Jan. 2020
[14] C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 184-199
[15] J. Kim, J. K. Lee, and K. Mu Lee, "Accurate image super-resolution using very deep convolutional networks," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2016, pp. 1646-1654
[16] J. Kim, J. K. Lee, and K. M. Lee, "Deeply-recursive convolutional network for image super-resolution," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2016, pp. 1637-1645
[17] Y. Tai, J. Yang, and X. Liu, "Image super-resolution via deep recursive residual network," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2017, pp. 3147-3155
[18] X. Mao, C. Shen, and Y.-B. Yang, "Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections," in Proc. Adv. Neural Inf. Process.g Syst., 2016, pp. 2802-2810
[19] C. Tian et al., "Lightweight image super-resolution with enhanced CNN," Knowl. Based Syst., vol. 205, Oct. 2020, Art. no. 106235
[20] C. Tian, L. Fei, W. Zheng, Y. Xu, W. Zuo, and C.-W. Lin, "Deep learning on image denoising: An overview," 2019. [Online]. Available: arXiv:1912.13171
[21] X. Fan, Y. Yang, C. Deng, J. Xu, and X. Gao, "Compressed multiscale feature fusion network for single image super-resolution," Signal Process., vol. 146, pp. 50-60, May 2018
[22] Z. Zhang, X. Wang, and C. Jung, "DCSR: Dilated convolutions for single image super-resolution," IEEE Trans. Image Process., vol. 28, no. 4, pp. 1625-1635, Apr. 2019
[23] K. Zhang, W. Zuo, S. Gu, and L. Zhang, "Learning deep CNN denoiser prior for image restoration," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2017, pp. 3929-3938
[24] P. Liu, H. Zhang, K. Zhang, L. Lin, and W. Zuo, "Multi-level wavelet- CNN for image restoration," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops, 2018, pp. 773-782
[25] C. Dong, C. C. Loy, and X. Tang, "Accelerating the super-resolution convolutional neural network," in Proc. Eur. Conf. Comput. Vis. 2016, pp. 391-407
[26] N. Ahn, B. Kang, and K.-A. Sohn, "Fast, accurate, and lightweight super-resolution with cascading residual network," in Proc. Eur. Conf. Comput. Vis., 2018, pp. 252-268
[27] Z. Hui, X. Gao, Y. Yang, and X. Wang, "Lightweight image superresolution with information multi-distillation network," in Proc. ACM Int. Conf. Multimedia, 2019, pp. 2024-2032
[28] X. Zhang, H. Dong, Z. Hu, W.-S. Lai, F. Wang, and M.-H. Yang, "Gated fusion network for joint image deblurring and super-resolution," 2018. [Online]. Available: arXiv:1807.10806
[29] C. Tian, Y. Xu, Z. Li, W. Zuo, L. Fei, and H. Liu, "Attentionguided CNN for image denoising," Neural Netw., vol. 124, pp. 177-129, Apr. 2020
[30] B. Du, T. Xinyao, Z. Wang, L. Zhang, and D. Tao, "Robust graph-based semisupervised learning for noisy labeled data via maximum correntropy criterion," IEEE Trans. Cybern., vol. 49, no. 4, pp. 1440-1453, Apr. 2019
[31] C. Wang, Z. Yan, W. Pedrycz, M. Zhou, and Z. Li, "A weighted fidelity and regularization-based method for mixed or unknown noise removal from images on graphs," IEEE Trans. Image Process., vol. 29, pp. 5229-5243, 2020
[32] D. Ren, W. Zuo, Q. Hu, P. Zhu, and Q. Hu, "Progressive image deraining networks: A better and simpler baseline," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 3937-3946
[33] W. Ren, J. Pan, H. Zhang, X. Cao, and M. Yang, "Single image dehazing via multi-scale convolutional neural networks with holistic edges," Int. J. Comput. Vis., vol. 128, no. 1, pp. 240-259, 2020
[34] K. Zhang, W. Zuo, and L. Zhang, "Deep plug-and-play super-resolution for arbitrary blur kernels," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 1671-1681
[35] C. Tian, Y. Xu, W. Zuo, B. Zhang, L. Fei, and C.-W. Lin, "Coarse-tofine CNN for image super-resolution," IEEE Trans. Multimedia, early access, Jun. 1, 2020, doi: 10.1109/TMM.2020.2999182
[36] S. Li, F. He, B. Du, L. Zhang, Y. Xu, and D. Tao, "Fast spatio-temporal residual network for video super-resolution," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 10522-10531
[37] X. Ding, Y. Guo, G. Ding, and J. Han, "ACNet: Strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks," in Proc. IEEE/CVF Conf. Comput. Vis., 2019, pp. 1911-1920
[38] J. Jin, A. Dundar, and E. Culurciello, "Flattened convolutional neural networks for feedforward acceleration," 2014. [Online]. Available: arXiv:1412.5474
[39] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2016, pp. 2818-2826
[40] S.-Y. Lo, H.-M. Hang, S.-W. Chan, and J.-J. Lin, "Efficient dense modules of asymmetric convolution for real-time semantic segmentation," in Proc. ACM Multimedia Asia, 2019, pp. 1-6
[41] W.-S. Lai, J.-B. Huang, N. Ahuja, and M.-H. Yang, "Deep Laplacian pyramid networks for fast and accurate super-resolution," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2017, pp. 5835-5843
[42] Z. Hui, X. Wang, and X. Gao, "Fast and accurate single image superresolution via information distillation network," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2018, pp. 723-731
[43] C. Xie,W. Zeng, and X. Lu, "Fast single-image super-resolution via deep network with component learning," IEEE Trans. Circuits Syst. Video Technol., vol. 29, no. 12, pp. 3473-3486, Dec. 2019
[44] W. Shi et al., "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2016, pp. 1874-1883
[45] W. Yang, X. Zhang, Y. Tian, W. Wang, J.-H. Xue, and Q. Liao, "Deep learning for single image super-resolution: A brief review," IEEE Trans. Multimedia, vol. 21, no. 12, pp. 3106-3121, Dec. 2019
[46] C. Douillard et al., "Iterative correction of intersymbol interference: Turbo-equalization," Eur. Trans. Telecommun., vol. 6, no. 5, pp. 507-511, 1995
[47] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1097-1105
[48] Y. Tai, J. Yang, X. Liu, and C. Xu, "MemNet: A persistent memory network for image restoration," in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2017, pp. 4539-4547
[49] B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, "Enhanced deep residual networks for single image super-resolution," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops, 2017, pp. 136-144
[50] R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, and L. Zhang, "NTIRE 2017 challenge on single image super-resolution: Methods and results," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops, 2017, pp. 114-125
[51] C. Tian, Y. Xu, W. Zuo, B. Du, C.-W. Lin, and D. Zhang, "Designing and training of a dual CNN for image denoising," 2020. [Online]. Available: arXiv:2007.03951
[52] D. Yuan, X. Li, Z. He, Q. Liu, and S. Lu, "Visual object tracking with adaptive structural convolutional network," Knowl. Based Syst., vol. 194, Apr. 2020, Art. no. 105554
[53] M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. Alberi-Morel, "Lowcomplexity single-image super-resolution based on nonnegative neighbor embedding," in Proc. Brit. Mach. Vis. Conf., 2012, pp. 1-10
[54] J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image super-resolution via sparse representation," IEEE Trans. Image Process., vol. 19, pp. 2861-2873, 2010
[55] D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proc. IEEE/CVF Int. Conf. Comput. Vis., vol. 2 2001, pp. 416-423
[56] J.-B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed self-exemplars," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2015, pp. 5197-5206
[57] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," 2014. [Online]. Available: arXiv:1412.6980
[58] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising," IEEE Trans. Image Process., vol. 26, pp. 3142-3155, 2017
[59] Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2018, pp. 2472-2481
[60] A. Hore and D. Ziou, "Image quality metrics: PSNR vs. SSIM," in Proc. Int. Conf. Pattern Recognit., 2010, pp. 2366-2369
[61] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770-778
[62] N. Ahn, B. Kang, and K. Sohn, "Image super-resolution via progressive cascading residual network," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 791-799
[63] L. Zhang, L. Zhang, X. Mou, and D. Zhang, "FSIM: A feature similarity index for image quality assessment," IEEE Trans. Image Process., vol. 20, pp. 2378-2386, 2011
[64] R. Timofte, V. De Smet, and L. Van Gool, "A+: Adjusted anchored neighborhood regression for fast super-resolution," in Proc. Asian Conf. Comput. Vis., 2014, pp. 111-126
[65] D. Dai, R. Timofte, and L. Van Gool, "Jointly optimized regressors for image super-resolution," Comput. Graph. Forum, vol. 34, no. 2, pp. 95-104, 2015
[66] Z. Wang, D. Liu, J. Yang, W. Han, and T. Huang, "Deep networks for image super-resolution with sparse prior," in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2015, pp. 370-378
[67] Y. Chen and T. Pock, "Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration," IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1256-1272, Jun. 2017
[68] Z. Lu et al., "Fast single image super-resolution via dilated residual networks," IEEE Access, vol. 7, pp. 109729-109738, 2018
[69] Y. Shi, K. Wang, C. Chen, L. Xu, and L. Lin, "Structure-preserving image super-resolution via contextualized multitask learning," IEEE Trans. Multimedia, vol. 19, no. 12, pp. 2804-2815, Dec. 2017
[70] H. Ren, M. El-Khamy, and J. Lee, "Image super resolution based on fusing multiple convolution neural networks," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops, 2017, pp. 54-61
[71] W. Bae, J. Yoo, and J. C. Ye, "Beyond deep residual learning for image restoration: Persistent homology-guided manifold simplification," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops, 2017, pp. 145-153
[72] J. Xu, M. Li, J. Fan, X. Zhao, and Z. Chang, "Self-learning superresolution using convolutional principal component analysis and random matching," IEEE Trans. Multimedia, vol. 21, no. 5, pp. 1108-1121, May 2019
[73] X. Yang et al., "DRFN: Deep recurrent fusion network for single-image super-resolution with large factors," IEEE Trans. Multimedia, vol. 21, no. 2, pp. 328-337, Feb. 2019
[74] C. Cruz, R. Mehta, V. Katkovnik, and K. O. Egiazarian, "Single image super-resolution based on Wiener filter in similarity domain," IEEE Trans. Image Process., vol. 27, no. 3, pp. 1376-1389, Mar. 2018
[75] F. Cao and B. Chen, "New architecture of deep recursive convolution networks for super-resolution," Knowl.-Based Syst., vol. 178, pp. 98-110, Aug. 2019
[76] Y. Fan et al., "Balanced two-stage residual networks for image superresolution," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops, 2017, pp. 161-168
[77] P. Bao, L. Zhang, and X. Wu, "Canny edge detection enhancement by scale multiplication," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 9, pp. 1485-1490, Sep. 2005
Data SourceWOS
Citation statistics
Cited Times [WOS]:0   [WOS Record]     [Related Records in WOS]
Document TypeJournal article
Identifierhttps://irepository.cuhk.edu.cn/handle/3EPUXD0A/2214
CollectionSchool of Science and Engineering
School of Data Science
Shenzhen Institute of Artificial Intelligence and Robotics for Society
Corresponding AuthorXu, Y.; Lin, C.
Affiliation
1.Department of Electrical Engineering, City University of Hong Kong, Hong Kong, SAR, China.
2.School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China, also with the Peng Cheng Laboratory, Shenzhen 518055, China, and also with the Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology Shenzhen, Shenzhen 518055, China (e-mail: [email protected
3.School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China, and also with the Peng Cheng Laboratory, Shenzhen 518055, China.
4.Department of Electrical Engineering and the Institute of Communications Engineering, National Tsing Hua University, Hsinchu 30013, Taiwan (e-mail: [email protected
5.School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen), Shenzhen 518172, China, and also with the Shenzhen Institute of Artificial Intelligence and Robotics for Society, The Chinese University of Hong Kong (Shenzhen), Shenzhen 518172, China.
Recommended Citation
GB/T 7714
Tian, C.,Xu, Y.,Zuo, W.et al. Asymmetric CNN for Image Superresolution[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems,2021,52(6):3718-3730.
APA Tian, C., Xu, Y., Zuo, W., Lin, C., & Zhang, D. (2021). Asymmetric CNN for Image Superresolution. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 52(6), 3718-3730.
MLA Tian, C.,et al."Asymmetric CNN for Image Superresolution".IEEE Transactions on Systems, Man, and Cybernetics: Systems 52.6(2021):3718-3730.
Files in This Item:
There are no files associated with this item.
Related Services
Usage statistics
Google Scholar
Similar articles in Google Scholar
[Tian, C.]'s Articles
[Xu, Y.]'s Articles
[Zuo, W.]'s Articles
Baidu academic
Similar articles in Baidu academic
[Tian, C.]'s Articles
[Xu, Y.]'s Articles
[Zuo, W.]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Tian, C.]'s Articles
[Xu, Y.]'s Articles
[Zuo, W.]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.