1

Beyond a Gaussian Denoiser: Residual Learning of

Deep CNN for Image Denoising
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang

Abstract—Discriminative model learning for image denoising random field (MRF) models [10], [11], [12]. In particular, the
has been recently attracting considerable attentions due to its NSS models are popular in state-of-the-art methods such as
favorable denoising performance. In this paper, we take one BM3D [2], LSSC [4], NCSR [6] and WNNM [13].
arXiv:1608.03981v1 [cs.CV] 13 Aug 2016

step forward by investigating the construction of feed-forward

denoising convolutional neural networks (DnCNNs) to embrace Despite their high denoising quality, most of the image
the progress in very deep architecture, learning algorithm, and prior-based methods typically suffer from two major draw-
regularization method into image denoising. Specifically, residual backs. First, those methods generally involve a complex op-
learning and batch normalization are utilized to speed up the timization problem in the testing stage, making the denoising
training process as well as boost the denoising performance. process time-consuming [6], [13]. Thus, most of the prior-
Different from the existing discriminative denoising models which
usually train a specific model for additive white Gaussian noise based methods can hardly achieve high performance without
(AWGN) at a certain noise level, our DnCNN model is able sacrificing computational efficiency. Second, the models in
to handle Gaussian denoising with unknown noise level (i.e., general are non-convex and involve several manually chosen
blind Gaussian denoising). With the residual learning strategy, parameters, providing some leeway to boost denoising perfor-
DnCNN implicitly removes the latent clean image in the hidden mance.
layers. This property motivates us to train a single DnCNN
model to tackle with several general image denoising tasks such To overcome the limitations of prior-based approaches,
as Gaussian denoising, single image super-resolution and JPEG several discriminative learning methods have been recently
image deblocking. Our extensive experiments demonstrate that developed to learn image prior models in the context of
our DnCNN model can not only exhibit high effectiveness in truncated inference procedure. The resulting models are able to
several general image denoising tasks, but also be efficiently get rid of the iterative optimization procedure in the test phase.
implemented by benefiting from GPU computing.
Schmidt and Roth [14] proposed a cascade of shrinkage fields
Index Terms—Image Denoising, Convolutional Neural Net- (CSF) method that unifies the random field-based model and
works, Residual Learning, Batch Normalization the unrolled half-quadratic optimization algorithm into a single
learning framework. Chen et al. [15], [16] proposed a trainable
I. I NTRODUCTION nonlinear reaction diffusion (TNRD) model which learns a
modified fields of experts [12] image prior by unfolding a fixed
Image denoising is a classical yet still active topic in low number of gradient descent inference steps. Some of the other
level vision since it is an indispensable step in many practical related work can be found in [17], [18]. Although CSF and
applications. The goal of image denoising is to recover a clean TNRD have shown promising results toward bridging the gap
image x from a noisy observation y which follows an image between computational efficiency and denoising quality, their
degradation model y = x + v. One common assumption is performance are inherently restricted to the specified forms of
that v is additive white Gaussian noise (AWGN) with standard prior. To be specific, the priors adopted in CSF and TNRD
deviation σ. From a Bayesian viewpoint, when the likelihood are based on the analysis model, which is limited in capturing
is known, the image prior modeling will play a central role in the full characteristics of image structures. In addition, the
image denoising. Over the past few decades, various models parameters are learned by stage-wise greedy training plus joint
have been exploited for modeling image priors, including fine-tuning among all stages, and many handcrafted parameters
nonlocal self-similarity (NSS) models [1], [2], [3], [4], sparse are involved. Another nonnegligible drawback is that they train
models [4], [5], [6], gradient models [7], [8], [9] and Markov a specific model for a certain noise level, and are limited in
blind image denoising.
This project is partially supported by HK RGC GRF grant (under no. PolyU
5313/13E) and the National Natural Scientific Foundation of China (NSFC) In this paper, instead of learning a discriminative model with
under Grant No. 61271093. an explicit image prior, we treat image denoising as a plain
K. Zhang is with the School of Computer Science and Technology, Harbin discriminative learning problem, i.e., separating the noise from
Institute of Technology, Harbin 150001, China, and also with the Department
of Computing, The Hong Kong Polytechnic University, Hong Kong. a noisy image by feed-forward convolutional neural networks
W. Zuo is with the School of Computer Science and Technology, Harbin In- (CNN). The reasons of using CNN are three-fold. First, CNN
stitute of Technology, Harbin 150001, China (e-mail: cswmzuo@gmail.com). with very deep architecture [19] is effective in increasing the
Y. Chen is with the Institute for Computer Graphics and Vision, Graz
University of Technology, 8010 Graz, Austria. capacity and flexibility for exploiting image characteristics.
D. Meng is with the School of Mathematics and Statistics and Ministry Second, considerable advances have been achieved on regu-
of Education Key Lab of Intelligent Networks and Network Security, Xi’an larization and learning methods for training CNN, including
Jiaotong University, Xi’an 710049, China.
L. Zhang is with the Department of Computing, The Hong Kong Polytech- Rectifier Linear Unit (ReLU) [20], batch normalization [21]
nic University, Hong Kong (e-mail: cslzhang@comp.polyu.edu.hk). and residual learning [22]. These methods can be adopted
 2

in CNN to speed up the training process and improve the ods in terms of both quantitative metrics and visual
denoising performance. Third, CNN is well-suited for par- quality.
allel computation on modern powerful GPU, which can be 3) Our DnCNN can be easily extended to handle general
exploited to improve the run time performance. image denoising tasks. We can train a single DnCNN
We refer to the proposed denoising convolutional neural model for blind Gaussian denoising, and achieve better
network as DnCNN. Rather than directly outputing the de- performance than the competing methods trained for a
noised image x̂, the proposed DnCNN is designed to predict specific noise level. Moreover, it is promising to solve
the residual image v̂, i.e., the difference between the noisy three general image denoising tasks, i.e., blind Gaussian
observation and the latent clean image. In other words, the denoising, SISR, and JPEG deblocking, with only a
proposed DnCNN implicitly removes the latent clean image single DnCNN model.
with the operations in the hidden layers. The batch normaliza- The remainder of the paper is organized as follows. Section
tion technique is further introduced to stabilize and enhance II provides a brief survey of related work. Section III first
the training performance of DnCNN. It turns out that residual presents the proposed DnCNN model, and then extends it to
learning and batch normalization can benefit from each other, general image denoising. In Section IV, extensive experiments
and their integration is effective in speeding up the training are conducted to evaluate DnCNNs. Finally, several conclud-
and boosting the denoising performance. ing remarks are given in Section V.
While this paper aims to design a more effective Gaussian
denoiser, we observe that when v is the difference between the
ground truth high resolution image and the bicubic upsampling II. R ELATED W ORK
of the low resolution image, the image degradation model
for Guassian denoising can be converted to a single image A. Deep Neural Networks for Image Denoising
super-resolution (SISR) problem; analogously, the JPEG im-
There have been several attempts to handle the denoising
age deblocking problem can be modeled by the same image
problem by deep neural networks. In [23], Jain and Seung
degradation model by taking v as the difference between the
proposed to use convolutional neural networks (CNNs) for
original image and the compressed image. In this sense, SISR
image denoising and claimed that CNNs have similar or even
and JPEG image deblocking can be treated as two special
better representation power than the MRF model. In [24], the
cases of a “general” image denoising problem, though in
multi-layer perceptron (MLP) was successfully applied for im-
SISR and JPEG deblocking the noise vs are much different
age denoising. In [25], stacked sparse denoising auto-encoders
from AWGN. It is natural to ask whether is it possible to
method was adopted to handle Gaussian noise removal and
train a CNN model to handle such general image denoising
achieved comparable results to K-SVD [5]. In [16], a trainable
problem? By analyzing the connection between DnCNN and
nonlinear reaction diffusion (TNRD) model was proposed
TNRD [16], we propose to extend DnCNN for handling
and it can be expressed as a feed-forward deep network by
several general image denoising tasks, including Gaussian
unfolding a fixed number of gradient descent inference steps.
denoising, SISR and JPEG image deblocking.
Among the above deep neural networks based methods, MLP
Extensive experiments show that, our DnCNN trained with
and TNRD can achieve promising performance and are able to
a certain noise level can yield better Gaussian denoising results
compete with BM3D. However, for MLP [24] and TNRD [16],
than state-of-the-art methods such as BM3D [2], WNNM [13]
a specific model is trained for a certain noise level. To the best
and TNRD [16]. For Gaussian denoising with unknown noise
of our knowledge, it remains uninvestigated to develop CNN
level (i.e., blind Gaussian denoising), DnCNN with a single
for general image denoising.
model can still outperform BM3D [2] and TNRD [16] trained
for a specific noise level. The DnCNN can also obtain promis-
ing results when extended to several general image denoising
B. Residual Learning and Batch Normalization
tasks. Moreover, we show the effectiveness of training only a
single DnCNN model for three general image denoising tasks, Recently, driven by the easy access to large-scale dataset
i.e., blind Gaussian denoising, SISR with multiple upscaling and the advances in deep learning methods, the convolutional
factors, and JPEG deblocking with different quality factors. neural networks have shown great success in handling various
The contributions of this work are summarized as follows: vision tasks. The representative achievements in training CNN
1) We propose an end-to-end trainable deep CNN for models include Rectified Linear Unit (ReLU) [20], tradeoff be-
Gaussian denoising. In contrast to the existing deep tween depth and width [19], [26], parameter initialization [27],
neural network-based methods which directly estimate gradient-based optimization algorithms [28], [29], [30], batch
the latent clean image, the network adopts the residual normalization [21] and residual learning [22]. Other factors,
learning strategy to remove the latent clean image from such as the efficient training implementation on modern
noisy observation. powerful GPUs, also contribute to the success of CNN. For
2) We find that residual learning and batch normalization Gaussian denoising, it is easy to generate sufficient training
can greatly benefit the CNN learning as they can not data from a set of high quality images. This work focuses
only speed up the training but also boost the denoising on the design and learning of CNN for image denoising. In
performance. For Gaussian denoising with a certain the following, we briefly review two methods related to our
noise level, DnCNN outperforms state-of-the-art meth- DnCNN, i.e., residual learning and batch normalization.
 3

1) Residual Learning: Residual learning [22] of CNN was A. Network Depth

originally proposed to solve the performance degradation Following the principle in [19], we set the size of con-
problem, i.e., even the training accuracy begins to degrade volutional filters to be 3 × 3 but remove all pooling layers.
along with the increasing of network depth. By assuming Therefore, the receptive field of DnCNN with depth of d
that the residual mapping is much easier to be learned than should be (2d+1)×(2d+1). Increasing receptive field size can
the original unreferenced mapping, residual network explicitly make use of the context information in larger image region.
learns a residual mapping for a few stacked layers. With such For better tradeoff between performance and efficiency, one
a residual learning strategy, extremely deep CNN can be easily important issue in architecture design is to set a proper depth
trained and improved accuracy has been achieved for image for DnCNN.
classification and object detection [22]. It has been pointed out that the receptive field size of
The proposed DnCNN model also adopts the residual denoising neural networks correlates with the effective patch
learning formulation. Unlike the residual network [22] that size of denoising methods [23], [24]. Moreover, high noise
uses many residual units (i.e., identity shortcuts), our DnCNN level usually requires larger effective patch size to capture
employs a single residual unit to predict the residual image. We more context information for restoration [34]. Thus, by fixing
further explain the rationale of residual learning formulation the noise level σ = 25, we analyze the effective patch size of
by analyzing its connection with TNRD [16], and extend it several leading denoising methods to guide the depth design of
to solve several general image denoising tasks. It should be our DnCNN. In BM3D [2], the non-local similar patches are
noted that, prior to the residual network [22], the strategy adaptively searched in a local widow of size 25 × 25 for two
of predicting the residual image has already been adopted in times, and thus the final effective patch size is 49×49. Similar
some low-level vision problems such as single image super- to BM3D, WNNM [13] uses a larger searching window and
resolution [31] and color image demosaicking [32]. However, performs non-local searching iteratively, resulting in a quite
to the best of our knowledge, there is no work which directly large effective patch size (361 × 361). MLP [24] first uses a
predicts the residual image for denoising. patch of size 39 × 39 to generate the predicted patch, and then
2) Batch Normalization: Mini-batch stochastic gradient de-
adopts a filter of size 9 × 9 to average the output patches, thus
scent (SGD) has been widely used in training CNN models.
its effective patch size is 47×47. The CSF [14] and TNRD [16]
Despite the simplicity and effectiveness of mini-batch SGD,
with five stages involves a total of ten convolutional layers with
its training efficiency is largely reduced by internal covariate
filter size of 7 × 7, and their effective patch size is 61 × 61.
shift [21], i.e., changes in the distributions of internal non-
Table I summarizes the effective patch sizes adopted in
linearity inputs during training. Batch normalization [21] is
different methods with noise level σ = 25. It can be seen
proposed to alleviate the internal covariate shift by incorpo-
that the effective patch size used in EPLL [33] is the smallest,
rating a normalization step and a scale and shift step before
i.e., 36×36. It is interesting to verify whether DnCNN with the
the nonlinearity in each layer. For batch normalization, only
receptive field size similar to EPLL can compete against the
two parameters per activation are added, and they can be
leading denoising methods. Thus, for Gaussian denoising with
updated with back-propagation. Batch normalization enjoys
a certain noise level, we set the receptive field size of DnCNN
several merits, such as fast training, better performance, and
to 35 × 35 with the corresponding depth of 17. For other
low sensitivity to initialization. For further details on batch
general image denoising tasks, we adopt a larger receptive
normalization, please refer to [21].
field and set the depth to be 20.
By far, no work has been done on studying batch normaliza-
tion for CNN-based image denoising. We empirically find that,
the integration of residual learning and batch normalization B. Network Architecture
can result in fast and stable training and better denoising The input of our DnCNN is a noisy observation y = x +
performance. v. Discriminative denoising models such as MLP [24] and
CSF [14] aim to learn a mapping function F(y) = x to predict
III. T HE P ROPOSED D ENOISING CNN M ODEL
the latent clean image. For DnCNN, we adopt the residual
In this section, we present the proposed denoising CNN learning formulation to train a residual mapping R(y) ≈ v,
model, i.e., DnCNN, and extend it for handling several general and then we have x = y − R(y). Formally, the averaged
image denoising tasks. Generally, training a deep CNN model mean squared error between the desired residual images and
for a specific task generally involves two steps: (i) network estimated ones from noisy input
architecture design and (ii) model learning from training
N
data. For network architecture design, we modify the VGG 1 X
network [19] to make it suitable for image denoising, and set `(Θ) = kR(yi ; Θ) − (yi − xi )k2F (1)
2N i=1
the depth of the network based on the effective patch sizes
used in state-of-the-art denoising methods. For model learning, can be adopted as the loss function to learn the trainable
we adopt the residual learning formulation, and incorporate parameters Θ in DnCNN. Here {(yi , xi )}N i=1 represents N
it with batch normalization for fast training and improved noisy-clean training image (patch) pairs. Fig. 1 illustrates the
denoising performance. Finally, we discuss the connection architecture of the proposed DnCNN for learning R(y). In
between DnCNN and TNRD [16], and extend DnCNN for the following, we explain the architecture of DnCNN and the
several general image denoising tasks. strategy for reducing boundary artifacts.
 4

TABLE I
T HE EFFECTIVE PATCH SIZES OF DIFFERENT METHODS WITH NOISE LEVEL σ = 25.

Methods BM3D [2] WNNM [13] EPLL [33] MLP [24] CSF [14] TNRD [16]
Effective Patch Size 49 × 49 361 × 361 36 × 36 47 × 47 61 × 61 61 × 61

!"#$%&'()* +*#",-(.%&'()*

/!01%2%3 %2%+*45

/!01%2%3 %2%+*45

/!01%2%3 %2%+*45
/!01%2%+*45

/!01
Ċ Ċ

Fig. 1. The architecture of the proposed DnCNN network.

1) Deep Architecture: Given the DnCNN with depth D, R(y) to predict v. According to [22], when the original map-
there are three types of layers, shown in Fig. 1 with three ping is more like an identity mapping, the residual mapping
different colors. (i) Conv+ReLU: for the first layer, 64 filters will be much easier to be optimized. Note that the noisy
of size 3 × 3 × c are used to generate 64 feature maps, and observation y is much more like the latent clean image x than
rectified linear units (ReLU, max(0, ·)) are then utilized for the residual image v (especially when the noise level is low).
nonlinearity. Here c represents the number of image channels, Thus, F(y) would be more close to an identity mapping than
i.e., c = 1 for gray image and c = 3 for color image. (ii) R(y), and the residual learning formulation is more suitable
Conv+BN+ReLU: for layers 2 ∼ (D − 1), 64 filters of size for image denoising.
3 × 3 × 64 are used, and batch normalization [21] is added Fig. 2 shows the average PSNR values obtained using these
between convolution and ReLU. (iii) Conv: for the last layer, two learning formulations with/without batch normalization
c filters of size 3 × 3 × 64 are used to reconstruct the output. under the same setting on gradient-based optimization algo-
To sum up, our DnCNN model has two main features: the rithms and network architecture. Note that two gradient-based
residual learning formulation is adopted to learn R(y), and optimization algorithms are adopted: one is the stochastic
batch normalization is incorporated to speed up training as gradient descent algorithm with momentum (i.e., SGD) and the
well as boost the denoising performance. By incorporating other one is the Adam algorithm [30]. Firstly, we can observe
convolution with ReLU, DnCNN can gradually separate image that the residual learning formulation can result in faster and
structure from the noisy observation through the hidden layers. more stable convergence than the original mapping learning. In
Such a mechanism is similar to the iterative noise removal the meanwhile, without batch normalization, simple residual
strategy adopted in methods such as EPLL and WNNM, but learning with conventional SGD cannot compete with the
our DnCNN is trained in an end-to-end fashion. Later we will state-of-the-art denoising methods such as TNRD (28.92dB).
give more discussions on the rationale of combining residual We consider that the insufficient performance should be at-
learning and batch normalization. tributed to the internal covariate shift [21] caused by the
2) Reducing Boundary Artifacts: In many low level vision changes in network parameters during training. Accordingly,
applications, it usually requires that the output image size batch normalization is adopted to address it. Secondly, we ob-
should keep the same as the input one. This may lead to the serve that, with batch normalization, learning residual mapping
boundary artifacts. In MLP [24], boundary of the noisy input (the red line) converges faster and exhibits better denoising
image is symmetrically padded in the preprocessing stage, performance than learning original mapping (the blue line). In
whereas the same padding strategy is carried out before every particular, both the SGD and Adam optimization algorithms
stage in CSF [14] and TNRD [16]. Different from the above can enable the network with residual learning and batch
methods, we directly pad zeros before convolution to make normalization to have the best results. In other words, it is
sure that each feature map of the middle layers has the same the integration of residual learning formulation and batch
size as the input image. We find that the simple zero padding normalization rather than the optimization algorithms (SGD
strategy does not result in any boundary artifacts. This good or Adam) that leads to the best denoising performance.
property is probably attributed to the powerful ability of the Actually, one can notice that in Gaussian denoising the
DnCNN. residual image and batch normalization are both associated
with the Gaussian distribution. It is very likely that residual
learning and batch normalization can benefit from each other
C. Integration of Residual Learning and Batch Normalization for Gaussian denoising1 . This point can be further validated
for Image Denoising by the following analyses.
The network shown in Fig. 1 can be used to train either the 1 It should be pointed out that this does not mean that our DnCNN can not
original mapping F(y) to predict x or the residual mapping handle other general denoising tasks well.
 5

29.2 29.2

29 29

28.8 28.8
Average PSNR(dB)

Average PSNR(dB)
28.6 28.6

28.4 28.4

28.2 28.2
With RL, with BN With RL, with BN
28 With RL, without BN 28 With RL, without BN
Without RL, with BN Without RL, with BN
27.8 Without RL, without BN 27.8 Without RL, without BN

27.6 27.6

1 5 10 15 20 25 30 35 40 45 50 1 5 10 15 20 25 30 35 40 45 50
Epochs Epochs

(a) SGD (b) Adam

Fig. 2. The Gaussian denoising results of four specific models under two gradient-based optimization algorithms, i.e., (a) SGD, (b) Adam, with respect to
epochs. The four specific models are in different combinations of residual learning (RL) and batch normalization (BN) and are trained with noise level 25.
The results are evaluated on 68 natural images from Berkeley segmentation dataset.

• On the one hand, residual learning benefits from batch from an abundant set of degraded-clean training image pairs.
normalization. This is straightforward because batch nor- Here N denotes the image size, λ is the regularization param-
malization offers some merits for CNNs, such as allevi- eter, fk ∗ x stands for the convolution of the image x with
ating internal covariate shift problem. From Fig. 2, one the k-th filter kernel fk , and ρk (·) represents the k-th penalty
can see that even though residual learning without batch function which is adjustable in the TNRD model. For Gaussian
normalization (the green line) has a fast convergence, it denoising, we set Ψ(z) = 12 kzk2 .
is inferior to residual learning with batch normalization The diffusion iteration of the first stage can be interpreted
(the red line). as performing one gradient descent inference step at starting
• On the other hand, batch normalization benefits from point y, which is given by
residual learning. As shown in Fig. 2, without residual
K 
learning, batch normalization even has certain adverse X ∂Ψ(z) 
effect to the convergence (the blue line). With residual x1 = y − αλ (f̄k ∗ φk (fk ∗ y)) − α , (3)
∂z z=0
k=1
learning, batch normalization can be utilized to speedup
the training as well as boost the performance (the red where f̄k is the adjoint filter of fk (i.e., f̄k is obtained by
line). Note that each mini-bath is a small set (e.g., 128) rotating 180 degrees the filter fk ), α corresponds to the
of images. Without residual learning, the input intensity 0
 and ρk (·) = φk (·). For Gaussian denoising, we have
stepsize
and the convolutional feature are correlated with their ∂Ψ(z) 
∂z  = 0, and Eqn. (3) is equivalent to the following
neighbored ones, and the distribution of the layer inputs z=0
also rely on the content of the images in each training expression
mini-batch. With residual learning, DnCNN implicitly K
X
removes the latent clean image with the operations in the v1 = y − x1 = αλ (f̄k ∗ φk (fk ∗ y)), (4)
hidden layers. This makes that the inputs of each layer are k=1
Gaussian-like distributed, less correlated, and less related
with image content. Thus, residual learning can also help where v1 is the estimated residual of x with respect to y.
batch normalization in reducing internal covariate shift. Since the influence function φk (·) can be regarded as point-
wise nonlinearity applied to convolution feature maps, Eqn. (4)
To sum up, the integration of residual learning and batch
actually is a two-layer feed-forward CNN. As can be seen from
normalization can not only speed up and stabilize the training
Fig. 1, the proposed CNN architecture further generalizes one-
process but also boost the denoising performance.
stage TNRD from three aspects: (i) replacing the influence
function with ReLU to ease CNN training; (ii) increasing
D. Connection with TNRD the CNN depth to improve the capacity in modeling image
characteristics; (iii) incorporating with batch normalization to
Our DnCNN can also be explained as the generalization of boost the performance. The connection with one-stage TNRD
one-stage TNRD [15], [16]. Typically, TNRD aims to train a provides insights in explaining the use of residual learning
discriminative solution for the following problem for CNN-based image restoration. Most of the parameters in
K X
N Eqn. (4) are derived from the analysis prior term of Eqn. (2). In
this sense, most of the parameters in DnCNN are representing
X
min Ψ(y − x) + λ ρk ((fk ∗ x)p ), (2)
x the image priors.
k=1 p=1
 6

It is interesting to point out that, even the noise is not Gaus- improvements. To train DnCNN for Gaussian denoising with
sian distributed (or the noise level of Gaussian is unknown), known noise level, we consider three noise levels, i.e., σ = 15,
we still can utilize Eqn. (3) to obtain v1 if we have 25 and 50. We set the patch size as 40 × 40, and crop

∂Ψ(z)  128 × 1, 600 patches to train the model. We refer to our
= 0. (5) DnCNN model for Gaussian denoising with known specific
∂z z=0
noise level as DnCNN-S.
Note that Eqn. (5) holds for many types of noise distributions, To train a single DnCNN model for blind Gaussian denois-
e.g., generalized Gaussian distribution. It is natural to assume ing, we set the range of the noise levels as σ ∈ [0, 55], and
that it also holds for the noise caused by SISR and JPEG the patch size as 50 × 50. 128 × 3, 000 patches are cropped
compression. It is possible to train a single CNN model to train the model. We refer to our single DnCNN model for
for several general image denoising tasks, such as Gaussian blind Gaussian denoising task as DnCNN-B.
denoising with unknown noise level, SISR with multiple For the test images, we use two different test datasets for
upscaling factors, and JPEG deblocking with different quality thorough evaluation, one is a test dataset containing 68 natural
factors. images from Berkeley segmentation dataset (BSD68) [12] and
Besides, Eqn. (4) can also be interpreted as the operations to the other one contains 12 images as shown in Fig. 3. Note that
remove the latent clean image x from the degraded observation all those images are widely used for the evaluation of Gaussian
y to estimate the residual image v. For these tasks, even denoising methods and they are not included in the training
the noise distribution is complex, it can be expected that our dataset.
DnCNN would also perform robustly in predicting residual In addition to gray image denoising, we also train the
image by gradually removing the latent clean image in the blind color image denoising model referred to as CDnCNN-
hidden layers. B. We use color version of the BSD68 dataset for testing and
the remaining 432 color images from Berkeley segmentation
E. Extension to General Image Denoising dataset are adopted as the training images. The noise levels
The existing discriminative Gaussian denoising methods, are also set into the range of [0, 55] and 128 × 3, 000 patches
such as MLP, CSF and TNRD, all train a specific model of size 50×50 are cropped to train the model.
for a fixed noise level [16], [24]. When applied to Gaussian To learn a single model for the three general image de-
denoising with unknown noise, one common way is to first noising tasks, as in [35], we use a dataset which consists
estimate the noise level, and then use the model trained with of 91 images from [36] and 200 training images from the
the corresponding noise level. This makes the denoising results Berkeley segmentation dataset. The noisy image is generated
affected by the accuracy of noise estimation. In addition, those by adding Gaussian noise with a certain noise level from
methods cannot be applied to the cases with non-Gaussian the range of [0, 55]. The SISR input is generated by first
noise distribution, e.g., SISR and JPEG deblocking. bicubic downsampling and then bicubic upsampling the high-
Our analyses in Section III-D have shown the potential of resolution image with downscaling factors 2, 3 and 4. The
DnCNN in general image denoising. To demonstrate it, we JPEG deblocking input is generated by compressing the image
first extend our DnCNN for Gaussian denoising with unknown with a quality factor ranging from 5 to 99 using the MATLAB
noise level. In the training stage, we use the noisy images from JPEG encoder. All these images are treated as the inputs to a
a wide range of noise levels (e.g., σ ∈ [0, 55]) to train a single single DnCNN model. Totally, we generate 128×8,000 image
DnCNN model. Given a test image whose noise level belongs patch (the size is 50 × 50) pairs for training. Rotation/flip
to the noise level range, the learned single DnCNN model can based operations on the patch pairs are used during mini-batch
be utilized to denoise it without estimating its noise level. learning. The parameters are initialized with DnCNN-B. We
We further extend our DnCNN by learning a single model refer to our single DnCNN model for these three general image
for several general image denoising tasks. We consider three denoising tasks as DnCNN-3. To test DnCNN-3, we adopt
specific tasks, i.e., blind Gaussian denoising, SISR, and JPEG different test set for each task, and the detailed description
deblocking. In the training stage, we utilize the images with will be given in Section IV-E.
AWGN from a wide range of noise levels, down-sampled 2) Parameter Setting and Network Training: In order to
images with multiple upscaling factors, and JPEG images capture enough spatial information for denoising, we set the
with different quality factors to train a single DnCNN model. network depth to 17 for DnCNN-S and 20 for DnCNN-B and
Experimental results show that the learned single DnCNN DnCNN-3. The loss function in Eqn. (1) is adopted to learn
model is able to yield excellent results for any of the three the residual mapping R(y) for predicting the residual v. We
general image denoising tasks. initialize the weights by the method in [27] and use SGD with
weight decay of 0.0001, a momentum of 0.9 and a mini-batch
size of 128. We train 50 epochs for our DnCNN models. The
IV. E XPERIMENTAL R ESULTS
learning rate was decayed exponentially from 1e − 1 to 1e − 4
A. Experimental setting for the 50 epochs.
1) Training and Testing Data: For Gaussian denoising with We use the MatConvNet package [37] to train the proposed
either known or unknown noise level, we follow [16] to DnCNN models. Unless otherwise specified, all the experi-
use 400 images of size 180 × 180 for training. We found ments are carried out in the Matlab (R2015b) environment
that using a larger training dataset can only bring negligible running on a PC with Intel(R) Core(TM) i7-5820K CPU
 7

Fig. 3. The 12 widely used testing images.

TABLE II
T HE AVERAGE PSNR( D B) RESULTS OF DIFFERENT METHODS ON THE BSD68 DATASET. T HE BEST RESULTS ARE HIGHLIGHTED IN BOLD .

Methods BM3D WNNM EPLL MLP CSF TNRD DnCNN-S DnCNN-B

σ = 15 31.07 31.37 31.21 - 31.24 31.42 31.73 31.61
σ = 25 28.57 28.83 28.68 28.96 28.74 28.92 29.23 29.16
σ = 50 25.62 25.87 25.67 26.03 - 25.97 26.23 26.23

3.30GHz and an Nvidia Titan X GPU. It takes about 6 non-local means based methods are usually better on images
hours, one day and three days to train DnCNN-S, DnCNN- with regular and repetitive structures whereas discriminative
B/CDnCNN-B and DnCNN-3 on GPU, respectively. training based methods generally produce better results on
images with irregular textures. Actually, this is intuitively
reasonable because images with regular and repetitive struc-
B. Compared Methods
tures meet well with the non-local similarity prior; conversely,
We compare the proposed DnCNN method with several images with irregular textures would weaken the advantages
state-of-the-art denoising methods, including two non-local of such specific prior, thus leading to poor results.
similarity based methods (i.e., BM3D [2] and WNNM [13]), Figs. 4-5 illustrate the visual results of different methods.
one generative method (i.e., EPLL [33]), three discrimina- It can be seen that BM3D, WNNM, EPLL and MLP tend
tive training based methods (i.e., MLP [24], CSF [14] and to produce over-smooth edges and textures. While preserving
TNRD [16]). Note that CSF and TNRD are highly efficient sharp edges and fine details, TNRD is likely to generate
by GPU implementation while offering good image quality. artifacts in the smooth region. In contrast, DnCNN-S and
The implementation codes are downloaded from the authors’ DnCNN-B can not only recover sharp edges and fine details
websites and the default parameter settings are used in our but also yield visually pleasant results in the smooth region.
experiments. The testing code of our DnCNN models can be For color image denoising, the visual comparisons between
downloaded at https://github.com/cszn/DnCNN. CDnCNN-B and the benchmark CBM3D are shown in Figs. 6-
7. One can see that CBM3D generates false color artifacts
C. Quantitative and Qualitative Evaluation in some regions whereas CDnCNN-B can recover images
with more natural color. In addition, CDnCNN-B can generate
The average PSNR results of different methods on the
images with more details and sharper edges than CBM3D.
BSD68 dataset are shown in Table II. As one can see, both
DnCNN-S and DnCNN-B can achieve the best PSNR results
0.9
than the competing methods. Compared to the benchmark
Improvement in PSNR(dB) over BM3D/CBM3D

Gray

BM3D, the methods MLP and TNRD have a notable PSNR 0.8
Color

gain of about 0.35dB. According to [34], [38], few methods 0.7

can outperform BM3D by more than 0.3dB on average. In 0.6

contrast, our DnCNN-S model outperforms BM3D by 0.6dB
0.5
on all the three noise levels. Particularly, even with a single
model without known noise level, our DnCNN-B can still 0.4

outperform the competing methods which is trained for the 0.3

known specific noise level. It should be noted that both 0.2

10 15 20 25 30 35 40 45 50
DnCNN-S and DnCNN-B outperform BM3D by about 0.6dB Noise level σ

when σ = 50, which is very close to the estimated PSNR

bound over BM3D (0.7dB) in [38]. Fig. 8. Average PSNR improvement over BM3D/CBM3D with respect to
different noise levels by our DnCNN-B/CDnCNN-B model. The results are
Table III lists the PSNR results of different methods on evaluated on the gray/color BSD68 dataset.
the 12 test images in Fig. 3. The best PSNR result for each
image with each noise level is highlighted in bold. It can be Fig. 8 shows the average PSNR improvement over
seen that the proposed DnCNN-S yields the highest PSNR BM3D/CBM3D with respect to different noise levels
on most of the images. Specifically, DnCNN-S outperforms by DnCNN-B/CDnCNN-B model. It can be seen that
the competing methods by 0.2dB to 0.6dB on most of the our DnCNN-B/CDnCNN-B models consistently outperform
images and fails to achieve the best results on only two images BM3D/CBM3D by a large margin on a wide range of noise
“House” and “Barbara”, which are dominated by repetitive levels. This experimental result demonstrates the feasibility of
structures. This result is consistent with the findings in [39]: training a single DnCNN-B model for handling blind Gaussian
 8

(a) Noisy / 14.76dB (b) BM3D / 26.21dB (c) WNNM / 26.51dB (d) EPLL / 26.36dB

(e) MLP / 26.54dB (f) TNRD / 26.59dB (g) DnCNN-S / 26.90dB (h) DnCNN-B / 26.92dB

Fig. 4. Denoising results of one image from BSD68 with noise level 50.

denoising within a wide range of noise levels. the GPU time, the proposed DnCNN achieves very appealing
computational efficiency, e.g., it can denoise an image of size
512 × 512 in 60ms with unknown noise level, which is a
D. Run Time distinct advantage over TNRD.
In addition to visual quality, another important aspect for
an image restoration method is the testing speed. Table IV E. Experiments on Learning a Single Model for Three General
shows the run times of different methods for denoising images Image Denoising Tasks
of sizes 256 × 256, 512 × 512 and 1024 × 1024 with noise In order to further show the capacity of the proposed
level 25. Since CSF, TNRD and our DnCNN methods are DnCNN model, a single DnCNN-3 model is trained for
well-suited for parallel computation on GPU, we also give the three general image denoising tasks, including blind Gaussian
corresponding run times on GPU. We use the Nvidia cuDNN- denoising, SISR and JPEG image deblocking. To the best
v5 deep learning library to accelerate the GPU computation of our knowledge, none of the existing methods have been
of the proposed DnCNN. As in [16], we do not count the reported for handling these three tasks with only a single
memory transfer time between CPU and GPU. It can be seen model. Therefore, for each task, we compare DnCNN-3 with
that the proposed DnCNN can have a relatively high speed the specific state-of-the-art methods. In the following, we
on CPU and it is faster than two discriminative models, MLP describe the compared methods and the test dataset for each
and CSF. Though it is slower than BM3D and TNRD, by task:
taking the image quality improvement into consideration, our • For Gaussian denoising, we use the state-of-the-art
DnCNN is still very competitive in CPU implementation. For BM3D and TNRD for comparison. The BSD68 dataset
 9

(a) Noisy / 15.00dB (b) BM3D / 25.90dB (c) WNNM / 26.14dB (d) EPLL / 25.95dB

(e) MLP / 26.12dB (f) TNRD / 26.16dB (g) DnCNN-S / 26.48dB (h) DnCNN-B / 26.48dB

Fig. 5. Denoising results of the image “parrot” with noise level 50.

TABLE III
T HE PSNR( D B) RESULTS OF DIFFERENT METHODS ON 12 WIDELY USED TESTING IMAGES .

Images C.man House Peppers Starfish Monar. Airpl. Parrot Lena Barbara Boat Man Couple Average
Noise Level σ = 15
BM3D [2] 31.91 34.93 32.69 31.14 31.85 31.07 31.37 34.26 33.10 32.13 31.92 32.10 32.372
WNNM [13] 32.17 35.13 32.99 31.82 32.71 31.39 31.62 34.27 33.60 32.27 32.11 32.17 32.696
EPLL [33] 31.85 34.17 32.64 31.13 32.10 31.19 31.42 33.92 31.38 31.93 32.00 31.93 32.138
CSF [14] 31.95 34.39 32.85 31.55 32.33 31.33 31.37 34.06 31.92 32.01 32.08 31.98 32.318
TNRD [16] 32.19 34.53 33.04 31.75 32.56 31.46 31.63 34.24 32.13 32.14 32.23 32.11 32.502
DnCNN-S 32.61 34.97 33.30 32.20 33.09 31.70 31.83 34.62 32.64 32.42 32.46 32.47 32.859
DnCNN-B 32.10 34.93 33.15 32.02 32.94 31.56 31.63 34.56 32.09 32.35 32.41 32.41 32.680
Noise Level σ = 25
BM3D [2] 29.45 32.85 30.16 28.56 29.25 28.42 28.93 32.07 30.71 29.90 29.61 29.71 29.969
WNNM [13] 29.64 33.22 30.42 29.03 29.84 28.69 29.15 32.24 31.24 30.03 29.76 29.82 30.257
EPLL [33] 29.26 32.17 30.17 28.51 29.39 28.61 28.95 31.73 28.61 29.74 29.66 29.53 29.692
MLP [24] 29.61 32.56 30.30 28.82 29.61 28.82 29.25 32.25 29.54 29.97 29.88 29.73 30.027
CSF [14] 29.48 32.39 30.32 28.80 29.62 28.72 28.90 31.79 29.03 29.76 29.71 29.53 29.837
TNRD [16] 29.72 32.53 30.57 29.02 29.85 28.88 29.18 32.00 29.41 29.91 29.87 29.71 30.055
DnCNN-S 30.18 33.06 30.87 29.41 30.28 29.13 29.43 32.44 30.00 30.21 30.10 30.12 30.436
DnCNN-B 29.94 33.05 30.84 29.34 30.25 29.09 29.35 32.42 29.69 30.20 30.09 30.10 30.362
Noise Level σ = 50
BM3D [2] 26.13 29.69 26.68 25.04 25.82 25.10 25.90 29.05 27.22 26.78 26.81 26.46 26.722
WNNM [13] 26.45 30.33 26.95 25.44 26.32 25.42 26.14 29.25 27.79 26.97 26.94 26.64 27.052
EPLL [33] 26.10 29.12 26.80 25.12 25.94 25.31 25.95 28.68 24.83 26.74 26.79 26.30 26.471
MLP [24] 26.37 29.64 26.68 25.43 26.26 25.56 26.12 29.32 25.24 27.03 27.06 26.67 26.783
TNRD [16] 26.62 29.48 27.10 25.42 26.31 25.59 26.16 28.93 25.70 26.94 26.98 26.50 26.812
DnCNN-S 27.03 30.00 27.32 25.70 26.78 25.87 26.48 29.39 26.22 27.20 27.24 26.90 27.178
DnCNN-B 27.03 30.02 27.39 25.72 26.83 25.89 26.48 29.38 26.38 27.23 27.23 26.91 27.206
 10

(a) Ground-truth (b) Noisy / 17.25dB (c) CBM3D / 25.93dB (d) CDnCNN-B / 26.58dB

Fig. 6. Color image denoising results of one image from the DSD68 dataset with noise level 35.

(a) Ground-truth (b) Noisy / 15.07dB (c) CBM3D / 26.97dB (d) CDnCNN-B / 27.87dB

Fig. 7. Color image denoising results of one image from the DSD68 dataset with noise level 45.

TABLE IV
RUN TIME ( IN SECONDS ) OF DIFFERENT METHODS ON IMAGES OF SIZE 256 × 256, 512 × 512 AND 1024 × 1024 WITH NOISE LEVEL 25. F OR CSF,
TNRD AND OUR PROPOSED D N CNN, WE GIVE THE RUN TIMES ON CPU ( LEFT ) AND GPU ( RIGHT ). I T IS ALSO WORTH NOTING THAT SINCE THE RUN
TIME ON GPU VARIES GREATLY WITH RESPECT TO GPU AND GPU- ACCELERATED LIBRARY, IT IS HARD TO MAKE A FAIR COMPARISON BETWEEN CSF,
TNRD AND OUR PROPOSED D N CNN. T HEREFORE , WE JUST COPY THE RUN TIMES OF CSF AND TNRD ON GPU FROM THE ORIGINAL PAPERS .

Methods BM3D WNNM EPLL MLP CSF TNRD DnCNN-S DnCNN-B

256×256 0.65 203.1 25.4 1.42 2.11 / - 0.45 / 0.010 0.74 / 0.014 0.90 / 0.016
512×512 2.85 773.2 45.5 5.51 5.67 / 0.92 1.33 / 0.032 3.41 / 0.051 4.11 / 0.060
1024×1024 11.89 2536.4 422.1 19.4 40.8 / 1.72 4.61 / 0.116 12.1 / 0.200 14.1 / 0.235

are used for testing the performance. For BM3D and Table V lists the average PSNR and SSIM results of
TNRD, we assume that the noise level is known. different methods for different general image denoising tasks.
• For SISR, we consider two state-of-the-art methods, i.e., As one can see, even we train a single DnCNN-3 model for the
TNRD and VDSR [35]. TNRD trained a specific model three different tasks, it still outperforms the nonblind TNRD
for each upscalling factor while VDSR [35] trained a and BM3D for Gaussian denoising. For SISR, it surpasses
single model for all the three upscaling factors (i.e., 2, 3 TNRD by a large margin and is on par with VDSR. For JPEG
and 4). We adopt the four testing datasets (i.e., Set5 and image deblocking, DnCNN-3 outperforms AR-CNN by about
Set14, BSD100 and Urban100 [40]) used in [35]. 0.3dB in PSNR and has about 0.1dB PSNR gain over TNRD
• For JPEG image deblocking, our DnCNN-3 is compared on all the quality factors.
with two state-of-the-art methods, i.e., AR-CNN [41] and
TNRD [16]. The AR-CNN method trained four specific Fig. 9 and Fig. 10 show the visual comparisons of different
models for the JPEG quality factors 10, 20, 30 and 40, methods for SISR. It can be seen that both DnCNN-3 and
respectively. For TNRD, three models for JPEG quality VDSR can produce sharp edges and fine details whereas
factors 10, 20 and 30 are trained. As in [41], we adopt TNRD tend to generate blurred edges and distorted lines.
the Classic5 and LIVE1 as test datasets. Fig. 11 shows the JPEG deblocking results of different meth-
ods. As one can see, our DnCNN-3 can recover the straight
 11

TABLE V R EFERENCES
AVERAGE PSNR( D B)/SSIM RESULTS OF DIFFERENT METHODS FOR
G AUSSIAN DENOISING WITH NOISE LEVEL 15, 25 AND 50 ON BSD68 [1] A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for
DATASET, SINGLE IMAGE SUPER - RESOLUTION WITH UPSCALING FACTORS image denoising,” in IEEE Conference on Computer Vision and Pattern
2, 3 AND 4 ON S ET 5, S ET 14, BSD100 AND U RBAN 100 DATASETS , JPEG Recognition, vol. 2, 2005, pp. 60–65.
IMAGE DEBLOCKING WITH QUALITY FACTORS 10, 20, 30 AND 40 ON [2] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by
C LASSIC 5 AND LIVE1 DATASETS . T HE BEST RESULTS ARE HIGHLIGHTED sparse 3-D transform-domain collaborative filtering,” IEEE Transactions
IN BOLD . on Image Processing, vol. 16, no. 8, pp. 2080–2095, 2007.
[3] A. Buades, B. Coll, and J.-M. Morel, “Nonlocal image and movie
Gaussian Denoising denoising,” International Journal of Computer Vision, vol. 76, no. 2,
Noise BM3D TNRD DnCNN-3 pp. 123–139, 2008.
Dataset
Level PSNR / SSIM PSNR / SSIM PSNR / SSIM
15 31.08 / 0.8722 31.42 / 0.8826 31.46 / 0.8826
[4] J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, “Non-local
BSD68 25 28.57 / 0.8017 28.92 / 0.8157 29.02 / 0.8190 sparse models for image restoration,” in IEEE International Conference
50 25.62 / 0.6869 25.97 / 0.7029 26.10 / 0.7076 on Computer Vision, 2009, pp. 2272–2279.
Single Image Super-Resolution [5] M. Elad and M. Aharon, “Image denoising via sparse and redundant
Upscaling TNRD VDSR DnCNN-3 representations over learned dictionaries,” IEEE Transactions on Image
Dataset Processing, vol. 15, no. 12, pp. 3736–3745, 2006.
Factor PSNR / SSIM PSNR / SSIM PSNR / SSIM
2 36.86 / 0.9556 37.56 / 0.9591 37.58 / 0.9590 [6] W. Dong, L. Zhang, G. Shi, and X. Li, “Nonlocally centralized sparse
Set5 3 33.18 / 0.9152 33.67 / 0.9220 33.75 / 0.9222 representation for image restoration,” IEEE Transactions on Image
4 30.85 / 0.8732 31.35 / 0.8845 31.40 / 0.8845 Processing, vol. 22, no. 4, pp. 1620–1630, 2013.
2 32.51 / 0.9069 33.02 / 0.9128 33.03 / 0.9128 [7] L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based
Set14 3 29.43 / 0.8232 29.77 / 0.8318 29.81 / 0.8321 noise removal algorithms,” Physica D: Nonlinear Phenomena, vol. 60,
4 27.66 / 0.7563 27.99 / 0.7659 28.04 / 0.7672
no. 1, pp. 259–268, 1992.
2 31.40 / 0.8878 31.89 / 0.8961 31.90 / 0.8961
BSD100 3 28.50 / 0.7881 28.82 / 0.7980 28.85 / 0.7981 [8] S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, “An iterative regu-
4 27.00 / 0.7140 27.28 / 0.7256 27.29 / 0.7253 larization method for total variation-based image restoration,” Multiscale
2 29.70 / 0.8994 30.76 / 0.9143 30.74 / 0.9139 Modeling & Simulation, vol. 4, no. 2, pp. 460–489, 2005.
Urban100 3 26.42 / 0.8076 27.13 / 0.8283 27.15 / 0.8276 [9] Y. Weiss and W. T. Freeman, “What makes a good model of natural
4 24.61 / 0.7291 25.17 / 0.7528 25.20 / 0.7521 images?” in IEEE Conference on Computer Vision and Pattern Recog-
JPEG Image Deblocking nition, 2007, pp. 1–8.
Dataset
Quality AR-CNN TNRD DnCNN-3 [10] X. Lan, S. Roth, D. Huttenlocher, and M. J. Black, “Efficient belief prop-
Factor PSNR / SSIM PSNR / SSIM PSNR / SSIM agation with learned higher-order Markov random fields,” in European
10 29.03 / 0.7929 29.28 / 0.7992 29.40 / 0.8026 Conference on Computer Vision, 2006, pp. 269–282.
20 31.15 / 0.8517 31.47 / 0.8576 31.63 / 0.8610 [11] S. Z. Li, Markov random field modeling in image analysis. Springer
Classic5
30 32.51 / 0.8806 32.78 / 0.8837 32.91 / 0.8861
40 33.34 / 0.8953 - 33.77 / 0.9003
Science & Business Media, 2009.
10 28.96 / 0.8076 29.15 / 0.8111 29.19 / 0.8123 [12] S. Roth and M. J. Black, “Fields of experts,” International Journal of
20 31.29 / 0.8733 31.46 / 0.8769 31.59 / 0.8802 Computer Vision, vol. 82, no. 2, pp. 205–229, 2009.
LIVE1
30 32.67 / 0.9043 32.84 / 0.9059 32.98 / 0.9090 [13] S. Gu, L. Zhang, W. Zuo, and X. Feng, “Weighted nuclear norm
40 33.63 / 0.9198 - 33.96 / 0.9247 minimization with application to image denoising,” in IEEE Conference
on Computer Vision and Pattern Recognition, 2014, pp. 2862–2869.
[14] U. Schmidt and S. Roth, “Shrinkage fields for effective image restora-
tion,” in IEEE Conference on Computer Vision and Pattern Recognition,
line whereas AR-CNN and TNRD are prone to generate 2014, pp. 2774–2781.
distorted lines. Fig. 12 gives an additional example to show [15] Y. Chen, W. Yu, and T. Pock, “On learning optimized reaction diffusion
processes for effective image restoration,” in IEEE Conference on
the capacity of the proposed model. We can see that DnCNN- Computer Vision and Pattern Recognition, 2015, pp. 5261–5269.
3 can produce visually pleasant output result even the input [16] Y. Chen and T. Pock, “Trainable nonlinear reaction diffusion: A flexible
image is corrupted by several distortions with different levels framework for fast and effective image restoration,” to appear in IEEE
transactions on Pattern Analysis and Machine Intelligence, 2016.
in different regions. [17] U. Schmidt, C. Rother, S. Nowozin, J. Jancsary, and S. Roth, “Discrim-
inative non-blind deblurring,” in IEEE Conference on Computer Vision
and Pattern Recognition, 2013, pp. 604–611.
[18] U. Schmidt, J. Jancsary, S. Nowozin, S. Roth, and C. Rother, “Cascades
V. C ONCLUSION of regression tree fields for image restoration,” IEEE Conference on
Computer Vision and Pattern Recognition, vol. 38, no. 4, pp. 677–689,
In this paper, a deep convolutional neural network was 2016.
proposed for image denoising, where residual learning is [19] K. Simonyan and A. Zisserman, “Very deep convolutional networks for
large-scale image recognition,” in International Conference for Learning
adopted to separating noise from noisy observation. The batch Representations, 2015.
normalization and residual learning are integrated to speed up [20] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification
the training process as well as boost the denoising perfor- with deep convolutional neural networks,” in Advances in Neural Infor-
mation Processing Systems, 2012, pp. 1097–1105.
mance. Unlike traditional discriminative models which train [21] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep
specific models for certain noise levels, our single DnCNN network training by reducing internal covariate shift,” in International
model has the capacity to handle the blind Gaussian denoising Conference on Machine Learning, 2015, pp. 448–456.
[22] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for
with unknown noise level. Moreover, we showed the feasibility image recognition,” in IEEE Conference on Computer Vision and Pattern
to train a single DnCNN model to handle three general image Recognition, 2016, pp. 770–778.
denoising tasks, including Gaussian denoising with unknown [23] V. Jain and S. Seung, “Natural image denoising with convolutional
noise level, single image super-resolution with multiple up- networks,” in Advances in Neural Information Processing Systems, 2009,
pp. 769–776.
scaling factors, and JPEG image deblocking with different [24] H. C. Burger, C. J. Schuler, and S. Harmeling, “Image denoising: Can
quality factors. Extensive experimental results demonstrated plain neural networks compete with BM3D?” in IEEE Conference on
that the proposed method not only produces favorable image Computer Vision and Pattern Recognition, 2012, pp. 2392–2399.
[25] J. Xie, L. Xu, and E. Chen, “Image denoising and inpainting with
denoising performance quantitatively and qualitatively but also deep neural networks,” in Advances in Neural Information Processing
has promising run time by GPU implementation. Systems, 2012, pp. 341–349.
 12

(a) Ground-truth (b) TNRD / 28.91dB (c) VDSR / 29.95dB (d) DnCNN-3 / 30.02dB

Fig. 9. Single image super-resolution results of “butterfly” from Set5 dataset with upscaling factor 3.

(a) Ground-truth (b) TNRD / 32.00dB (c) VDSR / 32.58dB (d) DnCNN-3 / 32.73dB

Fig. 10. Single image super-resolution results of one image from Urban100 dataset with upscaling factor 4.

(a) JPEG / 28.10dB (b) AR-CNN / 28.85dB (c) TNRD / 29.54dB (d) DnCNN-3 / 29.70dB

Fig. 11. JPEG image deblocking results of “Carnivaldolls” from LIVE1 dataset with quality factor 10.

(a) Input Image (b) Output Residual Image (c) Restored Image

Fig. 12. An example to show the capacity of our proposed model for three different tasks. The input image is composed by noisy images with noise level
15 (upper left) and 25 (lower left), bicubically interpolated low-resolution images with upscaling factor 2 (upper middle) and 3 (lower middle), JPEG images
with quality factor 10 (upper right) and 30 (lower right). Note that the white lines in the input image are just used for distinguishing the six regions, and the
residual image is normalized into the range of [0, 1] for visualization. Even the input image is corrupted with different distortions in different regions, the
restored image looks natural and does not have obvious artifacts.
 13

[26] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,

V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,”
in IEEE Conference on Computer Vision and Pattern Recognition, June
2015.
[27] K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers:
Surpassing human-level performance on imagenet classification,” in
IEEE International Conference on Computer Vision, 2015, pp. 1026–
1034.
[28] J. Duchi, E. Hazan, and Y. Singer, “Adaptive subgradient methods
for online learning and stochastic optimization,” Journal of Machine
Learning Research, vol. 12, no. Jul, pp. 2121–2159, 2011.
[29] M. D. Zeiler, “Adadelta: an adaptive learning rate method,” arXiv
preprint arXiv:1212.5701, 2012.
[30] D. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
in International Conference for Learning Representations, 2015.
[31] R. Timofte, V. De Smet, and L. Van Gool, “A+: Adjusted anchored
neighborhood regression for fast super-resolution,” in Asian Conference
on Computer Vision. Springer International Publishing, 2014, pp. 111–
126.
[32] D. Kiku, Y. Monno, M. Tanaka, and M. Okutomi, “Residual interpolation
for color image demosaicking,” in 2013 IEEE International Conference
on Image Processing. IEEE, 2013, pp. 2304–2308.
[33] D. Zoran and Y. Weiss, “From learning models of natural image patches
to whole image restoration,” in IEEE International Conference on
Computer Vision, 2011, pp. 479–486.
[34] A. Levin and B. Nadler, “Natural image denoising: Optimality and
inherent bounds,” in IEEE Conference on Computer Vision and Pattern
Recognition, 2011, pp. 2833–2840.
[35] J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using
very deep convolutional networks,” in IEEE Conference on Computer
Vision and Pattern Recognition, 2016, pp. 1646–1654.
[36] J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via
sparse representation,” IEEE Transactions on Image Processing, vol. 19,
no. 11, pp. 2861–2873, 2010.
[37] A. Vedaldi and K. Lenc, “Matconvnet: Convolutional neural networks
for matlab,” in Proceedings of the 23rd Annual ACM Conference on
Multimedia Conference, 2015, pp. 689–692.
[38] A. Levin, B. Nadler, F. Durand, and W. T. Freeman, “Patch complexity,
finite pixel correlations and optimal denoising,” in European Conference
on Computer Vision, 2012, pp. 73–86.
[39] H. C. Burger, C. Schuler, and S. Harmeling, “Learning how to combine
internal and external denoising methods,” in Pattern Recognition, 2013,
pp. 121–130.
[40] J.-B. Huang, A. Singh, and N. Ahuja, “Single image super-resolution
from transformed self-exemplars,” in IEEE Conference on Computer
Vision and Pattern Recognition, 2015, pp. 5197–5206.
[41] C. Dong, Y. Deng, C. Change Loy, and X. Tang, “Compression artifacts
reduction by a deep convolutional network,” in IEEE International
Conference on Computer Vision, 2015, pp. 576–584.