1792 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO.

5, MAY 2014

Two-Step Sparse Coding for the Pan-Sharpening of

Remote Sensing Images
Cheng Jiang, Hongyan Zhang, Member, IEEE, Huanfeng Shen, Member, IEEE, and
Liangpei Zhang, Senior Member, IEEE

Abstract—Remote sensing image pan-sharpening is an impor- and then utilize a pan-sharpening technique to produce a
tant way of enhancing the spatial resolution of a multispectral (MS) high-resolution multispectral (HRM) image by fusing the two
image by fusing it with a registered panchromatic (PAN) image. images.
The traditional pan-sharpening methods often suffer from color
distortion and are still far from being able to synthesize a real For the pan-sharpening task, the key problem is to find and
high-resolution MS image, as could be directly acquired by a better utilize the relationship between the LRM image and the HRP
sensor. Inspired by the rapid development of sparse representation image. A lot of pan-sharpening methods have been developed
theory, we propose a two-step sparse coding method with patch to date, and they can be categorized into four types: trans-
normalization (PN-TSSC) for image pan-sharpening. Traditional formation and substitution based, arithmetic based, ARSIS
one-step sparse coding has difficulty in choosing dictionary atoms
when the structural information is weak or lost. By exploiting the
[1] concept based (from its French name Amélioration de la
local similarity between the MS and PAN images, the proposed Résolution Spatiale par Injection de Structures), and restoration
sparse coding method deals with the dictionary atoms in two steps, based. The transformation and substitution based methods first
which has been found to be an effective way of overcoming this transform the resampled LRM image to a space where one of
problem. The experimental results with IKONOS, QuickBird, and its components is strongly correlated with the low-resolution
WorldView-2 data suggest that the proposed method can effec-
panchromatic (LRP) image degraded from the HRP image;
tively improve the spatial resolution of a MS image, with little color
distortion. The pan-sharpened high-resolution MS image outper- the HRP image is then stretched to replace the component;
forms those images fused by other traditional and state-of-the-art finally, an inverse transform is performed. The transformation
methods, both quantitatively and perceptually. and substitution based methods have been widely used, e.g.,
Index Terms—Image fusion, pan-sharpening, remote sensing intensity-hue-saturation (IHS or LHS) [2], principal component
image, two-step sparse coding. analysis (PCA) [3], and the Gram-Schmidt (GS) process [4].
As to the arithmetic based methods, the operations of mul-
tiplication, division, addition, and subtraction are combined
I. INTRODUCTION in different ways to achieve a better fusion performance [5].
Among them, the Brovey transform [5] is the most well known.

H IGH spatial resolution multispectral (MS) images are

favored by a lot of remote sensing applications, and
a number of high-resolution remote sensing systems have
For the ARSIS concept based methods, these approaches as-
sume that the missing spatial information in the LRM image
can be obtained from the high frequencies of the HRP image,
been launched and widely used, such as IKONOS, QuickBird, so the relationships between the high frequencies in the HRP
GeoEye-1, WorldView-1, and WorldView-2. Due to the phys- image and the LRM image are searched and exploited [1]. The
ical limitations, it is difficult to acquire a high-resolution MS typical ARSIS concept based methods are high-pass filtering
image directly. One commonly used strategy is to use two (HPF) [3] and the wavelet based methods [6]. The above three
sensors to separately acquire a high-resolution panchromatic types of methods can be viewed as traditional methods [7],
(HRP) image and a low-resolution multispectral (LRM) image, which tend to cause color distortions due to the wavelength
extension of the new satellite PAN images [7]. Inspired by
the rapid development of the single-image super-resolution
Manuscript received April 23, 2013; revised July 16, 2013 and August 31, technique, restoration based methods have recently become
2013; accepted September 20, 2013. Date of publication October 11, 2013;
date of current version June 13, 2014. This work was supported in part by popular. For the restoration based methods, the LRM image
the National Basic Research Program of China (973 Program) under Grant and the HRP image are both viewed as observations of the
2011CB707105, by the 863 program under Grant 2013AA12A301, by the HRM image via the image degradation model, and a prior of the
National Natural Science Foundation of China under Grants 61201342 and
40930532, and by Program for Changjiang Scholars and Innovative Research
target HRM image should be assumed [8]–[10]. In addition to
Team in University (IRT1278), the Fundamental Research Funds for the the four main types of methods, hybrid methods have also been
Central Universities. (Corresponding author: H. Zhang.) developed [11]–[13] and have shown improved performance.
C. Jiang, H. Zhang, and L. Zhang are with the State Key Laboratory of In-
formation Engineering in Surveying, Mapping, and Remote Sensing, Wuhan
Recently, the compressed sensing (CS) technique has been
University, P.R. China (e-mail: zhanghongyan@whu.edu.cn). applied to pan-sharpening [14]. Since the fused image patch
H. Shen is with the School of Resource and Environmental Science, Wuhan is adaptively estimated with different sets of dictionary atoms,
University, P.R. China.
Color versions of one or more of the figures in this paper are available online
the CS based method achieves impressive results. However, the
at http://ieeexplore.ieee.org. applicability is limited as the dictionary construction method
Digital Object Identifier 10.1109/JSTARS.2013.2283236 needs HRM training images, which may not be available. To

1939-1404 © 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
JIANG et al.: TWO-STEP SPARSE CODING FOR THE PAN-SHARPENING OF REMOTE SENSING IMAGES 1793

deal with this problem, Jiang et al. [15] constructed a joint dic- In fact, natural images tend to be sparse in the redundant image
tionary from upsampled LRM and HRP training images, which domain [21]. The redundant image domain is often called as the
are easily acquired from the available remote sensing systems. dictionary, where the image can be represented as a linear com-
The shortcoming of the method in [15] is that it still needs bination of only a small number of the dictionary atoms. For
a large collection of training images, and the trained dictio- practical applications, the images are processed patch by patch,
nary is also relatively large, which takes much computing time and it has been shown that a trained dictionary is better than the
for sparse solutions. Following the well-known coupled sparse fixed redundant bases such as discrete cosine transform (DCT)
coding technique for natural image super-resolution [16], the [22]. The sparse representation problem is often represented as:
latest work [17] brings the coupled sparse coding technique to
image pan-sharpening. The method in [17] shows good potential (1)
by only using the HRP image and its degraded version, which
has the same resolution as the LRM image. The coupled image where indicates the -norm, and denotes the -norm.
pairs are directly used, without training, to construct two cou- is a column vector representing a signal or a lexicographically
pled dictionaries, which consist of a high-resolution dictionary
ordered image patch. is a matrix representing the dictionary,
and a low-resolution dictionary. Since we assume that the sparse
with each column called as an atom. is the vector of the sparse
coefficients of the target HRM patch over the high-resolution
coefficients, with most of its coefficients being close to or zero.
(HR) dictionary are the same as the sparse coefficients of the
is the regularization parameter.
LRM patch over the low-resolution dictionary, the sparse coef-
Since (1) is well known as an NP-hard problem, greedy al-
ficients of every LRM patch should be estimated as accurately
gorithms such as orthogonal matching pursuit (OMP) are often
as possible. However, at the sparse coding stage, the method
used to tackle this problem. An alternative approach is to relax
in [17] relies strongly on the structural information of the LRM
(1) to an -norm convex optimization problem [19]:
patch, which may be lost or very weak, due to its low resolution,
so the coding of the LRM patch may be contaminated by unsuit-
able low-resolution (LR) dictionary atoms, for which the corre- (2)
sponding HR dictionary atoms cannot effectively represent the
ideal HRM patch. From the experimental results in Section IV, where indicates the -norm.
it is found that the method in [17] preserves the spectral prop- Many -norm optimization methods have been proposed to
erties well, but it shows poor spatial results in the regions with solve this convex problem [23], and the least angle regression
small objects or fine details. stagewise (LARS) algorithm [24] has been widely used due to
In view of this, we propose a novel sparse coding based re- its good performance and stability. In this paper, the LARS al-
mote sensing image pan-sharpening method. The main contri- gorithm package developed by Julien Mairal et al. [25], op-
bution of this paper is twofold. First, a two-step sparse coding is timized with multi-thread implementation, is adopted for our
used to obtain the sparse coefficients, instead of the traditional second sparse coding step.
one-step sparse coding. For the two-step sparse coding, we fully
utilize the strong correlation between the LRM patch and the
corresponding LRP patch, and thus overcome the problem of B. Sparse Representation With Coupled Dictionaries
structural information loss in some LRM patches. This greatly
Sparse representation was first applied to single natural image
improves the spatial details but may introduce a little color dis-
super-resolution by Yang et al. [16], and the main idea of the
tortion in some cases. To handle this issue, the patch normaliza-
method is to assume an occurrence relation in that the upsam-
tion strategy, which aims to improve the stability of the sparse
pled LR and HR image patch pairs share the same sparse co-
coding, is exploited and used to prevent the color distortion that
efficients with respect to their own dictionaries. Supposing that
may happen when using the two-step sparse coding on the image
coupled dictionaries are available (Section II-C describes how
patch directly, and further improves the fusion result. Moreover,
to construct the coupled dictionaries for pan-sharpening), then
since the proposed method deals with each patch independently,
for a certain upsampled LR image patch, the sparse coefficients
it can be fast with a multi-thread implementation. For simplicity,
of the LR image patch over the LR dictionary are first solved by
the proposed method also processes each MS band separately,
using (2). Then, with this assumption, the sparse coefficients of
as in [17], and thus less memory and computational burden are
the LR image patch are taken as the sparse coefficients of the
required.
HR image patch over the HR dictionary. Finally, the target HR
image patch is obtained by multiplying the HR dictionary with
II. PAN-SHARPENING WITH SPARSE REPRESENTATION the sparse coefficients. Recently, Zhu and Bamler [17] brought
this idea to the application of pan-sharpening, with each band
A. Sparse Representation of the LRM image being processed one by one. Due to its effec-
tiveness, this idea is also adopted in our proposed method. Note
Inspired by the research into the receptive fields, sparse rep- that in the work of Yang et al. [16], the first- and second-order
resentation has been proposed as a powerful statistical image gradients of the upsampled LR image patch are concatenated
modeling technique [18]. Since the emergence of CS theory into one vector to represent the LR image patch, and so are the
[19], [20], the theory and application of sparse representation, LR training samples when training the dictionaries. However, in
which is strongly correlated with CS, has been widely studied. our work, the images are used directly, which is more suitable.
1794 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 5, MAY 2014

Fig. 1. The flowchart of the proposed method.

C. Construction of the Coupled Dictionaries for stage, every patch of the LRM image is coded with one step
Pan-Sharpening by directly using (2) to obtain the sparse coefficients over the
For pan-sharpening, a corresponding LRM image and same LR dictionary. In our method, a two-step sparse coding
HRP image are available, and the goal is to produce the approach is utilized that fully considers the structural similarity
HRM image by fusing and . Unlike the traditional between the LRP patch and the LRM patch. In addition, the
sparse representation based methods, which need external patch normalization technique is used to improve the stability
image collections to train the dictionaries, the coupled dictio- of the sparse coding.
naries are constructed directly from the panchromatic (PAN) A. Patch Structural Similarity Property
image and its degraded version [17]. The HRP image is
For the th band of the LRM image , all the patches
first blurred and downsampled by a factor to get the LRP
are extracted with a patch size and a step size .
image , which has the same resolution and the same image
For the th patch (we use upper-case to represent an image
size as the LRM image . We extract all the patches from
and lower-case to denote a patch from it), it is strongly corre-
the LRP image with a patch size and a step size
lated with the corresponding patch from the LRP band, as
, then rearrange every patch to a vector. Finally, we
the patches share the same scene and are often acquired by dif-
build a matrix with these vectors, which is viewed as the
ferent sensors with an overlapping spectral domain. As shown
LR dictionary. Following the same method, the HR dictionary
in Fig. 2, despite the differences among the spectral responses,
is obtained from the HRP image with a patch size
each patch from the LRM bands tends to share a similar struc-
and a step size . In this way, the
tural pattern to the patch at the same location on the LRP band.
numbers of the dictionary atoms of the LR dictionary and
We name this important discovery the “patch structural simi-
the HR dictionary are the same.
larity property.” In addition, we call the th LR dictionary atom
(also denoted by ), which comes from the column stacked
III. PAN-SHARPENING WITH TWO-STEP SPARSE CODING
patch , the “adjoint-atom” of the LRM patch .
The overall flowchart of the proposed pan-sharpening
method is shown in Fig. 1. For the proposed pan-sharpening B. Two-Step Sparse Coding
method, the coupled dictionaries are constructed as described in For the traditional sparse coding, when the LR image patch
Section II-C, and the next task is to find the sparse representa- to be coded is weak in its structural information, it will be easily
tions of the LR patches over the LR dictionary, and reconstruct contaminated by unsuitable dictionary atoms, which may have
the fused patches with the HR dictionary, as mentioned in a negative effect on the performance of the image pan-sharp-
Section II-B. In the latest work [17], at the sparse coding ening. Therefore, an intuitive way is to use a two-step sparse
JIANG et al.: TWO-STEP SPARSE CODING FOR THE PAN-SHARPENING OF REMOTE SENSING IMAGES 1795

over the general LR dictionary , which takes all the patches

from the corresponding LRP image as atoms:

(5)

where is the vector of sparse coefficients by the second sparse

coding step, and is the regularization parameter. Equation
(5) is a traditional sparse coding problem. Since (5) cannot be
solved exactly with many dictionary atoms, we relax it to (6)
and use the LARS algorithm optimized with multi-thread im-
plementation (which can be downloaded from Julien Mairal’s
personal page [26]) to speed up the proposed method.

(6)

With the two-step sparse coding, the final sparse vector is

acquired by:

(7)

where is a vector with the same dimension as , with only

Fig. 2. Similar structural patterns among patches at the same location from the
LRM and LRP images. Each row of the 7 7 patches is randomly selected at the
the th element being one and the others being zero. Finally, as
same spatial location from the IKONOS image. B, G, R, and NIR represent the mentioned in Section II, the fused patch is reconstructed
blue, green, red, and near-infrared bands, respectively. Each image is magnified by:
10 times.
(8)

coding to distinguish the more reliable atoms from the less re- By using (4), (6), (7), and (8), all the patches from each band
liable ones. That is, at the first sparse coding step, the reliable of the LRM image are processed independently to acquire the
atoms are used to sparsely represent the vector, and at the second corresponding fused HRM patches, and the overlapping areas
sparse coding step, the rest of the atoms are used to sparsely of the fused patches are averaged.
represent the residual produced by the first sparse coding step. Note that in the first sparse coding step, the one-atom dictio-
How to find the more reliable atoms remains a problem, which nary varies with the current image patch being processed. How-
may differ between different applications. However, motivated ever, at the second sparse coding step, the dictionary is the same
by the patch structural similarity property of the LRP and LRM for all the LRM patches. One may wonder why the adjoint-atom
images, the adjoint-atom of the patch to be coded is sure to be a is still in the common dictionary at the second sparse coding
reliable atom. The proposed two-step sparse coding method is step, since it has already been coded in the first sparse coding
therefore simplified by using only one reliable atom at the first step. The reason for this is that in the first sparse coding stage,
sparse coding step. In this way, the adjoint-atom is treated the other dictionary atoms are not considered, so there may be
differently when we find the sparse coefficients of over the too much priority given to the atom , especially for the cases
LR dictionary . We first use this reliable atom to construct where the structural similarity between the LRM band patch and
a special one-atom dictionary (also denoted by ), and then use the LRP band patch is not very strong, e.g., the NIR patch with
(3) to sparsely represent the LRM patch at the first sparse the LRP patch in the first row in Fig. 2, and the blue, green, and
coding step: red band patch with the LRP patch in the fifth row in Fig. 2.
Therefore, at the second sparse coding step, the adjoint-atom is
still considered, to adaptively remedy this problem.
(3)
With the first sparse coding step, the proposed method puts
more emphasis on the adjoint-atom than a single-step sparse
where is the coefficient and is the regularization param- coding based method. Thus, the two-step sparse coding adap-
eter. Since the special dictionary consists of only one atom, tively utilizes the patch structural similarity property in a simple
is a constant, and the sparse term can be omitted. We then way, rather than only relying on the structural information of
get: the LRM image patch to infer the correct sparse representa-
tion. The experiments in Section IV show that this consideration
(4) improves the performance greatly over the traditional one-step
sparse coding method.

It is easy to solve (4) by letting its derivative be zero. Clearly, C. Patch Normalization
cannot be well represented by only one atom, so the second Patch normalization is widely used in regression problems
step is to represent the first step’s residual [27] to subtract a constant before the main procedures. It has
1796 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 5, MAY 2014

also been used in sparse representation based natural image pro- (monomodal or multimodal) and its reference [31]. The details
cessing to improve the numerical stability of sparse coding, and of these indices can be found in Appendix A.
has resulted in a better visual quality of the results [28]. We
adopt this strategy in the proposed method, and find that it is
A. Experiments With IKONOS Data
applicable to our two-step sparse coding based remote sensing
image pan-sharpening process. The IKONOS system simultaneously offers a 4 m-resolution
Patch normalization consists of two steps in the proposed MS image with four bands and a single-band 1 m-resolution
method. First, for each atom of the common LR dictionary and PAN image. The original MS and PAN images are low-pass fil-
HR dictionary, the mean intensity value of the vector is sub- tered and downsampled to acquire the 16 m-resolution and 4
tracted from all its elements. Second, for the LRM patch , m-resolution test images, respectively, and the original 4 m-res-
the mean intensity value is also subtracted from all its elements olution MS image is used as the reference. The data set used
before the first sparse coding step in (4), and is added back to in this experiment is shown in Fig. 3. For the AM method,
all the elements of the fused patch after the patch recon- the Huber parameter is set as 2, the adjustable parameter
struction in (8). With this simple processing strategy, the spec- between the enhancement of the spatial information
tral quality of the proposed two-step sparse coding is improved and the preservation of the spectral information is set as 10, and
greatly, and the resulting fused image looks even smoother. the termination threshold is set as . We choose a patch
size and step size of [7 7] and [3 3], respectively, for the SC,
TSSC, and PN-TSSC methods. For the SC method, in (2) is
IV. EXPERIMENTS AND DISCUSSIONS
set as . For the TSSC and PN-TSSC methods, in (6) is set as
In Wald’s [29] view, a synthetic image should be as similar as and , respectively. We set the parameters of the proposed
possible to the image that the corresponding sensor would ob- method by taking both performance and computation time into
serve at the highest spatial resolution. This idea is adopted here consideration, and the influence of these parameters is analyzed
to assess the different pan-sharpening methods. In this section, in Appendix B.
we use IKONOS, QuickBird, and WorldView-2 data to test the The experimental results of the seven pan-sharpening
performance of the proposed method. To quantitatively assess methods are shown in Fig. 4(b)–(h), respectively. To facilitate
the quality of the results, the original data are low-pass filtered a spatial comparison, detailed regions are illustrated in the
and degraded (for convenience, we use MATLAB’s “imresize” top-left corner of the images. From the middle-right vegetation
function, with the “antialiasing” parameter being “true”, for de- area, it can be clearly observed that the GIHS result suffers
grading images in this paper). The experiments are performed from severe color distortion when compared with the original
on the degraded data, and the original MS image is used as the MS image, and many details of the PAN image which do not
reference. All the data used in the following experiments are belong to the original MS image are contained in the fused
Geo format images, and the MS image and PAN image are well image. The AWLP result has good color, but some small details
corresponded with sub-pixel accuracy. All the images are 11-bit are lost, and the result does not look very sharp. For the GS
without dynamic range adjustment, and the final results are dis- method, the result seems good, perceptually, with high contrast.
played by ENVI 4.7. However, compared with the original MS image, some areas
We compare the proposed method with three popular tradi- are over-sharpened, and the color in the middle-right vegetation
tional methods: generalized IHS (GIHS) [2], GS [4] (imple- area is brighter. The result of the AM method is similar to the
mented with ENVI 4.7, and we select the average of the LRM GS result, with better color. The SC result shows a good spec-
as the LRP), AWLP [30], and the state-of-the-art adjustable tral quality, but contains blurring effects and outlier pixels in
model based method (denoted by AM) [10]. The method used in some regions with small objects or fine details. The main reason
the related study [17], which uses the traditional sparse coding for this is that the SC method only relies on the structural in-
for pan-sharpening, is also included (denoted by SC). To more formation of the LR patch to infer the structural information of
comprehensively examine the performance of the proposed ap- the HR patch. As shown in Fig. 3(a), the structural information
proach, we give two sets of results for the proposed method: the of some areas is almost lost, so it is difficult to infer the correct
results using only two-step sparse coding (denoted by TSSC), sparse representation of the LRM image patch via one-step
and the results using both two-step sparse coding and patch nor- sparse coding. By using the two-step sparse coding, which can
malization (denoted by PN-TSSC). For a fair comparison, the adaptively utilize the structural similarity property between the
SC method is implemented on every patch independently, with LRM patch and its corresponding LRP patch, the TSSC result
an averaging operation for the overlapping areas, and the same overcomes the blurring effects observed in the SC result and
multi-thread LARS is employed as the -norm optimization shows very good spatial detail information, but suffers from
tool. some color distortion when compared with the result of the SC
We evaluate the results both visually and quantitatively. To method. By simultaneously incorporating both the two-step
quantitatively evaluate the experiments, five widely used quality sparse coding approach and the patch normalization strategy,
indices are used: the correlation coefficient (CC), the structural the proposed method achieves the best overall performance
similarity metric (SSIM), the spectral angle mapper (SAM), the among all the methods. The color of the PN-TSSC method is
erreur relative globale adimensionnelle de synthèse (ERGAS), very similar to that of the SC method and the AWLP method,
and the Q4 index. Based on Wald’s view, these indices call upon and it is much better than the colors of the other methods, while
distances that measure the discrepancy between a fused image the spatial details of the PN-TSSC method are much better than
JIANG et al.: TWO-STEP SPARSE CODING FOR THE PAN-SHARPENING OF REMOTE SENSING IMAGES 1797

Fig. 3. Input 600 600 IKONOS images: (a) upsampled 16 m MS image; (b) 4 m PAN image.

Fig. 4. Pan-sharpening results with the degraded IKONOS image. (a) Original; (b) GIHS; (c) AWLP; (d) GS; (e) AM; (f) SC; (g) TSSC; (h) PN-TSSC.

those of the SC method, and are comparable with or even better AWLP method injects the high frequencies of the PAN image
than those of the other methods. into every MS band, in proportion to their original values, in
The quantitative assessment results are shown in Table I, in such a way that the spectral angle is not changed before and after
which the best results for each quality index are marked in bold, fusion for the MS image. Due to the blurring effect, SC fails to
and the second best are underlined. B, G, R, and NIR represent surpass the traditional methods. Comparing the results of SC,
the results of the blue, green, red, and near-infrared bands, re- TSSC, and PN-TSSC, the results improve successively from SC
spectively, and Avg. is the average result of the B, G, R, and NIR to PN-TSSC, which indicates the effectiveness of the two-step
results. In our experience, a difference of 0.01 for CC, SSIM, sparse coding approach and the patch normalization strategy for
and Q4, and a difference of 0.1 for SAM and ERGAS, indicate the sparse coding based pan-sharpening methods from a quanti-
significant differences which can be noticed by visual inspec- tative perspective. As a state-of-the-art method, AM is distinctly
tion. Bearing this in mind, it can be seen that the quantitative as- better than the traditional methods, and PN-TSSC is comparable
sessment results are consistent with the visual evaluations. The with or even better than AM in most of the quality measures.
1798 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 5, MAY 2014

TABLE I
THE QUANTITATIVE ASSESSMENT RESULTS OF THE SIMULATED EXPERIMENTS WITH THE IKONOS DATA SHOWN IN FIG. 4

Fig. 5. Input 600 600 QuickBird images: (a) upsampled 11.2 m MS image; (b) 2.8 m PAN image.

B. Experiments With QuickBird Data methods. It should also be noted that, despite the proposed
To test the proposed method with a different sensor, the four- method showing superiority in both spatial and spectral quali-
band 2.8 m-resolution MS image and the 0.7 m-resolution PAN ties, an exception in the tiny region inside the yellow rectangle
image of the QuickBird data are low-pass filtered and resampled in Fig. 6 can be observed. Here, there is still a blurring effect
to 11.2 m-resolution and 2.8 m-resolution test images, respec- for the small building surrounded by trees. The reason for this
tively, and the original 2.8 m-resolution MS image is used as the may be as follows: on the one hand, the small building covers
reference, as shown in Fig. 5. For the AM method, the Huber pa- an area of about 17 4 pixels, which is small compared with
rameter is set as 3, the adjustable parameter is set the reconstructed 28 28 image patch, and the sparse coding
as 120, and the termination threshold is set as . The patch step is only concerned with the overall reconstruction accu-
size and step size are set as [7 7] and [3 3], respectively, for SC, racy of the whole patch. On the other hand, the occasion of a
TSSC, and PN-TSSC. For the SC method, in (2) is set as . small building surrounded by trees is a relatively rare occur-
For the TSSC and PN-TSSC methods, in (6) is set as for rence in the whole image, which also means that dictionary
both methods. atoms similar to this kind of occasion will not be abundant.
The results of the seven different methods are presented Therefore, to preserve the spectral quality, tiny details are sac-
in Fig. 6(b)–(h), respectively. To facilitate a spatial compar- rificed with the PN-TSSC method, and further research into
ison, magnified regions are displayed in the top-left corner of adopting a multi-resolution dictionary may be able to solve
the images. Compared with the traditional methods, the SC this imperfection.
method shows a good color appearance, but blurring effects The quantitative assessment results are shown in Table II, in
and some outlier pixels are obvious. With the two-step sparse which the best results for each quality index are marked in bold,
coding, the TSSC method shows good spatial details, but the and the second best are underlined. Again, it can be seen that
result is accompanied by slight color distortion. By simulta- the quantitative assessment results are consistent with the vi-
neously exploiting the two-step sparse coding method and the sual evaluations, and are similar to those in Table I, which indi-
patch normalization strategy, the proposed PN-TSSC method cates that the proposed PN-TSSC method is stable with different
succeeds in achieving the best overall quality among all the kinds of sensors.
JIANG et al.: TWO-STEP SPARSE CODING FOR THE PAN-SHARPENING OF REMOTE SENSING IMAGES 1799

Fig. 6. Pan-sharpening results with the degraded QuickBird image. (a) Original; (b) GIHS; (c) AWLP; (d) GS; (e) AM; (f) SC; (g) TSSC; (h) PN-TSSC.

TABLE II
THE QUANTITATIVE ASSESSMENT RESULTS OF THE SIMULATED EXPERIMENTS WITH THE QUICKBIRD DATA SHOWN IN FIG. 6

TABLE III
THE SPECTRAL RANGES OF THE BANDS OF WORLDVIEW-2

C. Experiments With WorldView-2 Data 2 m-resolution PAN image, and the original 2 m-resolution MS
image is used as the reference. For the AM method, the Huber
To further test the proposed method with a different sensor parameter is set as 0.5, the adjustable parameter is
type and more bands, we utilize a 2 m-resolution MS image and set as 30, and the termination threshold is set as . The
the corresponding 0.5 m-resolution PAN image, as acquired by patch size and the step size are set as [7 7] and [3 3], respec-
the WorldView-2 earth observation satellite. The spectral ranges tively, for SC, TSSC, and PN-TSSC. For the SC method, in
of all the bands of the WorldView-2 data are given in Table III. (2) is set as . For the TSSC and PN-TSSC methods, in (6)
The original data are degraded to obtain the test images, as is set as and , respectively.
shown in Fig. 7. The results of all the pan-sharpening methods The results of the different methods are presented in Fig. 8,
are then obtained by fusing the 8 m-resolution MS image and the and zoomed regions are displayed in the top-left corner of the
1800 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 5, MAY 2014

Fig. 7. Input 600 600 WorldView-2 images: (a) upsampled 8 m MS image; (b) 2 m PAN image.

Fig. 8. Pan-sharpening results with the degraded WorldView-2 image. (a) Original; (b) GIHS; (c) AWLP; (d) GS; (e) AM; (f) SC; (g) TSSC; (h) PN-TSSC.

images to facilitate a spatial comparison. For the GS method, The quantitative assessment results are shown in Table IV, in
the blurring effect is obvious. This is because GS selects the av- which the best results for each quality index are marked in bold,
erage of all the LRM bands as the LRP, and the spectral ranges and the second best are underlined. C, B, G, Y, R, R-E, NIR-1,
of the coastal band, the NIR-1 band, and the NIR-2 band are and NIR-2 represent the results of the coastal, blue, green,
both out (almost out for NIR-1) of the range of the panchromatic yellow, red, red edge, near-infrared-1, and near-infrared-2
band. This also affects the AM method, which assumes that bands, respectively, and Avg. is the average result of the C,
the panchromatic band is a linear combination of all the mul- B, G, Y, R, R-E, NIR-1, and NIR-2 results. As with the visual
tispectral bands with the same resolution. Since the PN-TSSC evaluations, the proposed PN-TSSC method performs the best
method can process every band and every local patch adaptively among all the methods, in terms of the quantitative evaluations.
by choosing atoms from the dictionary, it is relatively robust to
this kind of problem. From the results, it can be easily concluded D. Cost of Time
that the proposed PN-TSSC method ranks first among all the The GS method is implemented in ENVI 4.7, AM is imple-
methods, with a much better spatial quality. mented in Microsoft Visual C++ 6.0, and the other methods are
JIANG et al.: TWO-STEP SPARSE CODING FOR THE PAN-SHARPENING OF REMOTE SENSING IMAGES 1801

TABLE IV
THE QUANTITATIVE ASSESSMENT RESULTS OF THE SIMULATED EXPERIMENTS WITH THE WORLDVIEW-2 DATA SHOWN IN FIG. 8

TABLE V
TIME COSTS FOR THE DIFFERENT METHODS WITH 600 600 DATA

implemented in MATLAB 2012a. The personal computer we enhancement and spectral information preservation are the two
use is a Dell T1500. The central processing unit (CPU) is an key issues in the remote sensing image pan-sharpening task. On
Intel Core i3 540 @ 3.07 GHz with dual-cores and four threads, the one hand, to improve the spatial resolution of the existing
the RAM is 3 Gb with 3.06 GHz, and the operating system is one-step sparse coding based method, a two-step sparse coding
32-bit Windows XP. The time costs for the experiments with method is utilized on the basis of the patch structural similarity
the IKONOS, QuickBird, and WorldView-2 data are shown in between the low-resolution panchromatic image patch and the
Table V. From Table V, it can be seen that the time cost of low-resolution multispectral image patch. On the other hand,
the proposed PN-TSSC method is comparable with the AM a patch normalization strategy is used to improve the stability
method; however, it should be noted that the proposed method of the sparse coding and preserve the spectral information. The
could be easily implemented with parallel processing. It is also experimental results with IKONOS, QuickBird, and World-
reasonable to believe that with the rapid development in com- View-2 data show that the performance of the proposed method
puter hardware and computation techniques, the time cost of the is competitive in both the spatial and spectral qualities, and
proposed method will soon no longer be an issue. For the pro- it outperforms both traditional and state-of-the-art methods
posed method, the whole of the PAN image is used to construct when considering the overall performance. Moreover, since a
the dictionary. Therefore, when the PAN image is very large, whole image can be processed in patches independently with
i.e., 1000 1000, or even larger, the dictionary will also be multi-thread calculation, the proposed method can also be fast.
large, which will increase the computation time for each patch. With further work, better results may be obtained by consid-
A simple solution is to partition the whole image into several ering all the multispectral bands together. A better dictionary
parts with overlaps, and process each part with the proposed should also be considered to speed up the method when the
method separately. In addition to this, a pre-trained dictionary processed image is very large.
pair for a certain kind of remote sensing system could be a better
choice, and this will be investigated in our future work.
APPENDIX A
V. CONCLUSIONS The five indices used in this paper are defined as follows.
This paper presents a novel sparse coding based pan-sharp- 1) The Correlation Coefficient (CC): The CC shows the sim-
ening method for remote sensing images. Spatial resolution ilarity in small-size structures between the reference image band
1802 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 5, MAY 2014

and the fused image band [29], [31]. As a measure of correla- and fused images. ERGAS fulfills three requirements: the inde-
tion, it should be as close as possible to 1, and it is defined as: pendence of units, the independence of the number of spectral
bands, and the independence of the ratio of the scales [33]. The
(9) ideal value for ERGAS is 0, and it is defined as:

where and are the th bands of the reference image and (15)
the fused image, respectively. is the covariance between
and . and represent the standard deviation of
and , respectively. where and are the spatial resolutions of the PAN image and
2) The Structural Similarity Metric (SSIM): Based on the the MS image, respectively. For the IKONOS, QuickBird, and
fact that the human visual system has evolved to extract struc- WorldView-2 data, is four times as large as , and is the
tural information from images, the SSIM [32] was developed number of the bands. The RMSE is defined as:
to capture the loss of image structure. It has been widely used
and is considered to be better than the root mean square error
(RMSE) since RMSE only measures the radiometric distortion
of the fused image band from the reference image band. The
value of the SSIM should be as close as possible to 1, and the 5) The Q4 Index: Based on the theory of quaternions, the
SSIM metric between the th band of the reference and that of Q4 index [34] is an extension of the Q index [35], which is a
the fused image is defined as: product of CC, mean bias, and contrast variation. The Q4 index
is suitable for testing both the spectral and spatial qualities of
fused images [14]. The ideal value for the Q4 index is 1, and it
(10) is defined as:
where , , and are the luminance,
contrast, and structural comparison measures. , , and are (16)
parameters used to define the relative importance of the three
where is defined as:
components, given as follows:
(17)
(11)

where and are the quaternions, defined as:

(12) , and .
denotes the modulus of the quaternion . is the hypercom-
(13) plex covariance between and . and are the square
roots of the variances of and , respectively. and are
the expected values of and , respectively. The first term of
where and are the mean values of and , re-
(17) is the modulus of the hypercomplex CC between and ,
spectively. , , and are small constants given as:
and the second and third terms measure contrast changes and the
, , and , respectively. is
mean bias on all bands, respectively.
the dynamic range of the pixel values. In this paper, we set
The conjugates of and are defined as
, , , and ,
and , respectively.
as suggested in [32].
As quaternions, and have the following rules:
3) The Spectral Angle Mapper (SAM): The SAM measures
the absolute angle between the spectral vectors of the reference
(18)
and fused images, which reflects the spectral distortion intro-
duced by the fusion process. The ideal value is 0, and it is de- (19)
fined as:
(20)
(21)
(22)
(14)
By using (18)–(22), (17) can be rewritten as:

where and are the th pixels of the th bands of the

reference and fused images, respectively. and are the
horizontal and vertical sizes of the image. (23)
4) The Erreur Relative Globale Adimensionnelle De Syn-
thèse (ERGAS): ERGAS is widely used and it gives a global where is obtained by averaging the pixel quaternions
depiction of the radiometric difference between the reference within a block. We choose in this paper, and
JIANG et al.: TWO-STEP SPARSE CODING FOR THE PAN-SHARPENING OF REMOTE SENSING IMAGES 1803

TABLE VI
INFLUENCE OF ON THE IKONOS DATA

TABLE VII
INFLUENCE OF ON THE QUICKBIRD DATA

TABLE VIII
INFLUENCE OF ON THE WORDVIEW-2 DATA

extend the Q4 index to Q8 for the 8-band WorldView-2 data by TABLE IX

redefining and as: INFLUENCE OF STEP SIZE ON THE IKONOS DATA
WITH PATCH SIZE [7 7] AND

(24)

(25)

Note that and are still hypercomplex numbers with only

their number of dimensions changing from 4 to 8, and we can
assume that we can still use (16) and (23) to calculate Q8 with
dimensional extension when calculating , , , with other patch sizes and step sizes. In fact, we can simply set
, , and , e.g., (18) is replaced with (26): , because for different data, the results converge with
, which has a very similar performance and consumes
only a little more time than the value we chose in this paper.
(26) To test the influence of the step size, we fix , and test
different step sizes with patch size [7 7] on the data sets used
in this paper. The results are shown in Table IX, Table X, and
Table XI, respectively. It can be seen that a smaller step size
APPENDIX B achieves an obviously better result; however, the computation
There are three parameters for the proposed method, i.e., the time will be greatly increased. Similar results are also observed
patch size, the step size, and the regularization parameter . with the other patch sizes. We chose the step size as [3 3] in this
For simplicity, the ERGAS and Q4 indices are used to evaluate paper because it is able to show the advantages of the proposed
the performance here. How these three parameters affect the method over the other methods, and the computation time is tol-
performance and computation time of the proposed method is erable with the basic PC used in this study. With much better
discussed as follows. Similar results were also observed for SC computation power offered by a workstation or distributed com-
and TSSC, but have been omitted to save space. putation technique, the step size could be set as [1 1] to achieve
We first set the patch size and step size as [7 7] and [3 3], the best performance.
respectively. The influence of on the IKONOS, QuickBird, Similarly, to test the influence of the patch size, we fix
and WorldView-2 data is shown in Table VI, Table VII, and and set the step size as [3 3]. The different results obtained by
Table VIII, respectively. By setting from 16384 to 1/16384, the different patch sizes with the three data sets are displayed
more dictionary atoms are allowed to be used to represent a cer- in Table XII, Table XIII, and Table XIV, respectively. It can be
tain patch; thus, the performance is first improved, then con- seen that a bigger patch size needs more computation time. The
verges because the patch has already been well represented with best patch size for the IKONOS data is [11 11], and the best
enough dictionary atoms. For the computation time, it increases patch size for the WordView-2 data is [7 7]. For the QuickBird
continuously with a smaller . Similar results are also observed data, a patch size of [7 7] can achieve the best Q4 index, while a
1804 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 5, MAY 2014

TABLE X ACKNOWLEDGMENT
INFLUENCE OF STEP SIZE ON THE QUICKBIRD DATA
WITH PATCH SIZE [7 7] AND The authors would like to thank Julien Mairal et al. for
sharing the multi-thread implementation of the LARS algo-
rithm. Thanks are also due to Ron Rubinstein and Michael Elad
for sharing the fast MEX implementation of the patch opera-
tions. The authors would also like to thank DigitalGlobe for
providing the WorldView-2 images used in this study, and the
IEEE GRSS Data Fusion Technical Committee for organizing
the 2006 [36] and 2012 Data Fusion Contest.

TABLE XI REFERENCES
INFLUENCE OF STEP SIZE ON THE WORDVIEW-2 DATA
[1] T. Ranchin, B. Aiazzi, L. Alparone, S. Baronti, and L. Wald, “Image
WITH PATCH SIZE [7 7] AND
fusion—The ARSIS concept and some successful implementation
schemes,” ISPRS J. Photogramm. Remote Sens., vol. 58, no. 1, pp.
4–18, Jun. 2003.
[2] T.-M. Tu, S.-C. Su, H.-C. Shyu, and P. S. Huang, “A new look at IHS-
like image fusion methods,” Inf. Fusion, vol. 2, no. 3, pp. 177–186,
Sep. 2001.
[3] P. S. Chavez, S. C. Sides, and J. A. Anderson, “Comparison of three
different methods to merge multiresolution and multispectral data:
Landsat TM and SPOT panchromatic,” Photogramm. Eng. Remote
Sens., vol. 57, no. 3, pp. 295–303, 1991.
[4] C. A. Laben and B. V. Brower, “Process for Enhancing the Spatial Res-
olution of Multispectral Imagery Using Pan-Sharpening,” U.S. Patent
TABLE XII 6,011,875, Jan. 4, 2000, Tech. Rep., Eastman Kodak Company.
INFLUENCE OF THE PATCH SIZE ON THE IKONOS DATA [5] Y. Zhang, “Problems in the fusion of commercial high-resolution satel-
WITH STEP SIZE [3 3] AND lite images as well as Landsat-7 images and initial solutions,” in Proc.
Int. Archives Photogramm. Remote Sens. Spatial Inf. Sci., 2002, vol.
34, no. 4, pp. 587–592, Part 4.
[6] K. Amolins, Y. Zhang, and P. Dare, “Wavelet based image fusion
techniques—An introduction, review and comparison,” ISPRS J.
Photogramm. Remote Sens., vol. 62, no. 4, pp. 249–263, Sep. 2007.
[7] Y. Zhang, “Understanding image fusion,” Photogramm. Eng. Remote
Sens., vol. 70, no. 6, pp. 657–661, Jun. 2004.
[8] M. V. Joshi, L. Bruzzone, and S. Chaudhuri, “A model-based approach
to multiresolution fusion in remotely sensed images,” IEEE Trans.
Geosci. Remote Sens., vol. 44, no. 9, pp. 2549–2562, Sep. 2006.
TABLE XIII [9] Z. Li and H. Leung, “Fusion of multispectral and panchromatic images
INFLUENCE OF THE PATCH SIZE ON THE QUICKBIRD DATA using a restoration-based method,” IEEE Trans. Geosci. Remote Sens.,
WITH STEP SIZE [3 3] AND vol. 47, no. 5, pp. 1482–1491, May 2009.
[10] L. Zhang, H. Shen, W. Gong, and H. Zhang, “Adjustable model-based
fusion method for multispectral and panchromatic images,” IEEE
Trans. Syst., Man, Cybern., Syst., vol. 42, no. 6, pp. 1693–1704, Dec.
2012.
[11] Y. Zhang and G. Hong, “An IHS and wavelet integrated approach to
improve pan-sharpening visual quality of natural colour IKONOS and
QuickBird images,” Inf. Fusion, vol. 6, no. 3, pp. 225–234, Sep. 2005.
[12] V. P. Shah, N. H. Younan, and R. L. King, “An efficient pan-sharpening
method via a combined adaptive PCA approach and contourlets,” IEEE
Trans. Geosci. Remote Sens., vol. 46, no. 5, pp. 1323–1335, May 2008.
TABLE XIV
[13] B. Luo, M. M. Khan, T. Bienvenu, J. Chanussot, and L. Zhang, “Deci-
INFLUENCE OF THE PATCH SIZE ON THE WORLDVIEW-2 DATA
sion-based fusion for pansharpening of remote sensing images,” IEEE
WITH STEP SIZE [3 3] AND
Geosci. Remote Sens. Lett., vol. 10, no. 1, pp. 19–23, Jan. 2013.
[14] S. Li and B. Yang, “A new pan-sharpening method using a compressed
sensing technique,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 2,
pp. 738–746, Feb. 2011.
[15] C. Jiang, H. Zhang, H. Shen, and L. Zhang, “A practical compressed
sensing-based pan-sharpening method,” IEEE Geosci. Remote Sens.
Lett., vol. 9, no. 4, pp. 629–633, Jul. 2012.
[16] J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution
via sparse representation,” IEEE Trans. Image Process., vol. 19, no.
11, pp. 2861–2873, Nov. 2010.
[17] X. Zhu and R. Bamler, “A sparse image fusion algorithm with appli-
patch size of [13 13] leads to the best ERGAS index. Therefore, cation to pan-sharpening,” IEEE Trans. Geosci. Remote Sens., vol. 51,
unlike the step size, the best patch size varies with different data. no. 5, pp. 2827–2836, May 2013.
[18] B. A. Olshausen and D. J. Field, “Emergence of simple-cell receptive
Moreover, in our experiments, we also found that the best patch field properties by learning a sparse code for natural images,” Nature,
size varied from [7 7] to [11 11] with different data acquired by vol. 381, no. 6583, pp. 607–609, 1996.
the same sensor. Since the computation time for [7 7] is much [19] E. J. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles:
Exact signal reconstruction from highly incomplete frequency infor-
less than for [9 9] or [11 11], we chose [7 7] for all the data in mation,” IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489–509, Feb.
this paper. 2006.
JIANG et al.: TWO-STEP SPARSE CODING FOR THE PAN-SHARPENING OF REMOTE SENSING IMAGES 1805

[20] D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, vol. Hongyan Zhang (M’13) received the B.S. degree
52, no. 4, pp. 1289–1306, Apr. 2006. in geographic information system and the Ph.D.
[21] M. Elad, M. A. T. Figueiredo, and Y. Ma, “On the role of sparse and degree in photogrammetry and remote sensing from
redundant representations in image processing,” Proc. IEEE, vol. 98, Wuhan University, Wuhan, China, in 2005 and 2010,
no. 6, pp. 972–982, Jun. 2010. respectively.
[22] M. Elad and M. Aharon, “Image denoising via sparse and redun- Since 2010, he has been a lecturer with the State
dant representations over learned dictionaries,” IEEE Trans. Image Key Laboratory of Information Engineering in Sur-
Process., vol. 15, no. 12, pp. 3736–3745, Dec. 2006. veying, Mapping, and Remote Sensing, Wuhan Uni-
[23] J. A. Tropp and S. J. Wright, “Computational methods for sparse versity. His current research interests focus on image
solution of linear inverse problems,” Proc. IEEE, vol. 98, no. 6, pp. reconstruction and remote sensing image processing.
948–958, Jun. 2010.
[24] B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle re-
gression,” Ann. Statist., vol. 32, no. 2, pp. 407–499, 2004.
[25] J. Mairal, F. Bach, J. Ponce, G. Sapiro, R. Jenatton, and G.
Obozinski, Sparse Modeling Software [Online]. Available: Huanfeng Shen (M’11) received the B.S. degree in
http://spams-devel.gforge.inria.fr/index.html Feb. 16, 2013 surveying and mapping engineering and the Ph.D.
[26] J. Mairal, Julien Mairal’s Homepage [Online]. Available: http://lear. degree in photogrammetry and remote sensing from
inrialpes.fr/people/mairal/ Feb. 16, 2013 Wuhan University, Wuhan, China, in 2002 and 2007,
[27] R. Tibshirani, “Regression shrinkage and selection via the lasso,” J. respectively.
Roy. Stat. Soc. B, vol. 58, no. 1, pp. 267–288, 1996. In July 2007, he joined the School of Resource and
[28] J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, “Non-local Environmental Science, Wuhan University, where he
sparse models for image restoration,” in Proc. IEEE Int. Conf. Com- is currently a full professor. His research interests in-
puter Vision., Tokyo, Japan, 2009, pp. 2272–2279. clude image processing (for quality improvement),
[29] L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of remote sensing application, data fusion and assimi-
different spatial resolutions: Assessing the quality of resulting images,” lation. Dr. Shen has published more than 60 research
Photogramm. Eng. Remote Sens., vol. 63, no. 6, pp. 691–699, 1997. papers. He has been supported by several talent programs, including the New
[30] X. Otazu, M. Gonzalez-Audicana, O. Fors, and J. Nunez, “Introduction Century Excellent Talents by the Ministry of Education of China (2011) and the
of sensor spectral response into image fusion methods. Application to Hubei Science Fund for Distinguished Young Scholars (2011).
wavelet-based methods,” IEEE Trans. Geosci. Remote Sens., vol. 43,
no. 10, pp. 2376–2385, Oct. 2005.
[31] C. Thomas and L. Wald, “Comparing distances for quality assessment
of fused products,” in Proc. 26th EARSeL Annu. Symp. New Develop. Liangpei Zhang (M’06–SM’08) received the B.S.
Chal-lenges Remote Sens., Z. Bochenek, Ed., Warsaw, Poland, May degree in physics from Hunan Normal University,
29–31, 2006, pp. 101–111, Rotterdam, The Netherlands: Balkema, ChangSha, China, in 1982, the M.S. degree in optics
2007. from the Xi’an Institute of Optics and Precision
[32] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Mechanics of Chinese Academy of Sciences, Xi’an,
quality assessment: From error measurement to structural similarity,” China, in 1988, and the Ph.D. degree in photogram-
IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004. metry and remote sensing from Wuhan University,
[33] L. Wald, “Quality of high resolution synthesized images: Is there a Wuhan, China, in 1998.
simple criterion?,” in Proc. Int. Conf. Fusion of Earth Data, Nice, He is currently with the State Key Laboratory of
France, Jan. 2000, vol. 1, pp. 99–105. Information Engineering in Surveying, Mapping and
[34] L. Alparone, S. Baronti, A. Garzelli, and F. Nencini, “A global quality Remote Sensing, Wuhan University, as the head of
measurement of pan-sharpened multispectral imagery,” IEEE Geosci. the Remote Sensing Division. He is also a “Chang-Jiang Scholar” Chair Pro-
Remote Sens. Lett., vol. 1, no. 4, pp. 313–317, Oct. 2004. fessor appointed by the Ministry of Education, China. He has more than 200
[35] Z. Wang and A. C. Bovik, “A universal image quality index,” IEEE research papers and 5 patents. He is now Principal Scientist for the China State
Signal Processing Lett., vol. 9, pp. 81–84, Mar. 2002. Key Basic Research Project (2011–2016) appointed by the Ministry of National
[36] L. Alparone, L. Wald, J. Chanussot, C. Thomas, P. Gamba, and L. Science and Technology of China to lead the remote sensing program in China.
Bruce, “Comparison of pansharpening algorithms: Outcome of the His research interests include hyperspectral remote sensing, high resolution re-
2006 GRSS data-fusion contest,” IEEE Trans. Geosci. Remote Sens., mote sensing, image processing and artificial intelligence.
vol. 45, no. 10, pp. 3012–3021, Oct. 2007. Dr. Zhang regularly serves as a Co-Chair of the series SPIE Conferences on
Multispectral Image Processing and Pattern Recognition (MIPPR), Conference
Cheng Jiang received the B.S. degree in remote on Asia Remote Sensing, and many other conferences. He edits several confer-
sensing science and technology from Wuhan Univer- ence proceedings, issues, and the Geoinformatics Symposiums. He also serves
sity, Wuhan, China, in 2010. He is currently pursuing as an Associate Editor of International Journal of Ambient Computing and In-
the Ph.D. degree at the State Key Laboratory of telligence (IJACI), International Journal of Image and Graphics, International
Information Engineering in Surveying, Mapping and Journal of Digital Multimedia Broadcasting, Journal of Geo-spatial Informa-
Remote Sensing (LIESMARS), Wuhan University, tion Science, and the Journal of Remote Sensing. He is a Fellow of IEE, Execu-
Wuhan, China. tive Member (Board of Governor) of the China National Committee of Interna-
His research interests include image fusion, tional Geosphere-Biosphere Programme and Executive Member for the China
denoising and super-resolution in remote sensing Society of Image and Graphics.
images.