HDF Master Data
HDF Master Data
Multispectral Analysis of
MASTER HDF Data
Table of Contents
OVERVIEW OF THIS TUTORIAL .....................................................................................................................................2
Background ......................................................................................................................................................2
SHORTWAVE INFRARED (SWIR) ANALYSIS ......................................................................................................................3
Open HDF File and Select a Dataset to Plot .........................................................................................................3
Open HDF File and Load Image ..........................................................................................................................3
Open HDF File with ENVI's MASTER Reader.........................................................................................................3
Spatially/Spectrally Subset Data .........................................................................................................................4
Empirical Reflectance Calibration ........................................................................................................................5
Display a Color Composite and Extract Spectra ....................................................................................................6
Compare Image Spectra to Spectral Library.........................................................................................................6
Image Processing with SAM ...............................................................................................................................8
LONGWAVE INFRARED (LWIR) ANALYSIS ..................................................................................................................... 10
View LWIR Color Composite and Subset Data .................................................................................................... 10
Animate LWIR Bands....................................................................................................................................... 10
Review LWIR Spectral Features ........................................................................................................................ 11
Design and Display Color Composites................................................................................................................ 11
Perform Decorrelation Stretch .......................................................................................................................... 12
Compare LWIR and SWIR Results..................................................................................................................... 13
Optional: Perform Combined SWIR/LWIR Analysis.............................................................................................. 13
Tutorial: Multispectral Analysis of MASTER HDF Data
Overview of This Tutorial
This tutorial introduces you to the Hierarchical Data Format (HDF) and analysis of multispectral imagery from the
MODIS/ASTER Airborne Simulator (MASTER) sensor. The first part of the tutorial shows you how to open and read HDF
files, extract spatial and spectral subsets of MASTER shortwave infrared (SWIR) data, compare spectra to spectral
libraries, and how to perform classic multispectral processing.
The second half of the tutorial explains the analysis of longwave infrared (LWIR) MASTER data, including how to examine
LWIR spectra to define key spectral bands, how to perform decorrelation stretching of color composites to enhance LWIR
spectral differences, and how to compare SWIR and LWIR mapping results
These and other MASTER data are available for purchase from NASA through the EROS Data Center, Sioux Falls, SD.
Background
MASTER is designed to collect simulation data in support of the ASTER and MODIS instrument teams for algorithm
development, calibration, and validation. See http://masterweb.jpl.nasa.gov/ for more information. MASTER is a 50-band
scanner, operated by NASA, that covers approximately 0.4 to 14 μm. The instrument operates on the NASA Beechcraft
B200, DC-8, or ER-2 aircraft to produce spatial resolutions of 5 to 50 m. Spectral band positions are designed to simulate
both ASTER and MODIS. MASTER data are delivered via EROS Data Center as HDF files.
HDF is a multi-object file format for the transfer of graphical and numerical data between machines. This format allows
you to create, access, and share scientific data in a form that is self-describing and network-transparent. “Self-describing”
means that a file includes information defining the data it contains. “Network-transparent” means that a file is
represented in a form that can be accessed by computers with different ways of storing integers, characters, and floating-
point numbers. HDF data supported in ENVI include raster format images, images stored in 2D or 3D scientific data
format, and plots stored in 1D scientific data format.
2
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
1. From the ENVI main menu bar, select File → Open External File → Generic Formats → HDF. A file selection
dialog appears.
2. Navigate to envidata\cup99mas and select 990051f.hdf. Click Open. An HDF Dataset Selection dialog
appears with a list of the available HDF datasets in the file.
3. Select the HDF dataset labeled (50x1):Left50%ResponseWavelength and click OK to plot the data in an
HDF 1-D Data Sets window. This dataset contains the center wavelengths for the MASTER HDF data.
4. Select Edit → Data Parameters from the HDF 1-D Data Sets plot window menu bar. A Data Parameters dialog
appears.
5. Select a symbol from the Symbol drop-down list. Enlarge the plot two or three times its initial size by dragging
one of the corners so that you can see the markers for the individual band centers.
6. Click and drag inside the plot window to read the wavelength values. The band center values appear at the
bottom of the plot as you move the cursor. The right number represents the band center of the selected band in
micrometers. The range of the MASTER data is approximately 0.4 to 14 μm.
2. In the HDF Dataset Selection dialog, select (716x50x2028):CalibratedData and click OK. An HDF Data Set
Storage Order dialog appears.
3. Select the BIL radio button and click OK to load the image data into the Available Band List dialog. This dataset
contains the image data for the Cuprite, Nevada, MASTER HDF data.
4. In the Available Bands List, select (716x50x2028):CalibratedData: Band 4 and click Load Band. ENVI
automatically reads the HDF data.
5. Examine the image and select File → Close All Files from the ENVI main menu bar when you are finished.
• Or, select File → Open Image File and double-click on 990051f.hdf. ENVI automatically recognizes
the file format. The 50 MASTER bands and their associated wavelengths appear in the Available Bands
List dialog.
3
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
2. Select the RGB Color radio button in the Available Bands List dialog, then select Band 5, Band 3, and Band 1.
Click Load RGB to display a true-color image.
2. Perform simultaneous spatial and spectral subsetting. From the ENVI main menu bar, select Basic Tools →
Resize Data (Spatial/Spectral). A Resize Data Input File dialog appears. Select 9900511f.hdr, but do not
click OK yet.
5. The red box shows the area currently viewed in the Image window. To subset this area, click OK, followed by OK
in the Select Spatial Subset dialog.
6. In the Resize Data Input File dialog, click Spectral Subset. A File Spectral Subset dialog appears.
7. Click Clear. Select Band 1, hold down the <Shift> key, then select Band 25. Click OK, followed by OK in the
Resize Data Input File dialog. A Resize Data Parameters dialog appears.
8. In the Enter Output Filename field, enter resized_image and click OK.
4
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
Empirical Reflectance Calibration
1. In the Available Bands List, select the RGB Color radio button. Select Band 5, Band 3, and Band 1 under
resized_image, and click Load RGB.
2. From the Display group menu bar, select Tools → Profiles → Z Profile (Spectrum). A Spectral Profile plot
window appears, showing a spectrum of MASTER bands 1-25 (0.46 - 2.396 μm). Note the shape of the spectrum,
corresponding to the shape of the solar irradiance spectrum modified by atmospheric absorption. Select File →
Cancel from the Spectral Profile menu bar.
Perform a rough atmospheric correction using the empirical Flat Field method, by choosing a spectrally flat region
of interest (ROI) and dividing the average spectrum for that ROI into the spectrum for each pixel in the image.
3. From the Display Group menu bar, select Overlay → Region of Interest. An ROI Tool dialog appears.
4. Draw a polygon ROI inside of Stonewall Playa, the white area near the bottom-center of the image. Click the left
mouse button to define polygon segments, then right-click to close the polygon. Right-click again to accept the
polygon ROI.
5. In the ROI Tool dialog, select the Off radio button to turn off ROI mouse control.
6. From the ENVI main menu bar, select Basic Tools → Preprocessing → Calibration Utilities → Flat Field. A
Calibration Input File dialog appears.
7. Select resized_image and click OK. A Flat Field Calibration Parameters dialog appears.
8. Select Region #1… In the Enter Output Filename field, enter calibration. Click OK to start the Flat Field
correction.
5
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
Display a Color Composite and Extract Spectra
1. In the Available Bands List, select the RGB Color radio button. Select Band 5, Band 3, and Band 1 under
calibration. Click Load RGB.
2. From the Display group menu bar, select Tools → Profiles → Z Profile (Spectrum). A Spectral Profile plot
window appears, showing a spectrum of MASTER bands 1-25 (0.46 - 2.396 μm). Note the corrected spectrum,
corresponding to apparent reflectance or relative reflectance (relative to the Flat Field spectrum).
3. Drag the cursor throughout the Image window and examine the spectra. Move the cursor to some of the red
areas in the image, which correspond to Fe+3 absorption features near 0.87 μm (Band 9), and examine their
spectra.
4. Click-and-hold the middle mouse button in the Spectral Profile window, and draw a box around the 2.0 - 2.4 μm
range. Drag the cursor throughout the Image window, and observe the absorption features near 2.2 and 2.3 μm
caused by clays and carbonates, respectively.
2. Click Open and select Spectral Library. A file selection dialog appears. Navigate to spec_lib\usgs_min and
select usgs_min.sli. Click Open.
3. Select usgs_min.sli in the Spectral Library Input File and click OK. A Spectral Library Viewer dialog appears.
4. Select alunite1.spc Alunite GDS84 Na03. A Spectral Library Plots window appears, showing the spectrum for
alunite. Then, select the following spectra:
5. Right-click in the Spectral Library Plots window and select Stack Plots.
6
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
6. Select MASTER spectra for the same minerals. From the Display group menu bar, select Tools → Pixel Locator.
7. In the Pixel Locator dialog, enter the pixel location (521, 1587), which is an alunite feature. Click Apply.
9. Enter the following pixel locations in the Pixel Locator dialog, and click Apply each time:
10. Right-click in the Spectral Profile window and select Stack Plots. Right-click again and select Plot Key.
11. In the Spectral Profile and Spectral Library Plots windows, zoom to a range of 2.0 to 2.4 μm, using the middle
mouse button to draw a box around the range.
12. Compare the image spectra to the library spectra. Again, note the absorption features near 2.2 and 2.3 μm.
13. For a more direct comparison, select Options → New Window: Blank from the Spectral Profile menu bar.
Click-and-drag a spectrum name from the Spectral Profile window into the ENVI Plot Window. Then, click-and-
drag the corresponding spectrum name from the Spectral Library Plots window into the ENVI Plot Window.
Following is an example for kaolinite:
7
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
Image Processing with SAM
Use spectra collected from the image to perform a Spectral Angle Mapper (SAM) classification of the MASTER data. SAM
measures the similarity of unknown and reference spectra in n-dimensions. The spectral angle is the angle between the
spectra (represented as vectors) in n-D space. This method assumes that the data have been reduced to apparent
reflectance and uses only the direction of the spectra, not their length. Thus, the SAM classification is insensitive to
illumination effects. See the Selected Hyperspectral Mapping Methods tutorial and ENVI Help for more information and
examples.
1. From the ENVI main menu bar, select Classification → Supervised → Spectral Angle Mapper. A
Classification Input File dialog appears.
3. Click-and-drag the spectra names from the Spectral Profiles window to the Endmember Collection:SAM dialog.
4. Click Select All and Plot to confirm that you have the correct endmember spectra.
6. In the Enter Output Filename field, enter sam_class_out. This is the SAM classification image that will be
created.
7. In the Enter Output Rule Filename field, enter sam_rule_out. Click OK. This is the set of SAM rule images
that will be created.
8. In the Available Bands List, click Display #1 and select New Display.
9. Select Sam under sam_class_out. Select the Gray Scale radio button, and click Load Band. The classes are
color-coded the same as the spectrum plot colors:
Mineral Color
Kaolinite Blue
Calcite Green
Alunite White
Buddingtonite Red
10. Compare the classification map to the true-color image. From a Display group menu bar, select Tools → Link →
Link Displays. Click in an Image window to toggle between the true-color image and the classification map.
11. From a Display group menu bar, select Tools → Profiles → Z Profile (Spectrum). Verify the spectral match.
12. You can also use the rule images to evaluate the spectral matches. The rule images show the best matches
(small angles) in black when first displayed. Since it is more intuitive to show the best matches as brighter values,
you can invert the colors in the rule images. Steps 13-15 explain this process.
13. In the Available Bands List, click Display #2 and select Display #1. Select a rule band under sam_rule_out
and click Load Band.
14. From the Display group menu bar, select Tools → Color Mapping → ENVI Color Tables.
15. Move the Stretch Bottom slider all the way to the right, and move the Stretch Top slider all the way to the
left.
8
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
16. From a Display group menu bar, select Tools → Link → Link Displays. A Link Displays dialog appears. Click
OK to link Display #1 (rule image) and Display #2 (classification image).
17. Click in an Image window to toggle between the rule image and classification image. Examine the spatial
locations of specific mapped minerals.
18. When you are finished, select Window → Close All Display Windows from the ENVI main menu bar.
9
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
2. In the Available Bands List, click Display #1 and select New Display.
3. In the Available Bands List, select any band under resized_image (the subset you created earlier), select the
Gray Scale radio button, and click Load Band.
4. From the ENVI main menu bar, select Basic Tools → Resize Data (Spatial/Spectral). A Resize Data Input
File dialog appears. Select 9900511f.hdf, but do not click OK yet.
5. Click Spatial Subset. A Select Spatial Subset dialog appears. You will use the same spatial subset you created
earlier, but with a different spectral subset.
7. Select resized_image and click OK, followed by OK in the Select Spatial Subset dialog.
8. In the Resize Data Input File dialog, click Spectral Subset. A File Spectral Subset dialog appears.
9. Click Clear. Select Band 41, hold down the <Shift> key, then select Band 50. Click OK, followed by OK in the
Resize Data Input File dialog. A Resize Data Parameters dialog appears.
10. In the Enter Output Filename field, enter resized_image_lwir and click OK.
2. Animate all of the subsetted LWIR bands and examine their variability. From the Display group menu bar, select
Tools → Animation. The Animation Image Parameters dialog appears. Click OK to accept the default values.
Ten LWIR bands are loaded and animated.
Most of the variability is caused by differential heating, not by spectral differences between bands. Ideally, these
data should be atmospherically corrected and converted to emissivity to enhance spectral differences. However,
for this tutorial, you will substitute selection of key image bands using LWIR spectra, and enhance them using a
Decorrelation Stretch.
4. If desired, display individual bands of interest in new display groups, and use image linking and dynamic overlays
to compare them.
10
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
Review LWIR Spectral Features
ENVI includes spectral libraries from Johns Hopkins University, which contain spectra for selected materials from 0.4 to 14
μm. The apparently seamless reflectance spectra over this region of rocks and soils were generated using two different
instruments, both equipped with integrating spheres for measuring directional hemispherical reflectance.
Under most conditions, the infrared portion of these data can be used to calculate emissivity using Kirchhoff’s Law. For
this tutorial, however, highs in the reflectance spectra from the spectral library can be considered equivalent to lows
expected in emissivity spectra.
1. From the ENVI main menu bar, select Spectral → Spectral Libraries → Spectral Library Viewer. A Spectral
Library Input File dialog appears.
2. Click Open and select Spectral Library. A file selection dialog appears. Navigate to spec_lib\jhu_lib and
select minerals.sli. Click Open.
3. Select minerals.sli in the Spectral Library Input File and click OK. A Spectral Library Viewer dialog appears.
4. Select the first Quartz option. A Spectral Library Plots window appears, showing the spectrum for quartz. Then,
select the second Calcite option.
5. From the Spectral Library Plots window menu bar, select Edit → Plot Parameters.
6. Enter Range values from 8 to 13 and click Apply. Quartz (silica) shows a maximum near 9 μm, while Calcite
does not. You can use this information to help find all of the silica-rich areas imaged by the MASTER data.
2. Bands 46, 44, and 41 (10.085, 9.054, and 7.793 μm, respectively) bracket the prominent silica feature near 9.0
μm. In the Available Bands List, select the RGB Color radio button, select Band 46, Band 44, and Band 41
under resized_image_lwir, and click Load RGB.
11
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
3. Relate the image colors to the expected relative contributions, based on the library spectra. The colors do not
match the expected colors very well because the effect of temperature overwhelms the spectral differences. The
bands are highly correlated because of differential heating of the rocks and soils.
1. Decorrelation stretching requires three bands (a stretched color composite) for input. You will use the currently
displayed color composite.
2. From the ENVI main menu bar, select Transform → Decorrelation Stretch. A Decorrelation Stretch Input File
dialog appears. Select Display #1 and click OK. A Decorrelation Stretch Parameters dialog appears.
3. In the Enter Output Filename field, enter lwir_stretched and click OK.
4. In the Available Bands List, select the RGB Color radio button. Select R DS, G DS, and B DS, and click Load
RGB. Try other color combinations.
12
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data
Tutorial: Multispectral Analysis of MASTER HDF Data
1. In the Available Bands List, click Display #1 and select New Display.
2. Select Sam under sam_class_out. Select the Gray Scale radio button, and click Load Band. As a reminder,
the classes are color-coded as follows:
Mineral Color
Kaolinite Blue
Calcite Green
Alunite White
Buddingtonite Red
3. From a Display group menu bar, select Tools → Link → Link Displays and click OK to link the two images.
4. Click in an Image window to toggle between the two images. Compare the distribution of minerals mapped from
the SWIR data versus the distribution of silica (red areas) on the decorrelated image.
1. From the file 990051f.hdf, extract the SWIR and LWIR bands using spectral and spatial subsetting to build a
combined data cube.
2. Create a Minimum Noise Transform (MNF) output file, and perform MNF analysis by looking at the spectral bands
and the eigenvalue plots. Select a reduced number of MNF bands for further analysis.
3. Run a Fast Pixel Purity Index (PPI) analysis with 10,000 iterations to find the key endmember spectra, thus
reducing the spectral dimensionality. Threshold the PPI to a ROI with around 5,000 pixels.
4. Use the n-D Visualizer to select endmembers. Rotate the scatter plot in many dimensions, and select extreme
pixels by drawing ROIs in the n-D Visualizer and exporting them to image ROIs.
5. Use an ENVI spectral mapping method with the combined datasets. Compare these results to the SWIR and LWIR
results above.
13
ENVI Tutorial: Multispectral Analysis of MASTER HDF Data