Geospatial Data Accuracy and the New

Mapping Accuracy Standard: New Era

Session #35

Dr. Qassim Abdullah, Woolpert, Inc.

Pierre Le Roux, Aerometric, Inc.
Becky Morton, Towill, Inc.

1
New ASPRS Positional Accuracy Standards for
Digital Geospatial Data

Drafting Committee:
Chair: Douglas L. Smith, David C. Smith & Associates, Inc.
Dr. Qassim A. Abdullah, Woolpert, Inc.
Dr. David Maune, Dewberry
Karl Hans Heidemann, USGS
REVISION 7, VERSION 1 NOVEMBER 14, 2014
(FINAL BOARD APPROVAED Version)

2
New ASPRS Positional Accuracy Standards for
Digital Geospatial Data
 Replaces:
• ASPRS Accuracy Standards for Large-Scale Maps (1990)
• ASPRS Guidelines, Vertical Accuracy Reporting for Lidar Data
(2004)
 Developed by:
ASPRS Map Accuracy Standards Working Group, PAD, PDAD and
LIDAR joint committee for map accuracy standard update
 In Final Approved Version
• REVISION 7, VERSION 1, Nov. 14, 2014
• Approved and adopted by ASPRS during the board meeting on
Monday Nov. 17, 2014 in Denver during ASPRS 2014 PECORA
conference

3
 New Standard for a New Era
Motivation Behind the New Standard:
• Legacy map accuracy standards, such as the ASPRS 1990 standard and the
NMAS of 1947, are outdated. (over 30 years since ASPRS1990 was
written)

• Many of the data acquisition and mapping technologies that these

standards were based on are no longer used.

• More recent advances in mapping technologies can now produce better

quality and higher accuracy geospatial products and maps.

• Legacy map accuracy standards were designed to deal with plotted or

drawn maps as the only medium to represent geospatial data.

4
 New Standard for a New Era
• Within the past two decades (during the transition period between the
hardcopy and softcopy mapping environments), most standard
measures for relating GSD and map scale to the final mapping accuracy
were inherited from photogrammetric practices using scanned film.

• New mapping processes and methodologies have become much more

sophisticated with advances in technology and advances in our
knowledge of mapping processes and mathematical modeling.

• Mapping accuracy can no longer be associated with the camera

geometry and flying altitude alone (focal length, xp, yp, B/H ratio,
etc.).

5
 New Standard for a New Era
• New map accuracy is influenced by many factors such as:
– the quality of camera calibration parameters;
– quality and size of a Charged Coupled Device (CCD) used in the
digital camera CCD array;
– amount of imagery overlap;
– quality of parallax determination or photo measurements;
– quality of the GPS signal;
– quality and density of ground controls;
– quality of the aerial triangulation solution;
– capability of the processing software to handle GPS drift and shift;
– capability of the processing software to handle camera self-calibration,
– the digital terrain model used for the production of orthoimagery.

• .
6
 New Standard for a New Era
• These factors can vary widely from project to project, depending on the
sensor used and specific methodology. For these reasons, existing
accuracy measures based on map scale, film scale, GSD, c-factor and
scanning resolution no longer apply to current geospatial mapping
practices.

• Elevation products from the new technologies and active sensors such
as lidar and IFSAR are not considered by the legacy mapping
standards. New accuracy standards are needed to address elevation
products derived from these technologies.

7
 ASPRS Positional Accuracy Standards for
Digital Geospatial Data
– Applicability:
• Defines specific accuracy classes and associated RMSE thresholds for
digital orthoimagery, digital planimetric data, and digital elevation data
• Intended to be technology independent
• Limited to accuracy thresholds and testing methodologies for any
mapping applications, and to meet immediate shortcomings in the
outdated 1990 and 2004 standards
• Is not intended to cover classification accuracy of thematic maps
• Does not specify the best practices or methodologies needed to meet the
accuracy thresholds
– Includes:
• Glossary, Symbols, examples, conversion to legacy standards

8
 New Standard Highlights
– Positional Accuracy Thresholds which are independent of published
GSD, map scale or contour interval
• digital orthoimagery
• digital elevation data
– Additional Accuracy Measures
• aerial triangulation accuracy,
• Ground controls accuracy,
• orthoimagery seam lines accuracy,
• lidar relative swath-to-swath accuracy,
• recommended minimum Nominal Pulse Density (NPD)
• horizontal accuracy of elevation data,
• delineation of low confidence areas for vertical data
• required number and spatial distribution of QA/QC check points based
on project area
9
 New Standard Highlights
– Positional Accuracy Thresholds which are independent of published
GSD, map scale or contour interval
• digital orthoimagery
• digital elevation data
– Additional Accuracy Measures
• aerial triangulation accuracy,
• Ground controls accuracy,
• orthoimagery seam lines accuracy,
• lidar relative swath-to-swath accuracy,
• recommended minimum Nominal Pulse Density (NPD)
• horizontal accuracy of elevation data,
• delineation of low confidence areas for vertical data
• required number and spatial distribution of QA/QC check points based
on project area
10
 New Standard Highlights
– It is All Metric!
– Unlimited Horizontal Accuracy Classes:

Horizontal Accuracy Standards for Geospatial Data

Horizontal
Horizontal Orthoimagery
RMSEx and Accuracy at 95%
Accuracy Mosaic Seamline
RMSEr (cm) Confidence Level
Class RMSEy (cm) Mismatch
(cm)
  (cm)
 
X-cm ≤X ≤1.41*X ≤2.45*X ≤ 2*X

11
 Horizontal Accuracy Orthoimage Mosaic Horizontal Accuracy at
Class RMSEx and RMSEr
Seamline Maximum the 95% Confidence Level
(cm) Mismatch (cm) (cm)
RMSEy (cm)

0.63 0.9 1.3 1.5

1.25 1.8 2.5 3.1
2.50 3.5 5.0 6.1
5.00 7.1 10.0 12.2
7.50 10.6 15.0 18.4

Common Horizontal 10.00 14.1 20.0 24.5

Accuracy Classes 12.50 17.7 25.0 30.6

15.00 21.2 30.0 36.7
according to the new
17.50 24.7 35.0 42.8
standard[1]
20.00 28.3 40.0 49.0
22.50 31.8 45.0 55.1
25.00 35.4 50.0 61.2
27.50 38.9 55.0 67.3
30.00 42.4 60.0 73.4
45.00 63.6 90.0 110.1
60.00 84.9 120.0 146.9
75.00 106.1 150.0 183.6
100.00 141.4 200.0 244.8
150.00 212.1 300.0 367.2
200.00 282.8 400.0 489.5
250.00 353.6 500.0 611.9
300.00 424.3 600.0 734.3
500.00 707.1 1000.0 1223.9
1000.00 1414.2 2000.0 2447.7
12
 Associated Horizontal
Accuracy According to
Legacy ASPRS 1990
Standard
Common ASPRS 1990
Associated
Orthoimager Accuracy
Map Scale
y Pixel Sizes Class
RMSEx and
RMSEx and RMSEy in
RMSEy (cm) terms of
pixels

Examples on Horizontal 1 1.3 2-pixels

Accuracy for Digital 0.625 cm 1:50 2 2.5 4-pixels

3 3.8 6-pixels
Orthoimagery interpreted from 1 2.5 2-pixels
ASPRS 1990 Legacy Standard. 1.25 cm 1:100 2 5.0 4-pixels
3 7.5 6-pixels
1 5.0 2-pixels
2.5 cm 1:200 2 10.0 4-pixels
3 15.0 6-pixels
1 10.0 2-pixels
5 cm 1:400 2 20.0 4-pixels
3 30.0 6-pixels
1 15.0 2-pixels
7.5 cm 1:600 2 30.0 4-pixels
3 45.0 6-pixels
1 30.0 2-pixels
15 cm 1:1,200 2 60.0 4-pixels
13
3 90.0 6-pixels
 Recommended
Horizontal
Common Accuracy Class
RMSEx and Orthoimage RMSEx and
Orthoimage
RMSEy in terms of Recommended use
ry Pixel RMSEy (cm)
Sizes pixels

≤1.3 ≤1-pixel Highest accuracy work

Standard Mapping and
2.5 2-pixels
1.25 cm GIS work
Visualization and less
≥3.8 ≥3-pixels
accurate work
≤2.5 ≤1-pixel Highest accuracy work
Digital Orthoimagery Standard Mapping and
5.0 2-pixels
Accuracy Examples for 2.5 cm GIS work

Current Large and Medium ≥7.5 ≥3-pixels

Visualization and less
accurate work
Format Metric Cameras ≤5.0 ≤1-pixel Highest accuracy work
Standard Mapping and
10.0 2-pixels
5 cm GIS work
Visualization and less
≥15.0 ≥3-pixels
accurate work
≤7.5 ≤1-pixel Highest accuracy work
Standard Mapping and
15.0 2-pixels
7.5 cm GIS work
Visualization and less
≥22.5 ≥3-pixels
accurate work
≤15.0 ≤1-pixel Highest accuracy work
Standard Mapping and
30.0 2-pixels
15 cm GIS work

≥45.0 ≥3-pixels
14
Visualization and less
accurate work
 Horizontal Accuracy/Quality Examples for High Accuracy Digital Planimetric Data

Equivalent to map  
ASPRS 2014
scale in  
 
Horizontal Horizontal  
Accuracy Class ASPRS ASPRS
RMSEr Accuracy at the Approximate GSD of Equivalent to map
RMSEx and 1990 1990
(cm) 95% Confidence Source Imagery (cm) scale in NMAS
Class 1 Class 2
RMSEy (cm) Level (cm)

0.63 0.9 1.5 0.31 to 0.63 1:25 1:12.5 1:16

1.25 1.8 3.1 0.63 to 1.25 1:50 1:25 1:32
2.5 3.5 6.1 1.25 to 2.5 1:100 1:50 1:63
5.0 7.1 12.2 2.5 to 5.0 1:200 1:100 1:127
7.5 10.6 18.4 3.8 to 7.5 1:300 1:150 1:190
10.0 14.1 24.5 5.0 to 10.0 1:400 1:200 1:253
12.5 17.7 30.6 6.3 to12.5 1:500 1:250 1:317
15.0 21.2 36.7 7.5 to 15.0 1:600 1:300 1:380
17.5 24.7 42.8 8.8 to 17.5 1:700 1:350 1:444
20.0 28.3 49.0 10.0 to 20.0 1:800 1:400 1:507
22.5 31.8 55.1 11.3 to 22.5 1:900 1:450 1:570
25.0 35.4 61.2 12.5 to 25.0 1:1000 1:500 1:634
27.5 38.9 67.3 13.8 to 27.5 1:1100 1:550 1:697
30.0 42.4 73.4 15.0 to 30.0 1:1200 1:600 1:760

15
 New Standard Highlights
– Unlimited Vertical Accuracy Classes:

Vertical Accuracy Standards for Digital Elevation Data

Absolute Accuracy Relative Accuracy (where applicable)

  Swath-to- Swath-to-
Vertical Within- Swath
RMSEz NVA at 95% Swath Swath
Accuracy VVA at 95th Hard Surface
Non- Confidence Non-Vegetated Non-Vegetated
Class Percentile Repeatability
Vegetated Level Terrain Terrain
  (cm) (Max Diff)
(cm) (cm) (RMSDz) (Max Diff)
(cm)
(cm) (cm)

X-cm ≤X ≤1.96*X ≤3.00*X ≤0.60*X ≤0.80*X ≤1.60*X

16
 Vertical Accuracy/Quality Examples for Digital Elevation Data

Absolute Accuracy Relative Accuracy (where applicable)

 
Vertical Within-Swath
RMSEz NVA Swath-to-Swath Swath-to-Swath
Accuracy VVA Hard Surface
Non- at 95% Non-Veg Terrain Non-Veg Terrain
Class at 95th Percentile Repeatability
Vegetated Confidence Level (RMSDz) (Max Diff)
  (cm) (Max Diff)
(cm) (cm) (cm) (cm)
(cm)
1-cm 1.0 2.0 3 0.6 0.8 1.6

2.5-cm 2.5 4.9 7.5 1.5 2 4

5-cm 5.0 9.8 15 3 4 8

10-cm 10.0 19.6 30 6 8 16

15-cm 15.0 29.4 45 9 12 24

20-cm 20.0 39.2 60 12 16 32

33.3-cm 33.3 65.3 100 20 26.7 53.3

66.7-cm 66.7 130.7 200 40 53.3 106.7

100-cm 100.0 196.0 300 60 80 160

333.3-cm 333.3 653.3 1000 200 266.7 533.3

17
 Vertical accuracy of the new ASPRS 2014 standard
compared with legacy standards

 
RMSEz Equivalent Class 1 contour Equivalent Class 2 Equivalent contour
Vertical
Non-Vegetated interval per ASPRS 1990 contour interval per interval per NMAS
Accuracy Class
(cm) (cm) ASPRS 1990 (cm) (cm)
 

1-cm 1.0 3.0 1.5 3.29

2.5-cm 2.5 7.5 3.8 8.22

5-cm 5.0 15.0 7.5 16.45

10-cm 10.0 30.0 15.0 32.90

15-cm 15.0 45.0 22.5 49.35

20-cm 20.0 60.0 30.0 65.80

33.3-cm 33.3 99.9 50.0 109.55

66.7-cm 66.7 200.1 100.1 219.43

100-cm 100.0 300.0 150.0 328.98

333.3-cm 333.3 999.9 500.0 1096.49

18
Examples on Vertical Accuracy and Recommended Lidar Point
Density for Digital Elevation Data according to the new ASPRS
2014 standard
Absolute Accuracy
 
Recommended Recommended
Vertical Accuracy RMSEz NVA Minimum NPD Maximum NPS7
Class Non-Vegetated at 95% Confidence (pts/m2) (m)
  (cm) Level (cm)

1-cm 1.0 2.0 ≥20 ≤0.22

2.5-cm 2.5 4.9 16 0.25

5-cm 5.0 9.8 8 0.35

10-cm 10.0 19.6 2 0.71

15-cm 15.0 29.4 1 1.0

20-cm 20.0 39.2 0.5 1.4

33.3-cm 33.3 65.3 0.25 2.0

66.7-cm 66.7 130.7 0.1 3.2

100-cm 100.0 196.0 0.05 4.5

333.3-cm 333.3 653.3 0.01 10.0

19
 Horizontal accuracy requirements for
elevation data
•• Photogrammetric
  elevation data:, the horizontal accuracy
equates to the horizontal accuracy class that would apply to
planimetric data or digital orthoimagery produced from the
same source imagery, using the same aerial triangulation/INS
solution.
 
• Lidar elevation data: use the following formula:

20
Expected horizontal errors (RMSEr) for Lidar
data in terms of flying altitude

Altitude Positional RMSEr Altitude Positional RMSEr

 
(m) (cm) (m) (cm)

500 13.1   3,000 41.6

1,000 17.5   3,500 48.0

1,500 23.0   4,000 54.5

2,000 29.0   4,500 61.1

2,500 35.2   5,000 67.6

21
 Low Confidence Areas in Lidar Dataset
Recommended Recommended Search Radius and Low Confidence
Vertical
Project Low Confidence Cell Size Polygons
Accuracy
Min NPD (pts/m2) Min NGPD (pts/m2) for Computing NGPD Min Area
Class
(Max NPS (m)) (Max NGPS (m)) (m) (acres (m2))

1-cm ≥20 (≤0.22) ≥5 (≤0.45) 0.67 0.5 (2,000)

2.5-cm 16 (0.25) 4 (0.50) 0.75 1 (4,000)

5-cm 8 (0.35) 2 (0.71) 1.06 2 (8,000)

10-cm 2 (0.71) 0.5 (1.41) 2.12 5 (20,000)

15-cm 1 (1.0) 0.25 (2.0) 3.00 5 (20,000)

20-cm 0.5 (1.4) 0.125 (2.8) 4.24 5 (20,000)

33.3-cm 0.25 (2.0) 0.0625 (4.0) 6.0 10 (40,000)

66.7-cm 0.1 (3.2) 0.025 (6.3) 9.5 15 (60,000)

100-cm 0.05 (4.5) 0.0125 (8.9) 13.4 20 (80,000)

333.3-cm 0.01 (10.0) 0.0025 (20.0) 30.0 25 (100,000)

22
 Accuracy requirements for aerial triangulation and
INS-based sensor orientation of digital imagery
• Accuracy of aerial triangulation designed for digital planimetric data
(orthoimagery and/or digital planimetric map) only:

RMSEx(AT) or RMSEy(AT) = ½ * RMSEx(Map) or RMSEy(Map)

RMSEz(AT) = RMSEx(Map) or RMSEy(Map) of orthoimagery

• Accuracy of aerial triangulation designed for elevation data, or

planimetric data (orthoimagery and/or digital planimetric map) and
elevation data production:

• RMSEx(AT), RMSEy(AT)or RMSEz(AT) = ½ * RMSEx(Map), RMSEy(Map)or

RMSEz(DEM)

23
 Accuracy requirements for ground control
used for aerial triangulation
• Accuracy of  ground controls designed for planimetric data
(orthoimagery and/or digital planimetric map)production only:

RMSEx or RMSEy = ¼ * RMSEx(Map) or RMSEy(Map),

RMSEz = ½ * RMSEx(Map) or RMSEy(Map)

• Accuracy of ground controls designed for elevation data, or

planimetric data and elevation data production:
 
RMSEx, RMSEy or RMSEz= ¼ * RMSEx(Map), RMSEy(Map) or
RMSEz(DEM)
24
Examples on Aerial Traingulation and
Ground Control Accuracy
Aerial Triangulation and Ground Control Accuracy Requirements,
Orthoimagery and/or Planimetric Data Only
A/T Accuracy Ground Control Accuracy
Product Accuracy
(RMSEx, RMSEy, or
RMSEx and RMSEy RMSEz RMSEx and RMSEy RMSEz
RMSEz)
(cm) (cm) (cm) (cm)
(cm)
50 25 25 12.5 12.5

Aerial Triangulation and Ground Control Accuracy Requirements,

Orthoimagery and/or Planimetric Data and Elevation Data

A/T Accuracy Ground Control Accuracy

Product Accuracy
(RMSEx, RMSEy) RMSEx and RMSEy RMSEz RMSEx and RMSEy RMSEz
(cm) (cm) (cm) (cm) (cm)
50 25 50 12.5 25

25
 Reporting Horizontal Accuracy
• “This data set was tested to meet ASPRS Positional Accuracy Standards
for Digital Geospatial Data (2014) for a ___ (cm) RMSEx / RMSEy
Horizontal Accuracy Class. Actual positional accuracy was found to be
RMSEx = ___ (cm) and RMSEy = ___ cm which equates to +/- ___ at
95% confidence level.”
 
• “This data set was produced to meet ASPRS Positional Accuracy
Standards for Digital Geospatial Data (2014) for a ___ (cm) RMSEx /
RMSEy Horizontal Accuracy Class which equates to +/- ___ cm at a 95%
confidence level.”
 

26
 Reporting Vertical Accuracy
• “This data set was tested to meet ASPRS Positional Accuracy Standards
for Digital Geospatial Data (2014) for a___ (cm) RMSEz Vertical Accuracy
Class. Actual NVA accuracy was found to be RMSEz = ___ cm, equating
to +/- ___ at 95% confidence level. Actual VVA accuracy was found to be
+/- ___ cm at the 95% percentile.”
 
• “This data set was produced to meet ASPRS Positional Accuracy
Standards for Digital Geospatial Data (2014) for a ___ cm RMSEz Vertical
Accuracy Class equating to NVA =+/-___cm at 95% confidence level and
VVA =+/-___cm at the 95% percentile

27
Recommended Number of Check Points
Based on Area
Horizontal Accuracy
Testing of Orthoimagery Vertical and Horizontal Accuracy Testing of Elevation
and Planimetrics Data sets
Project Area (Square
Kilometers) Total Number of Static Number of Static Number of Static Total Number of
2D/3D Check Points 3D Check Points 3D Check Points Static 3D Check
(clearly-defined points) in NVA in VVA Points
≤500 20 20 5 25
501-750 25 20 10 30
751-1000 30 25 15 40
1001-1250 35 30 20 50
1251-1500 40 35 25 60
1501-1750 45 40 30 70
1751-2000 50 45 35 80
2001-2250 55 50 40 90
2251-2500 60 55 45 100

28
 New Standard Highlights
– Not Yet Addressed:
• Methodologies for accuracy assessment of linear features (as opposed to
well defined points)
• Rigorous total propagated uncertainty (TPU) modeling (as opposed to --
or in addition to – ground truthing against independent data sources)
• Robust statistics for data sets that do not meet the criteria for normally
distributed data and therefore cannot be rigorously assessed using the
statistical methods specified herein
• Image quality factors, such as edge definition and other characteristics
• Robust assessment of check point distribution and density
• Alternate methodologies to TIN interpolation for vertical accuracy
assessment

29
 Example on Applying the New Standard
• User asked for orthoimagery with GSD =10 cm ortho, what
specifications he/she needs to ask for?
Answer:
1) According to the legacy standard of 1990, most probably will be:
Horizontal Accuracy RMSE = 20 cm (2 pixels) class I, Map scale = 1:800
2) According to the new Standard:
- RMSE = 10 cm (highest obtainable/"highest accuracy work“, entails
ground controls accurate to 2.5 cm and very accurate workflow)
- RMSE = 20 cm ( comparable to 1990)
Practical specs: RMSE = 15 cm, no scale to be assigned

30
 The Standard Web Site
• The final standard document is posted on the
web page:
http://www.asprs.org/PAD-Division/Map-Accura
cy-Standards-Working-Group.html

31
Thank You!

32