SAND2008-2571C

Comparison of Calibration Methods for a Surface Plate

Speaker: Orlando Espinosa
Sandia National Laboratories1
PO Box 5800, MS 0665
Albuquerque, New Mexico 87185
Phone: (505) 844-7651
Fax: (505) 844-4372
ocespin@sandia.gov

Authors: Orlando C. Espinosa, Porfirio J. Diaz, Margaret C. Baca, Barbara N. Allison,

Meghan Shilling
Sandia National Laboratories

1 Abstract
This paper presents equipment and methods that can be used to calibrate the flatness of large
surface plates to Federal Specification GGG-P-463c. The flatness calibration requirements in the
Federal Specification are discussed. Moody’s method for collecting data and calculating flatness
for surface plates is presented. Three measurement methods, electronic levels, autocollimator,
and laser interferometer, are used to collect data. The instrumentation and measurement
procedure for each of the three are presented and the flatness results are compared and discussed.

2 Surface Plate Specifications

Federal Specification GGG-P-463c [1] together with Amendment 1 [2] specify the material,
design, and construction requirements of granite surface plates used for precision locating,
layout, and inspection work. The specification also includes work surface workmanship, surface
texture, tolerances on repeat reading, and flatness. In this paper we focus flatness calibration for
granite surface plates.

In the Federal Specification, a flatness tolerance is listed for three different grades, AA, A, and
B, for a variety of common surface plate sizes. A portion of this chart, for rectangular surface
plates, is repeated in Table 1 and Table 2. For unlisted sizes, the flatness tolerance for Grade AA
D2
is calculated by 40 + μin (where D is the diagonal length of the surface plate in inches) or
25
1 + 1.6 D 2 × 10-6 μm (where D is the diagonal length of the surface plate in millimeters). The
flatness tolerance excludes a zone of 1.0-1.5 inches (25-38 mm) along the edge of the plate,
depending on size.

1
Sandia is a multi-program laboratory operated by Sandia Corporation, a Lockheed Martin
Company, for the United States Department of Energy’s National Nuclear Security
Administration under contract DE-AC04-94AL85000.

2008 NCSL International Workshop and Symposium

 Table 1: Total Flatness Tolerance in μin [1]
SIZE GRADE
Width
Length (in) AA A B
(in)
12 12 50 100 200
12 18 50 100 200
18 18 50 100 200
18 24 75 150 300
24 24 75 150 300
24 36 100 200 400
24 48 150 300 600
36 36 150 300 600
36 48 200 400 800
36 60 250 500 1000
36 72 300 600 1200
48 48 200 400 800
48 60 300 600 1200
48 72 350 700 1400
48 96 500 1000 2000
48 120 700 1400 2800
60 120 750 1500 3000
72 96 600 1200 2400
72 144 1100 2200 4400

2008 NCSL International Workshop and Symposium

 Table 2: Total Flatness Tolerance in μm [2]
SIZE GRADE
Width Length
AA A B
(mm) (mm)
300 300 1.3 2.6 5.2
300 450 1.5 2.9 5.9
450 450 1.6 3.3 6.6
450 600 1.9 3.8 7.6
600 600 2.2 4.3 8.6
600 900 2.9 5.7 11.5
600 1200 3.9 7.8 15.5
900 900 3.6 7.2 14.4
900 1200 4.6 9.2 18.4
900 1500 5.9 11.8 23.6
1500 1800 7.5 15.0 29.9
1200 1200 5.6 11.2 22.4
1200 1500 6.9 13.8 27.6
1200 1800 8.5 17.0 33.9
1200 2400 12.5 25.0 50.0
1200 3000 17.7 35.4 70.8
1500 3000 18.5 36.9 73.9
1800 2400 15.4 30.8 16.6
1800 3600 26.9 53.8 107.7

In the Federal Specification, testing is broken into two groups. Group A covers thickness,
stiffness, surface texture, squareness, and seams. Group B covers repeat reading measurement
and flatness of work surface. We will only cover the calibration of flatness.

Periodic flatness calibration is recommended with the interval varying depending on the grade of
the plate, wear resistance, conditions, and frequency of use. This may be a six month interval for
a busy manufacturing shop, or a year or longer in a laboratory environment. No interval is
specified in the GGG-P-463c standard. NCSLI RP-1 [3] provides guidance for determining
calibration intervals. However, the GGG-P-463c standard recommendeds that the flatness of the
surface plate be monitored with a repeat reading gage (shown in Figure 1), and recalibration be
performed when the results from the repeat reading gage differ from those found in the previous
calibration.

2008 NCSL International Workshop and Symposium

 Figure 1: Repeat reading gage

Surface plates must be given sufficient time in the calibration area before calibration to allow the
temperature of the surface plate to reach room temperature. The Federal Specification gives
guidance on soak-out time. Larger, thicker surface plates and those that have a large initial
temperature offset from the calibration area require more soak-out time. An equation that relates
the size and initial temperatures to soak-out time is given in the standard. In addition to soak-out
time, it is necessary to thoroughly clean and dry the surface plates before testing for flatness.

A flatness calibration test, as stated in the Federal Specification, contains readings in an eight
line grid pattern on the surface plate. The eight lines include two diagonal lines, four side lines,
and two center lines, as shown in Figure 2.

SECONDARY KEY POINT 1

POINT 2
SE
CO
ND
DI
AG
ON
AL
STICKER

N AL
O
IAG
D
Y
KE
SECONDARY
KEY POINT 2 POINT 1

Figure 2: Eight line calibration pattern for rectangular surface plates [1]

Flatness calibration elevation values are stated with respect to the reference plane of the surface
plate. The reference plane is defined at the center of the surface plate, and can be found by
evaluating the key diagonal profile and the intersection point of the two diagonal profiles. The
data points should be equally spaced and selected so that there is a data point in the middle of

2008 NCSL International Workshop and Symposium

each measurement line. The maximum vertical displacement above the reference plane locates
the roof plane. The roof plane is parallel to the reference plane. Similarly, the maximum vertical
displacement below the reference plane locates the base plane. The datum plane is also parallel
to the reference plane and located midway between the base plane and the roof plane. These
planes are shown in Figure 3 relative to the key diagonal, although the maximum and minimum
values that define the base and roof plane do not necessarily lie on the key diagonal. The
specification defines flatness as the distance between the roof plane and the base plane.

ROOF PLANE
WORK SURFACE PROFILE
CENTER OF PLATE, ALONG KEY DIAGONAL
ELEVATION ZERO

REFERENCE PLANE
DATUM PLANE

KEY POINT 1
KEY POINT 2

BASE PLANE

Figure 3: Plane Identification [1]

Several rules govern the selection of measurement points along these grid lines. The data points
must be equally spaced along each line, with a maximum spacing of 12 inches (305 mm) and a
minimum of 6 data points taken in any one line. The number of steps taken on any line must be
an even number (giving an odd number of data points) and the intersection points of all lines
must fall within a 0.3 inch (8 mm) diameter circle of each other. In most cases the last
requirement necessitates the use of adjustable supports to allow different spacing for the
perimeter/center lines and diagonal lines.

Most instruments do not work close enough to the edge (within 1-1.5 inches (25-38 mm),
depending on size) to meet requirements. If this is the case, the eight line pattern may be reduced
in size but the edge profile needs to be tied to the pattern using other instrumentation, such as a
master straight edge with an indicator.

3 Moody method of Calibration

J.C. Moody developed a method, including data analysis, for the calibration of surface plates [4].
This method meets the requirements of Federal Specification GGG-P-463c. The Moody method
is common in industry; frequently referenced in technical papers on the subject and used in
commercial software designed for surface plate calibration.

3.1 Methodology
The general procedure for surface plate calibration, as dictated by the Moody method [4], is as
follows:
1. Collect readings from two diagonals, four perimeter lines, and two center lines as specified in
Federal Standard GGG-P-463c. These measurements are angular deviations from one

2008 NCSL International Workshop and Symposium

 reading to the next. Sum the angular deviations cumulatively along each line in order to get
the angular displacement from the first point. This process is diagrammed in a later section
(Figure 6).
2. Adjust slope and height of the deviations on the diagonal lines to get zero displacement in the
center of each line and equal deviation at either end.
3. Adjust slope and height of each perimeter line to maintain consistency with the appropriate
corner point deviations on the diagonal lines (found in step 2).
4. Adjust slope and height of center lines to get consistency with the appropriate middle points
of the perimeter lines. The middle value for the center lines should be zero if everything is
done perfectly. This is unlikely due to the effect of slight errors in reading the
instrumentation. A certain amount of error, called closure error, should be permitted, Moody
specifies 0.0001 inch (2.5 μm) in [4]. The center lines can be shifted so the center point is
zero, and the error is propagated away from the center.
5. Convert the angular deviations to linear displacements by multiplying the sine of the angular
deviation by the distance between points. The maximum vertical displacement upward
locates the roof plane and the maximum vertical displacement downward locates the base
plane.

4 Calibration Methods
In this paper, three methods were used to acquire angular measurement data along the eight
prescribed lines. These measurement methods use: an autocollimator, a laser interferometer [5],
or electronic levels [6].

4.1 Electronic Levels

Electronic levels can be used to measure flatness by the Moody method. The electronic level
sensing head shown in Figure 4, operates on the pendulum principle. In the case of the
electronic level, a pendulum is attached to an extension block by two flexures (reed springs).
Tilting the level causes the shading loop to move away from the center of the core. This
movement causes an imbalance in the induced magnetic field between the primary and
secondary coils, delivering a signal proportional to the displacement of the pendulum. The
signal is translated into an angle and displayed on the amplifier meter.

2008 NCSL International Workshop and Symposium

 Figure 4: Electronic Level [6]

For measurement of flatness by electronic levels, at least one electronic level with readout, a
straightedge, and clamps are needed. The equipment used for the flatness measurement by
electronic levels presented in this paper is shown in Figure 5.

Figure 5: Equipment used for measurement by electronic levels

To operate the electronic level, the level is first adjusted so the meter reads zero at the starting
position (at the beginning of each line for evaluation). The electronic level is then translated
along a straight edge, which acts as guide. Measurements are taken at evenly spaced distances
along the line to be evaluated. The distance between measurements is determined by distance

2008 NCSL International Workshop and Symposium

between the feet on the base of the electronic level. A diagram of the measurement process is
shown in Figure 6.

Figure 6: Flatness measurements [6]

For our measurements, two electronic levels are used. The first level remains stationary and the
second is translated along the straight edge. The difference between the two measurements
recorded. If the surface plate tilts due to the weight of the translating level, the differential
measurement will not be effected, while an absolute measurement made with a single level
would be effected. The foot spacing of the base of the electronic levels used is 5.125 inches (130
mm). The foot spacing on our levels is not adjustable. The level is moved along the straight
edge a distance of 5.125 inches (130 mm) for each reading. The flatness measurement is taken
along the center of the width of the level. The set-up used is shown in Figure 7.

Figure 7: Electronic level set-up

4.2 Autocollimator
Autocollimators project a beam of collimated light. An external reflector reflects all or part of the
beam back into the instrument where the beam is focused and detected by a photodetector.
Changes in the inclination angle of the reflector cause changes in the position of the reflected

2008 NCSL International Workshop and Symposium

image, as seen in Figure 8. The angular deviation is determined by measuring the offset between
the initial reflected mage and the current reflected image.

detector
reflected beam for tilted reflector light source

reflected beam for perpendicular reflector

Figure 8: Autocollimator diagram

Figure 9: Autocollimator set-up

For measurement with our autocollimator, a zero value is established by reading the micrometer
drum with the mirror at the starting position (at the beginning of each line for evaluation). The
mirror and mirror mount are then translated in the same manner as the electronic level, with
measurements taken at intervals equal to the foot spacing of the mirror mount.

This set-up requires an autocollimator with readout, a mirror attached to a traveling mirror
mount, a straightedge and clamps. A mirror mount with 4 inch (101 mm) foot spacing was used.

2008 NCSL International Workshop and Symposium

A small surface plate mounted on a cart was used to minimize vertical drift. The set-up is shown
in Figure 9.

4.3 Laser Interferometer

The laser emits a single beam with two frequencies. The angular interferometer has a splitter to
separate the beam into two paths having different frequencies. One beam, through a beam
bender (turning mirror), is sent to the upper part of the angular reflector. The second beam is
sent to the lower part of the angular reflector. These beams return and are combined. The
difference in length between the two beam paths can be determined from the combined signal.
The difference in path length is directly related to the inclination angle of the angular reflector.
A diagram of the angular interferometer is shown in Figure 10.

Figure 10: Angular interferometer optics [5]

Like the electronic levels, the laser interferometer is zeroed at the beginning of each grid line to
be measured. The angular interferometer remains stationary and the angular reflector is
translated in the same manner as the electronic level and autocollimator, with measurements
taken at intervals equal to the foot spacing of the reflector mount.

This set-up requires a laser, an angular interferometer, and angular reflector and optics to alter
the beam path. The angular reflector mount with 4 inch (101 mm) foot spacing was used. The
laser was mounted on a tripod and remained stationary for the entire calibration. Turning mirrors
mounted on a base were used to keep the beam path in line with the movement of the angular
reflector. The set-up is shown in Figure 11.

2008 NCSL International Workshop and Symposium

 Figure 11: Set-up for laser interferometer calibration

4.4 Resolution and Accuracy

The resolution and accuracy of the three systems used in this paper are given in Table 3. The
three methods have similar resolution (the resolution of the autocollimator is at least one half of
the resolution of the divisions on the micrometer scale). Note that the accuracy specification of
the instrument is a tolerance, and does not indicate the actual measurement uncertainty of the
individual data points or the combined uncertainty in the analysis of flatness. Other sources of
measurement uncertainty include cleanliness, geometry of the foot pads, and geometry of the
contact between foot pad and surface plate.

Table 3: Resolution and accuracy of instruments used (per manufacturer specification or

instrument calibration)
Method Resolution Accuracy
Electronic Levels 0.1 arcsec (0.5 μrad) ±2 arcsec
Autocollimator 0.1 arcsec (0.5 μrad) ±0.2 arcsec
Laser Interferometer 0.1 arcsec (0.5 μrad) ±(0.2% of reading + 0.05
arcsec/m of travel)

5 Results
A 48 x 72 inches (1219 x 1829 mm) granite surface plate was measured by all three methods.
The flatness specification for grade AA, from Table 1, is 350 μin (8.9 μm). Computing the
flatness from the equation (when the size is not listed in the GGG-P-463c specification), one
obtains 340 μin (8.6 μm). Since this plate size is in the table, we use 350 μin (8.9 μm) as the
flatness specification for grade AA. The data from both the electronic levels and from the laser
interferometer were entered in commercial software in order to calculate the flatness of the
surface plate. According to their documentation, these software programs use the Moody
method for calculations. The software used with the laser interferometer uses the base plane as

2008 NCSL International Workshop and Symposium

the reference plane, unlike the other two software systems which locate the reference plane in
accordance with the standard. The data from the autocollimators were collected and analyzed
using software developed in-house for this specific task. This software uses the method Moody
established in [4], with slight changes in location of the reference plane in order to comply with
the Federal Specification. The software reports data in microinches.

The results from these three methods are displayed both as a plot of numerical deviations (Figure
12, Figure 13, Figure 14) and as a graphical representation of the numerical deviations (Figure
15, Figure 16, Figure 17). These plots are not necessarily to scale.

-91 -87 -64 -44 -30 -25 -29-35 -41 -67 -99 -155 -202 -238 -257

-95 -206
-100 -78 -37 -148 -208

-63 -100
-107 -48 -30 -57 -167

-38 -34
-116 -30 -17 -15 -126

-5 1
-133 -98 65 -37 -14 4 14 0 24 17 11 -1 -21 -40 -82

-1 5

-157 -10 13 13 -68

-31 15

-181 -67 13 15 -61

-104 6

-213 -142 26 -16 -77

-191 -51

-257 -199 -147 -96 -49 -1 30 39 47 51 31 12 -15 -52 -91

Figure 12: Results from electronic levels measurement (in μin)

2008 NCSL International Workshop and Symposium

 -84 -102 -102 -98 -93 -89 -89 -81 -22 -91 -109 -135 -159 -219 -245 -263 -273
-77 -229
-95 -64 28 -181 -243
-58 -137
-98 22 -197
-55 -95
-58 -65
-107 16 -160
-46 -43
-118 -33 12 -19 -118
-14 -4
-119 -76 -62 -44 -26 -17 -3 -7 0 8 12 17 9 7 -4 -22 -64
4 15
-160 -4 -16 12 -50
-32 8
-179 -37 -57
-58 5
-92 -10
-206 -53 -61
-124 -30
-234 -166 -67 -43 -69
-213 -64
-273 -283 -294 -304 -294 -303 -289 -244 -141 -175 -147 -138 -124 -110 -103 -89 -84

Figure 13: Results from autocollimator measurement (in μin)

Figure 14: Results from laser interferometer measurement (in μin)

2008 NCSL International Workshop and Symposium

 Figure 15: Plot of deviations from measurement with electronic levels

Figure 16: Plot of deviations from measurement with autocollimator

Figure 17: Plot of deviations from measurement with laser interferometer

2008 NCSL International Workshop and Symposium

The flatness measurements from the three methods are reported in Table 4. The flatness is
defined as the distance between the base plane and the roof plane. Closure error is the difference
between the center point calculated from the diagonal measurements (CG and AE on Figure 17)
and the center points calculated from the two center lines (north-south and east-west, or DH and
BF on Figure 17).

Table 4: Flatness and closure errors

Measurement Method Flatness Closure Error (NS) Closure Error (EW)
Electronic Levels 308 μin (7.8 μm) 19 μin (0.5 μm) 2 μin (0.05 μm)
Autocollimator 332 μin (8.4 μm) 123 μin (3.1 μm) 40 μin (1.0 μm)
Laser Interferometer 235 μin (6.0 μm) 43 μin (1.1 μm) 3 μin (0.07 μm)

It is possible to see from the results that the overall shape of the surface plate is consistent
between the three measurement methods. The flatness values vary from 235 μin (6.0 μm) to 332
μin (8.4 μm) - all meeting the AA flatness specification for a surface plate of this size. The
variation in flatness values between measurement methods can most likely be attributed to the
differences in the geometries of the equipment and the fact that the data points were taken in
slightly different locations for each of the measurement methods.

Moody [4] specifies a satisfactory calibration as having closure error less than 100 μin (2.5 μm).
Based on this, both the laser interferometer calibration and the electronic levels calibration can
be considered satisfactory. In the autocollimator calibration, the closure error for the north-south
center line is slightly higher than what can be considered satisfactory (123 μin, 3.1 μm). This
calibration would need to be repeated according the Moody’s criteria.

6 Conclusions
It is not possible to recommend a method based on the presented results. All three measurement
tools have similar resolution. Their measurement uncertainties are all similar. However, based
on ease of use, the electronic levels can be recommended. The levels do not require alignment of
optics like the other two methods. Minimal set-up time is required and an experienced operator
can perform the calibration in approximately one hour.

The autocollimator needs to be repositioned and re-aligned several times in order to measure all
of the required lines. In addition, the autocollimator requires the operator to view the
microscope and adjust the micrometer drum in order to determine the angular displacements.
This is more time consuming than the other two methods which simply display the angular
displacement. In addition, the autocollimator is sensitive to air currents passing between it and
the reflector. A knowledgeable operator can complete the measurement by autocollimator in
approximately 5 hours.

Like the autocollimator, the laser interferometer requires repositioning and alignment of optics
between runs. The operator must also be careful not to break the beam path during
measurement. This measurement takes approximately 5 hours with an experienced operator.

This paper presents the Federal Specification for surface plate flatness calibration. A
methodology for collecting and analyzing data in accordance with the Federal Specification,

2008 NCSL International Workshop and Symposium

Moody’s method, is discussed. The results from the three different tools, electronic levels, an
autocollimator, and a laser interferometer, used to collect the necessary data for analysis do not
indicate any one tool as being better than the others. However, the electronic levels provide the
calibration data in one hour as opposed to 5 hours for the other methods.

7 Acknowledgements
Sandia is a multi-program laboratory operated by Sandia Corporation, a Lockheed Martin
Company, for the United States Department of Energy’s National Nuclear Security
Administration under contract DE-AC04-94AL85000.

8 References
1. Federal Specification GGG-P-463c, ‘Federal Specification: Plate, Surface, (Granite),’
September 1973.
2. Federal Specification GGG-P-463c Amendment -1, ‘Federal Specification: Plate, Surface,
(Granite) (Inch and Metric)’, June 1977.
3. NCSLI, “Establishment and Adjustment of Calibration Intervals,” Recommended Practice
RP-1, January 1996.
4. J.C. Moody, “How to Calibrate Surface Plates in the Plant,” reprint from The Tool Engineer,
October 1955.
5. Hewlett Packard 5528A Laser Measurement System User’s Guide, 1986 (Part No. 00528-
90014)
6. Federal Electronic Gaging Products Catalog 190.

2008 NCSL International Workshop and Symposium