Proposed new tests for evaluating CMM performance

Pereira, P. H. & Beutel, D. E.

Technical Services Division, Caterpillar Inc., Peoria, IL

Introduction

The current series of international standards for performance evaluation of Coordinate

Measuring Machines (CMMs), ISO 10360, outline tests that allow manufacturers
(OEMs) to publish specifications in such a way that it is more difficult for the non-expert
CMM user to compare different brands of machines. The intent is to improve competitive
advantage by having a very attractive, yet many times non-representative, value denoting
the machine possible accuracy. Typically these values can only be achieved under very
strict conditions. Once values are established and published in accordance with current
standards, it is very difficult to challenge them without executing (expensive) tests.

The current test procedure is open enough that, for example, a CMM manufacturer can
use artifacts made of very low coefficient of thermal expansion (CTE) materials to carry
out their tests without the need for tight environmental control obtaining better results to
be published on their literature. However, users do not, for the most part, measure parts
made of low CTE materials and cannot afford tight temperature control. Recently, a
manufacturer started advertising a new precision CMM with specifications according to
ISO 10360-2. Their specification for MPEE is around 1.5 µm for a 1 m length with a
claimed temperature range of ± 2 oC. However, just the uncertainty of the test considering
the effect of temperature on a normal CTE (steel) artifact is almost as large as the
specification. Such specification is possible, i.e. it is compliant, because, according to the
current version of the Standard [1], the manufacturer is not required to disclose the actual
test conditions and is free to use any material for the artifacts.

Modified tests are necessary to level the playing field while at the same time, keeping the
overall task of performance evaluation to a low, manageable cost. The overall objective is
to have the specifications convey a realistic picture of the machine behavior when
measuring real parts. One possibility for such tests is explained and results are shown
comparing existing and proposed procedures.

Proposed tests and results

In order to minimize the possibilities of misinterpretation of performance values for

machines, two main modifications are proposed. First, that a specification for a normal
(close to that of steel) CTE artifact should be provided by the manufacturer, with
additional specifications for a low CTE artifact being optional. Since the current version
of the Standard states the manufacturer is to choose the probe stylus, obviously the most
used one is a short straight down configuration because it is more rigid and the results are
generally better. This is the reason for the second proposed modification where an offset
probe should be required for the volumetric tests.
By adding two plane diagonals with five length measurements each repeated three times
to the existing seven positions [1], and requiring them to be measured with an offset
probe, it is possible to realize the mentioned benefits.

A supplementary useful parameter that could be extracted from the same data is a
repeatability value. Calculated from the range of results from the repeated measurements,
this repeatability parameter provides more information about the machine without
additional testing.

Figure 1 shows the result on a MMZ 20 30 16 machine with a Vast scanning probe head
evaluated as per the current Standard [1]. The artifact was a 1 m steel step gage. Seven
positions were used, including three parallel to each of the axes and four body diagonals.
The plot shows only the worst of three repetitions for each of five lengths on the step
gage. A total of 105 measurements were taken. The largest error is 6.4 µm.

Zeiss CMM - results as per current Standard

(largest of 3 repetitions with 1 m step gage)

0.006

0.004

0.002
Error [mm]

0.000
0 5 10 15 20 25 30 35
-0.002

-0.004

-0.006

-0.008
measurement

Figure 1 – MMZ results with probe straight down

Figure 2 has the results from the same MMZ machine tested using a laser ball step gage
(LBSG) [3] with a 200 mm offset probe. Five lengths were measured three times each on
two plane diagonals for a total of 30 measurements all shown on the plot. With the
LBSG, longer lengths were used covering a larger volume of the machine. The worst-
case error for this test was 18.8 µm thus three times that of the current Standard.
Maximum repeatability, or worst case variation, was 0.8 µm.
 Zeiss CMM - 203016
Test with laser ball step gage
0.02

0.015

0.01

0.005
Error [mm]

0
0 500 1000 1500 2000 2500 3000
-0.005
XZ diagonal
-0.01 YZ diagonal

-0.015

-0.02

-0.025
Length [mm]

Figure 2 – MMZ results with offset probe

Figure 3 is a photo of the test setup for the proposed tests on the MMZ subject machine.
To actually use the LBSG artifact, which is very practical for larger machines to achieve
the required coverage of 66% of the body diagonal, another minor addition is needed to
the Standard. In its current form, only artifacts with parallel faces, like a mechanical step
gage, are allowed for this test. The averaging effect when measuring a sphere must be
properly accounted for to allow different artifacts, used with the same procedure, to
output comparable results.

Figure 3 – Test setup on MMZ 20 30 16 – YZ diagonal

A 1 m steel step gage was used to evaluate a Cygnus 20 12 10 machine equipped with a
PMM scanning probe head according to existing Standard [1]. Seven positions including
three aligned with the machine axes and four body diagonals were used. Five lengths
were measured three times. Figure 4 displays the results of this evaluation. The largest
error comes to 2.9 µm.

Cygnus CMM - results as per current Standard

(largest of 3 repetitions with 1 m step gage)
0.001

0.000
0 5 10 15 20 25 30 35
Error [mm]

-0.001

-0.002

-0.003
measurement

Figure 4 – Cygnus results with probe straight down

Cygnus CMM - 201210

Test with step gage
0.006

0.004

0.002
Error [mm]

0
0 100 200 300 400 500 600 700 800 900 1000
-0.002

-0.004

-0.006
XZ diagonal
YZ diagonal
-0.008
Length [mm]

Figure 5 – Cygnus results with offset probe

Figure 5 shows the results of the same Cygnus machine evaluated with a 200 mm offset
probe on two plane diagonals. Again five lengths of a step gage were measured three
times each on both positions. The plot has all 30 measurements. Maximum non-
repeatability, or worst-case variation, was 0.3 µm. The maximum error was 6.1 µm or
twice that of the evaluation according to current Standard.

Discussion

Table 1 below displays the summary of the test results for comparison. There is clearly a
difference where the existing tests show better values. For both machines, the offset
probe certainly uncovers more geometric errors. For the MMZ machine, with the LBSG,
a considerably larger volume of the machine was evaluated perhaps causing the more
substantial increase in the error. The repeatability values are useful to demonstrate how
repeatable the machine can be without an extra test.

Table 1 – Summary of results

E as per Proposed
Machine current Repeatability Volumetric
Standard error (E)
MMZ 6.4 µm 0.8 µm 18.8 µm
Cygnus 2.9 µm 0.3 µm 6.1 µm

Conclusions

As normal, change comes with certain resistance. Users and particularly vendors may
state that they are comfortable to read results as per the current Standard and that all
setups (fixtures, artifacts, programs, etc) are in place to test machines as such. However,
it is also very important that the evaluation tests represent real life applications. From the
results shown:

- The current tests minimize the errors making the machine better than it really is

- Thermal effects are not taken into account

- Parallel face artifacts are not adequate for larger machines

- Requiring uncertainty statements to be shown [5] for performance evaluation tests

have not significantly changed anything yet

It is argued the modified tests will generate values more closely related to actual part
measurements that will be more useful to all involved.
References

1. ISO 10360:2-2001, Geometrical product specifications (GPS) – Acceptance and

reverification test for coordinate measuring machines (CMM) – Part 2: CMMs
used for measuring size
2. ISO 10360:5-2000, Geometrical product specifications (GPS) – Acceptance and
reverification tests for coordinate measuring machines (CMM) – Part 5: CMMs
using multiple-stylus probing systems
3. Phillips, S.D.; Sawyer, D.; Borchardt, B.R.; Ward, D.; Beutel, D.E., “A novel
artifact for testing large coordinate measuring machines”, Precision Engineering,
Vol. 25, n. 1, Jan/2001
4. ASME B89.4.1-1997, "Methods for performance evaluation of coordinate
measuring machines", The American Society of Mechanical Engineers.
5. ISO 14253 series, Geometrical product specifications (GPS) – Inspection by
measurement workpieces and measuring equipment

Keywords: CMM, standards, performance tests, CTE uncertainty