See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/321769343

DETERMINING SAMPLE SIZE FOR SPECIFICATION LIMITS VERIFICATION

WITH TOLERANCE INTERVALS

Preprint · December 2017

CITATIONS READS

0 387

1 author:

Alessandro Vasta
Baxter International
2 PUBLICATIONS   14 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Clinical Risk Analysis for dialytic therapies View project

Sample Size Determination for Verification Contexts View project

All content following this page was uploaded by Alessandro Vasta on 18 March 2019.

The user has requested enhancement of the downloaded file.

PREPRINT

Determining sample size for specification limits verification with

tolerance intervals

Alessandro Vastaa
This is a Manuscript of an article submitted to Taylor & Francis in Quality Engineering
Journal on 09-Mar-2019 , for possible online publication at: http://www.tandfonline.com/

ARTICLE HISTORY
Compiled March 19, 2019

ABSTRACT
In traditional industrial verification process, when the aim is the conformance to
specifications, it’s difficult finding an affordable statistical method reaching this
purpose.
Most of data tables included in industrial procedures and standards could not
be suitable for verification tests or otherwise they could lead to unknown or very
high values of Type I error. The practitioner could be tempted referring distribution
free intervals leading to very high sample sizes. A two-sided tolerance interval in a
normal universe, combined with a bivariate statistical hypothesis test can be used
to address this problem. The proposed approach leads to a determination of the
sample size with pre established probabilities of Type I and Type II errors, essential
for risk and reliability estimations.
Finally, a novel approach for the tolerance interval testing factors determination is
proposed. This approach calculates the testing factors based on the deviation of the
mean and the variance from the null hypothesis, when a specified value of Type II
error is met. The deviations of the mean and variance are determined in such a way
that a certain proportion of the population falls within the specification limit rather
than a specific proportion of the population within the sampled tolerance interval.

KEYWORDS
Tolerance Interval; Sample Size; Confidence; Population; Significance; Power;
Verification; Risk Analysis; Reliability

1. Introduction

In accordance with ISO 3534-1[1] a statistical tolerance interval, is an interval deter-

mined from a random sample in such a way that one may have a specified level of
confidence γ that the interval covers at least a specified proportion of the sampled
population P.
The interest in determining sample sizes for tolerance intervals was already present
in 1941 with the article of Wilks [3] who presented both a non-parametric approach
and a method based on the assumption of a normal distribution. The sample size is
determined controlling the precision of the proportion of population P such that the
probability is at least p that P will lie between two given number, say Pb & Pc .
The problem of constructing tolerance limits for a normal universe is considered by
Wald & Wolfowitz [4] proposing a method that achieves a good approximation of such

CONTACT Alessandro Vasta. Email: alesnic@icloud.com

tolerance limits. This is also the method used in R language simulation programs in
this study.
In the article of Ellison [5] the Wald & Wolfowitz method is reviewed stating that the
confidence level attained by their approximation converges to the nominal confidence
level with increasing sample size N, and that difference is 0(1/N) rather than 0(1/N2
).
Similar approach to [3] is adopted also in Kirkpatrick article [6], where the sample
size is selected to obtain a tolerance interval which will bracket (P ± d) of the popula-
tion such that: the probability is γ that the interval contains between (P - d) and (P
+ d) of the population, the probability is (1+γ)/2 that the interval contains at least
(P - d) of the population, and the probability is δ = (1-γ)/2 that the interval contains
more than (P + d) of the population.
A two-sided β-expectation tolerance interval is defined by Odeh, Chou and Owen
[7], where the expectation of tolerance interval with endpoints µ − kσ and µ + kσ
is β, that means on average the coverage is β. They explained also the difference
between a β-expectation tolerance interval and a β-content tolerance interval, with the
latter constructed to contain at least 100β% of the population with a stated degree of
confidence.
Hauck and Shaikh [8] proposed an approach employed in pharmaceutical industry
where using a two-sided tolerance interval they accept a given batch if the tolerance
interval falls entirely within an acceptable interval. They also dealt with the sample
size, significance level α and power determination for such an approach. The proposed
method of this article is based on a similar approach but with a different statistical
hypothesis test. In this article the power is evaluated by two alternative hypotheses in
two particular situations: as displacement of the mean from the null hypothesis (∆µ
= µ1 -µ0 ) or as displacement of the standard deviation from the null hypothesis (∆σ
= σ 1 /σ 0 ), (see sect.2.6).
With an example where is possible to compare the population coverage for a confi-
dence interval, a prediction interval and a tolerance interval, Gryze [9] showed a study
for an ordinary linear regression application.
Two types of tolerance intervals are provided in standard ISO 16269-6 [2]: paramet-
ric, based on a normal distribution and distribution-free. For normal distributions,
all combinations are addressed including one-sided, two-sided tolerance intervals,
with known and unknown variance. Any combination leads to a table including many
instances of the tolerance interval normalized half-width k factor. All tables are
calculated by computer programs that report the ”exact” numerical computation of
the k factors for different values of sample size n, population P and confidence level
γ . In accordance with ISO16269-6 introduction, the tolerance limits discussed in
this document can be used to compare the natural capability of a process with given
specification limits. Nevertheless, setting the specification limits by the tabulated k
factor for a specific sample size, further sample batches with the same sample size
could fit within the specification limits with an unknown probability and in any
case very low. For instance, supposing µ=0, σ=1, P=0.95, γ=0.95 for a sample size
n=8 and for a two-sided interval, the tabulated k factor is 3.746. Thus, setting a
nominal specification interval with [LSL=-3.746; USL=3.746] coincident with the
tolerance interval width, the Type I error probability has been calculated as high as
53%. Alternatively, relying on distribution-free table, the Type I error probability
becomes practically 0%, but at the expense of a much larger sample size n= 93.
With the proposed approach in this article instead, it will be possible to find that
setting the Type I error to 5% for the above example, the sample size will be n=19

2
and setting the Type I error to 1%, the sample size will be n=25, (this values can
be calculated with the attached R program ”Sample size by spec.r” setting all input
parameters according the above example and the parameter mode sample_find = T).

The R library ”EnvStats” [12] with the function tolIntNormN() compute the sample
size necessary to achieve a specified half-width of a tolerance interval for a normal
distribution, given the estimated standard deviation, coverage, and confidence level.
Nevertheless, if these half-width values will be used to set specification limits running
a verification test, as in the previous example, high values of Type I error probability
should be expected.
Regarding hypothesis testing, Ryan [10] points out that the null hypothesis are usu-
ally defined in comparing two means that are equal, which implies that the difference
between them is zero. In practice the difference almost certainly won’t be zero, so the
difference between means in the null hypothesis shouldn’t be specified as zero. Equiv-
alence testing should be formalized in place of equality test. So, if the difference of the
two means is less than ∆µ = µ1 -µ0 (i.e. with µ1 > µ0 ), then the means are equivalent.
µ1 could be an alternative hypothesis, with the null hypothesis µ0 such that the dif-
ference between the two means is lower or equal to ∆µ. This equivalence concept is
one of two specific cases that will be considered in the next sections (see sect. 2.2 and
2.6) about null and alternative hypothesis definition H0 , Haµ1 & Haσ1 .
Many references Meeker [11], Wilks, Kirkpatrick [3, 6] and Faulkenberry [14] are still
based on a population proportion precision criteria where the sample size is chosen
large enough such that:
- The probability is γ (large) that at least a proportion P of the population will be
included within tolerance interval and
0 0
- The probability is δ (small) that more than a proportion P with P > P will be
included.
It’s worth noting that the probability δ is a decreasing function of the sample size n, so
higher the sample size lower is the uncertainty determining the population proportion.
Another recent reference, Young [15], propose a criteria in choosing P’ and δ avoiding
subjectivity, in particular P’ is determined as a function of the margin between the
specification limits and the expected mean µ̃ derived from historical data; thus guiding
the practitioner achieving an acceptable result concerning Type I error and minimum
sample size.
The aim of this paper, is to achieve an exact solution, starting from pre established
values of Type I (α) and also Type II (β) error probabilities. This solution is expect-
ed to find the minimum sample size so that at least an assigned proportion of the
population falls within the specification limits with the specified power (1 - β).

3
1.1. Basic formulation
Let’s suppose we verify a requirement where a measurable entity x, represented by
a random variable with a normal probability distribution with known mean µ and
standard deviation σ, shall be found inside a proper specification range
[LSL; USL]. The Figure 1 shows a particular situation where the above condition is
met with a probability 100γ% = 95%.

Figure 1. Probability Interval

Thus, the probability P[LSL ≤ x ≤ U SL]= γ, where x is a generic single observa-

tion of the process under examination.
If µ and σ are unknown and an accurate estimate of the process is required, a signif-
icant number of data points shall be collected to achieve a negligible error estimation
of the mean µ and the standard deviation σ.
In order to perform a test with a practicable sample size a tolerance interval can be
considered to model the uncertainty about the true value of the mean and standard
deviation.

1.2. Tolerance interval parameters

Let x be the variable under consideration and xi with i = 1, 2,...,n a sample of n
independent observations in x belonging to a normal distribution.
x̄ = ni=1 xni is the sample mean;
P
q Pn
i=1 (xi −x̄)
2
s= n−1 is the sample standard deviation.
Two-sided tolerance intervals are calculated in this way:

LT L = x̄ − k2 ∗ s (1)
U T L = x̄ + k2 ∗ s

4
 Where:

LTL is the Lower Tolerance Limit

UTL is the Upper Tolerance Limit
k2 is a factor for a two-sided tolerance interval that is a function of the sample size
required to cover the desired proportion P of the population with a preassigned value
of probability γ (the Tolerance Interval Confidence).
In this paper the k2 approximation proposed by Wald & W. [4] has been referred and
extensively used in several statistical calculations performed by R programs supporting
the thesis of this article, so:

s
n−1
k2 = r (2)
χ2n−1,γ

where:
n is the number of observations (the sample size)
χ2 n−1,γ denotes the number for which the probability that χ2 distribution with n-1
degrees of freedom will exceed this number is γ
r is the solution of the following equation:

1
Z √ +r
1 n −t2
√ e 2 dt = P
2π √
1
n
−r

P is the proportion of population covered by the tolerance interval.

It can be verified [4] that the k2 factor does not depend on the unknown parameters
µ and σ. Throughout the article exposition we assume without loss of generality that
µ = 0.
Finally, the proper sample size we are looking for in order to verify the specifications,
shall satisfy the following relationships:
LTL > LSL
UTL < USL

Each UTL and LTL are random processes, then the proposed methodology discussed
in the next sections, will deal with the determination of the probability distribution
of LTL and UTL.

2. Methodology

Starting from a working assumption based on a normal process, the expectation is

finding the observation in a symmetrical range; for this reason, we consider in this
article only symmetrical specification limits centered on the expected mean µ, and in
our particular case, with µ = 0, we have:
USL = -LSL = ASL.
Let’s call ASL the absolute value of the symmetrical specification limits.
Given this specification goal, its verification implies verifying both tolerance limits
LTL and UTL falling within [LSL; USL] range, i.e.:

5
 max(abs(LTL), UTL) = ATL ≤ ASL
In the relationships above we derived the one-sided tolerance limit ATL starting
from the original two-sided interval; therefore equations (1) can be rewritten in this
way:

AT L = |x̄| + k2 ∗ s (3)

Probability density histograms have been calculated by an R program running e-

quation (3) with N = 100,000 reiterations.
Figure 2 shows the histogram of the simulation for the very small sample size n =
3, with xi ∈ N orm(0, 1)

Figure 2. Histogram of PDF with n=3

The Figure 2 shows that the probability density is not normal and strongly asym-
metrical.
The Figure 3 below shows instead the histogram with a large sample size n = 1000.

Figure 3. Histogram of PDF with n=1000

This figure suggests that the probability distribution is almost normally distributed.
This can be evaluated applying the Anderson Darling normality test; the results for

6
n=1000 are reported below:

Results of Hypothesis Test

————————–
Alternative Hypothesis:
Test Name: Anderson-Darling normality test
Data: sample n1000
Test Statistic: A = 0.3066074
P-value: 0.5644184
This is a consequence of the central limit theorem since the sum of n independent
random variables with a finite mean and variance tends towards a normal distribution.

2.1. Determining ATL probability density function

Aim of this section is finding the Probability Density Function (PDF) of ATL term
in equation (3). The PDF of the sum of two independent functions is the convolution
integral of the two PDF terms, let’s consider now these two terms.
Let’s call f|x̄|| (x, n) the PDF of first term |x̄|. If we consider a generic value of the
distribution |xi |, this implies two possible occurrences xi > 0 and -xi .
Under the working assumption, we have:
P DF (|xi |) = N orm(xi , 0, σ) + N orm(−xi , 0, σ) , with xi ≥ 0, so being the normal
PDF symmetrical we can rewrite the above equation as follows:
P DF (|xi |) = 2N orm(xi , 0, σ) , now extending the results to the mean, we have:

σ
f|x̄| (x, n) = 2N orm(x, 0, √ ); x ≥ 0 (4)
n

Let’s call instead fk2s (x, n) the PDF of the second term k2*s.
Starting from the working hypothesis, with xi the independent observation from a
normal distribution, Cochran’s theorem [13] shows that s2 follows a scaled chi-square
distribution where: √
(n−1) 2 2 (n − 1), that implies : n−1 s ∼ χ(n − 1)
σ2 s ∼ χ σ
Then, we can write:

√ √
n−1 n−1
fk2s (x, n) = χ(x , n − 1) (5)
k2 ∗ σ k2 ∗ σ
√
n−1
It’s worth noting that the scaling factor k2∗σ included in χ() function argument, in
order to refer the term k2*s in the abscissa axis, shall be appliedR also to the ordinate
+∞
axis to maintain the following distribution probability property: 0 fk2s (x, n)dx = 1

Finally the PDF of ATL is the convolution of the two above mentioned functions
f|x̄| (x, n) and fk2s (x, n), that is:

Z ∞
T IH0 (AT L, n) = fk2s (AT L − x, n)f|x̄| (x, n)dx (6)
0

Figure 4 shows in the dashed line the plot of PDF function defined for ATL accord-

7
ing (6), overlapped to histogram of equation (3) computation with N=100,000 and n
= 3. Both of them have been calculated with R program ”Sample size by spec.r” in
attachment.

Figure 4. PDF function of TIH0 (ATL, 3) overlapped to Histogram

2.2. Determining ASL for each sample size n

Given a normal process with mean µ = 0, let’s find ASL, the absolute value of the
symmetrical two-sided specification limit that, given the following null hypothesis:
H0 : (µ0 = 0; σ0 ), will imply ATL(n) ≤ ASL(n). Where ATL(n), as defined in sect. 2
and calculated with equation (3) on a sample of n measures xi , is the maximum of the
absolute values of the two-sided tolerance interval limits including a proportion P of
the population with a level of confidence γ.
Let’s now introduce the new parameter: ”significance level” = α as the maximum
probability of rejecting the null hypothesis H0 when in fact H0 is true, (the Type I
error probability). Aim of this section is the determination of ASL being the absolute
specification limit supporting the null hypothesis H0 : (µ = µ0 ; σ = σ0 ) with the
parameters set (n, P, γ, α).
ASL is the solution of the following equation:

Z ASL
(1 − α) = T IH0 (z, n)dz (7)
0

The equation (7) provide the value of ASL depending on the set of parameters (n,
µ0 , σ0 , P, γ, α). It’s worth noting that in equation (7) there is a dependency on n, P
and γ embedded in factor k2 in the term fk2s (x, n) .
Equation (7) has been solved numerically in R and, the related statistical results to
evaluate the experimental values of significance level α, are reported in next section 2.4.

8
2.3. Extension to general case µ 6= 0
Thanks to the independency of µ from k2 as already discussed in section 1.2, it follows
that:

U SL = µ + ASL (8)
LSL = µ − ASL

2.4. Experimental Tests by Simulation

In Table 1 are summarized the results of a simulation program performed in R en-
vironment where tolerance limit are calculated by equation (1) starting from sets of
simulated observations xi (i=1, 2,. . . ,n) obtained by a random number generating
function and belonging to a normal process with mean µ=0, standard deviation σ=1
and different sample sizes n. All these calculations are reiterated N = 100,000 times
to evaluate the experimental values of the significance level α.
In this program, the specification limits are calculated according equation (7) previ-
ously examined. For each sample size n the experimental αe % = αe *100 is calculated
as follows:
nout
αe % = N 100

Where nout is the number of occurrences that doesn’t satisfy in N reiterations the
following relationships:
USL = ASL ≥ UTL
LSL = -ASL ≤ LTL
The additional parameters used in this experimental test are:

• Tolerance Interval population P = 0.95

• Tolerance interval confidence γ = 0.95
• Significance level α = 0.05

Moreover, the following additional outcomes are included in Table 1:

UTLmean : is the mean of all UTLj limits with j=1, 2,. . . ,N. (Due to the symmetry
of the test, only the outcomes related to the upper tolerance limit are shown).
ASL: is the absolute value for the specification limit calculated by numerical solution
of (7)
s(UTL): is the sample standard deviation of UTL
UTLmax : is the maximum value of UTL recorded in all N reiterations.
MAX: is the maximum value recorded in all N reiterations for the observation xi .
Pxout%: calculated in similar way as αe %, is the experimental % probability to find
the observation xi outside the specification limits LSL USL
k2: (2) is the factor for two-sided tolerance interval calculation in (3) that is a
function of (n, P, γ) and has been presented in subsection 1.2.

9
Table 1. Simulation results and ASL calculation for P=0.95; γ=0.95; α=0.05
n UTLmean ASL s(UTL) UTLmax αe % MAX Pxout % k2
3 8.8 17.6 4.65 35.8 5.11 4.34 0 9.92
4 5.86 10.7 2.53 18.5 4.93 4.45 0 6.37
5 4.77 8.21 1.79 14.2 5.06 4.41 0 5.08
6 4.21 6.93 1.42 10.9 4.99 4.82 0 4.41
7 3.84 6.14 1.19 9.4 4.89 4.52 0 4.01
8 3.6 5.61 1.04 8.41 5.01 4.76 0 3.73
9 3.43 5.22 0.93 7.8 5.02 4.76 0 3.53
18 2.78 3.81 0.534 5.23 4.96 5.03 0.252 2.82
42 2.41 3.01 0.307 3.91 5.06 5.13 10.4 2.43
100 2.31 2.59 0.169 3.14 4.97 5.35 61.7 2.23

Let’s consider now the results in the column αe %. The expected theoretical value
is 5%, the results are close to this value but never equal. Nevertheless, if we perform
an hypothesis test of one binomial proportion for an event with 5% probability of
occurrence over a number of trials N=100,000, we obtain the following 95% confidence
interval:

P[4.87% ≤ αe % = (nout /N) *100 ≤ 5.14%] = 0.95

In Table 2 the same results as Table 1 are shown, but with the following parameters
changes:

• Tolerance Interval Population P = 0.99

• Tolerance interval confidence γ = 0.99
• Significance level α = 0.01

Table 2. Simulation results and ASL calculation for P=0.99; γ=0.99; α=0.01
n UTLmean ASL s(UTL) UTLmax αe % MAX Pxout % k2
3 25.8 62.8 13.5 104 1.04 4.54 0 29.1
4 13.4 28.7 5.66 42.5 0.971 4.5 0 14.5
5 9.63 19.1 3.52 27.9 1.01 4.57 0 10.3
6 7.9 14.8 2.58 21.6 0.994 4.79 0 8.3
7 6.9 12.4 2.06 16.8 1.06 4.64 0 7.19
8 6.24 10.8 1.73 14.4 0.977 4.64 0 6.47
9 5.78 9.75 1.5 13.8 1 6.08 0 5.97
18 4.24 6.26 0.769 7.62 1.02 5 0 4.31
42 3.46 4.54 0.415 5.66 1.01 4.9 0.03 3.49
100 3.17 3.71 0.229 4.18 1.04 5.52 1.99 3.1

Similar considerations to Table 1 can be done also for Table 2.

The expected theoretical value for αe % is 1%. The hypothesis test of one binomial
proportion for an event with 1% probability of occurrence over a number of trials
N=100,000, provides the following 95% confidence interval:

P[0.939% ≤ αe % = (nout /N) *100 ≤ 1.064%] = 0.95

Refer to attached R program ”Tolerance interval analysis.r” to reproduce the above

results or your favorite examples.

10
2.5. Sample Size Determination
In ”Sample Size by Spec.r” R program, an iterative algorithm determines the sam-
ple size n assuming the function ASLtest (n), solution of equation (9), a monotone
decreasing function.

Z ASLtest (n)
(1 − α) = T IH0 (z, n)dz (9)
0

The solution is calculated by a reiterated algorithm with successive dichotomies for

the sample size n, as the minimum integer satisfying the following:

ASLtest (n) ≤ ASL = (U SL − µ0 ) = (µ0 − LSL) (10)

Where USL and LSL are the desired specification limit and µ0 = (USL + LSL) / 2
the expected mean.
Being n an integer value, the above relationship (10) cannot be generally satisfied
with = operator, this means that the specified significance level is correspondent to
ASLtest . If instead ASL is used for the test limits, a value ≤ α for the significance level
should be expected.
As an example, Table 3 reports the results of ASLtest , k2 and sample size n, ( in the
rightmost columns gruoup), for some specific values of input data, ( in the leftmost
columns gruoup), with σ0 = 1.
Table 3. Sample size determination
ASL Population Confidence Significance ASLtest k2 Sample Size
P% γ% α% n
4 95 95 5 3.983 2.903 16
4 95 95 1 3.956 2.723 21
4 99 99 5 3.997 3.344 54
4 99 99 1 4 3.231 69
4 99.9 99.9 1 3.999 3.666 446
5 95 95 5 4.923 3.379 10
5 95 95 1 4.828 3.081 13
5 99 99 5 4.99 3.904 25
5 99 99 1 5 3.706 31
5 99.9 99.9 1 4.991 4.202 101

Finally, the proposed sample size determination approach requires to define a Null
Hypothesis H0 :(µ0 ; σ0 ) with respectively µ0 , σ0 mean and standard deviation of a
normal process that, combined with statistical parameter (P, γ, α), provide the sample
size for null hypothesis testing.
It’s worth noting that a preliminary estimation of µ0 and σ0 is required. This can
be performed with a pilot study with data external to the data batch under test or
also internal; for instance, Ryan [10] in various application, such as medical, deals with
sequential sample size selection because more than one phase can be involved, such as
in clinical trials, and sequential sample size determination can also be used within a
given phase. Thus, sample size reestimation can occur as a study progresses.

11
2.6. Considerations about the Power of the Test
This section will take into consideration two particular alternative hypotheses Haµ1
and Haσ1 , where in the first case: Haµ1 :(µ 6= µ0 ; σ0 ). That is, only the mean µ deviates
from the null hypothesis.
So, we would like finding that µ1 = µ0 ± ∆µ leading to an assigned Type II error
probability β. According what already discussed in sect. 1.2, without loss of generality
we can assume µ0 = 0 and µ1 = ∆µ > 0.
In order to calculate µ1 , leading to a β% probability to accept the null hypothesis,
we need to find the probability distribution as already done in sect. 2.1.
In this particular case the equation (4) becomes:

σ0 σ0
f|x̄| (x, n, µ1 ) = N orm(x, µ1 , √ ) + N orm(−x, µ1 , √ ); x ≥ 0 (11)
n n

While for the term k2*s, the previous function discussed in 2.1 fk2s (x,n) is still valid:

√ √
n−1 n−1
fk2s (x, n) = χ(x , n − 1) (12)
k2 ∗ σ0 k2 ∗ σ0

Finally, the PDF of ATL is the convolution of the two above mentioned functions
f|x̄| (x, n, µ1 ) and fk2s (x, n), that is:

Z +∞
T IHaµ1 (AT L, n, µ1 ) = fk2s (AT L − x, n)f|x̄| (x, n, µ1 )dx (13)
0

Let’s now introduce the parameter β as the maximum probability of accepting the
null hypothesis H0 when in fact Haµ1 is true.
µ1 is the solution of the following equation:

Z ASL
β= T IHaµ1 (z, n, µ1 )dz (14)
0

With n determined according the iterative process described in sect. 2.5 by

equations (9) and (10)
Finally the test will have a power = 1 - β when the alternative hypothesis will imply
the offset ∆µ = µ1 from the null hypothesis mean.

With a similar approach we can take into consideration another particular alter-
native hypothesis Haσ1 where: Haσ1 :(µ0 = 0 ; σ1 > σ0 ). That is, only the standard
deviation σ deviates from the null hypothesis.
More in detail we would like finding that σ1 = σ0 *∆σ with ∆σ >1 leading to an
assigned Type II error probability β.
In order to calculate σ1 , leading to a β% probability to accept the null hypothesis,
we need to follow the same approach above described.
The equation (11) becomes:

12
 σ1
f|x̄| (x, n, σ1 ) = 2N orm(x, 0, √ ); x ≥ 0 (15)
n

While for the term k2*s:

√ √
n−1 n−1
fk2s (x, n, σ1 ) = χ(x , n − 1) (16)
k2 ∗ σ1 k2 ∗ σ1

Then:

Z +∞
T IHaσ1 (AT L, n, σ1 ) = fk2s (AT L − x, n, σ1)f|x̄| (x, n, σ1 )dx (17)
0

σ1 is the solution of the following equation:

Z ASL
β= T IHaσ1 (z, n, σ1 )dz (18)
0

Finally the test will have a power = 1 - β when the alternative hypothesis will
imply a standard deviation ratio ∆σ = σ1 /σ0

2.7. Numerical solution and simulation of alternative hypothesis

In similar way to section 2.4, the following section will show the simulation results for
the three Hypothesis:

• H0 : (µ0 = 0; σ0 = 1); The null hypothesis

• Haµ1 : (µ1 ; σ0 = 1); The alternative hypothesis based on µ deviation
• Haσ1 : (µ0 = 0; σ1 ); The alternative hypothesis based on σ deviation

Every sample iteration is always obtained calculating tolerance limits with equa-
tion (1) for each sample size n.
In Table 4 and 5 are reported output parameters and simulation results for all
sample sizes and data input combinations already taken in consideration for Table 3,
where the following additional parameters are shown :

• Pwr = (1 - β) = the probability to reject the null hypthesis when the alteranative
hypothesis Haµ1 or Haσ1 are true.
• µ1 = the mean deviation correspondent to the specified Power.
• σ1 = the σ deviation correspondent to the specified Power.
• Pµ1 = the proportion of population falling within the specification limits when
the alternative hypothesis Haµ1 is true.

13
 • Pσ1 = the proportion of population falling within the specification limits when
the alternative hypothesis Haσ1 is true.
• αe is the experimental % probability to reject the Null hypothesis H0 when H0
in fact is true
• Pwrµ1 = 100 - β(%) is the experimental % probability to reject the null hypoth-
esis H0 when in fact Haµ1 is true.
• Pwrσ1 = 100 - β(%) is the experimental % probability to reject the null hypoth-
esis H0 when in fact Haσ1 is true.

In Tables 4 the testing factors in the rightmost columns group are calculated solving
the equations (7), (14) and (18), while in the rightmost columns group of Table 5 are
reported the simulation results always obtained by N=100,000 trials. All of these
results are obtained by the attached R program ”Tolerance interval hypothesis test
sigma.r”. The population Pµ1 and Pσ1 are calcuated according the following equations:

U SL − µ1 LSL − µ1
Pµ1 = Φ( ) − Φ( ) (19)
σ0 σ0
U SL LSL
Pσ1 = Φ( ) − Φ( )
σ1 σ1

where Φ(·) indicates the the standard normal cumulative distribution function.
Table 4. Testing factors and alterantive hypotheis deviations
ASL P% γ% α% Pwr% ASLtest k2 n µ1 σ1 Pµ1 % Pσ1 %
4 95 95 5 95 3.983 2.903 16 2.061 1.822 97.4 97.2
4 95 95 1 95 3.956 2.723 21 2.041 1.841 97.5 97
4 99 99 5 99 3.997 3.344 54 1.465 1.488 99.4 99.3
4 99 99 1 99 4 3.231 69 1.467 1.498 99.4 99.2
4 99.9 99.9 1 99.9 3.999 3.666 446 0.737 1.205 99.94 99.91
5 95 95 5 95 4.923 3.379 10 2.978 2.17 97.8 97.9
5 95 95 1 95 4.828 3.081 13 2.899 2.176 98.2 97.8
5 99 99 5 99 4.99 3.904 25 2.453 1.81 99.5 99.4
5 99 99 1 99 5 3.706 31 2.466 1.83 99.4 99.4
5 99.9 99.9 1 99.9 4.991 4.202 101 1.739 1.481 99.94 99.93

Table 5. Simulation results for H0 , Haµ1 and Haσ1

ASL P% γ% α% Pwr% αe % Pwrµ1 % Pwrσ1 %
4 95 95 5 95 5.07 95.1 95.0
4 95 95 1 95 0.96 95.1 95 0
4 99 99 5 99 5.06 99.0 99.0
4 99 99 1 99 1.03 99.0 99.0
4 99.9 99.9 1 99.9 1.00 99.9 99.9
5 95 95 5 95 5.09 95.1 95.0
5 95 95 1 95 1.06 94.9 95.0
5 99 99 5 99 5.07 99.0 99.0
5 99 99 1 99 0.96 99.0 99.0
5 99.9 99.9 1 99.9 1.02 99.9 99.9

14
 From Table 4 and 5 we can observe that the populations Pµ1 and Pσ1 falling within
the specification limits LSL USL, in relation to the alternative hypothesis Haµ1 and
Haσ1 , are always greater than the correspondent population of the tolerance interval.
Furthermore these alternative hypothesis occurrences impliy an high rejection proba-
bility, the test power. Based on these considerations we can derive that the tolerance
interval coverage P and the confidence γ are no more crucial parameters for our aim,
but rather the minimum population proportion falling inside the specification limits
when the correspondent alterantive hypothesis can be detected with a specified power.
It’s worth noting that in this section the specification limit ASLtest , µ1 and σ1 are
output results of input data ASL, n, P, γ α and β.
ASLtest limit, that is always lower or equal the initial specification, can be used for
the verification test based on the following limits:

U SLtest = µ0 + ASLtest
LSLtest = µ0 − ASLtest

The R program ”Tolerance interval hypothesis test sigma.r”, starting from inputs
n, P, γ α an β, calculates µ1 , σ1 , ASLtest and k2 for test execution.

3. A new approach

Starting from the considerations of the previous sections, we can derive that we are no
more interested about the tolerance interval population coverage and confidence of the
tolerance interval itself, but we are looking for testing factors determination starting
from different constraints.
What we are looking for is the set of testing factors: ASLtest , k2 and the minimum
sample size n, satisfying the followings desired constraints:
• When the null hypothesis H0 is true the probability to reject it is equal to the
assigned value α (the Type I error).
• When the alternative hypothesis Haσ1 is true, we want the populations Pσ1 falling
within the specification limits LSL, USL not lower an assigned value Pσ1set , with
a probability not higher than the assigned value β (the Type II error), to accept
the null hypothesis with Pσ1 < Pσ1set .
• When the alternative hypothesis Haµ1 is true, we want the populations Pµ1 falling
within the specification limits LSL, USL not lower an assigned value Pµ1set , with
a probability not higher than the assigned value β (the Type II error), to accept
the null hypothesis with Pµ1 < Pµ1set .
Given Pσ1set and Pµ1set data input, we can obtain σ1set and µ1set as deviation
parameters, solving the following equations:

U SL − µ1set LSL − µ1set

Pµ1set = Φ( ) − Φ( ) (20)
σ0 σ0
U SL LSL
Pσ1set = Φ( ) − Φ( )
σ1set σ1set

The testing factors ASLtest , k2 and n are calculated finding the solutioon of the
following equations system:

15
 Z ASLtest
(1 − α) = T IH0 (z, n, µ0 , σ0 , k2)dz (21)
0

Z ASLtest
β= T IHaσ1 (z, n, µ0 , σ1set , k2)dz (22)
0

Z ASLtest
β= T IHaµ1 (z, n, µ1set , σ0 , k2)dz (23)
0

where the probability density functions T IH0 , T IHaσ1 and T IHaµ1 are calculated
according the former (6), (17) and (13). The above equantions system is solved by a
numerical reiterated algorithm with three nested loops. In the innermost loop ASLtest
is changed in order to fulfill the equation (21), in the middle loop the sample size n
is changed to fulfill the equation (22), while in the outermost loop the k2 factor is
changed to fulfill the equation (23). In the attached R program ”Testing factors with
sym.r” the solutions accuracy is controlled as desired setting an appropriate number
of iterations for each loop. It’s worth noting that for the middle loop (22), solving
the sample size n, the n solution accuracy implies an error , due to n rounded by
excess to the nearest integer, with 0 ≥  < 1. So, the correspondent deviation σ1 will
be generally lower than σ1 set, moreover the difference σ1set − σ1 and consequently
Pσ1 − Pσ1set , increases with decreasing of n.
Concerning instead the remaining solutions ASLtest and k2, the related errors can be
reduced as far as needed, properly increasing the specific loop iterations counter.

According this new approach, we refer in Table 6 a similar set of input parameter
as in Table 4, where we replaced the tolerance interval population P and confidence γ,
with populations Pµ1set and Pσ1set , both of them assumed equal to the correspondent
P value of Table 4.
Pµ1set and Pσ1set fall within the specification limits when the related alternative hy-
potheses Haµ1 and Haσ1 imply a power Pwr = 1 - β to reject the null hypothesis.
The Table 6 below shows the solution parameters solving the above equations sys-
tem (21), (22) and (23)
Table 6. Testing factors according the new approach
ASL Pµ1set % Pσ1set % α% Pwr% ASLtest k2 n µ1 σ1 Pµ1 % Pσ1 %
4 95 95 5 95 4.071 2.841 12 2.355 2.013 95.00 95.31
4 95 95 1 95 4.148 2.726 16 2.355 2.013 95.00 95.31
4 99 99 5 99 4.263 3.519 45 1.674 1.547 99.00 99.03
4 99 99 1 99 4.286 3.409 58 1.674 1.553 99.00 99.00
4 99.9 99.9 1 99.9 4.812 4.402 402 0.91 1.215 99.90 99.90
5 95 95 5 95 5.114 3.379 8 3.355 2.412 95.00 96.18
5 95 95 1 95 5.122 3.101 10 3.355 2.443 95.00 95.93
5 99 99 5 99 5.079 3.895 21 2.674 1.92 99.00 99.08
5 99 99 1 99 5.071 3.667 26 2.674 1.938 99.00 99.01
5 99.9 99.9 1 99.9 5.247 4.38 90 1.91 1.517 99.90 99.90

16
 Now, examining Table 6, we can notice that ASLtest is no more constrained to
be lower than ASL, but is always greater, while the sample sizes n are lower than
the values seen in Table 4. On the other hand, now the popuations Pµ1 and Pσ1
falling inside the specification limits are very close to the correspondent expected
values Pµ1set and Pσ1set . In particular only Pσ1 shows a slight mismatch due to the
considerations described above about the sample size rounding.

As done in Table 5 of the previous section, in Table 7 below are shown the simulation
results with the testing factors of Table 6
Table 7. Simulation results for H0 , Haµ1 and Haσ1 for the new approach
ASL Pµ1set % Pσ1set % α% Pwr% αe % Pwrµ1 % Pwrσ1 %
4 95 95 5 95 5.01 95.0 95.0
4 95 95 1 95 0.95 95.2 95 1
4 99 99 5 99 4.99 99.1 98.9
4 99 99 1 99 1.03 99.0 99.0
4 99.9 99.9 1 99.9 0.98 99.9 99.9
5 95 95 5 95 4.98 95.0 95.0
5 95 95 1 95 1.02 95.0 95.0
5 99 99 5 99 4.94 99.0 99.1
5 99 99 1 99 1.01 99.0 99.0
5 99.9 99.9 1 99.9 1.04 99.9 99.9

In the next Table 8 a significant sample size reduction is obtained applying asym-
metric values for Pµ1set and Pσ1set , where in particular is Pσ1set slightly lower than
Pµ1set .
Table 8. Testing factors with asymmetric deviation sensitivity
ASL Pµ1set % Pσ1set % α% Pwr% ASLtest k2 n µ1 σ1 Pµ1 % Pσ1 %
4 95 90 5 95 3.35 2.117 8 2.355 2.398 95.00 90.47
4 95 90 1 95 3.507 2.086 11 2.355 2.344 95.00 91.21
4 99 95 5 99 2.706 1.953 19 1.674 2.012 99.00 95.32
4 99 95 1 99 2.746 1.867 24 1.674 2.028 99.00 95.14
4 99.9 99 1 99.9 2.068 1.654 92 0.908 1.549 99.90 99.02
5 95 90 5 95 4.516 2.779 6 3.355 2.828 95.00 92.29
5 95 90 1 95 4.444 2.422 7 3.355 2.952 95.00 90.97
5 99 95 5 99 3.773 2.576 11 2.674 2.55 99.00 95.01
5 99 95 1 99 3.874 2.463 14 2.674 2.515 99.00 95.32
5 99.9 99 1 99.9 3.476 2.598 38 1.91 1.935 99.90 99.02

Now, ASLtest is alawys lower than ASL. This asymmetric approach could be at-
tractive when a lower sensitivity to σ deviation is acceptable. In Table 9 below, the
simulation results for this case are shown.

17
Table 9. Simulation results for H0 , Haµ1 and Haσ1 in asymmetric case
ASL Pµ1set % Pσ1set % α% Pwr% αe % Pwrµ1 % Pwrσ1 %
4 95 90 5 95 5.08 95.0 95.0
4 95 90 1 95 1.06 94.9 95 1
4 99 95 5 99 4.96 99.1 99.0
4 99 95 1 99 1.06 99.0 99.0
4 99.9 99 1 99.9 1.01 99.9 99.9
5 95 90 5 95 4.96 95.0 94.9
5 95 90 1 95 1.00 95.0 95.1
5 99 95 5 99 5.09 99.0 99.0
5 99 95 1 99 1.01 99.0 99.0
5 99.9 99 1 99.9 0.97 99.9 99.9

4. Conclusions

When a verification program requires testing a bivariate null hypothesis referring the
mean µ and standard deviation σ of a normal process, a novel tolerance interval based
approach is proposed.
The sample size is determined such that the desired Type I = α and Type II = β
errors probabilities, however always taking into account the borderline populations
Pµ1 and Pσ1 falling within the specification limits, with the assigned statistical power
(1 - β) to reject the null hypothesis.
Finally the input data are:
• Null hypothesis H0 by (µ0 , σ0 ). Where µ0 and σ0 are preliminary estimations
obtained with a pilot study or theoretical considerations.
• The borderline population Pµ1 falling within the specification limits LSL, USL.
This is consequence of the deviation µ1 in case of alternative hypothesis
Haµ1 :(µ0 + µ1; σ0 ).
• The borderline population Pσ1 falling within the specification limits LSL, USL.
This is consequence of the deviation σ1 /σ0 in case of alternative hypothesis
Haσ1 :(µ0 ; σ1 ).
• Significance level α (Type I error). That is the probability to reject the null
hypothesis when in fact it is true.
• Power = 1 -β (1 - Type II error). That is the probability to reject the null
hypotheis when in fact one of the alternative hypothesis Haµ1 or Haσ1 is true.
The population covered by the specification limits will be always greater than Pµ1
and Pσ1 when the null hypothesis is true or when the alternative hypotheses will be
met with deviation lower than µ1 and σ1.
From a risk management perspective, we can summarize the problem in this way:
- There is a manufacturer risk that is provided by the combination of significance level
α probability and the verification cost.
- There is a design (or customer) risk, based on the combination of β probability and
the potential harms due to the alternative hypothesis deviations µ1 and σ1. The risk
management of the two issues above mentioned can be an essential element for the
sample size selection. Finally, the claim of this paper is: a tolerance interval with a
confidence γ to cover the population P inside its random limits LTL UTL, unlikely
leads to an optimal criterion to verify if a minimum expected population is falling
inside the assigned specification limits LSL USL.

18
The proposed approach instead provides the following advantages:
• sets a minimum population that falls within the specifications, implying the test
failing with the pre established power if this requirement is not met
• provides the parameters to gain full control about costs (n) and risks (n ∗ α;
µ1 ∗ β; σ1 ∗ β).
• provides accurate results when specific level of safety or reliability are required.

References

[1] Komitee ISO/TC 69, ISO3534-1:Statistics-Vocabulary and symbols-Part 1: General sta-

tistical terms and terms used in probability, DIN Deutsches Institut fr Normung, Berlin,
Deutschland,2006, available at http://www.din.de/
[2] Komitee ISO/TC 69, ISO16269-6: Statistical interpretation of data - Part 6: Determina-
tion of statistical tolerance intervals, DIN Deutsches Institut fr Normung Berlin, Deutsch-
land, 2009, available at http://www.din.de/
[3] S.S. Wilks, Determination of Sample Sizes for Setting Tolerance Interval Limits, The
Annals of Mathematical Statistics, Vol. 12 (1941), pp. 91–96.
[4] A. Wald and J. Wolfowitz, Tolerance Limits for a Normal Distribution, The Annals of
Mathematical Statistics, Vol. 17,(1946), pp. 208-215
[5] B.E. Ellison, On Two-Sided Tolerance Intervals for a Normal Distribution, The Annals
of Mathematical Statistics, Vol. 35 (2003), pp. 762–772.
[6] R.L. Kirkpatrick Sample Sizes to Set Tolerance Limits, Journal of Quality Technology,
Vol. 9, (1977), pp 6-12
[7] R.E. Odeh and Y-M Chou and D.B. Owen, Sample-Size Determination for Two-Sided
beta-Expectation Tolerance Intervals for a Normal Distribution, Vol. 31, Technometrics,
(1989), pp. 461-468
[8] W. Hauck and R. Shaikh, Sample Sizes for Batch Acceptance from Single and Multistage
Design using Two-Sided Normal Tolerance Intervals with Specified Content,Journal of
Biopharmaceutical Statistics, Vol. 11, (2001), pp. 335-346
[9] S. De Gryze and I. Langhans and M. Vandebroek, Using the correct intervals for predic-
tion: A tutorial on tolerance intervals for ordinary least-squares regression, Chemomet-
rics and intelligent laboratory systems, Journal of Biopharmaceutical Statistics, Vol. 87,
(2007), pp. 147-154
[10] T.P. Ryan, Sample Size Determinatiobn and Power, John Wiley & Sons, Hoboken, New
Jersey, US, (2013)
[11] W.Q. Meeker and G.J.Hahn and L.A. Escobar, Statistical Intervals: A Guide for Practi-
tioners and Researchers, John Wiley & Sons, Hoboken, New Jersey, US, (2017)
[12] Steven P. Millard, Package for Environmental Statistics, Including US EPA Guidance,
Repository CRAN, (2017).
[13] K. Knight, Mathematical Statistics, Cahapman & Hall/CRC, Boca Raton, Florida, US,
(1999)
[14] Faulkenberry, G. D., and D. L. Weeks, Sample size determination for tolerance limits,
Technometrics 10 (2): 343348, (1968)
[15] Derek S. Young, Charles M. Gordon, Shihong Zhu & Bryan D. Olin Sample size determi-
nation strategies for normal tolerance intervals using historical data, Quality Engineering,
(2016)

19
 5. Appendices

Appendix A. Tolerance interval analysis.r

See attached file

Appendix B. Sample size by spec.r

See attached file

Appendix C. Tolerance Interval hypothesis test sigma.r

See attached file

Appendix D. Testing factors with sym.r

See attached file

20

View publication stats