This article was downloaded by: [Michigan State University]

On: 26 December 2014, At: 12:57

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41
Mortimer Street, London W1T 3JH, UK

Journal of the American Statistical Association

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/uasa20

A Test of Missing Completely at Random for Multivariate

Data with Missing Values
a
Roderick J. A. Little
a
Department of Biomathematics , School of Medicine, University of California , Los Angeles , CA ,
90024 , USA
Published online: 12 Mar 2012.

To cite this article: Roderick J. A. Little (1988) A Test of Missing Completely at Random for Multivariate Data with Missing Values,
Journal of the American Statistical Association, 83:404, 1198-1202

To link to this article: http://dx.doi.org/10.1080/01621459.1988.10478722

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the
publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or
warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed
by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with
primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings,
demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly
in connection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is
expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
 A Test of Missing Completely at Random for
Multivariate Data With Missing Values
RODERICK J. A. LITTLE*

A common concern when faced with multivariate data with missing values is whether the missing data are missing completely
at random (MCAR); that is, whether missingness depends on the variables in the data set. One way of assessing this is to
compare the means of recorded values of each variable between groups defined by whether other variables in the data set are
missing or not. Although informative, this procedure yields potentially many correlated statistics for testing MCAR, resulting
in multiple-comparison problems. This article proposes a single global test statistic for MCAR that uses all of the available
data. The asymptotic null distribution is given, and the small-sample null distribution is derived for multivariate normal data
with a monotone pattern of missing data. The test reduces to a standard t test when the data are bivariate with missing data
confined to a single variable. A limited simulation study of empirical sizes for the test applied to normal and nonnormal data
suggests that the test is conservative for small samples.
KEY WORDS: Incomplete data; Multivariate normal distribution; Nonresponse.

1. INTRODUCTION birthpill (1 = user, 2 = nonuser), cholesterol, albumin,

Downloaded by [Michigan State University] at 12:57 26 December 2014

and calcium. About 20% of the values of each of the latter

Many statistical analyses of data with missing values
four variables were randomly deleted; weight was missing
make the assumption that data are missing completely at
for two cases in the original data set. Of the p ( p - 1) =
random (MCAR), in the sense that missingness does not
(6)(5) = 30 possible pairwise t statistics for testing MCAR,
depend on the values of variables in the data set subject
the five that split the sample by whether age was missing
to analysis. Nevertheless, formal tests of MCAR have not
are vacuous, since age is never missing. Also, the five that
received much attention. When missing values are con-
split the sample by whether weight was missing are dis-
fined to a single variable y , the standard procedure is to
carded, since for these the nonrespondent group consists
compare the distributions of fully observed variables for
of only two cases. A stem-and-leaf plot of the remaining
respondents and nonrespondents to y , either informally or
20 t statistics is shown in Figure 1; for the sample size
formally, via t tests for the differences in means. In a
studied they can be viewed as normal deviates. Note that
regression setting, Simon and Simonoff (1986) considered
the extreme t statistics (-2.3, 2.4, 2.7, 3.3) might be re-
a sensitivity analysis of deviations from the MCAR as-
garded as evidence against MCAR, although an MCAR
sumption (which they call missing at random) when miss-
deletion mechanism was in fact employed.
ing values are confined to a single independent variable.
Dixon (1983) in the program BMDP8D extended the t- I propose a single test statistic for testing MCAR and
test approach to multivariate data with missing values on show that its null distribution is asymptotically chi-squared.
any of p variables. For each variable with missing values, For the data in the example, the statistic takes the value
the sample is split into cases with that variable observed 76.5 on 60 df ( P = .074), suggesting that the evidence
and cases with that variable missing. The means of ob- against random missingness is in fact weak. Before de-
served values of the other variables in the two groups are scribing the test statistic, I discuss the MCAR assumption
then compared by two sample t tests. Significant differ- in more detail.
ences between these means are evidence that the data are
not MCAR. This procedure is informative, but yields up 2. FORMAL DEFINITIONS OF RANDOMLY
to ( p - 1) t tests for each variable in the data set, or up MISSING DATA
to p ( p - 1) t statistics for assessing the MCAR assump- Let y denote an ( n x p ) data matrix of n observations
tion. The difficulties of simultaneous inference are con- on p variables and r an ( n x p ) missingness indicator
siderable, since the t statistics are correlated with a com- matrix, such that r,, = 1 if y,, is missing and 0 otherwise.
plex correlation structure depending on the pattern of A full model for the data and the missing-data mechanism
missing data and the correlation matrix of the y variables. specifies a distribution f(y I 0) for y, indexed by unknown
Example: Blood Chemistry Data W i t h Values De- parameters 0, and a distribution f(r I y, JI) for r, given y,
leted. As an illustration, I consider the Werner blood- indexed by unknown parameters JI. Write y = (yo,,,, y,,,),
chemistry data (Werner, Tolls, Hultin, and Mellecker 1970; where Yob, represents the observed values of y and y,,,
see Dixon 1983, table 5.1) with values randomly deleted. represents the missing values. Rubin (1976) defined the
The data record eight variables for n = 188 women. Six missing data as MCAR if f(r I Yobs, ymls,JI) = f(r I JI) for
of the variables were selected for our purposes: age, weight, all yobsand y,,,; that is, missingness does not depend on
the observed or missing values of y. Rubin also defined a
weaker condition for the missing-data mechanism, calling
* Roderick J. A. Little is Professor, Department of Biomathematics,
School of Medicine, University of California, Los Angeles, CA 90024.
This work was supported by National Institute of Mental Health Grant 0 1988 American Statistical Association
USPHS MH 37188. The author thanks the referees and associate editors Journal of the American Statistical Association
for several helpful suggestions. December 1988, Vol. 83, No. 404, Theory and Methods

1198
 Little: A Test c Missing Completely at Random 1199

3-3.9 3 3. A TEST OF MCAR FOR MULTIVARIATE DATA

2-2.9 047
1-1.9 4 3.1 Notation
0-.9 3457888 I use the following notation: y, = (1 x p ) vector of
-.9--0 322
-1.9--1 500 values for case i, in the absence of missing data. r, = (1
-2.9--2 31 x p ) vector of missing-data indicators for case i. J =
- 3.9-- 3 number of distinct missing-data patterns ifin the data set.
- 4.9-- 4 Fully observed cases, if present, count as a pattern. S, =
Figure 1. Distribution of Pairwise t Statistics for Werner Data With
set of cases with missing-data pattern j ( j = 1, . . . , J ) .
Data Deleted by an MCAR Mechanism. m, = number of cases in S,; Cm, = n. p , = number of
observed variables for cases in S,. D, = ( p x p,) matrix
indicating which variables are observed for pattern j . The
the missing data missing at random (MAR) if f(r I Yobs, matrix has one column for each variable present, consist-
ymis, +) = f(r I Yobs, +) for all ymis; that is, missingness ing of p - 1 0s and one 1 corresponding to the variable
does not depend on the missing values ymisof y but may identified. Yobs., = (1 x p , ) vector of values of observed
depend on observed values in the data set. Rubin showed variables in case i. Fobs,, = m;' EfES,Yobs,, = (1 x p,) vector
+
that if the missing data are MAR and 8 and are distinct, of means of observed variables for pattern j . p, I; = (1
then likelihood inference for 8 can be based on the like-
Downloaded by [Michigan State University] at 12:57 26 December 2014

X p ) population mean vector and ( p x p ) covariance

lihood obtained by integrating ymisout of the density f(yobs, matrix of y,. fi, 2 = ML estimates of p and I;, assuming
ymis I O), without including a term for the missing-data the y, are iid normal and the missing-data mechanism is
mechanism in the likelihood. Under these conditions, Rubin ignorable. 2 = n%/(n - l ) , the ML estimate of I; with
called the missing-data mechanism ignorable for likeli- a correction for degrees of freedom. pobs,, = pD, = (1 X
hood-based inferences. p,) vector of means of observed variables in patternj. I;obs,,
Tests for the MAR assumption occur in the literature = D,TD, = ( p , x p,) covariance matrix of observed vari-
on models for selectivity bias (see Amemiya 1985; Heck- ables in pattern j .
man 1976; Olsen 1980) but are highly sensitive to model
misspecification (e.g. , see Little 1985). The procedures in 3.2 A Likelihood Ratio Test Statistic, Assuming 2
this article test the stronger MCAR assumption. Such tests Is Known
are too restrictive for testing whether the missing-data
To motivate the test statistic I first consider the (un-
mechanism is ignorable for likelihood inferences, since the
realistic) case where I; is known. Let p* denote the ML
latter requires MAR, not MCAR. Nevertheless, they are
estimate of p, assuming the missing data are MAR and
useful for other purposes. known 2, and let p&, = p*D,. I propose the following
First, many simple missing-data methods, including re- test statistic for the MCAR assumption:
striction to complete cases and pairwise methods, gener-
ally require the MCAR assumption (Little and Rubin 1987, J
chap. 3). Second, maximum likelihood (ML) estimation di = j = l mj(?obs., - p,*bs.j)I;Lb's.j(?obs.j - p,*s,j)T. (1)
from data with missing values based on ignorable missing-
data models do not require the MCAR assumption, but Suppose that yi is multivariate normal distributed with
are more sensitive to model misspecification when the data mean p and covariance matrix I;. If the data are MCAR,
are not MCAR. In particular, ML estimation for the multi- then conditional on ri,
variate normal model (Orchard and Woodbury 1972) pro-
vides consistent estimates of 8 under mild assumptions
(finite fourth moments) when the data are MCAR, but
uses the multivariate normal assumptions when the data (2)
are MAR but not MCAR, and does not in general supply If the data are not MCAR then the means of the observed
consistent estimates when the data are not MAR. Finally, variables can vary across the patterns, suggesting the al-
standard errors for the parameter estimates based on the ternative model
expected information matrix are valid only if the data are
MCAR. Standard errors based on the observed infor- (Y0bs.i I ri) i;;d N ( v o b s . , , I;obs.j), sj7 j I7
mation matrix are preferable when the data are not MCAR,
since they remain valid when the data are MAR but not (3)
MCAR; however, for the multivariate normal model they where {vobs.j, j = 1, . . . , J } are (1 x pi) vectors of mean
require more computation, particularly for the mean pa- parameters for observed variables that (unlike p 0 b s . j ) are
rameters. Hence a test for MCAR provides guidance as distinct for each pattern j . Note that the variances and
to when standard errors based on the expected information covariances are assumed the same for each pattern; the
matrix are adequate. [Note that even when the data are case where they too are allowed to vary is considered in
MCAR arguments can be advanced for preferring stan- Section 4.
dard errors based on the observed information; see Efron The statistic dg tests Model (2) against the alternative
and Hinkley (1978).] model (3), as the following lemma shows.
 1200 Journal of the American Statistical Association, December 1988

Lemma. (a) d i is the likelihood ratio statistic for test- swept covariance matrix computed by EM (see Little and
ing Model (2) against the alternative (3). (b) Under (2), Rubin 1987).
d i has a chi-squared distribution with f = Cp, - p df. (c)
If the data are MCAR and y, has any distribution with 3.4 The Test Statistic for Monotone Missing Data
mean p and covariance matrix 2,d i is asymptotically chi- The small-sample null distribution of d2 is extremely
squared with f df. That is, for large samples the assumption complex for a general pattern of missing data, but sim-
of normality in (2) can be relaxed. plifies for particular missing-data patterns. Consider first
Proof. The log-likelihood under (3) (to within a con- the special case of p = 2 variables Yl and Y2,where Yl is
stant) is observed for all n cases and Y2is observed for n2< n cases,
say i = 1, . . . , n2. There are J = 2 patterns: Pattern 1
l(v I Yobs)
denotes cases with Yl and Y2present and pattern 2 denotes
1 ’ cases with only Yl present. Then, Y0bs.i = (yil, yiz) for i =
= -- 2
J-1
mj(yobs,, - vobs,j)Zob&(yobs.j - vobs.j)T, (4) 1, . . . , n2 and yobs,i= yil for i = n2 + 1, . . . , n; yobs,l
= (Fl, J 2 ) , the sample means of Yl and Y2based on the
where v denotes the set of means for all of the patterns.
- first n2 cases; yobs,2 = jjt, the sample mean of Yl based on
Substituting ML estimates = Y0bs.l yields l(* I yobs) =
the last n - n2 cases; Zobs.1 = 2; and 2 0 b s . 2 = 611.n u s
0. Substituting ML estimates of the means under Model
( 5 ) becomes
(2) yields 1(p*I Yobs) = -di/2. Hence the likelihood ratio
Downloaded by [Michigan State University] at 12:57 26 December 2014

test statistic is -2[1(p* I - I(+ I Yobs)] = d i , proving

(4.
To prove (b), concatenate the pattern mean vectors
{Fobs,], j = 1, . . . , J } into a single 1 x Cp, vector, and
note that under Model (2) this vector is normal with mean + (n - n2)(LT - Fd2/6l1, (6)
pX and known covariance matrix r(Z),where X is a which can be rewritten as
known p x Cp, matrix of 0s and Is, and r is a known
matrix, since 2 is assumed known. It is easily seen that n2G1 - PI)’ + 472 - Fz - Ij2l.dL1 - P111’
p* is the generalized least squares estimate of p and d i 611 622 1
is the residual sum of squares. Hence by standard least
squares theory, under (2) the distribution of d i conditional + (n - nz)(L: - P1)’
* (7)
on the response pattern (rl, . . . , rn) is chi-squared with g11

f = Cp, - p df. Explicit ML estimates of the parameters are available for

Finally, to prove (c) let the number of observations for this problem (see Anderson 1957; Little and Rubin 1987,
each observed pattern tend to infinity, ignoring patterns chap. 6). Substituting these in (7) yields, after a little al-
that do not appear in the n observed cases. The pattern gebra,
mean vectors Fobs, tend to normality by the central limit
theorem, so d i tends to a chi-squared deviate with f df, d2 = [n2(% - P1)’ + (n - n2)(Ll* - P 1 ) 2 1 / 6 1 1
by the proof of (b). = SSBl/MSTI = (n - 1)F/(n - 2 + F),
3.3 A Test Statistic When I; Is Unknown where SSBl, MSTl, and Fare, respectively, the between-
groups sum of squares, the total mean square, and the F
When Z is not known, I propose replacing p* and Z in
statistic from the analysis of variance (ANOVA) of Yl on
(1) with fi and 2 from the multivariate normal ML al-
the missing-data pattern. Since there are just two patterns
gorithm, yielding the test statistic
here, F = t2, where t is the t statistic for comparing pattern
J
means discussed in Section 1. Hence the test based on d 2
d2 = mj(Foh.j - f i o b s , , ) ~ ~ b ~ . ) ( ~ o b -
s . j fiobs.j)T. ( 5 ) is equivalent to the t test. Under the null hypothesis of
,=1
MCAR and assuming that the values of Yl are normal, F
Suppose that the observed data contain information on all has an F distribution with 1 and n - 2 df.
pairs of variables so that all of the means, variances, and More generally, suppose that the data can be arranged
covariances are estimable. If the data are MCAR and the in a monotone pattern, where variable Yqis more observed
distribution of yl has finite fourth moments, 3 is a consis- than Yq-lfor q = 1, . . . ,p - 1 (Rubin 1974). Then, if
tent estimate of Z. Hence under these conditions, d 2 ,like n4 is the number of cases for which Yq is observed, n =
d i , is asymptotically chi-squared distributed withfdf. This n12 n2 2 2 np.A generalization of the previous analysis
-3.

result follows from the multivariate analog of a theorem yields

of Cramer (1946, sec. 20.6). Thus a large-sample test of
the MCAR assumption compares d2 with a chi-squared +
d 2 = SSBl/MSTl SSB2.1/MST2,1 +
distribution with f df, rejecting when d 2 is large.
The computation of fi and 2 is iterative, but the EM
algorithm is easy to program with the aid of the SWEEP
operator and is available in current software (see the
BMDPAM program of Dixon 1983). The additional cal-
culation for ( 5 ) is trivial, since $obis, is a submatrix of the
 Little: A Test of Missing Completely at Random 1201

where nq is the number of cases with Yq observed; k, is Table 1 shows the empirical sizes of the test for each
the number of patterns with Y, observed; SSB1, MSTl, problem. For example, 20.2 in the table indicates that for
and Fl are, respectively, the between-groups sum of squares, this problem the null hypothesis of MCAR was rejected
total mean square, and F statistic from the ANOVA of Yl at the 20% level in 202 out of 1,000 data sets. Superscripts
on all kl patterns; SSB2I , MST, and F2 I are the between- a and b indicate that the empirical size differs significantly
groups sum of squares, total mean square, and F statistic from the nominal size at, respectively, the 1% and 5%
from the analysis of covariance of Y2 on all k2 patterns levels of significance.
with Y2observed, adjusting for Y l ;and the remaining terms The empirical sizes do not differ significantly from nom-
are defined similarly. Under normality and MCAR, each inal levels for the data sets with 80 observations. For the
of the contributions in (8) is independent, so the small- smaller sample sizes the test appears overly conservative,
sample null distribution of d2 is a sum of functions of particularly at the lower nominal levels. An encouraging
independent F statistics. In large samples, these functions feature of the results is the relatively small impact of non-
become chi-squared distributed, and d2has the asymptotic normality (in the form of long tails or skewness) on the
chi-squared distribution discussed in Section 3.3. empirical sizes, suggesting a fair degree of robustness for
the method. This reinforces the fact that asymptotically
3.5 Simulation Study the test does not require normality.
A limited simulation study was conducted to examine These results on size should be treated as suggestive
Downloaded by [Michigan State University] at 12:57 26 December 2014

the empirical size of tests based on d2, for incomplete rather than definitive, given the modest scope of the sim-
multivariate normal, skewed (lognormal), and long-tailed ulation study. Power calculations are not included, since
(multivariate r3) data sets with n = 20, 40, or 80 obser- the power depends greatly on what departures from the
vations. To generate the multivariate normal and multi- MCAR assumption are contemplated. For example, in the
variate r data sets, observations i on p = 4 variables Y l , bivariate monotone case of Section 3.4, power may be
Y2,Y,, and Y, were generated from independent standard high if missingness of Y, depends on the fully observed
normal deviates Z1,Z2,Z3,and Z4,using variable Y l . On the other hand, if missingness of Y2de-
pends on Y,, then the test statistic only has good power
Yll = z 1 , / V z ,
if Yl and Y2are highly correlated. For a general pattern
of missing data, the global nature of the test leads to a
y, = z l l m + z 1 2 m , loss of power relative to others that test specific alternative
y13 = Z I l + Z~ I 2 2 / - l / q r + z 1 3 m ,
hypotheses, such as an alternative that specifies that miss-
ingness is a function of a particular variable. In most cir-
and cumstances such specific alternative hypotheses are hard
to formulate with much certainty, so this loss of power
y,, = - Z l 1 +~ z 1 2 m may be tolerated in the interests of achieving a single
+ z 1 3 W z14-. + global test statistic. Since power may be low, it is prudent
to keep in mind that accepting the null hypothesis of MCAR
For the multivariate normal data sets qi = 1 for all i; for does not imply its correctness.
the multivariate r3 data sets qi equals a chi-squared deviate
with 3 df. The resulting data sets all have mean vector (0, 4. DISCUSSION
0, 0, 0) and variances (1, 1, 1, 1). The correlations are p12 I conclude by discussing some limitations of the pro-
= .9487, pi3 = .4472, p23 = .5243, = w.7746, p24 = posed test and outlining some alternative procedures.
- .5767, and pM = .0763, thus encompassing a range of Close relatives of the test statistic d2 in Equation ( 5 ) are
values. The lognormal data sets were obtained by expo- d:, where 3 is replaced by the ML estimate of Z under
nentiating the values (yij) generated for the multivariate the alternative model (3), and d i , the likelihood ratio test
normal case.
Missing data were then created in the data set by an Table 1. Percent Empirical Sizes for a Test of the MCAR
MCAR mechanism such that for every data set exactly Assumption, From N = 1,000 Simulated Data Sets
40% of the cases were complete (i.e., had the pattern ri
~~~

Nominal level of test

= OOOO), 10% of the cases had Y4missing (OOOl), and 10% Sample
of the cases had each of the patterns 0011, 0010, 0110, size Distribution 20% 10% 5% 1%
0100, and 0101. Note that all cases had at least two vari- 80 Normal 20.2 10.9 4.9 .5
ables present. For each of the nine problems generated Lognormal 18.9 8.8 3.7 .7
by the combinations of distribution and sample size, N = ton3df 21.2 11.2 5.5 1.o
1,000 incomplete data sets were generated using the 40 Normal 21.2 9.5 2.5" .2b
Lognormal 18.8 8.9 3.2b .3b
GGNPM and GGUBS subroutines in the IMSL library. t on 3 df 20.8 9.6 4.1 .8
The same set of random numbers was used for each of the
20 Normal 20.3 6.8" 2.8" .3b
nine problems to sharpen comparisons between problems. Lognormal 21.1 8.3 3.5b .5
For each data set the MCAR test statistic was calculated; ton3df 21.6 8.lb 2.0" .3b
for the chosen missing-data pattern it has 15 df. Accept- Standard errors 1.27 .95 .69 .315
ance or rejection of the MCAR hypothesis was recorded a 1% level of significance.
for the 20%, lo%, 5%, and 1%nominal levels. 5% level of significance.
 1202 Journal of the American Statistical Association, December 1988

statistic obtained by subtracting twice the maximized log- lett’s test for comparing dispersions, I expect the test to
likelihoods for Models (3) and (2). These statistics have be sensitive to departures from the normality assumption,
the same asymptotic null distribution as d2. They both and even under normality the asymptotic null distribution
require ML estimates to be calculated under both (2) and seems unlikely to be reliable unless the sample size is large.
(3), and hence involve a bit more computation.
[ReceivedJuly 1986. Revised March 1988.1
We have seen that the test based on d2 is valid asymp-
totically without normal assumptions; however, it seems
most appropriate when the variables are quantitative. If REFERENCES
they are categorical, with data forming a contingency table Amemiya, T. (1984), “Tobit Models: A Survey,” Journal of Economet-
with supplemental margins, more appropriate large-sam- rics, 24, 3-61.
ple tests of the MCAR assumption can be based on the Anderson, T. W. (1957), “Maximum Likelihood Estimators for the
Multivariate Normal Distribution When Some Observations Are Miss-
chi-squared statistics in Fuchs [1982, eqs. (4.1) and (4.2)], ing,” Journal of the American Statistical Association, 52, 200-203.
with estimates of the cell probabilities based on the sat- Cramer, H. (1946), Mathematical Methods in Statistics, Princeton, NJ:
urated model. Princeton University Press.
Dixon, W. J. (ed.) (1983), BMDP Statistical Software, Berkeley: Uni-
An important limitation of d2, d:, and di is that they versity of California Press.
are derived from an alternative hypothesis (3) that allows Efron, B., and Hinkley, D. V. (1978), “Assessing the Accuracy of the
missingness to affect the means but constrains the vari- Maximum Likelihood Estimator: Observed Versus Expected Infor-
mation,” Biometrika, 65, 457-487.
ances and covariances to be the same for all patterns. A
Downloaded by [Michigan State University] at 12:57 26 December 2014

Fuchs, C. (1982), “Maximum Likelihood Estimation and Model Selec-

referee proposed relaxing this limitation. In particular, tion in Contingency Tables With Missing Data,” Journal ofthe Amer-
consider the alternative model ican Statistical Association, 77, 270-278.
Heckman, J. D. (1976), “The Common Structure of Statistical Models
(Y0bs.i I ri) iz N(l)obs.j, robs,j), E sj, i ZS J7 (9) of Truncation, Sample Selection and Limited Dependent Variables
and a Simple Estimator for Such Models,” Annals of Economic and
Social Measurement, 5 , 475-492.
where now the covariance matrices {robs.j},like the means Little, R. J. A. (1985), “A Note About Models for Selectivity Bias,”
{v,,bs,j}, contain distinct parameters for each pattern j . A Econometrica, 53, 1469-1474.
standard calculation yields the following likelihood ratio Little, R. J. A , , and Rubin, D. B. (1987), StatisticalAnalysisWith Missing
Data, New York: John Wiley.
test statistic for (9) relative to (2): Olsen, R. J. (1980), “A Least Squares Correction for Selectivity Bias,”
I Economerrica , 48, 1815- 1820.
Orchard, T., and Woodbury, M. A. (1972), “Missing Information Prin-
= d2 + mj[tr(sobs,jed.j) - pj ciple: Theory and Applications,” in Proceedings of the Sixth Berkeley
j=l Symposium on Mathematical Statistics and Probability (Vol. l), Berke-
ley: University of California Press, pp. 697-715.
Rubin, D. B. (1974), “Characterizing the Estimation of Parameters in
Incomplete Data Problems,” Journal of the American Statistical As-
where S0bs.j is the sample covariance matrix of the observa- sociation, 69, 467-474.
tions with pattern j and aug is augmented. Under (2) d:ugis (1976), “Inference and Missing Data,” Biometrika, 63,581-592.
+
asymptotically chi-squared with Cjpj(pj 3)/2 - p ( p + Simon, G. A , , and Simonoff, J. S. (1986), “Diagnostic Plots for Missing
Data in Least Squares Regression,” Journal of the American Statistical
3)/2 df; this can be a large number (for the data in the Association, 81, 501-509.
simulation study it is 42), raising concerns about the power Werner, M., Tolls, R., Hultin, J., and Mellecker, J. (1970), “Sex and
of the test. Also, data for patterns j with mi5 p j need to Age Dependence of Serum Calcium, Inorganic Phosphorus, Total Pro-
tein, and Albumin in a Large Ambulatory Population,” in Fifth Tech-
be discarded, since for those patterns S0bs.jis singular; thus nical International Congress on Automation, Advances in Automated
S,-d,,jin (10) cannot be computed. By analogy with Bart- Analysis (Vol. 2), Mount Kisco, NY: Futura Publishing, pp. 59-65.