What are Cross-Classified Data?

• Subjects are classified by two or more types of clusters, but

Multilevel Cross-Classified and Multi-Membership Models clusters are not hierarchical or nested within one another

Don Hedeker data collected on who are clustered within

Division of Epidemiology & Biostatistics students schools and neighborhoods
Institute for Health Research and Policy
School of Public Health patients providers and hospitals
University of Illinois at Chicago
Here,
email: hedeker@uic.edu
http://www.uic.edu/∼hedeker/ml.html
• students from the same neighborhoods go to different schools
• patients can be seen by the same provider at different hospitals

1 2

Example: Leckie, G. (2013). Cross-Classified Multilevel Models. LEMMA VLE

Nested or Hierarchical Structure Module 12. (http://www.bristol.ac.uk/cmm/learning/course.html)
School 1 School 2 School 3 School 4 Data from 3,435 children who attended 148 primary schools and 19 secondary
Neighborhood 1 xxx xx schools in Scotland.
Neighborhood 2 xxxx xxxxx • VRQ: A verbal reasoning score from tests pupils took when they entered
secondary school
students nested within schools within neighborhoods
• ATTAIN: Attainment score of pupils at age 16
• PID: Primary school identifying code
Crossed Structure • SEXF: Pupil’s gender (0 = boy and 1 = girl)
School 1 School 2 School 3 School 4 • SC: Pupil’s social class scale (continuous score from low to high social class)
Neighborhood 1 x x x x xx xxx xx • SID: Secondary school identifying code
Neighborhood 2 x x x xx xxxx • FED: Father’s education (0,1)
students nested within crossing of schools and neighborhoods • CHOICE: Choice of secondary school that they attend (1=first choice, ...
4=fourth choice)

⇒ interest is on assessing effects of schools and neighborhoods on • MED: Mother’s education (0,1)
student outcomes
3 4

Excel file: xwdata 9var.xlsx

SAS code

PROC IMPORT OUT= WORK.xw DATAFILE="xwdata_9var.xlsx"

DBMS=EXCEL; GETNAMES=YES;
RUN;

PROC MEANS DATA=xw;

RUN;
The MEANS Procedure

Variable Label N Mean Std Dev Minimum Maximum

------------------------------------------------------------------------------------------
VRQ VRQ 3435 97.8043668 13.2929071 70.0000000 140.0000000
ATTAIN ATTAIN 3435 5.6786026 3.0585043 1.0000000 10.0000000
PID PID 3435 70.7377001 45.0257180 1.0000000 148.0000000
SEXF SEXF 3435 0.4937409 0.5000336 0 1.0000000
SC SC 3435 6.8448326 10.8876068 0 31.0000000
SID SID 3435 10.2195051 5.5569402 1.0000000 19.0000000
FED FED 3435 0.2754003 0.4467808 0 1.0000000
data are sorted by Primary School ID (PID), but appear unsorted in CHOICE
MED
CHOICE
MED
3435
3435
1.1953421
0.3420670
0.6502375
0.4744710
1.0000000
0
4.0000000
1.0000000
terms of Secondary School ID (SID) ------------------------------------------------------------------------------------------

5 6
PROC MIXED analyses (using default REML estimation) Alternative (simpler) syntax (using default REML estimation)
Null model Null model
PROC MIXED DATA=xw COVTEST; PROC MIXED DATA=xw COVTEST;
CLASS pid sid ; CLASS pid sid ;
MODEL attain = / S; MODEL attain = / S;
RANDOM INT / SUB=pid; RANDOM pid sid;
RANDOM INT / SUB=sid; RUN;
RUN;
Model including verbal reasoning score as a covariate
Model including verbal reasoning score as a covariate
PROC MIXED DATA=xw COVTEST;
PROC MIXED DATA=xw COVTEST; CLASS pid sid;
CLASS pid sid; MODEL attain = vrq / S;
MODEL attain = vrq / S; RANDOM pid sid;
RANDOM INT / SUB=pid; RUN;
RANDOM INT / SUB=sid;
RUN; SAS abbreviation: S=SOLUTION

SAS abbreviations: S=SOLUTION, SUB=SUBJECT, INT=INTERCEPT

7 8

The Mixed Procedure 103 104 105 106 107 108 109
110 111 112 113 114 115 116
Model Information 117 118 119 120 121 122 123
124 125 126 127 128 129 130
Data Set WORK.XW 131 132 133 134 135 136 137
Dependent Variable ATTAIN 138 139 140 141 142 143 144
Covariance Structure Variance Components 145 146 147 148
Subject Effects PID, SID SID 19 1 2 3 4 5 6 7 8 9 10 11 12 13
Estimation Method REML 14 15 16 17 18 19
Residual Variance Method Profile
Fixed Effects SE Method Model-Based
Degrees of Freedom Method Containment Dimensions

Covariance Parameters 3
Class Level Information Columns in X 1
Columns in Z Per Subject 167
Class Levels Values Subjects 1
Max Obs Per Subject 3435
PID 148 1 2 3 4 5 6 7 8 9 10 11 12 13
14 15 16 17 18 19 20 21 22 23
24 25 26 27 28 29 30 31 32 33 Number of Observations
34 35 36 37 38 39 40 41 42 43
44 45 46 47 48 49 50 51 52 53 Number of Observations Read 3435
54 55 56 57 58 59 60 61 62 63 Number of Observations Used 3435
64 65 66 67 68 69 70 71 72 73 Number of Observations Not Used 0
74 75 76 77 78 79 80 81 82 83
84 85 86 87 88 89 90 91 92 93
94 95 96 97 98 99 100 101 102

9 10

NULL MODEL WITH NO COVARIATES MODEL WITH VERBAL REASONING AS COVARIATE

Covariance Parameter Estimates Covariance Parameter Estimates

Standard Z Standard Z
Cov Parm Subject Estimate Error Value Pr > Z Cov Parm Subject Estimate Error Value Pr > Z

Intercept PID 1.1300 0.2074 5.45 <.0001 Intercept PID 0.2747 0.06171 4.45 <.0001
Intercept SID 0.3722 0.1743 2.14 0.0164 Intercept SID 0.01436 0.02402 0.60 0.2749
Residual 8.1107 0.2004 40.46 <.0001 Residual 4.2546 0.1050 40.54 <.0001

Fit Statistics Fit Statistics

-2 Res Log Likelihood 17150.8 -2 Res Log Likelihood 14859.1

AIC (smaller is better) 17156.8 AIC (smaller is better) 14865.1
AICC (smaller is better) 17156.8 AICC (smaller is better) 14865.1
BIC (smaller is better) 17150.8 BIC (smaller is better) 14859.1

Solution for Fixed Effects Solution for Fixed Effects

Standard Standard
Effect Estimate Error DF t Value Pr > |t| Effect Estimate Error DF t Value Pr > |t|

Intercept 5.5017 0.1787 18 30.79 <.0001 Intercept -10.0257 0.2782 18 -36.04 <.0001
VRQ 0.1600 0.002766 3268 57.87 <.0001

11 12
SPSS syntax after reading in Excel file (using default REML) Alternative (simpler) SPSS syntax

Null Model Null Model

MIXED MIXED
attain attain BY pid sid
/PRINT = SOLUTION TESTCOV /PRINT = SOLUTION TESTCOV
/RANDOM INTERCEPT | SUBJECT(pid) /RANDOM pid sid .
/RANDOM INTERCEPT | SUBJECT(sid) .
Model with vrq as covariate
Model with vrq as covariate MIXED
MIXED attain WITH vrq BY pid sid
attain WITH vrq /FIXED = vrq
/FIXED = vrq /PRINT = SOLUTION TESTCOV
/PRINT = SOLUTION TESTCOV /RANDOM pid sid .
/RANDOM INTERCEPT | SUBJECT(pid)
/RANDOM INTERCEPT | SUBJECT(sid) .

13 14

SPSS output Type III Tests of Fixed Effects

|---------|------------|--------------|--------|----|
Mixed Model Analysis |Source |Numerator df|Denominator df|F |Sig.|
|---------|------------|--------------|--------|----|
Model Dimension |Intercept|1 |2179.567 |1298.827|.000|
|------------------------|----------------|--------------------|--------------------|-----------------| |---------|------------|--------------|--------|----|
| |Number of Levels|Covariance Structure|Number of Parameters|Subject Variables| |VRQ |1 |3356.599 |3348.771|.000|
|--------------|---------|----------------|--------------------|--------------------|-----------------| |---------------------------------------------------|
|Fixed Effects |Intercept|1 | |1 | | a Dependent Variable: ATTAIN.
| |---------|----------------|--------------------|--------------------|-----------------|
| |VRQ |1 | |1 | |
|--------------|---------|----------------|--------------------|--------------------|-----------------| Estimates of Fixed Effects
|Random Effects|Intercept|1 |Variance Components |1 |PID | |---------|----------|----------|--------|-------|----|-----------------------------------|
| |---------|----------------|--------------------|--------------------|-----------------| |Parameter|Estimate |Std. Error|df |t |Sig.|95% Confidence Interval |
| |Intercept|1 |Variance Components |1 |SID | | | | | | | |-----------------------|-----------|
|------------------------|----------------|--------------------|--------------------|-----------------| | | | | | | |Lower Bound |Upper Bound|
|Residual | | |1 | | |---------|----------|----------|--------|-------|----|-----------------------|-----------|
|------------------------|----------------|--------------------|--------------------|-----------------| |Intercept|-10.025716|.278189 |2179.567|-36.039|.000|-10.571259 |-9.480173 |
|Total |4 | |5 | | |---------|----------|----------|--------|-------|----|-----------------------|-----------|
|-----------------------------------------------------------------------------------------------------| |VRQ |.160036 |.002766 |3356.599|57.869 |.000|.154614 |.165458 |
a Dependent Variable: ATTAIN. |-----------------------------------------------------------------------------------------|
a Dependent Variable: ATTAIN.

Information Criteria
|------------------------------------|---------| Estimates of Covariance Parameters
|-2 Restricted Log Likelihood |14859.140| |----------------------------------|--------|----------|------|----|-----------------------------------|
|------------------------------------|---------| |Parameter |Estimate|Std. Error|Wald Z|Sig.|95% Confidence Interval |
|Akaike’s Information Criterion (AIC)|14865.140| | | | | | |-----------------------|-----------|
|------------------------------------|---------| | | | | | |Lower Bound |Upper Bound|
|Hurvich and Tsai’s Criterion (AICC) |14865.147| |----------------------------------|--------|----------|------|----|-----------------------|-----------|
|------------------------------------|---------| |Residual |4.254604|.104958 |40.536|.000|4.053785 |4.465372 |
|Bozdogan’s Criterion (CAIC) |14886.563| |-------------------------|--------|--------|----------|------|----|-----------------------|-----------|
|------------------------------------|---------| |Intercept [subject = PID]|Variance|.274656 |.061712 |4.451 |.000|.176821 |.426624 |
|Schwarz’s Bayesian Criterion (BIC) |14883.563| |-------------------------|--------|--------|----------|------|----|-----------------------|-----------|
|----------------------------------------------| |Intercept [subject = SID]|Variance|.014365 |.024020 |.598 |.550|.000542 |.380736 |
The information criteria are displayed in smaller-is-better form. |------------------------------------------------------------------------------------------------------|
a Dependent Variable: ATTAIN. a Dependent Variable: ATTAIN.

15 16

Alternative way to run cross-classified multilevel models Alternative way to run cross-classified multilevel models

• SAS & SPSS perform cross-classified analyses seamlessly, but not • Identify cluster level with fewest number of clusters; here, 148
all software can primary schools and 19 secondary schools
• Create indicator variables for the secondary schools,
• Some hierarchical multilevel software programs can be “tricked” secs1-secs19 (0/1), which indicate the secondary school that a
into running cross-classified models if they allow student belongs to (each student belongs to only one)
– 3-level models • Create a variable cons that equals 1 for all observations in the
– Equality constraints on variances of random effects dataset
– Zero covariances of random effects • At the third level, specify cons as the level-3 ID variable, and the
19 indicator variables secs1-secs19 as random effects with
EQUAL variance and zero covariances
• At the second level, specify the primary school ID nested within
the level-3 ID (pid(cons) in SAS) and specify a random
intercept

17 18
Secondary School Indicator Variables DATA xw2; SET xw;
cons=1;
ID secs1 secs2 secs3 secs4 secs5 secs6 secs7 ... secs19
sid = 1 1 0 0 0 0 0 0 ... 0
ARRAY secschool(19) secs1-secs19;
sid = 2 0 1 0 0 0 0 0 ... 0 DO i = 1 TO 19;
sid = 3 0 0 1 0 0 0 0 ... 0 secschool(i)=0;
sid = 4 0 0 0 1 0 0 0 ... 0 IF sid=i THEN secschool(i)=1;
sid = 5 0 0 0 0 1 0 0 ... 0 END;
sid = 6 0 0 0 0 0 1 0 ... 0
sid = 7 0 0 0 0 0 0 1 ... 0
... ... ... ... ... ... ... ... ... ... PROC MIXED DATA=xw2 COVTEST;
... ... ... ... ... ... ... ... ... ... CLASS pid cons;
sid = 19 0 0 0 0 0 0 0 ... 1 MODEL attain = / S;
RANDOM INT / SUB=pid(cons);
RANDOM secs1-secs19 / SUB=cons TYPE=TOEP(1);
⇒ 19 indicator variables for the 19 secondary schools
RUN;

TOEP(1) is a structure in which all of the random effect variances

(for the variables secs1-secs19) are equal and uncorrelated
19 20

The Mixed Procedure 94 95 96 97 98 99 100 101 102

103 104 105 106 107 108 109
Model Information 110 111 112 113 114 115 116
117 118 119 120 121 122 123
Data Set WORK.ONE 124 125 126 127 128 129 130
Dependent Variable ATTAIN 131 132 133 134 135 136 137
Covariance Structures Variance Components, 138 139 140 141 142 143 144
Banded Toeplitz 145 146 147 148
Subject Effects PID(cons), cons cons 1 1
Estimation Method REML
Residual Variance Method Profile
Fixed Effects SE Method Model-Based Dimensions
Degrees of Freedom Method Containment
Covariance Parameters 3
Columns in X 1
Class Level Information Columns in Z Per Subject 167
Subjects 1
Class Levels Values Max Obs Per Subject 3435

PID 148 1 2 3 4 5 6 7 8 9 10 11 12 13
14 15 16 17 18 19 20 21 22 23 Number of Observations
24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 Number of Observations Read 3435
44 45 46 47 48 49 50 51 52 53 Number of Observations Used 3435
54 55 56 57 58 59 60 61 62 63 Number of Observations Not Used 0
64 65 66 67 68 69 70 71 72 73
74 75 76 77 78 79 80 81 82 83
84 85 86 87 88 89 90 91 92 93

21 22

Covariance Parameter Estimates Stata for cross-classified multilevel

Standard Z
Cov Parm Subject Estimate Error Value Pr > Z
• Stata uses this 3-level approach for cross-classified multilevel
Intercept
Variance
PID(cons)
cons
1.1300
0.3722
0.2074
0.1743
5.45
2.14
<.0001
0.0164
models in xtmixed
• It creates the school indicator variables, so you don’t have to
Residual 8.1107 0.2004 40.46 <.0001

Fit Statistics • Uses ML estimation by default

-2 Res Log Likelihood 17150.8 • Stata is case-sensitive (the variable names were all upper case in
AIC (smaller is better) 17156.8
AICC (smaller is better) 17156.8 this example after reading in the Excel file)
BIC (smaller is better) 17150.8

Solution for Fixed Effects . import excel "C:\mixdemo\xwdata_9var.xlsx", sheet("Sheet1") firstrow

Standard . xtmixed ATTAIN || _all: R.SID || PID:, reml

Effect Estimate Error DF t Value Pr > |t|

Intercept 5.5017 0.1787 147 30.79 <.0001

23 24
Performing EM optimization: ------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
Performing gradient-based optimization: -----------------------------+------------------------------------------------
_all: Identity |
Iteration 0: log restricted-likelihood = -8575.3795 sd(R.SID) | .6101043 .1428677 .3855483 .965449
Iteration 1: log restricted-likelihood = -8575.3795 -----------------------------+------------------------------------------------
PID: Identity |
Computing standard errors: sd(_cons) | 1.063027 .0975383 .88806 1.272467
-----------------------------+------------------------------------------------
Mixed-effects REML regression Number of obs = 3435 sd(Residual) | 2.847926 .0351921 2.77978 2.917744
------------------------------------------------------------------------------
----------------------------------------------------------- LR test vs. linear regression: chi2(2) = 280.57 Prob > chi2 = 0.0000
| No. of Observations per Group
Group Variable | Groups Minimum Average Maximum Note: LR test is conservative and provided only for reference.
----------------+------------------------------------------
_all | 1 3435 3435.0 3435
PID | 148 1 23.2 72
-----------------------------------------------------------

Wald chi2(0) = .
⇒ Stata estimates SDs (and not variances) for the random effects by
Log restricted-likelihood = -8575.3795 Prob > chi2 = . default; for variance estimates add the option variance
------------------------------------------------------------------------------
ATTAIN | Coef. Std. Err. z P>|z| [95% Conf. Interval] . xtmixed ATTAIN || _all: R.SID || PID:, reml variance
-------------+----------------------------------------------------------------
_cons | 5.501727 .1786809 30.79 0.000 5.151519 5.851935
------------------------------------------------------------------------------

25 26

Intraclass correlations Crossed random-effects model with random interaction

For the null model (no covariates), Residual variance = 8.1107,

• For this, need combinations of primary and secondary schools
Primary school variance = 1.13, Secondary school variance = 0.3722
where there are several observations
• Can relax assumption of additive random effects
Students from same primary school, but different secondary schools
1.13 • Some secondary schools might benefit students from particular
ICC = = 0.118 primary schools more than other primary schools
8.1107 + 1.13 + 0.3722

Students from same secondary school, but different primary schools

0.3722
ICC = = 0.039
8.1107 + 1.13 + 0.3722

Students from same primary and secondary schools

1.13 + 0.3722
ICC = = 0.156
8.1107 + 1.13 + 0.3722
27 28

Crossed random effects with interaction

Crossed structure with singletons only - CANNOT estimate
interaction SAS
Secondary 1 Secondary 2 Secondary 3 Secondary 4 PROC MIXED COVTEST;
CLASS pid sid ;
Primary 1 x x x
MODEL attain = / S;
Primary 2 x x x RANDOM pid sid pid*sid;
Primary 3 x x x x
Primary 4 x x x SPSS
MIXED
Crossed structure with duplications - CAN estimate attain BY pid sid
/PRINT = SOLUTION TESTCOV
interaction
/RANDOM pid sid pid*sid .
Secondary 1 Secondary 2 Secondary 3 Secondary 4
Primary 1 xxx xx xxx STATA
Primary 2 xx x xxx . xtmixed ATTAIN || _all: R.SID || PID: || SID:, reml variance
Primary 3 x xxx xx xx
Primary 4 xxxx x xxxx
29 30
Intraclass correlations - model with RE interaction Crossed random effect models - show me the equations!

Null model (no covariates), Residual var = 8.0873, Primary var = Subject k nested within crossing of primary schools i and secondary
0.9144, Secondary var = 0.3376, Interaction var = 0.2335 schools j

model with additive random effects

Students from same primary school, but different secondary schools
0.9144 yijk = xijk β + υi + υj + ijk
ICC = = 0.096
8.0873 + 0.9144 + 0.3376 + 0.2335
υi ∼ N (0, σP2 ) υj ∼ N (0, σS2 ) ijk ∼ N (0, σ2)
Students from same secondary school, but different primary schools
0.3376
ICC = = 0.035 model with random effects interaction
8.0873 + 0.9144 + 0.3376 + 0.2335
yijk = xijk β + υi + υj + υij + ijk
Students from same primary and secondary schools
0.9144 + 0.3376 + 0.2335 υi ∼ N (0, σP2 ) υj ∼ N (0, σS2 ) υij ∼ N (0, σP2 S ) ijk ∼ N (0, σ2)
ICC = = 0.155
8.0873 + 0.9144 + 0.3376 + 0.2335
31 32

Multi Membership Models Nested or Hierarchical Structure weights

• Subjects are nested within more than one cluster Teacher 1 Teacher 2 Teacher 3 Teacher 4
Student 1 1 0 0 0
data collected on who are clustered within Student 2 0 1 0 0
students more than one teacher Student 3 0 0 1 0
patients more than one provider Student 4 0 0 0 1
students nested within teachers
Here,
• assume there are known weights that represent the degree of
Multi Membership Structure weights
membership for a subject to the different clusters
Teacher 1 Teacher 2 Teacher 3 Teacher 4
• sum of weights equals one Student 1 .25 .25 .25 .25
• possibly do sensitivity analysis to examine how different choices Student 2 0 1 0 0
for weights affect results Student 3 .33 .33 0 .33
Student 4 0 0 .5 .5
students nested within multiple teachers

33 34

Example: Leckie, G. (2013). Multiple Membership Multilevel Models. LEMMA Excel file: nursedat2.xlsx - some subjects seen by only 1 nurse
VLE Module 13. (http://www.bristol.ac.uk/cmm/learning/course.html)
Simulated data from 1,000 patients who were treated in all by 25 nurses: 400 treated by only one
nurse, 300 treated by two nurses, 200 by three nurses, and 100 by four nurses.
• patient: Patient ID
• satis: Patient post-op satisfaction (mean=0, std=1)
• assess: Patient pre-op assessment (mean=0, std=1); higher scores are better
• nurses: Number of nurses seen by the patient (1 to 4)
• n1st: Nurse ID for patient’s 1st nurse
• n2nd: Nurse ID for patient’s 2nd nurse
• n3rd: Nurse ID for patient’s 3rd nurse
• n4th: Nurse ID for patient’s 4th nurse
• p1: Proportion of time with nurse 1
• p2: Proportion of time with nurse 2
. ..
• .. .
• p25: Proportion of time with nurse 25 value of 1 for only one of the variables p1 to p25 (all others equal 0)
• h1: Job Happiness score for nurse 1
• h2: Job Happiness score for nurse 2
. ..
• .. .
• h25: Job Happiness score for nurse 25

35 36
Excel file: nursedat2.xlsx - some subjects seen by 2 nurses Excel file: nursedat2.xlsx - Job Happiness values (mean=0, sd=1) of
nurses

Two of the variables p1 to p25 are non-zero and their sum equals 1

Similarly, for patients seen by 3 (or 4) nurses: three (or four) of the
variables p1 to p25 are non-zero and their sum equals 1 Nurse-level covariate doesn’t change value within a column (same
nurse), only across columns h1 to h25
37 38

PROC IMPORT OUT=WORK.nursedat2 DATAFILE="C:\temp\nursedat2.xlsx" The MEANS Procedure

DBMS=EXCEL; Variable Label N Mean Std Dev Minimum Maximum
GETNAMES=YES; --------------------------------------------------------------------------------------------
RUN; patient patient 1000 500.5000000 288.8194361 1.0000000 1000.00
satis satis 1000 -0.0387974 0.9809753 -2.9819665 3.0405710
assess assess 1000 0.0204101 0.9700845 -2.9099002 3.2279503
PROC MEANS DATA=nursedat2; nurses nurses 1000 2.0000000 1.0005004 1.0000000 4.0000000
n1st n1st 1000 12.8140000 7.0523145 1.0000000 25.0000000
RUN; n2nd n2nd 1000 7.8110000 8.5426141 0 25.0000000
n3rd n3rd 1000 3.7510000 6.9372274 0 25.0000000
DATA ndats; n4th n4th 1000 1.2640000 4.5328634 0 25.0000000
p1 p1 1000 0.0442800 0.1633541 0 1.0000000
SET nursedat2; p2 p2 1000 0.0487800 0.1786310 0 1.0000000
cons = 1; p3 p3 1000 0.0348300 0.1418933 0 1.0000000
p4 p4 1000 0.0364100 0.1521412 0 1.0000000
ARRAY ps(25) p1-p25;
p5 p5 1000 0.0410500 0.1640424 0 1.0000000
ARRAY hs(25) h1-h25; p6 p6 1000 0.0386500 0.1468190 0 1.0000000
happiness = 0; p7 p7 1000 0.0396500 0.1549309 0 1.0000000
p8 p8 1000 0.0373400 0.1517833 0 1.0000000
DO i = 1 TO 25; p9 p9 1000 0.0429000 0.1720581 0 1.0000000
happiness = happiness + ps(i)*hs(i); p10 p10 1000 0.0440600 0.1767704 0 1.0000000
END; p11 p11 1000 0.0323600 0.1407880 0 1.0000000
p12 p12 1000 0.0497800 0.1763305 0 1.0000000
p13 p13 1000 0.0487400 0.1728985 0 1.0000000
PROC MEANS DATA=ndats; p14 p14 1000 0.0442600 0.1630725 0 1.0000000
p15 p15 1000 0.0434600 0.1710687 0 1.0000000
RUN; p16 p16 1000 0.0353200 0.1456603 0 1.0000000

39 40

p17 p17 1000 0.0298700 0.1288965 0 1.0000000 h20 h20 1000 1.0973895 0 1.0973895 1.0973895
p18 p18 1000 0.0405200 0.1626130 0 1.0000000 h21 h21 1000 -0.9833028 0 -0.9833028 -0.9833028
p19 p19 1000 0.0388300 0.1544260 0 1.0000000 h22 h22 1000 -0.2067423 0 -0.2067423 -0.2067423
p20 p20 1000 0.0327900 0.1439258 0 1.0000000 h23 h23 1000 -0.6376706 0 -0.6376706 -0.6376706
p21 p21 1000 0.0469600 0.1701150 0 1.0000000 h24 h24 1000 0.1376409 0 0.1376409 0.1376409
p22 p22 1000 0.0401100 0.1623559 0 1.0000000 h25 h25 1000 0.2539080 0 0.2539080 0.2539080
p23 p23 1000 0.0376800 0.1503983 0 1.0000000 cons 1000 1.0000000 0 1.0000000 1.0000000
p24 p24 1000 0.0357600 0.1463942 0 1.0000000 happiness 1000 -0.0231534 0.8019001 -1.9858233 1.7678539
p25 p25 1000 0.0356100 0.1463354 0 1.0000000 --------------------------------------------------------------------------------------------
h1 h1 1000 1.7678539 0 1.7678539 1.7678539
h2 h2 1000 -0.0044529 0 -0.0044529 -0.0044529
h3 h3 1000 -0.0398703 0 -0.0398703 -0.0398703
h4 h4 1000 -0.3772039 0 -0.3772039 -0.3772039
h5 h5 1000 -1.3820966 0 -1.3820966 -1.3820966
h6 h6 1000 0.9227986 0 0.9227986 0.9227986
h7 h7 1000 -0.1143707 0 -0.1143707 -0.1143707
h8 h8 1000 -1.3080722 0 -1.3080722 -1.3080722
h9 h9 1000 0.7863722 0 0.7863722 0.7863722
h10 h10 1000 1.0340058 0 1.0340058 1.0340058
h11 h11 1000 1.5630561 0 1.5630561 1.5630561
h12 h12 1000 -1.4213433 0 -1.4213433 -1.4213433
h13 h13 1000 0.5874567 0 0.5874567 0.5874567
h14 h14 1000 -0.3803294 0 -0.3803294 -0.3803294
h15 h15 1000 1.2916487 0 1.2916487 1.2916487
h16 h16 1000 -1.9858233 0 -1.9858233 -1.9858233
h17 h17 1000 1.0496269 0 1.0496269 1.0496269
h18 h18 1000 -1.0102353 0 -1.0102353 -1.0102353
h19 h19 1000 -0.7836885 0 -0.7836885 -0.7836885

41 42
SAS syntax for multi-membership multilevel models NULL MODEL
The Mixed Procedure

PROC MIXED DATA=ndats COVTEST METHOD=ML; Model Information

CLASS cons; Data Set WORK.NDATS

Dependent Variable satis
MODEL satis = / SOLUTION; Covariance Structure Banded Toeplitz
RANDOM p1-p25 / SUBJECT=cons TYPE=TOEPLITZ(1); Subject Effect cons
Estimation Method ML
Residual Variance Method Profile
Fixed Effects SE Method Model-Based
PROC MIXED DATA=ndats COVTEST METHOD=ML; Degrees of Freedom Method Containment
CLASS cons;
Dimensions
MODEL satis = assess / SOLUTION;
Covariance Parameters 2
RANDOM p1-p25 / SUBJECT=cons TYPE=TOEPLITZ(1); Columns in X 1
Columns in Z Per Subject 25
Subjects 1
PROC MIXED DATA=ndats COVTEST METHOD=ML; Max Obs Per Subject 1000
CLASS cons;
MODEL satis = assess happiness / SOLUTION; Number of Observations

RANDOM p1-p25 / SUBJECT=cons TYPE=TOEPLITZ(1); Number of Observations Read 1000

Number of Observations Used 1000
RUN; Number of Observations Not Used 0

43 44

WITH PATIENT COVARIATE ASSESS

Covariance Parameter Estimates

Covariance Parameter Estimates
Standard Z
Cov Parm Subject Estimate Error Value Pr > Z Standard Z
Cov Parm Subject Estimate Error Value Pr > Z
Variance cons 0.2323 0.07503 3.10 0.0010
Residual 0.8145 0.03689 22.08 <.0001 Variance cons 0.2550 0.07893 3.23 0.0006
Residual 0.5875 0.02661 22.08 <.0001

Fit Statistics
Fit Statistics
-2 Log Likelihood 2686.0
AIC (smaller is better) 2692.0 -2 Log Likelihood 2368.8
AICC (smaller is better) 2692.0 AIC (smaller is better) 2376.8
BIC (smaller is better) 2686.0 AICC (smaller is better) 2376.9
BIC (smaller is better) 2368.8

Solution for Fixed Effects

Standard Solution for Fixed Effects

Effect Estimate Error DF t Value Pr > |t|
Intercept -0.02653 0.1006 975 -0.26 0.7921 Standard
Effect Estimate Error DF t Value Pr > |t|

Intercept -0.03317 0.1039 974 -0.32 0.7497

assess 0.4904 0.02531 974 19.37 <.0001

45 46

WITH NURSE COVARIATE JOB HAPPINESS SPSS syntax

Covariance Parameter Estimates GET FILE=’C:\MixDemo\nursedat2.sav’.
Standard Z
Cov Parm Subject Estimate Error Value Pr > Z
COMPUTE cons=1.

Variance cons 0.1689 0.05413 3.12 0.0009 COMPUTE happy = 0.

Residual 0.5874 0.02660 22.08 <.0001
VECTOR ps = p1 TO p25 / hs = h1 TO h25.
LOOP i = 1 TO 25.
Fit Statistics
+ COMPUTE happy = happy + ps(i) * hs(i).
-2 Log Likelihood 2359.4 END LOOP.
AIC (smaller is better) 2369.4
AICC (smaller is better) 2369.5
BIC (smaller is better) 2359.4 MIXED
satis WITH assess happy p1 TO p25
Solution for Fixed Effects
/FIXED = assess happy
/METHOD = ML
Standard /PRINT = SOLUTION TESTCOV
Effect Estimate Error DF t Value Pr > |t|
/RANDOM p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12
Intercept -0.03135 0.08579 974 -0.37 0.7148 p13 p14 p15 p16 p17 p18 p19 p20 p21 p22
assess 0.4912 0.02530 974 19.42 <.0001 p23 p24 p25 | SUBJECT(cons) COVTYPE(ID).
happiness 0.2936 0.08710 974 3.37 0.0008

47 48
 Type III Tests of Fixed Effects

SPSS output - Dependent Variable: satis |---------|------------|--------------|-------|----|

|Source |Numerator df|Denominator df|F |Sig.|
|---------|------------|--------------|-------|----|
Model Dimension
|Intercept|1 |23.106 |.134 |.718|
|------------------------------------|----------------|--------------------|--------------------|-----------------|
|---------|------------|--------------|-------|----|
| |Number of Levels|Covariance Structure|Number of Parameters|Subject Variables|
|assess |1 |981.019 |377.001|.000|
|--------------|---------------------|----------------|--------------------|--------------------|-----------------|
|---------|------------|--------------|-------|----|
|Fixed Effects |Intercept |1 | |1 | |
|happy |1 |25.414 |11.362 |.002|
| |---------------------|----------------|--------------------|--------------------|-----------------|
|--------------------------------------------------|
| |assess |1 | |1 | |
| |---------------------|----------------|--------------------|--------------------|-----------------|
| |happy |1 | |1 | |
Estimates of Fixed Effects
|--------------|---------------------|----------------|--------------------|--------------------|-----------------|
|---------|--------|----------|-------|------|----|-----------------------------------|
|Random Effects|p1 + p2 + ... + p25 |25 |Identity |1 |cons |
|Parameter|Estimate|Std. Error|df |t |Sig.|95% Confidence Interval |
|------------------------------------|----------------|--------------------|--------------------|-----------------|
| | | | | | |-----------------------|-----------|
|Residual | | |1 | |
| | | | | | |Lower Bound |Upper Bound|
|------------------------------------|----------------|--------------------|--------------------|-----------------|
|---------|--------|----------|-------|------|----|-----------------------|-----------|
|Total |28 | |5 | |
|Intercept|-.031354|.085791 |23.106 |-.365 |.718|-.208781 |.146073 |
|-----------------------------------------------------------------------------------------------------------------|
|---------|--------|----------|-------|------|----|-----------------------|-----------|
As of version 11.5, the syntax rules for the RANDOM subcommand have changed. Your command syntax may yield
|assess |.491215 |.025299 |981.019|19.417|.000|.441569 |.540862 |
results that differ from those produced by prior versions. If you are using version 11 syntax, please
|---------|--------|----------|-------|------|----|-----------------------|-----------|
consult the current syntax reference guide for more information.
|happy |.293575 |.087095 |25.414 |3.371 |.002|.114348 |.472803 |
|-------------------------------------------------------------------------------------|
Information Criteria
|------------------------------------|--------|
|-2 Log Likelihood |2359.406|
Estimates of Covariance Parameters
|------------------------------------|--------|
|----------------------------------------------|--------|----------|------|----|-----------------------------------|
|Akaike’s Information Criterion (AIC)|2369.406|
|Parameter |Estimate|Std. Error|Wald Z|Sig.|95% Confidence Interval |
|------------------------------------|--------|
| | | | | |-----------------------|-----------|
|Hurvich and Tsai’s Criterion (AICC) |2369.467|
| | | | | |Lower Bound |Upper Bound|
|------------------------------------|--------|
|----------------------------------------------|--------|----------|------|----|-----------------------|-----------|
|Bozdogan’s Criterion (CAIC) |2398.945|
|Residual |.587433 |.026604 |22.081|.000|.537538 |.641959 |
|------------------------------------|--------|
|----------------------------------------------|--------|----------|------|----|-----------------------|-----------|
|Schwarz’s Bayesian Criterion (BIC) |2393.945|
|p1 + p2 + ... + p25 [subject = cons] Variance |.168940 |.054127 |3.121 |.002|.090160 |.316558 |
|---------------------------------------------|
|------------------------------------------------------------------------------------------------------------------|
The information criteria are displayed in smaller-is-better form.

49 50

Stata syntax Performing EM optimization:

Performing gradient-based optimization:

. import excel "C:\mixdemo\nursedat2.xlsx", sheet("Sheet1") firstrow
Iteration 0: log likelihood = -1179.7032
Iteration 1: log likelihood = -1179.7032
. forvalues j = 1/25 {
2. generate p‘j’Xh‘j’ = p‘j’*h‘j’ Computing standard errors:
3. }
Mixed-effects ML regression Number of obs = 1000
Group variable: _all Number of groups = 1
. egen happiness = rsum(p1Xh1-p25Xh25)
Obs per group: min = 1000
. xtmixed satis assess happiness || _all: p1-p25, nocons covariance(identity) avg = 1000.0
max = 1000
mle variance

Wald chi2(2) = 386.22

Log likelihood = -1179.7032 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
satis | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
assess | .4912155 .0252988 19.42 0.000 .4416306 .5408003
happiness | .2935754 .0870952 3.37 0.001 .1228719 .4642789
_cons | -.0313536 .0857909 -0.37 0.715 -.1995006 .1367934
------------------------------------------------------------------------------

51 52

------------------------------------------------------------------------------ Multi-Membership Multilevel Model

Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
_all: Identity | Patient j nested within nurse(s) i (i = 1, . . . , N )
var(p1..p25)(1) | .1689407 .0541275 .0901602 .3165583
-----------------------------+------------------------------------------------
var(Residual) | .5874327 .0266036 .5375377 .6419591 Model with only patient covariates
------------------------------------------------------------------------------
N
yj = xj β + pij υi + ij

LR test vs. linear regression: chibar2(01) = 121.39 Prob >= chibar2 = 0.0000
(1) p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16 p17 p18 p19 p20 p21 p22 p23 p24 p25 i=1
N
υi ∼ N (0, συ2 ) ij ∼ N (0, σ2) pij = 1 ∀ j


i=1

Model with a nurse covariate xi

⎛ ⎞
N N
yj = xj β + ⎜ 
pij xi⎟⎟⎠ βxi + pij υi + ij

⎜
⎝
i=1 i=1

Model with N × p nurse covariate matrix X and N × 1 nurse

weight vector pj for patient j
yj = xj β + pj Xβ X + pj υi + ij
53 54
Intraclass correlation - Leckie (2013)
Summary
• pairwise correlation of patients DV within a given cluster
(conditional on covariates) • Cross-classified models are useful when subjects are classified in
• ICCs vary depending on patient weights two or more cluster types, but cluster types are not nested within
each other; more than 1 level of (crossed) clustering
Correlation for two patients cared for by the same nurse for the entire – Students within primary and secondary schools
hospital stay (null model: nurse var = .2323, error var = .8145)
συ2 .2323 • Multi-membership models apply when subjects are clustered
ICC = = = 0.222 within potentially multiple clusters (at the same level)
συ + σ2
2 .2323 + .8145
– Patients seen by multiple nurses
Correlation for two patients j and j  cared for by the same nurse,
say nurse 2, for half their stays (p2j = p2j  = .5), but different nurses • Can be computationally demanding because of specification of
for the rest of their stays, say p5j = .5 and p8j  = .5 many random effects; SAS HPMIXED is potentially useful for this
p2j p2j  συ2 .25 × .2323
ICC =

 = = 0.062
p22j + p25j συ2 + σ2 p22j  + p28j  συ2 + σ2 .5 × .2323 + .8145

55 56