Using Predictive

Modeling to Target
Interventions
Barry P. Chaiken, MD, MPH
Chief Medical Officer
ABQAURP - PSOS

Overview
Cost and Quality Trends
Disease Management and Modeling
Predictive Modeling Fundamentals
Accuracy of Models - Case Study

Change Per Capita In Health Care

Spending and GDP
Percent

Health Care Spending

8.5%

GDP
2.9%

* Data for January through June 2003, compared with corresponding months in 2002
Source: B. Strunk and P. Ginsburg, Tracking Health Care Costs: Trends Stabilize But Remain High in 2002,
Health Affairs (Web Exclusive June 11, 2003); B. Strunk and P. Ginsburg, Tracking Health Care Costs: Trends
Slow in First Half of 2003, Center for Studying Health System Change, December 2003.

Growth in Per Enrollee Premiums and

Benefits
Percent
Premiums per enrollee
Benefits per enrollee

* Data for growth between Spring 2002 and Spring 2003

Source: Heffler et al., Health Spending Projections for 2002-2012, Health Affairs (Web Exclusive February
7, 2003) for 19852001; Employer Health Benefits 2003 Annual Survey, The Kaiser Family Foundation and
Health Research and Educational Trust, September 2003 for 20022003.

Drivers of Care Management

50% - preventive care
30% - lack recommended acute care
40% - lack recommended chronic care
30% - receive contraindicated acute

care

20% - receive contraindicated chronic

care

Recommended Care and Quality Varies

Percent Receiving Recommended Care

Source: McGlynn et al., The Quality of Health Care Delivered to Adults in the United States, The New
England Journal of Medicine (June 26, 2003): 26352645.

Health Care Costs Concentrated in Sick Few

0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%

Expenditure Threshold
(1997
Dollars)

1%
5%
10%

27%
50%

55%
69%
97%

U.S. Population

$27,914

$7,995
$4,115

$351

Health Expenditures

Source: AC Monheit, Persistence in Health Expenditures in the Short Run: Prevalence and Consequences,
Medical Care 41, supplement 7 (2003): III53III64.

Overview
Cost and Quality Trends
Disease Management and Modeling
Predictive Modeling Fundamentals
Accuracy of Models - Case Study

DM Defined
Knowledge-based process intended to
improve continuously the value of health
care delivery from the perspectives of
those who receive, purchase, provide,
supply and evaluate it.
--James B. Couch, MD

DM Criteria Specific
High dollar and volume
Preventable complications
Short time frame for results
Treatment variability
Extensive patient non-compliance
Practical guidelines
Measurable quality metrics

10

DM Processes
Identify patients
Develop therapeutic programs
Improve outcomes
Achieve acceptable cost levels
Provide evidence based care

11

Spectrum of Care

Disease Management
Lower Costs

Higher Costs

Self-Directed

Primary

Secondary

Tertiary

Long Term
Care

Patients
Provider
Home Care

PCP
Allied Health
Professionals

Specialist
Outpatient
Clinics

Hospitals
Centers of
Excellence

Institutions
Long Term
Care
Nursing Homes

12

Risk Measurement Pyramid

High
Disease
Burden

Case
Mgmt.

Single High
Impact
Disease

Users

Disease
Mgmt.

Needs
Assessment

Practice
Resource
Quality
Mgmt. Improvement
Payment/
Finance

Users & Non-Users

Population Segment

Source: Weiner JP. Presentation at National BCBS

Association meeting, Chicago, 1/30/03.

13

Overview
Cost and Quality Trends
Disease Management and Modeling
Predictive Modeling Fundamentals
Accuracy of Models - Case Study

14

Predictive Modeling Defined

Use of clinical information available
for all members of a population to
predict future healthcare needs,
overall or for specific types of
services.

Source: CB Forrest, Population-based Predictive Modeling Using ACGs: Application to Disease and Case
Management, International ACG Users Conference, November 11, 2003

15

Reasons for Predictive

Modeling
Existing high utilization by the few
Uses available data to identify high-

risk

Allows intervention early in disease

cycle

Enhances case and disease

management

16

Predictive Modeling Focus

Case management targeting
Identify

persons for care programs

Disease management risk

stratification
Intensity

tiers

Financial forecasting
Actuarial

risk

Source: CB Forrest, Population-based Predictive Modeling Using ACGs: Application to Disease and Case
Management, International ACG Users Conference, November 11, 2003

17

What Predictive Modeling

Does
Stratify members

Enhance impact of interventions

Probabilistic identification of high utilizers

Assign risk scores

Describe comparative severity of illness

Identify members not receiving proper care

Highlight inconsistency of care
Prospectively identify adverse events
Allow focused interventions

Maximize benefits of disease management

Discover inefficient care

18

Additional Uses of Modeling

Influence adoption of best practices
Track effectiveness of interventions
Establish pay for performance
Set more accurate premiums
Develop contracts with providers
Actuarial

Help plan network composition

Based

on member needs

19

Target Populations
Risk stratify
Subpopulation
Risk

factors

Identify

most likely to benefit

Develop specific, targeted interventions

Probabilities

for certain outcomes

Practice

guidelines

Practice

standardization

Decrease variation
20

Modeling Utilizes Available

Data
Traditional demographic data
Age,

sex, occupation, prior costs

Clinical data from claims

Pharmacy data
Diagnostic tests
Health risk appraisals
Questionnaires

21

Components of Modeling
Entire population data
Baseline assessment used to predict future
Risk assessment period
Static:

year 1 predicts year 2

Rolling: assign score every period

Outcomes of interest
Changes in health status
Costly healthcare events
Overall healthcare expenditures
Source: CB Forrest, Population-based Predictive Modeling Using ACGs: Application to Disease and Case
Management, International ACG Users Conference, November 11, 2003

22

Statistics in Predictive
Modeling
+
Screening
Test

+
-

TP
(True
Positive)
FN
(False
Negative)

FP
(False
Positive)
TN
(True
Negative)

Predicative Pos.
Value
Predicative Neg.
Value
Sensitivity

TP/TP + FP
TN/FN + TN

Specificity

TN/FP + TN

TP/TP+FN
23

Sensitivity Versus Predictive

Value
% True Cases
Among Group

% All True
Cases
Identified

Top 1%

69%

14%

Top 5%

36%

36%

Top 10 %

25%

51%

Score CutPoint

Predictive Pos.
Value

Sensitivity
24

First Generation Modeling

Utilize demographic data
Age,

sex, diagnoses

Rely upon historical financial data

Predict risk

25

Second Generation Modeling

Utilize first generation data sources
Incorporate second generation

sources

Pharmacy
Lab
Test

data

data
data

Predictions based on risk adjustment

DCGs,

ACGs, ETGs

26

Third Generation Modeling

Utilize first and second generation sources
Incorporate other sources and models
Health

risk appraisals, questionnaires

Surveys
Regional variability
ACGs, DCGs, ETGs

Model the models

Choose the best modeling of models for results
Learn from previous data modeling

27

Third Generation Process

Perform data cleanup
Split data into 2 years
Use Year1 data to predict Year2 cost
Drugs
Episodes of
Care (ETGs)

Risk
Score

Patient Characteristics
Age
Gender
Enrollment
Insurance type

Provider Specialty

MEDais
Clinical
Groupings
Selected
Resource
Measures

Timing &
Frequency of
Services

Identify modeling clusters and select best drivers

Diseases, enrollment groups, product line

Source: MEDai Inc.

28

Modeling of Models

Select model for optimum training of each cluste

Linear & Nonlinear / Regression, Neural Networks.,

Source: MEDai Inc.

29

Overview
Cost and Quality Trends
Disease Management and Modeling
Predictive Modeling Fundamentals
Accuracy of Models - Case Study

30

Accuracy of Third Generation

Model
40000

35000

Future Dollars

30000

25000

Predicted Costs By Member

20000

versus
Actual Costs Experienced that Year

Predicted
Actual

15000

10000

5000

Each data point represents a single group of members within a range of predicted costs
from the lowest predicted group to the highest predicted group (100 groups each with 1900 members)

100

97

94

91

88

85

82

79

76

73

70

67

64

61

58

55

52

49

46

43

40

37

34

31

28

25

22

19

16

13

10

Source: MEDai Inc.

31

Diabetes Model Accuracy

90000
80000
70000

Actual $

60000
50000
40000
30000
20000
10000
0
0

20000

40000

60000

80000

100000

120000

140000

Predicted $

95 members per observation

Source: MEDai Inc.

32

Depression Model Accuracy

40000
35000
30000

Actual $

25000
20000
15000
10000
5000
0
0

10000

20000

30000

40000

50000

60000

70000

Predicted $

105 members per observation

Source: MEDai Inc.

33

Hypertension Model Accuracy

230 members per observation

Source: MEDai Inc.

34

Underwriting Impact for Employer

Groups
Sample of
Employer Group

Members

MEDai forecast
$PMPM

Internal Actuary
Premium SPMPM

Difference

80

$114

$119

78

$151

$139

($12)

78

$121

$149

$28

61

$167

$124

($43)

55

$114

$93

($21)

51

$145

$113

($32)

48

$170

$134

($36)

44

$111

$126

$15

41

$131

$135

$5

39

$204

$168

$118

$124

36

$6

($36)
$6

Health plan XYZ compared their premiums for a sample of employer groups using actuary vs. MEDais.
The actuarial model underestimated on the majority of small employer groups in comparison to MEDai.
This creates substantial losses, since 80% of the employers are small-group. The MEDai forecasting provides a
savings opportunity that approximated $11 million for 100,000 lives.
The client states that in the final quarter of 2001, the actual cost for these groups shows clear
Source: MEDai Inc.
underestimation by the internal actuary forecasting.
35

36

37

Successful Predictive Modeling

Identify clear goals
Models fit some better than others
Actuarial versus care management
Assess available data inputs
Demographic,

claims, pharmacy, lab values

Secure a product champion

Key to any successful implementation
Apply effective change management
Adjustment

of approach to care
management

38

References
Monheit AC, Persistence in health expenditures in the

short run: Prevalence and consequences, Medical Care 41,

supplement 7 (2003): III53III64.
Strunk B, Ginsburg P, Tracking health care costs: Trends
stabilize but remain high in 2002, Health Affairs (Web
Exclusive June 11, 2003)
Strunk B, Ginsburg P, Tracking health care costs: Trends
slow in first half of 2003, Center for Studying Health
System Change, December 2003.
Heffler et al., Health spending projections for 2002-2012,
Health Affairs (Web Exclusive February 7, 2003) for 1985
2001.
Heffler et al., Employer health benefits 2003 annual survey,
The Kaiser Family Foundation and Health Research and
Educational Trust, September 2003 for 20022003.
McGlynn et al., The Quality of Health Care Delivered to
Adults in the United States, The New England Journal of
Medicine (June 26, 2003): 26352645
39

References
Weiner JP, Predictive modeling and risk measurement:

Paradigms, potential and pitfalls. Presented at a

symposium on Predictive Modeling sponsored by the
National Blue Cross/Blue Shield Association, Chicago,
January 30, 2003.
Forrest CB, Population-based predictive modeling using
ACGs: Application to disease and case management,
International ACG Users Conference, November 11, 2003.
LeGrow G, Metzger J, E-Disease Management, First
Consulting Group, 2001, California HealthCare
Foundation
Couch JB, The Physicians Guide to Disease Management .
1997, Aspen Publishing, Inc., Gaithersburg, MD
Kongstevdt PR, The Managed Health Care Handbook, 3 rd.
1996, Aspen Publishers, Inc., Gaithersburg, MD
www.medai.com
40