494750

research-article2013
AJMXXX10.1177/1062860613494750American Journal of Medical QualityPrentice et al

Article
American Journal of Medical Quality

Which Outpatient Wait-Time Measures

2014, Vol. 29(3) 227­–235
© 2013 by the American College of
Medical Quality
Are Related to Patient Satisfaction? Reprints and permissions:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/1062860613494750
ajmq.sagepub.com

Julia C. Prentice, PhD,1 Michael L. Davies, MD,2

and Steven D. Pizer, PhD1

Abstract
Long waits for appointments decrease patient satisfaction. Administrative wait-time measures are used by managers,
but relationships between these measures and satisfaction have not been studied. Data from the Veterans Health
Administration are used to examine the relationship between wait times and satisfaction. Outcome measures include
patient-reported satisfaction and timely appointment access. Capacity and retrospective and prospective time stamp
measures are calculated separately for new and returning patients. The time stamp measures consist of the date
when the appointment was created in the scheduling system (create date [CD]) or the date the appointment was
desired as the start date for wait-time computation. Logistic regression models predict patient satisfaction using these
measures. The new-patient capacity, new-patient time stamp measures using CD, and the returning-patient desired-
date prospective measure were significantly associated with patient satisfaction. Standard practices can be improved
by targeting wait-time measures to patient subpopulations.

Keywords
wait-time measures, patient satisfaction, VA, VHA, access to care

More than a decade ago, the Institute of Medicine iden- most scheduling systems but they do not measure how
tified timely access to health care as an essential way to long an individual patient actually waits. Variation in
improve health care quality in the United States. provider practice schedules and clinic limitations
Appointment wait times continue to be an essential related to appointment types also may make these mea-
measure of access as the health care system continues sures less reliable.16,17
to struggle with long wait times.1,2,3 For example, a As the Veterans Health Administration (VHA) shifted ser-
national survey in 2009 found an average wait time of vices from inpatient to outpatient care,18 stakeholders devel-
20.3 days for an appointment in family practice.4 Wait oped a strong interest in knowing the timeliness of
times for outpatient care are expected to further increase appointments for individual veterans. Consequently, the
with the implementation of the Patient Protection and VHA uses a wider variety of wait-time measures than the pri-
Affordable Care Act that expands health insurance cov- vate sector. VHA also consistently measures patient satisfac-
erage.5-7 Negative consequences of delayed access to tion through patient surveys. This article compares the ability
care include poor health outcomes, especially among of alternative measures of wait times to reliably predict
older and more vulnerable patient populations,7-11 and patient satisfaction. To place the forthcoming analyses in
lower patient satisfaction.12-14 context, the following section describes each of the wait-time
Reliable wait-time measures are underdeveloped in measures along with their advantages and disadvantages.
the United States. One way to measure wait times uses
physician surveys that ask how long it would take to get
an appointment for patients with a nonemergency con-
1
dition.4,15 Unfortunately, survey data are expensive to VA Boston Healthcare System, Boston University School of Public
Health and School of Medicine, Boston, MA
obtain and do not continuously monitor changes in 2
National Director of Systems Redesign, Department of Veterans
wait times. As an alternative, proponents of schedul- Affairs, Fort Meade, SD
ing interventions that are focused on decreasing wait
Corresponding Author:
times (eg, Advanced Clinic Access [ACA]) have sug-
Julia C. Prentice, PhD, VA Boston Healthcare System, Boston
gested capacity measures, such as how many days University School of Public Health and School of Medicine, 150 S
until the third next available appointment for a physical Huntington Avenue, Mailstop 152H, Boston, MA 02130.
exam.16,17 Capacity measures are easily calculated from Email: Julia.Prentice@va.gov

Downloaded from ajm.sagepub.com at NORTH DAKOTA STATE UNIV LIB on March 13, 2015
228 American Journal of Medical Quality 29(3)

Wait-Time Measures in the VHA The first starting point used was based on the CD. CD
is the date that an appointment is created (ie, made) or
Capacity Measures the date the patient is entered into an electronic waiting
list (Table 1). The main strength of this measure is that
In response to complaints about long waits for VHA
the CD time stamp is captured automatically, without
care, Congress requested information on outpatient wait-
human discretion. Once created, the only way the CD
ing times starting in 1999.19,20 Early performance metrics
time stamp can be changed is by the patient canceling
focused on capacity measures such as the first next avail-
and rescheduling or the patient not showing up for the
able appointment (FNA; M. Davies, e-mail communica-
appointment.
tion, September 2010). This is a prospective wait-time
The principal limitation of CD concerns follow-up
measure that uses the day an appointment is being cre-
appointments for returning patients. Because the CD time
ated as the starting point and measures the time between
stamp captures the creation of an appointment, the results
that day and the day the first available open appointment
of measuring CD are believed to reflect the pattern of
slot occurs (Table 1). This measure counts only the days
booking appointments. For example, suppose a patient
the clinics are open, ignoring weekends, holidays, or
comes in for a checkup and agrees to schedule a follow-
unavailable days for part-time providers. FNA is consid-
up appointment in 6 months. If the clinic creates the fol-
ered a marker of the amount of backlogged appointments
low-up appointment on the day of the initial appointment
in the system, in that it measures how far into the future
(“on today”), the resulting measured wait time will be 6
a scheduler has to look before finding an open
months. Alternatively, the clinic might contact the vet-
appointment.
eran 5 months from “today” and create the intended
The strengths of FNA include the ability to bench-
6-month follow-up appointment, resulting in a measured
mark performance with other organizations that use sim-
wait time of 1 month. Another limitation of this measure
ilar capacity measures, but there are a number of
is that it does not take patient preferences into account.
limitations. Like all capacity measures, FNA does not
For example, a new patient may want the certainty of
reflect how long patients actually wait but rather the
making an appointment “on today” but “for” a future time
capacity of the clinic to have open appointments.
after a holiday or family gathering.
Individual patients may not actually want the FNA
Recent VHA policy has attempted to overcome these
appointment because the appointment length or type
limitations by focusing on DD as the initial “start date”22
does not meet their needs or because they want a follow-
(M. Davies, e-mail communication, September 2010;
up appointment in the future. The latter case is more
Table 1). This time stamp designates the ideal time “a
problematic for returning patients who wish to schedule
patient or provider wants the patient to be seen.”22 If the
a follow-up than for new patients who most likely wish
patient has an established relationship with the provider
to be seen as soon as possible.21
and agrees to return for a future appointment (ie, internal
FNA also may overestimate appointment availability.
demand in ACA literature),17 the date the patient and pro-
An open appointment may be the result of a late cancella-
vider agree on as the desired return date is the DD. If this
tion in an otherwise backlogged clinic, and without the
returning patient requests an unanticipated appointment
cancellation, there would be little open space in the
or if a new patient requests their first appointment, the
schedule. FNA also varies because of differences in how
scheduling clerk is instructed to ask the patient when they
clinics display appointments in the scheduling system.
would like to be seen (regardless of when they are able to
Some clinics display multiple schedule profiles for the
see an open slot). The answer to this question establishes
same provider (eg, one schedule for new patients and one
the DD for this “external demand” situation.17 The
schedule for returning patients). The computer does not
strength of the DD time stamp measure is that it reflects
consult all available profiles for a provider when calculat-
the patient’s or provider’s wishes. Additionally, it is not
ing FNA, so the FNA may incorrectly determine that the
influenced by differences in local scheduling practices.
provider has open appointments in one profile when, in
For example, in the case of the patient who was sched-
reality, this time slot is committed.
uled to come back in 6 months, the DD for the follow-up
appointment would be the date 6 months into the future
Time Stamp Create Date (CD) and Desired regardless of when the appointment ultimately was
scheduled.
Date (DD)
The principal limitation of this measure is its reliance
The limitations of FNA led VHA managers to consider on schedulers to accurately determine DDs. Initial audits
time stamp wait-time measures (M. Davies, e-mail of VHA’s scheduler performance in 2005 found that DD
communication, September 2010). Time stamps require was correctly entered 40% to 60% of the time. Follow-up
choices in what to use as starting and ending points. audits after educational efforts found that DD was entered

Downloaded from ajm.sagepub.com at NORTH DAKOTA STATE UNIV LIB on March 13, 2015
Prentice et al. 229

Table 1. Summary of Wait-Time Measures.

Measure Algorithm Example Calculation Strengths

First next available FNA − Appt Patient X requests first available appt on Comparable to private sector
appt (FNA-capacity) request 1/1/2010 and this is 1/15/2010 measure
1/15/2010 − 1/1/2010 = 15 days Captured by scheduling system in
contrast to scheduler entry of
dates
Retrospective, create Completed appt Patient X requests an appt on 1/1/2010, cannot Captured by scheduling system in
date (CD) date − Appt CD take the appt offered 1/15/2010, so the contrast to scheduler entry of
scheduler schedules and patient completes an dates
appt on 1/21/2010
1/21/2010 − 1/1/2010 = 21 days
Patient Y requests an appt on 1/1/2010 and
accepts a 1/10/2010 appt date. Patient Y does
not show up for the 1/10/2010 appt. This appt
is never included in retrospective wait-time
calculations
Prospective, CDa Scheduled appt Patient X has a scheduled appt for 1/21/2010 Captured by scheduling system in
date − Appt CD that was created on 1/1/2010 contrast to scheduler entry of
1/21/2010 − 1/1/2010 = 21 days dates
Patient Y has a scheduled appt for 1/10/2010 Includes all scheduled appts
that was created on 1/1/2010 compared to only completed
1/10/2010 − 1/1/2010 = 10 days appts
Retrospective, DD Completed appt Patient X wanted an appt on 1/15/2010 and Captures when patient desires appt
date − DD was scheduled for and completed an appt on in contrast to clinical capacity or
1/21/2010 clinic booking patterns
1/21/2010 − 1/15/2010 = 6 days
Patient Y wanted an appt on 1/20/2010 and was
offered and agreed to an appt on 1/27/2010.
Patient Y canceled the 1/27/2010 appt and
never rescheduled. This appt is never included
in retrospective wait-time calculations
Prospective, DDa Scheduled appt Patient X has a scheduled appt for 1/21/2010 Captures when patient desires appt
date − DD and this patient desired this appt on 1/15/2010 in contrast to clinical capacity or
1/21/2010 − 1/15/2010 = 6 days clinic booking patterns
Patient Y wanted an appt on 1/20/2010 and was Includes all scheduled appts
offered and agreed to an appt on 1/27/2010 compared with only completed
1/27/2010 − 1/20/2010 = 7 days appts

Abbreviation: Appt(s), appointment(s).

a
A snapshot of all pending appts in the system is taken on the 1st and 15th of each month to calculate prospective wait-time measures.

correctly more than 90% of the time (M. Davies, oral on the DD, causing high patient satisfaction. An example
communication, April 2012). of this situation is a returning patient who schedules a
Combinations of the time stamp measures described future appointment. In contrast, a backlogged clinic that
above are thought to reflect the patient experience. It was cannot accommodate the follow-up appointment when
hypothesized that patients who receive appointments desired for this returning patient may have lower patient
closest to when they are desired have higher satisfaction satisfaction.
levels. A patient may have their CD, DD, scheduled
appointment (SA), and completed appointment (CA) all
Completed Versus Scheduled Appointments
on (or close to) the same day. An example of this is when
patients walk in to request an appointment, and the clinic To calculate a wait time, in addition to the “start date,” an
gives them an appointment on the same day. A back- ending point time stamp also must be established (Table 1).
logged clinic unable to accommodate this “today” patient One ending point is the CA date collected automatically
may have lower patient satisfaction. Alternatively, an by the computer. The CD and DD to CA measures are
appointment may be created at an earlier point in time retrospective measures that include only successfully
than desired, while still being scheduled and completed completed appointments. If a patient does not show up

Downloaded from ajm.sagepub.com at NORTH DAKOTA STATE UNIV LIB on March 13, 2015
230 American Journal of Medical Quality 29(3)

for the appointment or the appointment is canceled and clinic stops and VHA facilities are averaged together, and
never rescheduled, the appointment is excluded from this facility-level wait time is used in the analysis.
these retrospective wait-time measures (S. Campbell, This study focuses on wait times for 50 appointment
e-mail communication, May 2012). types used for performance measurement by VHA opera-
Wait times may also be measured prospectively by tion managers and in previous research linking wait times
examining appointments that have not occurred yet (M. to health outcomes (the list of appointment types is avail-
Davies, e-mail communication, September 2010). The able from the authors on request). These appointment
VHA pending appointment list keeps track of all SAs, types have high volume, include 93% of patient-provider
and a snapshot measure of this list is taken bimonthly. interactions (vs other services such as labs or telephone
Waits are calculated by subtracting the original CD or DD consultations), and cover all major medical subspecialties
from the SA date. Prospective measures do not reflect (eg, mental health, orthopedics).9,10 Appointment types
future actions such as cancellations or no-shows, so all are weighted by national utilization and averaged together
appointments are included (Table 1). Consequently, pro- at each VHA medical center. Missing wait times are
spective measure results may be very different from ret- imputed with 0 when appropriate.7-10,26
rospective measures. For example, if there are 2
appointments scheduled when a report is pulled and the
waits are calculated to be 10 days and 28 days, the aver-
Patient Satisfaction
age SA wait time is 19 days (38/2). If the 28-day wait- The dependent variables measuring satisfaction come
time appointment turns out to be a no-show, the average from the 2010 Survey of Healthcare Experiences of
CA wait time would be only 10 days. Patients (SHEP) that is modeled after the Consumer
A weakness of any wait-time measure used to reward Assessment of Healthcare Providers and Systems family
performance, as done in the VHA, is that the measures of survey instruments. Human subjects institutional
can be thwarted.23-25 Individuals could inappropriately review board approval was obtained from the VA Boston
hold open an FNA appointment, manage the times Healthcare System. Managed by the VHA Office of
appointments are created, enter incorrect DD data, or Quality and Performance, SHEP is an ongoing nation-
cancel appointments inappropriately. Educational efforts, wide survey that seeks to obtain patient feedback on
mandatory quality reviews and feedback, and inspections recent episodes of VHA inpatient or outpatient care to
are used to ensure the integrity of the system. improve health care quality.27 For outpatient care, a sim-
ple random sample of patients with completed appoint-
ments at VHA facilities is selected each month. The
Methods overall response rate was 53%, and there were 221 540
Facility-Level New- and Returning-Patient respondents included in this study who had valid satisfac-
tion data. Respondents came from all VHA medical cen-
Wait-Time Measures
ters (n = 129). The median number of patients from each
Wait-time measures were obtained from 2010. These facility was 1805 (interquartile range = 1314-2725). (A
include the FNA, retrospective CA measures using CD list of the facilities included in the study along with the
and DD, and prospective SA measures using CD and number of patients from each facility is available from
DD (Table 1). Facility-level wait times were needed for the authors on request.)
analysis, not individual wait times, because individual
satisfaction with waits is likely to be simultaneously
Dependent Variables
determined with individual health status. Individuals in
poor health tend to report lower satisfaction and also Five different patient satisfaction measures are taken
tend to have shorter wait times because clinics triage from SHEP. Satisfaction with timeliness of care is mea-
cases and arrange to see more urgent cases more sured by asking respondents how often they were able to
quickly. It is a mistake to conclude that shorter waits for get VHA appointments as soon as they thought they
these patients caused lower satisfaction. To avoid this needed care, excluding times they needed urgent care.
problem facility-level averages were computed for each Access to VHA tests or treatments and appointments
measure.7-10,26 with VHA specialists is measured by asking how easy it
Each measure is calculated separately for new and was to get this care in the past 12 months. Response
returning patients. The scheduling system examines options for these 3 measures include always, usually,
whether an individual has been seen in a specific clinic sometimes, and never. General satisfaction is measured
(eg, cardiology) in the previous 24 months (going back to by asking respondents to rate VHA health care in the
2008 data); if not, the patient is defined as new.22 Wait past 12 months on a scale of 0 to 10 and their satisfac-
times of all new patients and returning patients within tion with their most recent VHA visit using a Likert

Downloaded from ajm.sagepub.com at NORTH DAKOTA STATE UNIV LIB on March 13, 2015
Prentice et al. 231

scale ranging from 1 to 7, with higher numbers indicat- FNA appointment and DD measures. The wait-time mea-
ing greater satisfaction. sures that rely on DD were the shortest for new-patient
measures, with the mean wait time for the FNA appoint-
ment capacity measure being similar to the CD new-
Risk Adjustment patient measures. The retrospective DD returning-patient
Risk adjustors include age, sex, race/ethnicity, education measure had the shortest waits followed by the FNA
level, number of visits to a doctor’s office in the past 12 appointment capacity measure and the prospective DD
months, and self-reported health status, all obtained from measure for returning-patient measure.
the 2010 SHEP. Patients visiting VHA facilities with shorter new-
patient FNA or CD waits (retrospective or prospective)
were more satisfied because the odds ratio for wait times
Analyses
in the second, third, and fourth quartile were significantly
STATA 10.0 (StataCorp LP, College Station, TX) was lower compared with the odds ratio in the first quartile for
used to run logistic regression models that predicted all 5 satisfaction measures (Table 4). For example,
patient satisfaction. For the timeliness of care, access to patients visiting VHA facilities with the longest retro-
treatment, and specialist measures, outcomes are coded spective new-patient CD waits (Q4) were 17% to 34%
as always or usually versus sometimes or never. Models less satisfied compared with patients visiting facilities
predict ratings of 9 or 10 versus less than 9 for the mea- with the shortest retrospective waits. In contrast, there
sure “rating of the VHA in the past 12 months,” and 6 or was no consistent relationship between the new-patient
7 versus less than 6 for the measure “satisfaction with the retrospective DD measure and patient satisfaction.
most recent visit.” Longer waits using the new-patient prospective DD mea-
The completed appointment date that the VHA uses to sure were significantly associated with lower patient sat-
target individuals for the SHEP sample was matched to isfaction for 2 of the 5 measures (VHA rating and
each of the wait-time measures. For prospective mea- treatment access).
sures (FNA and SA using CD and DD), the wait time in There was a consistent and significant relationship
the month before the targeted appointment date is between individuals visiting VHA facilities with longer
assigned to reflect waits when the appointments are waits, using the returning-patient prospective DD mea-
requested or desired. This specification results in having sure, and decreased satisfaction (Table 5). Patients visit-
11 instead of 12 months of data in these models (because ing facilities in the highest quartile of waits using the
the first month has no previous month in these data). For returning-patient FNA measure were between 7% and
the retrospective wait-time measures, the wait time in the 10% less satisfied than patients visiting facilities in the
current month of the targeted appointment is assigned (so lowest quartile, depending on the satisfaction outcome.
all 12 months could be used in analysis). Wait-time mea- The other 4 returning-patient wait-time measures did not
sures are categorized into quartiles, with the lowest quar- reliably predict patient satisfaction.
tile used as the reference group.
Discussion
Results This study associates operational measures of administra-
The SHEP respondents in this study generally reflected tive wait times with commonly used measures of patient
the larger VHA patient population. Respondents were satisfaction. Findings suggest that health care systems
predominantly male, in poor health, and frequent health should utilize a wider variety of wait-time measures than
care users. Satisfaction levels with VHA care were high. are popular in current practice because different new and
More than 80% of respondents reported obtaining returning wait-time measures were associated with
appointments as soon as they wanted them and found it patient satisfaction.
easy to obtain treatments or specialist appointments. Longer waits using a new-patient capacity measure
More than 75% of the respondents rated VHA care in the (FNA) and the retrospective and prospective new-patient
past 12 months in the top 2 categories, and more than CD wait-time measures were significantly associated with
80% did the same for satisfaction with the most recent patient satisfaction for timely VHA appointments, ease of
VHA visit (Table 2). access obtaining treatments or specialist appointments,
There was significant variation in measured wait times rating of VHA care, and satisfaction with the VHA at the
using the different methods of measurement for new and last visit (Table 4). The capacity measure finding is con-
established patients (Table 3). Wait-time measures that sistent with past research that finds a significant causal
rely on the CD for appointments were the longest for both relationship between longer FNA waits and poorer health
new and returning-patient measures compared with the outcomes, especially among older and more vulnerable

Downloaded from ajm.sagepub.com at NORTH DAKOTA STATE UNIV LIB on March 13, 2015
232 American Journal of Medical Quality 29(3)

Table 2. Descriptive Statistics of Individuals in the SHEP Sample.

Demographics (n = 221 540) Mean or Percentage

Age (years) 66.96
Male 95%
Had some college 53%
White 79%
Black 10%
Hispanic 5%
Other 6%
≥5 Visits to a doctor’s office in the past 12 months 31%
Excellent/Very good self-reported health status in the past 12 months 25%
Patient satisfaction measures
Timely visit: receiving an appointment as soon as you thought you needed one
   Always or usually versus sometimes or never (n = 158 841)a 83%
VHA rating: rate all VHA care in the past 12 months on a scale of 0 to 10 (10, highest rating)
   9 or 10 versus <9 (n = 219 772) 78%
Treatment access: how often was it easy to get treatment or tests?
   Always or usually versus sometimes or never (n = 181 250) 85%
Specialist access: how often was it easy to get an appointment with a specialist?
   Always or usually versus sometimes or never (n = 121 721) 82%
VHA satisfaction: satisfaction with VHA care at most recent visit on scale of 1 to 7 (7, most satisfied)
   6 or 7 versus less than 6 (n = 218 677) 82%

Abbreviations: SHEP, Survey of Healthcare Experiences of Patients; VHA, Veterans Health Administration.
a
Sample sizes differ between models because not all SHEP respondents answered every satisfaction question.

Table 3. Descriptive Statistics of Facility-Level Wait-Time Measures.

New-Patient Measures Meana 25%a 50%a 75%a

First next availableb 20.06 17.24 19.53 22.14
Retrospective create dateb 17.97 15.61 17.50 19.96
Prospective create dateb 31.13 26.65 30.19 34.77
Retrospective desired dateb 4.72 2.77 4.69 6.14
Prospective desired dateb 15.65 12.11 15.42 18.61

Returning-patient measures
First next availableb 7.88 5.41 8.01 10.17
Retrospective create dateb 30.90 23.60 29.05 34.95
Prospective create dateb 72.26 50.54 66.71 86.90
Retrospective desired dateb 2.72 1.91 2.60 3.36
Prospective desired dateb 17.19 13.75 16.79 19.91
a
Means and quartiles are calculated using facility-months weighted by the number of observations at each facility.
b
n = 201 819 For capacity and prospective wait-time measures, and n = 221 540 for retrospective wait-time measures. Sample sizes differ between
types of wait-time measures because December 2009 wait-time data (for January 2010) are missing on the capacity and prospective measures.

veterans.7-10 New patients typically want to be seen as Returning patients are more complicated because they
soon as possible, often because of a change in health status may not be interested in obtaining the next available
that is causing concern.21 Consequently, it is not surprising appointment for follow-up care. Surveys of patients have
that capacity or time stamp measures that use the date that found that scheduling future appointments at convenient
an appointment request was made as the start date (Table times or maintaining continuity of provider may out-
1) were successful predictors of patient satisfaction. These weigh concerns about long waits for appointments for
wait-time measures can be calculated easily from most follow-up care.21,28,29 Recognizing these complexities,
scheduling systems to help health care providers continu- VHA policy makers recently shifted to using a DD
ally track access for new patients.16,17 approach to measure wait times (ie, schedulers ask

Downloaded from ajm.sagepub.com at NORTH DAKOTA STATE UNIV LIB on March 13, 2015
Prentice et al. 233

Table 4. Logistic Regressions Predicting Patient Satisfaction Using New-Patient Wait-Time Measures.

Timely Visit VHA Rating Treatment Access Specialist Access VHA Satisfaction
a b
FNA (reference = Q1) (n = 144 538) (n = 200 207) (n = 165 053) (n = 110 807) (n = 199 219)
Q2 0.89c,d 0.96d 0.93d 0.94d 0.95d
Q3 0.82d 0.92d 0.83d 0.84d 0.91d
Q4 0.74d 0.86d 0.73d 0.74d 0.85d
Retrospective CD (reference = Q1) (n = 158 841) (n = 219 772) (n = 181 250) (n = 121 721) (n = 218 677)
Q2 0.84d 0.95d 0.87d 0.86d 0.93d
Q3 0.78d 0.91d 0.80d 0.81d 0.89d
Q4 0.66d 0.83d 0.65d 0.66d 0.81d
Prospective CD (reference = Q1) (n = 144 538) (n = 200 207) (n = 165 053) (n = 110 807) (n = 199 219)
Q2 0.88d 0.97d 0.88d 0.85d 0.96d
Q3 0.85d 0.95d 0.86d 0.85d 0.91d
Q4 0.73d 0.88d 0.72d 0.71d 0.85d
Retrospective DD (reference = Q1) (n = 158 841) (n = 219 772) (n = 181 250) (n = 121 721) (n = 218 677)
Q2 1.06d 1.00 1.01 1.05d 1.01
Q3 1.10d 0.98 1.06d 1.10d 1.01
Q4 1.06d 1.01 1.02 1.08d 1.02
Prospective DD (reference = Q1) (n = 144 538) (n = 200 207) (n = 165 053) (n = 110 807) (n = 199 219)
Q2 1.00 0.96d 0.96d 1.02 1.01
Q3 0.95d 0.92d 0.94d 0.96 0.95d
Q4 0.89d 0.89d 0.86d 0.89d 0.90d

Abbreviations: Q, quartile; VHA, Veterans Health Administration; FNA, first next available; CD, create date; DD, desired date; SHEP, Survey of
Healthcare Experiences of Patients.
a
For the range in days of each quartile for the wait-time measures, refer to Table 3.
b
Sample sizes differ between models because of the retrospective versus prospective wait-time measures and because not all SHEP respondents
answered every satisfaction question.
c
Reported numbers are odds ratios.
d
P < .05.

patients what day they desire their appointment).22 between longer wait times and patient satisfaction are
Results from this study provide some support for the causal because omitted variables may be responsible for
validity of these policy changes. the observed relationship. For example, a flu epidemic
Patients visiting facilities with longer returning-patient may increase waits for care and also decrease satisfaction
prospective DD waits were significantly less satisfied on levels because patients do not feel well. In this case, lower
all 5 patient satisfaction measures (Table 5). In contrast, satisfaction cannot be blamed entirely on access. Because
the returning-patient retrospective DD measure did not of the cross-sectional nature of this study, facility quality
consistently predict patient satisfaction. For this measure, could not be controlled for through facility fixed effects,
if the patient never comes for an appointment (no show and the findings of the present study should be confirmed
rates are ˜ 12.5%) or if a patient or clinic cancels an in future longitudinal studies. Despite this reservation,
appointment and never reschedules it, the appointment is past research has found that longer wait times using capac-
excluded, whereas the prospective DD measure includes ity measures cause poorer health outcomes, especially
all appointments on the day a report is pulled. The longer among older and more vulnerable populations,7-11 so it is
waits in the returning-patient prospective DD wait-time plausible that administrative wait times are causally linked
measure compared to the retrospective DD measure to lower patient satisfaction. The sample consists predom-
(Table 3) combined with the significant relationships inantly of adult males, so results may not be generalizable
between the returning-patient prospective DD and satis- to women and children. Finally, the data are combined
faction (Table 5) suggest that prospective DD is a more from a nationwide sample of facilities, so patient experi-
accurate reflection of access to the system for returning ences at specific facilities may not be comparable to each
patients. Future research should confirm the reliability of other.
DD by examining whether the association holds when The recent popularity of interventions such as ACA
predicting other health outcomes. has encouraged clinics to better utilize information
This study has several limitations. The main limitation available in the scheduling system because performance
is that one cannot be certain that the identified relationships metrics based on the scheduling system are much

Downloaded from ajm.sagepub.com at NORTH DAKOTA STATE UNIV LIB on March 13, 2015
234 American Journal of Medical Quality 29(3)

Table 5. Logistic Regressions Predicting Patient Satisfaction Using Returning-Patient Wait-Time Measures.

Timely Visit VHA Rating Treatment Access Specialist Access VHA Satisfaction
a b
FNA (reference = Q1) (n = 144 538) (n = 200 207) (n = 165 053) (n = 110 807) (n = 199 219)
Q2 1.01c 0.97 0.99 1.00 0.99
Q3 1.00 0.95d 1.02 1.03 0.98
Q4 0.90d 0.90d 0.90d 0.90d 0.93d
Retrospective CD (reference = Q1) (n = 158 841) (n = 219 772) (n = 181 250) (n = 121 721) (n = 218 677)
Q2 0.92d 0.96d 0.89d 0.94d 0.98
Q3 0.92d 0.95d 0.87d 0.89d 0.93d
Q4 1.01 1.06d 0.95d 0.99 1.01
Prospective CD (reference = Q1) (n = 144 538) (n = 200 207) (n = 165 053) (n = 110 807) (n = 199 219)
Q2 0.92d 0.92d 0.87d 0.90d 0.92d
Q3 0.93d 0.94d 0.89d 0.89d 0.91d
Q4 1.12d 1.07d 1.03 1.05 1.02
Retrospective DD (reference = Q1) (n = 158 841) (n = 219 772) (n = 181 250) (n = 121 721) (n = 218 677)
Q2 0.99 0.94d 0.97 0.98 0.97
Q3 1.05d 0.98 1.03 1.02 1.00
Q4 0.96d 0.95d 0.94d 0.97 0.98
Prospective DD (reference = Q1) (n = 144 538) (n = 200 207) (n = 165 053) (n = 110 807) (n = 199 219)
Q2 0.91d 0.93d 0.91d 0.88d 0.92d
Q3 0.84d 0.86d 0.85d 0.81d 0.86d
Q4 0.78d 0.85d 0.75d 0.74d 0.83d

Abbreviations: Q, quartile; VHA, Veterans Health Administration; FNA, first next available; CD, create date; DD, desired date; SHEP, Survey of
Healthcare Experiences of Patients.
a
For the range in days of each quartile for the wait-time measures, refer to Table 3.
b
Sample sizes differ between models because of the retrospective versus prospective wait-time measures and because not all SHEP respondents
answered every satisfaction question.
c
Reported numbers are odds ratios.
d
P < .05.

cheaper and easier to obtain than provider surveys taken References

at sporadic intervals.16,17,21 The VHA has a long history
1. Fortney JC, Burgess JF, Bosworth HB, Booth BM, Kaboli
of using a wide variety of wait-time measures and now PJ. A re-conceptualization of access for 21st century health-
http://www.merritthawkins.com/compensation-surveys care. J Gen Intern Med. 2011;26(suppl 2):639-647.
.aspx#; is investing in research to link these measures to 2. Agency for Healthcare Research and Quality. National
patient outcomes. The results of this work suggest that Healthcare Quality Report 2009. Rockville, MD: US
fairly simple modifications to current scheduling sys- Department of Health and Human Services; 2010. Report
tems can support improved wait-time measures that will No.: 10-0003.
better predict patient satisfaction across all patient 3. Institute of Medicine. Crossing the Quality Chasm: A New
populations. Health System for the 21st Century. Washington, DC:
National Academies Press; 2001.
Declaration of Conflicting Interests 4. Merritt Hawkins & Associate. http://www.merritthawkins
.com/compensation-surveys.aspx#; 2009 Survey of physi
The authors declared no potential conflicts of interest with cian appointment wait times. http://www.merritthawkins.
respect to the research, authorship, and/or publication of this com/pdf/mha2009waittimessurvery.pdf. Accessed January
article. 13, 2012.
5. Galewitz P. Health bills in Congress won’t fix doctor short-
Funding age. http://www.kaiserhealthnews.org/stories/2009/octo-
The authors disclosed receipt of the following financial support ber/12/primary-care-doctor-shortage.aspx?referrer=search.
for the research and/or authorship of this article: Funding for Accessed February 5, 2012.
this research was provided by the Department of Veterans 6. Pizer SD, Prentice JC. Time is money: outpatient waiting
Affairs, Veterans Health Administration Systems Redesign times and health insurance choices of elderly veterans in
office. the United States. J Health Econ. 2011;30:626-636.
The views expressed in this article are those of the authors 7. Pizer SD, Prentice JC. What are the consequences of wait-
and do not necessarily represent the position or policy of the ing for health care in the veteran population? J Gen Intern
Department of Veterans Affairs or Boston University. Med. 2011;26(suppl 2):676-682.

Downloaded from ajm.sagepub.com at NORTH DAKOTA STATE UNIV LIB on March 13, 2015
Prentice et al. 235

8. Prentice JC, Fincke BG, Miller DR, Pizer SD. Outpatient 20. United States General Accounting Office. More national
wait times and diabetes care quality improvement. Am J action needed to reduce waiting times, but some clinics have
Manag Care. 2011;17:e43-e54. made progress. Report No. 22 GAO-01-953. http://gao.gov/
9. Prentice JC, Pizer SD. Delayed access to health care and assets/240/232610.pdf. Accessed October 11, 2012.
mortality. Health Serv Res. 2007;42:644-662. 21. Rose KD, Ross JS, Horwitz LI. Advanced access schedul-
10. Prentice JC, Pizer SD. Waiting times and hospitalizations ing outcomes. Arch Intern Med. 2011;171:1150-1159.
for ambulatory care sensitive conditions. Health Serv 22. Department of Veterans Affairs. VHA outpatient schedul-
Outcomes Res Methodol. 2008;8:1-18. ing processes and procedures; 2010. http://www.va.gov/
11. Subramanian U, Ackermann RT, Brizendine EJ, et al. vhapublications/ViewPublication.asp?pub_ID=2252.
Effect of advanced access scheduling on processes and Accessed January 15, 2013.
intermediate outcomes of diabetes care and utilization. J 23. United States General Accounting Office. Audit of the
Gen Intern Med. 2009;24:327-333. Veterans Health Administration’s outpatient scheduling
12. Fogarty C, Cronin P. Waiting for healthcare: a concept procedures. Report No. 04-02887. http://www.va.gov/
analysis. J Adv Nurs. 2008;61:463-471. oig/52/reports/2005/VAOIG-04-02887-169.pdf. Accessed
13. Jatulius D, Bundek N, Legorreta A. Identifying predictors February 1, 2013.
of satisfaction with access to medical care and quality of 24. United States General Accounting Office. Audit of the
care. Am J Med Qual. 1997;12:11-18. Veterans Health Administration’s outpatient waiting times.
14. Van Deusen-Lukas C, Meterko M, Mohr D, Seibert MN. Report No. 07-00616-199. http://www.va.gov/oig/52/
The Implementation and Effectiveness of Advanced Clinic reports/2007/vaoig-07-00616-199.pdf. Accessed February
Access. Boston, MA: Office of Research and Development, 5, 2013.
Department of Veteran Affairs; 2004. 25. VA Office of Inspector General. Review of veterans’
15. Resneck J Jr, Pletcher MJ, Lozano N. Medicare, Medicaid, access to mental health care. Report No. 12-00900-168.
and access to dermatologists: the effect of patient insur- http://www.va.gov/oig/pubs/VAOIG-12-00900-168.pdf.
ance on appointment access and wait times. J Am Acad Accessed May 1, 2012.
Dermatol. 2004;50:85-92. 26. Prentice JC, Graeme Fincke B, Miller DR, Pizer SD.
16. Gupta D, Potthoff S, Blowers D, Korlett J. Performance met- Primary care and health outcomes among older patients
rics for advanced access. J Healthc Manag. 2006;51:246- with diabetes. Health Serv Res. 2012;47:46-67.
258. 27. Burnett-Zeigler I, Zivin K, Ilgen MA, Bohnert ASB.
17. Murray M, Berwick DM. Advanced access: reducing wait- Perceptions of quality of health care among veterans
ing and delays in primary care. JAMA. 2003;289:1035- with psychiatric disorders. Psychiatric Services. 2011;
1040. 62:1054-1059.
18. United States General Accounting Office. VA health care: 28. Gerard K, Salisubury C, Street D, Pope C, Baxter H. Is fast
status of efforts to improve efficiency and access. Report access to general practice all that should matter? A discrete
No. GAO/HEHS-98-48. http://www.gao.gov/archive/1998/ choice experiment of patient preferences. J Health Serv Res
he98048.pdf. Accessed January 15, 2013. Policy. 2008;13(suppl 2):3-10.
19. United States General Accounting Office. VA needs better 29. Salisbury C, Goodall S, Mongromery AA, et al. Does
data on extent and causes of waiting times. Report No. GAO/ advanced access improve access to primary health
HEHS-00-90. http://www.gao.gov/assets/240/230347.pdf. care? Questionnaire survey of patients. Br J Gen Pract.
Accessed December 22, 2012. 2007;57:615-621.

Downloaded from ajm.sagepub.com at NORTH DAKOTA STATE UNIV LIB on March 13, 2015