Lester & Goggin
Lester & Goggin
net/publication/237116150
CITATIONS READS
119 3,591
2 authors, including:
Malcolm Goggin
University of Colorado
35 PUBLICATIONS 1,276 CITATIONS
SEE PROFILE
All content following this page was uploaded by Malcolm Goggin on 11 February 2014.
POLICY
CURRENTS
The University of New Mexico NEWSLETTER OF THE PUBLIC POLICY SECTION
Institute for Public Policy
AMERICAN POLITICAL SCIENCE ASSOCIATION
1
Back to the Future September 1998
One would hope that the discipline of political sci- The Reformers
ence would continue to contribute something of use to Reformers are those who are quite positive about
decisionmakers in the contemporary “implementation the need to continue implementation research, but they
era” of the 1990s and beyond. Thus, in this paper, we also see the need to modify our conceptual or method-
are motivated both by a desire to advance the theoreti- ological approaches. These individuals have consis-
cal understanding of this concept as well as to continue tently supported implementation research, but they
to assist policy implementors in carrying out their re- also see the need to build upon and/or modify what ex-
sponsibilities during the period of the “devolution ists in the extant literature.13
revolution” in the 1990s.11 Other reformers include Robert Stoker, who ar-
Before we propose a means of energizing the gues that his “implementation regime framework,”
study of public policy implementation, we need to re- provides a useful framework to invigorate the study of
consider past evaluations of implementation research public policy implementation.14 He suggests that “a
as well as to re-establish the importance of this area of new perspective is required if implementation is to be
the policy cycle and the need for a discipline like po- seen as a problem of governance in which cooperation
litical science to contribute toward a more complete between independent authorities must be induced.”15
understanding of that process. What follows is our at- Finally, Richard Matland, in an effort to reconcile
tempt to stimulate a renaissance of interest in policy the two major schools of thought on policy implemen-
implementation research and to redirect research in tation (i.e., the “top-downers” versus the “bottom-up-
more fruitful ways. pers”), provides an alternative conceptual framework
II. Looking Backwards: Evaluating the Evolution based on the theoretical significance of ambiguity and
of Implementation Studies conflict.16 Collectively, these scholars have never
As we suggested above, a number of policy schol- doubted the utility of implementation research; however
ars have dismissed the study of public policy imple- they see the need to reform it in constructive ways.17
mentation for various reasons. As we see it, various The Testers
researchers may be categorized according to a typol- Testers appear to be quite comfortable with the
ogy that differentiates among them across two dimen- way policy implementation has been studied in the
sions. The first continuum is whether (or not) they are past. Examples of the “testers,” include such scholars
positive about the continuation of implementation as Mazmanian and Sabatier (in their earlier works)18
studies, and the second concerns whether (or not) they and a number of individuals who tested their imple-
advocate modifying the conceptual or methodological mentation framework.19
approaches to implementation studies. In other words, For example, Deborah McFarlane tested their
implementation scholars tend to take either a positive model in the context of federal family planning pro-
or a negative view about the utility of continuing re- grams.20 In addition, Lester and Bowman applied both
search in this area and they either see a need to change the Mazmanian-Sabatier framework and their own
our conceptual or methodological approaches or not to framework toward an explanation of the implementa-
do so. Based on this typology, it appears as though tion of the Resource Conservation and Recovery Act
there are at least four archetypes of scholarship. Our typol- of 1976 in the fifty American states.21 Fundamentally,
ogy is presented in Figure 1 and we discuss each of these these scholars are rather neutral as far as any criticisms
implementation types, with prime examples, below:12 of implementation research are concerned; instead,
their primary purpose is to test a particular framework
Figure 1: A Typology of Implementation Scholars and to identify the “crucial variables” believed to af-
fect policy implementation.22 Whatever criticisms they
voice are placed within the context of the model they
are testing; they do not advocate new directions in
Continuation of Implementation Research
methodological approaches or conceptual frameworks.
Positive Negative The Skeptics
Modifications Needed
jettisoning the contemporary approach to implementa- analysis. As we view implementation research, it has
tion research, he advocates combining the “participa- made significant progress since its inception in the
tory approach” to policy analysis with the early 1970s and it is on the verge of major theoretical
“post-positivist approach.” Both of these develop- breakthroughs today. Our major concern is that far too
ments, he argues, “can be seen as being strongly con- many scholars of public policy have seemingly ig-
trary to most implementation research and could have nored the promising leads advanced by a number of
a direct effect on a renewed study of policy implemen- implementation researchers and have instead opted to
tation.”24 pursue other lines of research inquiry, such as policy
In addition, he is joined by some postmodernists change or policy design. This behavior is not unique to
who argue that the analysis of policy implementation the field of public policy, but it has serious conse-
is simply too complex for a positivist approach and in- quences for the advancement of the study of imple-
stead they advocate a more intuitive or interpretive ap- mentation, multiple consequences for the discipline of
proach.25 For example, Charles Fox argues that political science, and far-reaching consequences for
implementation has been far too preoccupied with decisionmakers who are responsible for the successful
positivist epistemology and that future research would implementation of policies adopted in the last thirty
greatly benefit from the application of more intuitive years.30
approaches.26 The Importance of Implementation Research in the
The Terminators Next Millennium
The terminators are those who want to discontinue For several reasons, the late-1990s are a particu-
implementation research and/or change the approach larly exciting time to examine policy implementation
to analysis of this part of the policy cycle altogether. in the American context (or foreign contexts for that
They are distinguished from the skeptics in that they matter).31 First, the 1980s-1990s were a period of
have no desire at all to see research on policy imple- implementation of many federal policies enacted dur-
mentation continue. The best-known example of the ing the previous two decades. For example, in the en-
“terminators” category is Paul Sabatier, who in the vironmental area, these policies included the Clean Air
late-1980s, argued that “the implementation literature Act Amendments of 1970, the Federal Water Pollution
seems to have reached somewhat of a dead end.”27 In- Control Act Amendments of 1972, the Resource Con-
stead of a continuation of research on policy imple- servation and Recovery Act of 1976, the Safe Drinking
mentation, Sabatier believes that we must move on to Water Act of 1974, the Surface Mining Control and
the study of “policy change and learning,” and he and Reclamation Act of 1977, the Comprehensive Envi-
his colleague, Hank Jenkins-Smith, have introduced a ronmental Response, Compensation and Liability Act
model and a research agenda to move scholars toward of 1980, the Hazardous and Solid Waste Amendments
that area of research.28 of 1984, and the Superfund Amendments and Reau-
Other terminators, like Helen Ingram, argue that thorization Act of 1986. Like the 1980s, the decade of
the field of implementation studies “has not yet the 1990s has been an “implementation era” in envi-
achieved conceptual clarity,” and she (together with ronmental policy, as well as in many other areas of
Anne Schneider) moved on to the study of policy de- public policy.
sign.29 In that sense, then, they share the perspective Second, intergovernmental relations have recently
of other terminators that the study of policy implemen- taken on greater significance than ever before. During
tation is passé (at best) or not at all worthwhile (at the 1980s, and continuing into the 1990s, the doctrine
worst). After these assorted dismissals, the study of of “New Federalism” has stressed devolution of au-
policy implementation appeared to have died a quiet thority from the federal level to the state and local lev-
death (or so it seemed). els in many areas of public policy. As part of the
III. The Rumor of Our Death is Greatly Exagger- legacy of the Reagan, Bush, and Clinton presidencies,
states and local communities are taking on many re-
ated: The Premature Demise of Implementation sponsibilities that were previously the province of the
Studies federal government.
We will argue in this article (and later in a forth- Third, the states themselves have undergone a
coming book), that the study of public policy imple- number of important transformations in terms of their
mentation is very important at this particular time, that institutional capacities for implementing federal pro-
it has great practical utility to decisionmakers who grams. For example, the states have improved their
must implement public policies, and that it is clearly revenue systems, strengthened the Governor’s office,
not at an “intellectual dead-end.” To the contrary, professionalized their legislatures, reformed their
there are a number of useful theoretical insights from courts, consolidated their bureaucracies, etc.32 Pre-
recent research that can be put to work to further assist sumably, states are no longer the “weak link” in the in-
our understanding of this important concept in policy tergovernmental system of the United States. These
3
Back to the Future September 1998
enormous changes of the past two decades are so far- former sought to introduce new theoretical constructs
reaching as to constitute a definite break with the past into the analysis of interorganizational implementa-
and they suggest that the states are now more able than tion, the latter sought to examine whether devolution
ever before to assume important roles in the delivery of policies and programs to the states was a good idea.
of public policies in environmental (and other) area(s). Both lines of inquiry, we argue, have provided us with
Therefore, it is essential that we better understand the a number of fresh theoretical insights that could ener-
theoretical and practical implications of federal devo- gize the study of public policy implementation in the
lution of authority. As Robert Putnam once remarked, remainder of this century and beyond.
“America is in the midst of an historic transfer of au- For example, in Implementation Theory and Prac-
thority from the federal government to the states, with- tice, Malcolm Goggin and his associates utilize “com-
out any serious assessment of the possible costs (as munications theory” as a theoretical guide to the study
well as benefits) and there may be cause to wonder of policy implementation.40 Communications theory
whether the current bipartisan consensus on the virtues provides a means of understanding the relationships in
of devolution may not be profoundly and dangerously intergovernmental (especially federal-state) policy
misguided.”33 Scholars over the next few decades will implementation. State level implementors form the
likely be engaged in the study of intergovernmental re- nexus for the communications channels and these
lations (and especially federal-state relations) as they implementors are the target of implementation-related
affect the formulation, implementation, and impact of messages transmitted from both federal and local level
many public policies. For example, these future inves- senders. As recipients, state level implementors must
tigators are likely to be concerned with the extent to interpret a barrage of messages. Structuring the inter-
which the fifty American states are providing leader- pretation process are the form and content of the mes-
ship in the environmental area or conversely, the ex- sage and the legitimacy and reputation of the sender.
tent to which they are not effectively implementing Therein lies the key to implementation’s variability.
federal laws dealing with pollution or other areas of More specifically, they incorporated in their
public policy.34 conceptualization of implementation a recognition of
Fourth, the literature on knowledge utilization has the joint nature of decisions and actions of interdepen-
undergone tremendous growth over the past fifteen dent institutions at the subnational level of govern-
years or so. Most of the work has been concerned with ment, and the bargaining that takes place among them
answering the following question: What are the char- and between them and the national level of govern-
acteristics of social science research studies that make ment. Given the assumptions underpinning their model
them most “useful” for decisionmaking? According to of intergovernmental policy implementation, communi-
Weiss and Bucuvalas “useful” involves: (1) whether cations theory offers a means of synthesizing the “top-
the content makes an intrinsic contribution to the work down” and “bottom-up” approaches that dominate (and
of an agency; and (2) whether government officials divide) the implementation literature today.
say they would be likely to take that research into ac- In Reluctant Partners, Robert Stoker introduces
count in decisionmaking.35 Both national and the concept of “regime theory” as a theoretical guide
subnational implementors continue to seek useful to the understanding of implementation.41 He argues
policy research and guidance for policy implementa- that “a new perspective is required if implementation
tion. Due to the above considerations, implementation is to be seen as a problem of governance in which co-
research is particularly relevant and much sought after operation between independent authorities must be
at the present time. Indeed, it is one of the most useful induced.”42 A “regime perspective” establishes an ar-
areas for decisionmakers who seek policy analysis and rangement among implementing participants that
advice.36 identifies the values to be served during the imple-
Finally, there are a number of insights from imple- mentation process and provides an organizational
mentation research conducted in the 1990s that, while framework to promote those values.43 He further ar-
useful to both the theory and practice of policy imple- gues that the key misconception that exists in the
mentation, have nevertheless been largely ignored. Be- implementation literature is that the likelihood of co-
ginning in the early 1990s, a “third generation” of operation is inseparable from the intensity of conflict
implementation researchers (including both senior of interest. Thus, cooperation from the implementors
scholars and younger members of the profession) be- is unlikely whenever conflict exists over the objec-
gan a concentrated effort to better understand the tives of national policy. In response, one must either
policy implementation process.37 Some of their contri- suppress conflict or surrender control of policy in or-
butions were theoretically-driven in terms of their re- der to realize cooperation. Implementation regimes
search goals,38 while others were empirically-based may thus be arranged so that cooperation is more
studies of federal-state implementation.39 While the likely.44
4
Back to the Future September 1998
Laurence O’Toole has suggested the possibilities Finally, Denise Scheberle argues that implementa-
(as well as the constraints) posed by the use of rational tion success depends upon differing levels of “trust
choice theory (especially game theory) as a useful way among implementing officials” and “involvement by
to analyze interorganizational implementation.45 More oversight personnel.”52 She suggests that four patterns
specifically, he argues that rational-choice approaches are likely under these conditions: 1) “pulling together
have been essentially neglected in the study of policy and synergistic,” characterized by high trust and high
implementation. Perspectives like game theory offer involvement; 2) “cooperative but autonomous,” char-
several advantages to the study of policy implementa- acterized by high trust and low involvement; 3) “com-
tion, such as including the rigor of deductive theory, ing apart with avoidance,” characterized by low trust
the potential to unite strengths of both “top-down” and and low involvement; and 4) “coming apart and conten-
“bottom-up” perspectives, and an apparatus to assist in tious,” characterized by low trust and high involvement.53
conceptualizing interdependence across “games” as Collectively, these scholars present us with an ar-
well as across actors and decisions.46 ray of potentially important variables that are believed
Finally, several other scholars prefer to discuss to affect implementation success or failure and should
theoretical frameworks based on contingencies, rather be taken into account in future implementation research.
than to try to build a combined or synthetic model.47 IV. Where Do We Go From Here? Charting a
For example, James Lester suggests that successful
New Direction in Implementation Research
state implementation of environmental policy depends
Despite the potential utility of implementation re-
upon two crucial factors: a) a state’s overall commit-
search, a new direction for implementation research
ment to environmental protection, and b) a state’s
cannot be charted until we accomplish three things.
overall institutional capacity to implement federal en-
First, we must clearly define (and collectively agree
vironmental laws. Based on this typology, Lester ar-
upon) what we mean by implementation. One of the
gues that the states may be divided into four categories
most intractable problems in implementation research
with different patterns (i.e., levels of success) of
has been how to measure the concept of successful
implementation. These four categories include the
implementation.54 In our view, policy implementation
“progressive” states (with high commitment and high
is a process, a series of subnational decisions and ac-
capacity), the “struggler” states (with high commit-
tions directed toward putting a prior authoritative fed-
ment but low capacity), the “delayer” states (with low
eral decision into effect. The essential characteristic of
commitment and high capacity), and the “regressive”
the implementation process, then, is the timely and
states (with low commitment and low capacity).48
satisfactory performance of certain necessary tasks re-
Similarly, William Lowry develops an implementation
lated to carrying out the intent of the law. This means
framework based on “vertical” involvement by the
rejecting a dichotomous conceptualization of imple-
federal government and “horizontal” potential for in-
mentation as simply success or failure.
terstate competition.49 His basic thesis is that state
leadership in environmental policy implementation is Second, we must identify the implementors and
affected by intergovernmental dimensions of the policy their respective roles in the larger political and admin-
involved. To wit, “policies are not simply created by na- istrative system. The primary responsibility for imple-
tional officials and then routinely implemented by state mentation rests with administrators in the executive
and local governments as if they were unquestioning au- branch. But rarely is it the case that administrators act
tomatons in some Weberian machine. Rather, state offi- alone. The President, members of the U.S. House of
cials make policy and adjust national efforts to match Representatives and Senate, state legislators, judges,
parochial circumstances.”50 His typology thus produces spokespersons for organized interests, community or-
four different styles of implementation. ganizations, and members of the public often constrain
the choices of administrators during the policy imple-
In addition, Richard Matland argues that the “am-
mentation process. While all these actors are involved
biguity-conflict model,” which he developed, presents
in policy implementation, we still need to ascertain
us with a more comprehensive and coherent model of
their relative import.
policy implementation than what has existed hereto-
fore. More specifically, four implementation perspec- Moreover, in order to understand more fully the
tives are developed in the model, based on a policy’s strategic choices of implementors and to be able to ex-
ambiguity and conflict levels: 1) administrative imple- plain and predict implementation outcomes, we also
mentation, characterized by low levels of conflict and need to know what are the interests, motives, and re-
ambiguity; 2) political implementation, characterized sources of individual implementors and to whom are
by high conflict and low ambiguity; 3) experimental implementors accountable?55 In other words, what
implementation, characterized by low conflict and role orientations do they ultimately adopt and whose
high ambiguity; and 4) symbolic implementation, char- interests are served? For example, in performing
acterized by high levels of conflict and ambiguity.51 implementation tasks, the bureaucrat acts as an agent,
5
Back to the Future September 1998
responding to directives from principals such as the MA: MIT Press, 1977); George C. Edwards,
President, the Congress, the Courts, and the public. Implementing Public Policy (Washington, D.C.:
The conventional wisdom is that politicians exercise Congressional Quarterly Press, 1980); Carl Van
considerable power and control over bureaucratic deci- Horn, Policy Implementation in the Federal System
sion making.56 But with multiple constituencies in the (Lexington, MA: D.C. Heath, 1979); Robert T.
implementor’s external environment, whose interests Nakamura and Frank Smallwood, The Politics of
are to be served? A number of recent empirical studies Policy Implementation (New York, NY: St. Martin’s
attempt to answer this question.57 While we are aware Press, 1980); Walter Williams, The Implementation
of various role orientations adopted by bureaucrats, we Perspective (Berkeley, CA: University of California
still do not know which roles are paramount and which Press, 1980); and Steven Kelman, “Using Imple-
are less important. mentation Research to Solve Implementation
Finally, and most importantly, we must develop a Problems,” Journal of Policy Analysis and Man-
parsimonious, yet complete, theory of policy imple- agement, Vol. 4, (1984): 75-91.
mentation and a set of testable hypotheses that explain 5
See Paul A. Sabatier and Hank C. Jenkins-Smith,
variations in the way implementors behave. In other Policy Change and Learning: An Advocacy
words, what accounts for variations in the behavior of Coalition Approach (Boulder, CO: Westview Press,
implementors across time, across policies, and across 1993), p. 1.
units of government? By incorporating the insights of
6
communications theory, regime theory, rational choice See Paul A. Sabatier, “An Advocacy Coalition
theory (especially game theory), and contingency theo- Framework of Policy Change Within Subsystems:
ries, a “meta-theory” may perhaps be developed. The Effects of Exogenous Events, Strategic
While this new theory of policy implementation Interaction, and Policy-Oriented Learning Over
should build upon previous work to chart a path to the Time,” A Paper Prepared for the Annual Meeting of
future, it should also recognize the contributions of the Western Political Science Association, Las
those involved in the study of implementation in the Vegas, Nevada, March 28, 1985; see also Paul A.
decade of the 1990s. Sabatier and Hank C. Jenkins-Smith, Policy
In conclusion, we suggest that policy scholars Change and Learning: An Advocacy Coalition
who stress the challenging theoretical aspects of Approach (Boulder, CO: Westview Press, 1993).
implementation studies are more likely to save or 7
See Helen M. Ingram, “Implementation: A Review
strengthen it than are the critics who diminish it (or and Suggested Framework,” in Naomi Lynn and
terminate it) by their premature and excessive dismiss- Aaron Wildavsky, eds., Public Administration: The
als. As John Stuart Mill reminds us, the meaning of a State of the Art (Chatham, NJ: Chatham House
doctrine is in danger of being lost if it ceases to be dis- Publishers, 1990); see also Helen M. Ingram and
cussed or if a “public truth” is not reexamined over Anne Schneider, “Social Construction of Target
and over again. Populations,” American Political Science Review,
Vol. 87, (June, 1993): 334-346; and Anne L.
Notes
Schneider and Helen M. Ingram, Policy Design for
1
See William A. Glazer, “The Type and Uses of Democracy (Lawrence, KN: University of Kansas
Political Theory,” Social Research Vol. 22 (1955): Press, 1997).
275-296. 8
See Peter DeLeon, “The Missing Link Revisited:
2
See Richard S. Ruderman, “Aristotle and the Contemporary Implementation Research,” A Paper
Recovery of Political Judgement.” American Prepared for the Annual Meeting of the American
Political Science Review 91 (June 1997): 409-420. Political Science Association, Washington, D.C.,
3 August 27-31, 1997.
See Stuart Nagel, “Evaluating Public Policy Evalua-
tion,” Policy Studies Review, Vol. 16, (Winter, 9
See, for example, Charles J. Fox, “Implementation
1987): 219-233. Research: Why and How to Transcend Positivist
4 Methodologies,” in Dennis J. Palumbo and Donald
Indeed, this was the point made by some of the first
J. Calista, eds., Implementation and the Policy
scholars to write about public policy implementa-
Process: Opening Up the Black Box (Westport, CT:
tion. See, for example, Martha Derthick, “Defeat at
Greenwood Press, 1990), pp. 199-212; see also
Ft. Lincoln,” The Public Interest, Vol. 20, (Summer,
DeLeon, op. cit.
1970): 3-39; Jeffrey Pressman and Aaron
Wildavsky, Implementation (Berkeley, CA: 10
See James M. Rogers, “Social Science Disciplines
University of California Press, 1973); Eugene and Policy Research: The Case of Political Sci-
Bardach, The Implementation Game (Cambridge,
6
Back to the Future September 1998
ence,” Policy Studies Review, Vol. 9, (Autumn, 479; and Soren Winter, “Integrating Implementa-
1989): 13-28. It is interesting to note, however, that tion Research,” in Palumbo and Calista, op. cit.
a recent survey of public policy scholars did not 18
See, for example, Daniel A. Mazmanian and Paul A.
even mention policy implementation, per se, as an
important component of a good public policy Sabatier, Implementation and Public Policy
(Glenview, IL: Scott, Foresman, 1983).
program. Rather, they emphasized only policy
formulation. See Hank C. Jenkins-Smith, “Design- 19
For a list of the scholars who applied their frame-
ing Public Policy Programs,” Policy Currents, Vol. work, see Paul A. Sabatier, “Top-Down and
8, (June, 1998): 1. Bottom-Up Approaches to Implementation Re-
11 search: A Critical Analysis and Suggested Synthe-
See Richard P. Nathan, “The Devolution Revolution,”
in Symposium: American Federalism Today (Albany, sis,” Journal of Public Policy, Vol. 6, (1986): 21-48.
NY: Rockefeller Institute, 1996), p. 5; and Enid 20
See Deborah McFarlane, “Testing the Statutory
Beaumont, “Status of the Devolution Revolution,” Coherence Hypothesis: Implementation of Family
The Public Manager, (Winter, 1996-97): 23-26. Planning Policy in the States,” Administration and
12 Society, Vol. 20, (1989): 395-422; See also Kenneth
This typology does not mean to imply that these
Meier and Deborah McFarlane, “Statutory Coher-
implementation scholars are static in terms of their
research agendas. It is possible, and indeed quite ence and Policy Implementation: The Case of
Family Planning,” Journal of Public Policy
common, for implementation scholars to move
(September-December, 1995).
from one category to another category over the
course of their professional lifetimes. For example, 21
See James P. Lester and Ann O’M. Bowman,
Paul Sabatier has moved from the “tester” category “Implementing Environmental Policy in a Federal
to the “terminator” category; Lester and Bowman System: A Test of the Sabatier-Mazmanian Model,”
have moved from the “tester” category to the Polity, Vol. 21, (Summer, 1989): 731-753; see also
“reformer” category. This fact, however, does not James P. Lester, “Hazardous Waste and Policy
diminish the utility of the typology because at any Implementation: The Subnational Role,” Hazard-
given point in time, individual implementation ous Waste and Hazardous Materials, Vol. 2, (Fall,
scholars tend to occupy a single cell of the typol- 1985): 381-397; and Ann O’M. Bowman and James
ogy. P. Lester, “Policy Implementation in a Federal
13 System: A Comparative State Analysis,” in Fred
See Malcolm L. Goggin, et al., Implementation
Meyer and Ralph Baker, eds., State Policy Problems
Theory and Practice: Toward a Third Generation
(New York, NY: Harper Collins, 1990); see also (Chicago, IL: Nelson-Hall, 1993), pp. 156-176.
James P. Lester, et al., “Public Policy Implementa- 22
See Thomas A. Sinclair, “Implementation Theory
tion: Evolution of the Field and Agenda for Future and Practice: Uncovering Policy and Administra-
Research,” Policy Studies Review Vol. 7, (Autumn, tion Linkages in the 1990s,” International Journal
1987): 200-216. Reprinted in Stuart S. Nagel, ed., of Public Administration (Forthcoming, 1999);
Research in Public Policy Analysis and Manage- Charlyn Cassady, et. al, “Measuring the Implemen-
ment (Greenwich, CT: JAI Press, 1995). tation of Injury Prevention Programs in State
14 Health Agencies,” Injury Prevention, Vol. 3, (June,
See Robert P. Stoker, Reluctant Partners: Imple-
1997): 94-99; Malcolm Goggin and Steven
menting Federal Policy (Pittsburgh, PA: The
University of Pittsburgh Press, 1991); see also Laubacher, “Administrative Initiative in Policy
Implementation: Mental Retardation
Robert Stoker, “A Regime Framework for Imple-
Deinstitutionalization Policy in Texas,” A Paper
mentation Analysis,” Policy Studies Review, Vol. 9,
(Autumn, 1989). Prepared for the Annual Meeting of the Midwest
Political Science Association, Chicago, Illinois,
15
Ibid., p. 44. April 5-7, 1990; Lucinda M. Deason, “Bureaucratic
16 Discretion: An Examination of External Influences
See Richard E. Matland, “Synthesizing the Imple- on State Implementation of Federal Mandates,” A
mentation Literature: The Ambiguity-Conflict
Paper Prepared for the Annual Meeting of the
Model of Policy Implementation,” Journal of
Midwest Political Science Association, Chicago,
Public Administration Research and Theory, Vol. 5, Illinois, April 23-25, 1998; Marjorie Sarbaugh-
(April, 1995): 145-174.
Thompson, “Change From Below: Integrating
17
See also Yeheskel Hasenfeld and Thomas Brock, Bottom-Up Entrepreneurship Into a Program
“Implementation of Social Policy Revisited,” Development Framework,” American Review of
Administration and Society, Vol. 22, (1991): 451- Public Administration, Vol. 28 (March, 1998): 3-
7
Back to the Future September 1998
29
25; and N. Ryan, “Some Advantages of an Inte- See Helen M. Ingram and Anne Schneider, “Improv-
grated Approach to Implementation Analysis,” ing Implementation Through Framing Smarter
Public Administration, Vol. 74, (1996): 737-753. Statutes,” Journal of Public Policy (1990); see also
23 James E. Garrett, “Public Administration and
See Peter DeLeon, “The Missing Link Revisited: Policy Implementation: A Social Work Perspec-
Contemporary Implementation Research,” A Paper
tive,” Journal of Public Administration, Vol. 16,
Prepared for the Annual Meeting of the American
(1993): 1247-1263.
Political Science Association, Washington, D.C.,
30
August 27-31, 1997, pp. 3-4. See Nagel, op. cit.; and David Ricci, The Tragedy of
24 Political Science: Politics, Scholarship, and
Ibid., p. 15. see also Peter DeLeon, “Participatory
Democracy (New Haven, CT: Yale University
Policy Analysis: Prescriptions and Precautions,” Press, 1984).
The Asian Journal of Public Administration, Vol.
31
12, (June, 1990): 29-54; and Peter DeLeon, Advice This section borrows heavily from James P. Lester,
and Consent: The Development of the Policy “A New Federalism? Environmental Policy in the
Sciences (New York, NY: Russell Sage, 1988). States,” in Norman Vig and Michael Kraft, eds.,
25 Environmental Policy in the 1990s (Washington,
See Fox, op. cit; see also Dvora Yanow, “Tackling
D.C.: Congressional Quarterly Press, 1994).
the Implementation Problem: Epistemological
32
Issues in Implementation Research,” in Palumbo See Ann O’M. Bowman and Richard C. Kearney,
and Calista, op. cit., pp. 213-227. The Resurgence of the States (Englewood Cliffs,
26 NJ: Prentice-Hall, 1986).
See Fox, op. cit. These suggestions, it seems to us,
33
would add very little in the way of conceptual See his comments on John D. Donahue’s Disunited
improvements since they are more concerned with States: What’s at Stake as Washington Fades and
epistemological preferences that lean toward the States Take the Lead (New York, NY: Basic
“bottom-up” approaches. As such, they do not Books, 1997) as reported in the American Political
present us with a reconciliation of the two opposing Science Review.
approaches; rather, they continue the debate. 34
See James P. Lester and Emmett N. Lombard, “The
Moreover, critics of participatory approaches to
Comparative Analysis of State Environmental
policymaking often argue that increased citizen
involvement in policymaking and implementation Policy,” Natural Resources Journal, Vol. 30
(Summer, 1990): 301-319.
would lead to an increase in group dissensus over
program goals and procedures; that it will also lead 35
This section draws on James P. Lester, The Utiliza-
to needless delays in policy formation and imple- tion of Policy Analysis by State Agency Officials,”
mentation; that the costs of policymaking and Knowledge: Creation, Diffusion, Utilization, Vol.
implementation would increase dramatically; and 14 (March, 1993): 267-290; see also C. Weiss and
that disaffected interests would seek to obstruct M. J. Bucuvalas, “The Challenge of Social Re-
programs through litigation or recourse to Con- search to Decisionmaking,” in C. Weiss, ed., Using
gress. In addition to these concerns, where such Social Research in Public Policymaking. (Lexing-
participatory experiments have been tried in ton, MA: Lexington Books, 1977).
Europe, confusion and conflict among public 36
participants increases. On these two points, see See James M. Rogers, “Social Science Disciplines
Walter A. Rosenbaum, “The Paradoxes of Participa- and Policy Research: The Case of Political Sci-
tion,” Administration and Society, Vol. 8 (1976): ence,” Policy Studies Review (1989): 13-28.
355-383 and Dorothy Nelkin, Technical Decisions 37
The idea of “three generations” of implementation
and Democracy (Beverly Hills, CA: Sage, 1977). research has been popularized by Goggin, et al., op.
27
See Paul A. Sabatier, “Beyond Implementation: The cit.; see also T. Younis and Ian Davidson, “The
Need for a Longer Time Perspective,” A Paper Study of Implementation,” in T. Younis, ed.,
Prepared for the Annual Meeting of the Midwest Implementation in Public Policy (Aldershot, UK:
Political Science Association, Chicago, Illinois, Dartmouth, 1990).
April 13-15, 1989. 38
See Goggin, et al., op. cit; Stoker, op. cit; O’Toole,
28
See Paul A. Sabatier and Hank Jenkins-Smith, eds., op. cit; Matland, op. cit;
Policy Change and Learning: An Advocacy 39
See William Lowry, The Dimensions of Federalism
Coalition Approach (Boulder, CO: Westview Press, (Durham, NC: Duke University Press, 1992); Evan
1993). Ringquist, Environmental Protection at the State
8
Back to the Future September 1998
Level (Armonk, NY: M.E. Sharpe, 1993); Donahue, Press, 1990), pp. 59-79. A revised version was
op. cit; Susan Hunter and Richard W. Waterman, published in the second edition in 1994.
Enforcing the Law: The Case of the Clean Water 49
See William Lowry, The Dimensions of Federalism:
Acts (Armonk, NY: M.E. Sharpe, 1996); Caroline
M. Orth, “State Implementation of Air and Water State Governments and Pollution Control Policies
(Duke University Press, 1992), especially pp. 1-26.
Pollution Control Policy: An Analysis of State
Ecological Capacity,” A Paper Prepared for the 50
Ibid., pp. 3-4.
Annual Meeting of the Midwest Political Science 51
Association, Chicago, Illinois, April 10-12, 1997; Matland, op. cit., pp. 159-162.
and Denise Scheberle, Federalism and Environ- 52
See Denise Scheberle, Federalism and Environmen-
mental Policy (Washington, D.C.: Georgetown tal Policy: Trust and the Politics of Implementation
University Press, 1997). For a review of the (Washington, D.C.: Georgetown University Press,
primary contributions of some of these studies, see 1997).
James P. Lester, “Comparative State Environmental 53
Politics and Policy: The Evolution of a Literature,” Ibid., pp. 16-23.
Policy Studies Journal, Vol. 22 (Winter, 1994): 696- 54
See Dennis J. Palumbo, Steven Maynard-Moody, and
700. Paula Wright, “Measuring Degrees of Successful
40
Goggin, et al., op. cit., pp. 31-41; see also Richard J. Implementation,” Evaluation Research, Vol. 8,
Stillman, Public Administration: Concepts and (1984): 45-74; see also, Goggin, et al., op. cit., pp. 44-
Cases (Boston, MA: Houghton Mifflin, 1992), pp. 47 and 173-174; and Scheberle, op. cit., pp. 24-26.
391-403. While we find that our previous work has 55
This issue has been taken up by a number of authors.
been heavily cited during the period from 1991- See, for example, Emmette S. Redford, Democracy
1998, few have taken up the challenge to do “third in the Administrative State (New York, NY: Oxford
generation” implementation research. This type of University Press, 1969); Charles S. Hyneman,
implementation research is very expensive in that it Bureaucracy in a Democracy (New York, NY:
requires a combination of field work and extensive Harper, 1950); and Douglas Yates, Bureaucratic
data analysis. Few, if any, scholars have been able Democracy: The Search for Democracy and
to secure the necessary funding thus far from the Efficiency in American Government (Cambridge,
National Science Foundation (or other sources) to MA: Harvard University Press, 1982).
undertake such a study.
56
41
For recent studies on these questions, see Robert F.
Stoker, op. cit., pp. 95-110. Durant, “Public Policy, Overhead Democracy, and
42
Ibid., p. 44. the Professional State Revisited,” Administration
and Society, Vol. 27, (1995): 165-202; see also B.
43
Ibid, p. 55. Dan Wood and Richard W. Waterman, Bureaucratic
44
See Stoker, op. cit., pp. xiv. Dynamics: The Role of Bureaucracy in a Democ-
racy (Boulder, CO: Westview Press, 1994);
45
See Laurence J. O’Toole, “Rational Choice and Rosemary O’Leary, Environmental Change:
Policy Implementation,” American Review of Federal Courts and the EPA (Philadelphia, PA:
Public Administration, Vol. 25, (March, 1995): 43- Temple University Press, 1993); Richard Waterman
57; see also John Chubb, “The Political Economy and Kenneth J. Meier, “Principal-Agent Models:
of Federalism,” American Political Science Review, An Expansion?” Journal of Public Administration
Vol. 79, (1985): 994-1015. Research and Theory, Vol. 8, (1998): 173-202; For
46 a brief overview of these materials, see Scott R.
See O’Toole, op. cit., p. 54.
Furlong, “Political Influence on Bureaucracy: The
47
See, for example, Andrew Dunsire, Implementation Bureaucracy Speaks,” Journal of Public Adminis-
in a Bureaucracy (Oxford, UK: Martin Roberson, tration Research and Theory, Vol. 8 (1998): 39-65.
1984); Harald Saetren, The Implementation of For an alternative view, see David B. Spence,
Public Policy (Oslo, Norway: Universitetsforlaget, “Agency Policymaking and Political Control:
1983); and Matland, op. cit. Modeling Away the Delegation Problem,” Journal
48 of Public Administration Research and Theory, Vol.
See James P. Lester, “A New Federalism? Environ-
7 (1997): 199-219.
mental Policy in the States,” in Norman J. Vig and
57
Michael E. Kraft, eds., Environmental Policy in the See Furlong, op. cit.
1990s (Washington, D.C.: Congressional Quarterly
9
Announcement/Skewered September 1998
10
General Information September 1998
Rosemary O’Leary
Indiana University
11
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
September 1998
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
NON PROFIT - ORG.
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
US POSTAGE
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
PAID
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
THE UNIVERSITY OF NEW MEXICO Albuquerque, NM
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
PERMIT NO. 39
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
Institute for Public Policy
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1805 Sigma Chi Rd., NE
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
Albuquerque, NM 87131-1121
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
(505) 277-1099
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
12
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345
1234567890123456789012345678901212345678901234567890123456789012123456789012345678901234567890121234567890123456789012345678901212345