International Journal of Accounting Information Systems 7 (2006) 18 35

Organizational performance effects of ERP systems usage: The impact of post-implementation changesB
Andreas I. Nicolaou a,*, Somnath Bhattacharya b
a

Department of Accounting and MIS, Bowling Green State University, USA b School of Accounting, Florida Atlantic University, USA

Received 11 March 2005; received in revised form 3 December 2005; accepted 10 December 2005

Abstract This paper examines the long-term financial performance effects of ERP system changes/revisions for firms that have previously reported ERP adoptions. The study is motivated by the mixed results of recent studies examining the financial effects of ERP systems and by studies which demonstrate that ERP implementations are modular in nature and thus may yield organizational performance gains or losses as a result of discrete changes that might be initiated after the initial system implementation. This study empirically examines the extent to which discrete changes to ERP systems over a post-implementation time-frame impact on firms ability to deliver long-run financial performance. It further examines whether the timing and nature of system transformation during the post-implementation period presents a significant moderating condition of ERP performance outcomes. We specifically consider the ERP-adopting firms previously examined by Nicolaou [Nicolaou, A.I. 2004a. Firm performance effects in relation to the implementation and use of enterprise resource planning systems. Journal of Information Systems 18 (Fall): 79105] and track them for changes/revisions in the form of enhancements, upgrades, abandonments and switches during the post-implementation period. Two research hypotheses are developed in the study which posit that both the nature and the timing of system implementation changes represent significant conditions for ERP post-implementation success. From the original sample of 247 firms used in Nicolaou [Nicolaou, A.I. 2004a. Firm performance effects in relation to the implementation and use of enterprise resource planning systems. Journal of Information Systems 18 (Fall): 79105], a total of 83 firms were identified to report 182 discrete changes during their ERP system post-implementation period. The results suggest that, in general, subsequent changes in ERP systems often help resolve or surface implementation issues that affect subsequent use of and success from the use of such systems. Specific findings indicate that ERP-adopting firms, which initiate early

B Best Paper Award at the 2nd International Conference on Enterprise Systems and Accounting, Thessalonike, Greece, July 2005. * Corresponding author. Tel.: +1 419 372 2932; fax: +1 419 372 2875. E-mail addresses: anicol@cba.bgsu.edu (A.I. Nicolaou), sbhatt@fau.edu (S. Bhattacharya).

1467-0895/$ - see front matter D 2006 Elsevier Inc. All rights reserved. doi:10.1016/j.accinf.2005.12.002

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

19

enhancements in the form of either add-ons or upgrades, may enjoy superior differential financial performance in comparison to other ERP-adopting firms differential performance. Late enhancements and both early and late abandonments lead to apparent differential performance deterioration for the ERPadopting firms. These findings support the theoretical propositions developed in the study The studys findings provide an important contribution to the ERP systems success literature and present important insights for future research in this area. D 2006 Elsevier Inc. All rights reserved.
Keywords: ERPS post-implementation; Financial performance; ERPS change events

1. Introduction This paper examines the long-term financial performance effects of ERP revisions (enhancements, upgrades, abandonments and switches) for firms that have previously reported ERP adoptions. The study is motivated by the mixed results of recent studies examining the financial effects of ERP systems. Prior studies such as Hunton et al. (2003) found that while ERP adopters demonstrated no change in their post-ERP financial performance, the financial performance of non-adopters deteriorated during the post-implementation period. Poston and Grabski (2000, 2001), on the other hand, reported mixed and somewhat contrarian differences between ERP adopters and non-adopters on a number of financial performance measures. While ERP-adopting firms did no better than their counterparts on some measures, the non-adopting firms actually improved their performance on others. In a long-term examination of differences between ERP adopters and non-adopters, Nicolaou (2004a) reported that a lag of at least 2 years was necessary before adopters would begin to demonstrate positive differential financial performance in comparison to their non-adopting peers. While the time lag may partly explain the results obtained by Poston and Grabski (2000, 2001), a number of conditions were also found to moderate the degree to which use of ERP systems would result in positive differential returns for adopting organizations. In this paper we examine the change/revision condition as a moderating factor to ERP systems success. We posit that the complexity and scope of ERP systems is such that the implementation process typically does not end when the system goes live. In fact, the implementation of ERP systems is considered a strategic investment decision (Cooke and Peterson, 1998; Wah, 2000) with benefits expected to accrue over several periods of time as opposed to one-time windfall gains (Nicolaou 2004a,b; Shang and Seddon, 2000). Additionally, ERP implementations are often modular in nature and thus may yield productivity gains or losses as a result of discrete changes that might be initiated after the initial system implementation (Holland and Light, 2001; Markus et al., 2000; OLeary, 2000). Past research overwhelmingly reports that the immediate after-effects of ERP implementations are fraught with productivity and profitability problems (Davenport, 1998; Poston and Grabski, 2000, 2001; Hitt et al., 2002; Hunton et al., 2003; Nicolaou, 2004a,b). These are thought to be due to possibly severe systems integration problems, misalignment between people, processes and technology, and overall change management issues during and shortly after the implementation process (Nicolaou, 2004a; Murray and Coffin, 2001; Peterson et al., 2001; Ross and Vitale, 2000; Scott and Vessey, 2000; Soh et al., 2000; Stephanou, 2000). Similar productivity paradoxes between IT investment and firm performance have also been recorded with other IT implementations (Bhattacharya et al., 1997; Barua et al., 1995; Brynjolfsson, 1993; Weill,

20

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

1992). With ERP, the temporary deterioration in firm performance and productivity is theorized to be the result of primary value-chain activities being more complex than previously planned (Brown and Vessey, 1999), a lack of understanding of the system by users (Peterson et al., 2001), inadequate training and support for end-users to help them understand the newly adopted business processes and workflows, inadequate system testing and inadequate communication of system objectives (Nicolaou, 2004a). Other causes for this dip include ineffective change management (Motwani et al., 2002; Peterson et al., 2001) and the severity of the implementation mode (Nicolaou, 2004b; Motwani et al., 2002). However, prior evidence also suggests that this dip in performance and productivity following ERP implementations is somewhat short-lived and after a lag of time ERP firms often emerge to higher productivity and performance in the long run. Such lag-led re-emergence of performance gains is often a product of job redefinition, the establishment of new procedures, fine-tuning of the ERP software, and the taking charge of new streams of information created by the ERP system (Nicolaou, 2004a). As mentioned earlier, these are often achieved in discrete steps facilitated by the strong modularity of ERP systems that allow for organizations to gain (lose) from revisions and additions to their original ERP implementations. This study, therefore, empirically examines the extent to which discrete changes to ERP systems over a post-implementation time-frame impact on firms ability to deliver long-run organizational financial performance. Hence, we specifically consider the ERP-adopting firms previously examined by Nicolaou (2004a) and track them for changes/revisions in the form of enhancements, upgrades, abandonments and switches during the post-implementation period. The rest of this paper is divided into the following sections: In the next section we discuss the prior literature on ERP and other system changes and formalize the hypotheses for the study. This is followed by a discussion on the methodology employed in Section 3. Section 4 presents the results of the study and Section 5 presents the studys conclusions, implications, and avenues for future research. 2. Literature review and hypotheses development The results of studies such as those of Poston and Grabski (2000, 2001), Hunton et al. (2003) and Nicolaou (2004a) suggest that the full effects of ERP adoptions for firms do not surface until after a considerable time-lag. Hence, studies such as Poston and Grabski (2000, 2001) that examine performance differences between adopting and non-adopting firms immediately following such adoption do not find any significant differences in the financial performance of adopting and non-adopting firms. On the other hand, studies that incorporate time-lags into their design such as Hunton et al. (2003) and Nicolaou (2004a) appear to find greater evidence of such differences between ERP-adopting and non-adopting firms. But the question remains: Do these performance differences sustain or do they change following post-adoption revisions to the ERP systems undertaken by the same ERP-adopting firms? Prior literature on information technology investments suggests that strategic IT investment such as ERP give firms the ability to gain tangible and intangible benefits that help sustain operational efficiencies in the long run (Nicolaou, 2004a; Kettinger et al., 1994; Mata et al., 1995). Furthermore, Nicolaou (2004a) has shown that these performance benefits typically accrue to ERP-adopting firms only after a lag of approximately 2 years from the date of original rollout. Professional literature also suggests that ERP implementations are typically not unitary and one-time events but involve a series of modular upgrades and enhancements to the original system (Holland and Light, 2001; Markus et al., 2000; OLeary, 2000). While changes in the form of upgrades and enhancements are well documented, evidence also suggests that such

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

21

implementations sometimes go awry and are followed by somewhat less publicized events such as switches to other ERP vendors and/or even total abandonments of such systems. This, therefore, leads us to question whether the firm performance (non)-effects surfaced by earlier studies of ERP implementation sustain over periods over which these same firms undertake ERP revisions/changes. This is reflected in the first hypothesis in its null form: H1. ERP-adopting firms differential performance following post-implementation changes will be no different from their differential performance prior to the adoption of their ERP systems. While research hypothesis H1 is designed to capture the overall effect of a postimplementation transformation, it is important to further examine whether the timing and nature of system transformation during the post-implementation period is also significant. We consider four specific kinds of changes to define the bnature of changeQ criterion. These include enhancements to ERP systems that typically take the form of (i) modular additions to the original implementation and (ii) upgrades that occur as a result of vendor-supported version changes. We also consider (iii) system switches whereby firms switch from one ERP system to another and (iv) more drastic abandonments whereby firms decide to totally abandon their ERP-related processes and revert to non-ERP systems. In terms of timing, we consider (i) early changes which include any of the above mentioned four types of changes that occur in the time period when ERP implementation is completed and the system is rolled over or in the year following system completion, and (ii) late changes that again include all four types of changes which occur at a time starting in the second year after ERP implementation. The nature and timing of such post-implementation changes warrant investigation because of the following reasons: a. Enhancements (upgrades/add-ons) that happen early in the post-implementation period (during the year of completion or in the year following) may signify that the postimplementation review process is well managed and has either identified deficiencies in the initial implementation that need correction or has identified areas of improvement and the system is expanded to better fit needs. To the extent, this improves system acceptance and system reach, an observed early enhancement to an implemented system should enhance the overall positive impact of the ERP adoption event. Further, past literature also indicates that system migrations (upgrades) that are carried out in a timely manner might help an organization differentiate itself from competitors and gain higher differential returns (Kremers and Van Dissel, 2000, p. 56). b. Enhancements (upgrades/add-ons) that occur later in the post-implementation period (at least 2 years after system completion) may signify that the system has actually been well accepted in the organization and now serves as the basic infrastructure for launching other strategic initiatives, such as customer relationship management or supply chain management. Prior literature such as Kremer and Van Dissel (2000), however, suggest that late enhancements are perhaps more likely to signify that necessary adjustments to the system were not performed in a timely manner, which in turn may be indicative of less than optimal control by the firm over its ERP processes. As Kremer and Van Dissel (2000) indicate, organizations which do not undertake timely system migrations may lose on important system benefits with a negative impact on their overall performance. Hence, in giving greater credence to prior empirical findings, we expect that late enhancements have an overall negative impact on ERP-enabled firm performance. c. Vendor switchings and/or system abandonments typically provide signals of prior implementation failure or significant prior implementation difficulties (Nicolaou, 2004b).

22

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

As a result, the earlier this change happens, the sooner the adopting company may be seen to have identified the difficulties and responded to it. This is because firms that struggle with ERP system implementations must initiate dramatic changes in order to reverse the negative effects of faulty ERP processes. The earlier this change takes place, the lesser the negative impact on firm performance in the post-implementation period. To the extent, this improves system acceptance and systems reach, an observed early switch/abandonment should enhance the initial positive impact of the ERP adoption event. d. Late switches and abandonments represent the worst-case scenario. Here the changes represent almost chaotic transformations/reversals whereby firms may be viewed as having had little or no control over the ERP-related change management process from the get-go. Such firms may be viewed as having begun the ERP implementation with significant difficulties that exacerbate over the ERP use period. Hence, such switches and abandonments may represent the sort of horror stories in the popular media that report companies that spend upwards of $40 $50 million to implement ERP systems only to abandon them during the post-implementation period (McAfee, 2003). The above scenarios suggest that the timing and nature of an observed ERP system change should significantly influence the strength of the original ERP system adoption effect. The combined influence of the ERP adoption along with the change events would therefore moderate a firms differential performance in the years following initial system implementation. As noted above, our expectation is that early changes defined as occurring in the year of the adoption or in the year after represent rapid firm response to the perceived need for ERP changes. While early enhancements in the shape of upgrades and add-ons may be viewed positively as representing firms rapid and early movement toward cementing ERP-driven productivity and performance gains, later enhancements may be viewed more negatively. While it is certainly possible that such later enhancements serve as platforms for future modular growth, they may very well be indicative of a firms inability to make rapid positive transformations to their installed ERP base. ERP switches/abandonments, whether early or late, represent pre-installation problems that manifest themselves in the post-implementation period. However, such changes initiated by firms within the year of installation or in the year after it (early changes) represent firm awareness of the problem and rapid reaction to it. Late switches/abandonments, on the other hand, represent situations whereby such firms appear incapable of stemming the effects of faulty ERP implementations until very late in the post-implementation period. All of this suggests that, early changes, irrespective of their form (upgrades, add-ons, switches or abandonments), represent firms ability to respond rapidly to organizational needs for ERP transformation. Therefore, we expect that the differential performance for firms undertaking such early changes will complement and further the differential changes in performance brought about by the ERP adoptions themselves. On the other hand, late changes, irrespective of their form, may represent firms inability to execute timely response to ERP-related organizational issues. Hence, they present a potentially more complex scenario where late upgrades/add-ons may be viewed in a negative manner overall, but late switches/abandonments will almost always be cause for high levels of concern.1 As a result, our second hypothesis, again in null form, states:
1 A counter argument may be that a firm which upgrades/extends its ERPs later has done a much better job during systems planning and analysis than one which has to do so earlier. While late decisions to abandon/switch a system are more convincingly indicative of ineffective systems management, our contention may not hold for upgrades/add-ons irrespective of their form. This, therefore, makes this an empirically testable issue which is why we express the second hypothesis in its null-form as well.

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

23

H2. ERP-adopting firms differential performance following early (late) post-implementation changes will be no different from their differential performance prior to the adoption of their ERP systems. 3. Methodology We use the original Nicolaou (2004a) data set to identify our sample of 247 ERP adopting and non-adopting firm pairs.2 Nevertheless, there is considerable attrition in the original sample size obtained there because not all firms made subsequent revisions to their adopted systems. As a result, our search of the Lexis/Nexis database for post-ERP implementation changes was constrained to the sample of 247 ERP-adopting firms reported in Nicolaou (2004a) and included the following search terms: abandonment, upgrade, add-on and switching. As mentioned earlier, while add-ons to ERP systems typically take the form of modular additions to the original implementation, upgrades occur as a result of vendor-supported version changes. Enhancements may also occur when other-vendor add-ons embellish systems. While these add-ons are not original-vendor supported, they typically improve the ERP systems functionality in customerspecific ways. Of the four changes targeted in this study, we found abandonments and switches more difficult to trace with the former being several degrees more difficult than the latter. This situation arose because both ERP vendors and their customers have vested interests in minimizing publicity related to ERP abandonments.3 While switches are not as stigmatized as pure-play abandonments, they are, nevertheless, often viewed as precursors and predictors of ERP-related trouble. Hence, these too were relatively more difficult to surface when compared to the more favorable enhancement and upgrade criteria. Eventually, we identified 182 discrete changes with the breakdown of 148 add-ons, 15 upgrades, 11 switches and 8 abandonments (see Table 1). 3.1. Identification of ERP completion and post-implementation periods As described in Nicolaou (2004a), a total of 105 of the firms in our sample disclosed completion dates for their ERP projects in both announcements and in the MD&A sections of their annual reports, thus providing useful information about their completion years. For firms that did not disclose their completion year, we adopted Nicolaous (2004a) methodology to assume that their expected completion time was equal to Nicolaous (2004a) mean expected completion time of 9.92 months. Therefore, for those firms, the expected completion year was assumed to be the year following the adoption decision. This assumption allowed for an average completion time period of at least 1 full year. For instance, if a firm had adopted an ERP system sometime in 1997, the completion year was assumed to be the following year 1998. This year of completion (go-live date) was coded as the time period bt 0Q, to indicate the starting (base) period of ERP use.
2 See Nicolaou (2004a) for a detailed discussion of the manner in which the sample of ERP adopting firms was developed and then matched against peer non-ERP adopting firms. 3 It could also be argued that abandonments are rare simply because of the enormous negative impact of these changes especially if they occur after the go-live date. There is some empirical evidence of the enormous impact of canceling ERPS projects before the go-live date (e.g., Davenport, 1998), so it is not surprising that firms try to improve their systems rather than to abandon them and implement a new one. Those, which decide to do so nevertheless, will almost inevitably face a very negative impact on firm performance.

24

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

Table 1 ERP change-types and construction of dichotomous measures for model testing Change type Add-ons Upgrades Switches Abandonments Total Dummy Dummy Dummy Dummy Dummy Dummy Dummy Dummy Dummy variables for Model 1 testing 1 = total number of firms with any change 2 = number of firms with add-ons or upgrades 3 = number of firms with switches or abandonments variables for Model 2 testing variable early add-ons/upgrades variable late add-ons/upgrades variable early switches/abandonments variable late switches/abandonments Discrete count 148 15 11 8 182

83 74 16

74 47 15 6

ERP-adopting firms were then tracked for post-implementation changes in Lexis/Nexis. As can be seen in Table 1, there were 83 change-firms that were identified as such if they made any kinds of changes. Many of these firms engaged in multiple changes, including multiple upgrades and add-ons. We tracked the changes and the specific years of the changes and then compared them to the system completion year. If a change occurred in the system completion year or the year following system completion, we classified such changes as bearlyQ changes. All other changes were classified as blateQ. As can be seen in Table 1, we identified a total of 89 early changes (including 74 early add-ons/upgrades and 15 early switches/abandonments), while a total of 53 late changes were identified (47 late add-ons/upgrades and 6 late switchings/abandonments). 3.2. Performance measures We used the same performance measures as those used by Nicolaou (2004a). Our choice of the same performance measures as Nicolaou (2004a) was motivated not only by our desire to maintain continuance with that study but also by prior literature that has expressed the need for ba process-oriented model of the enterprise to understand the creation of IT impacts, and a scientific approach to measuring the economic consequences of IT investments.Q (Barua et al., 1995). The operational measures as used by Nicolaou (2004a) and repeated here recognize that the primary impact of IT investments is first felt at the lower operational levels of an enterprise. These represent the so-called first-order effects because they measure gains/losses in firms operational efficiency. These first-order effects are expected to eventually contribute to the overall profitability of the enterprise (Bhattacharya et al., 1997). Table 2 shows the different performance measures used in this study. 3.3. Statistical models 3.3.1. Model 1 H1 predicts that the differential performance of ERP-adopting firms versus the performance of non-adopting matched firms will be positively affected through the moderation of ERP adoption

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835 Table 2 Performance measures Performance variable Return on assets (ROA) Compustat definition

25

Income before extraordinary items Available for common, divided by total assets, multiplied by 100. Return on investment (ROI) Return on investment is income before extraordinary items Available for common, divided by total invested capital, which is the sum of the following items: total long-term debt; preferred stock; minority interest; and total common equity. This is then multiplied by 100. Operating return on assets (OIA) Operating income over assets: operating income before depreciation charges (Compustat Item A13), divided by total assets, multiplied by 100. Return on sales (ROS) Income before extraordinary items available for common, divided by net sales, multiplied by 100. Operating income over sales Operating income before depreciation charges (Compustat Item A13), divided (OIS) by net sales (Compustat item A12), multiplied by 100. Cost of goods sold divided by All costs directly allocated by the company to production, such as material, sales (CGSS) labor and overhead, divided by net sales, multiplied by 100. Selling, general and administrative All operational selling, general and administrative expenses not allocated to cost expenses over sales (SGAS) of goods sold, divided by net sales, multiplied by 100. Number of employees divided This item represents the number of company workers as reported to shareholders, by sales (ES) divided by net sales, multiplied by 100.

by the presence of a post-implementation change event. As a result, the statistical model developed here considers both the performance of an adopting firm subsequent to the adoption of an ERP system relative to that same firms own performance before adoption, as well as the hypothesized incremental improvement of the adopting firms performance in relation to a matched non-adopting firm. In the model, we analyze how the differential prepost performance of each adopting/matched firm pair is affected not only by the ERP adoption event but also by a change event. We therefore examine whether the effect of the ERP adoption event is moderated by the presence or absence of different types of changes in the use of an ERP system. To account for differential performance comparisons, the statistical model was applied on a panel of data and was constructed as follows: DPERFiPOST f DPERPi;t1 ; ERP Adopt Event; ERP Change Event; Interaction e 1 where DPERFiPOST is the difference in average performance in the ith pair between the ERP implementing company and control company for the 4 years after initial ERP system implementation. Each one of the following performance indicators is considered: DROA, DROI, DOIA, DROS, DOIS, DCGSS, DSGAS, and DES; DPERFi,t-1 is the difference in the performance in the ith pair of ERP adopting and matched firm for the year immediately preceding the ERP adoption decision (t - 1); ERP Adopt Event is a dummy variable, which takes the value of 1 if the differential performance is considered as a difference between the adopting firms performance minus the matched firms performance, and takes the value of 0 when differential performance is calculated as the difference between the matching firms performance minus the adopting firms performance; ERP Change Event is a dummy variable, which takes the value of 1 if a change has been observed in the adopting firms ERPS subsequent to initial implementation and takes the value of 0 if no change has been identified in the initial ERPS

26

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

implementation. The model accounts for two types of ERP Change Events: add-ons or upgrades (ENHANCEMENT), and abandonments or switchings (ABANDON); Interaction is the product of the ERP Adopt Event and the ERP Change Event. The inclusion of differential performance prior to the ERP adoption event represents a control measure and it helps ensure that differences between ERP adopters and matched firms are not due to an omitted variables problem, thus minimizing misspecification of the test statistics (Barber and Lyon, 1996). It is also reported in prior studies that a bhaloQ effect may exist in a performance measure due to that performance in a given time period being affected by performance in a prior period (Brown and Perry, 1994). As a result, inclusion of prior performance in model (1) helps to adjust for any such effects and control for the presence of a possible halo effect. The inclusion of the main effects for the ERP adopt and change events in model (1) is designed to prevent conclusions of the existence of an interaction effect when such an effect may be solely due to main effects (Haartman and Moers, 1999; Jaccard et al., 1990). If such main effects are found in the empirical results, however, their interpretation is made impossible in the presence of the interaction term. As a result, the ERP event and implementation factors in model (1) will be centered in order to facilitate the interpretation of coefficients for the main effects and to minimize potential multicollinearity problems. Centering is the proper linear transformation procedure recommended by authorities in the field (e.g., Cohen and Cohen, 1983; Jaccard et al., 1990) and does not affect the level of significance of the interaction term. The ERP Event dummy is designed to capture the direction of the difference in the dependent variable in model (1). This is necessary in a matched pairs analysis, where the dependent variable is a made up of a difference score in the performance of the ERP adopting versus the corresponding performance of a matched firm. Not including paired differences in the postadoption and pre-adoption performance measures in model (1) does not ensure that the statistical test provides a direct comparison of differences within each matched pair; further, non-inclusion of the ERP Event dummy variable does not test if any such differences are due to better performance of the ERP-adopting firms or due to the better performance of the matched non-adopting firms. Since the ERP Event dummy solely depends on the independent observation as to whether a firm is an ERP adopter or not, model (1) avoids concerns of model endogeneity. Four types of ERP post-implementation changes are considered: add-ons to the ERPs, upgrades to new versions of the same ERPs, switching to a different ERPs provided by an entirely different vendor, and abandonment of the whole ERP project. For model testing purposes, we combine add-ons and upgrades into a single categorical variable (Enhancement), and switchings and abandonments into a second categorical variable (Abandon). 3.3.2. Model 2 Research hypothesis H2 predicts that the nature and timing of a change will significantly influence the strength of the moderating influence in model (1). As a result, the statistical model developed to test H2 assumes that the combined influence of the ERP adoption and the timing and nature of the change events moderates a firms differential performance in the years following initial system implementation. To account for differential performance comparisons, the following statistical model was also applied on a panel of data and was constructed as follows: DPERFiPOST f DPERFi;t1 ; ERP Adopt Event; Early Addon; 2 Late Addon; Early Abandon; Late Abandon; Interactions e

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

27

where DPERFiPOST as in model (1); DPERFI,t-1 as in model (1); ERP Adopt Event as in model (1); Early/Late Change Events are dummy variables, which take their values as explained before. The model accounts for four types of ERP Change Events: early/late addons or upgrades (Enhancement), and early/late abandonments or switchings (Abandon); Interaction is the product of the ERP Adopt Event with each of the four early/late ERP Change Events. In the model, all types of changes are evaluated simultaneously and thus allow for a complete analysis of differential returns firm pairs (adopt/non-adopt) which are associated with any of the four types of changes specified above. 4. Results 4.1. Results on research hypothesis H1 The research method in this study allows for the control of ex ante differences among adopting and non-adopting firms in testing significant performance improvements between matched firm pairs on an ex post basis. As a result, the identification of a significant difference represents a strong test for the first research hypothesis (H1). As tested in H1, ERP change firms may be expected to improve their differential performance as a result of changes completed after the implementation of the ERP system. Given the expected interaction between the implementation phase and any subsequent changes, we mainly focus on the interaction rows of Table 3. The results indicate a highly significant interaction between the adoption of ERP event and the bEnhancementQ change event for the ROA differential dependent variable (t = 3.73; p b .01). Similarly, the interaction between the adoption event and the bAbandonQ change variable is significant (t = 2.06; p b .05). The negative sign on the interaction coefficient, in this case, indicates that switchings and abandonments negatively moderate the influence of ERP adoption on differential ROA performance. The Adopt Abandon interaction, however, only holds for the ROA differential and is not significant in any other differential measures of performance. Statistically significant results are also observed for the Adopt Enhancement interaction for the ROI differential, the OIA differential, the SGAS differential, and the ES differential. With regard to the direction of the effects, all significant Adopt Enhancement interactions have a positive sign, indicating that the presence of an ERP enhancement event (whether add-on or upgrade), overall, has a positive impact on the relationship between ERP adoption and differential firm performance. As shown in Table 3, the pre-adoption control included in the model represented the corresponding performance metric in the year prior to the ERP adoption event. Similar results are also obtained when the 3-year average pre-adoption performance is used instead of performance in the 1 year prior to adoption. These results support the propositions made in H1 (the null is rejected). 4.2. Results on research hypothesis H2 The research method for this hypothesis also allows for the control of ex ante differences among adopting and non-adopting firms in testing significant performance improvements between matched firm pairs on an ex post basis. As a result, the identification of a significant difference again represents a strong test for the second research hypothesis (H2) which examines the influence of the nature and timing of post-implementation changes on differential

28 A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

Table 3 Results on testing research hypothesis H1 Independent variables Intercept Performance t 1 ERP-adopt event ERP bEnhanceQ change event ERP bAbandonQ change event Adopt Enhance interaction Adopt Abandon interaction Model F (df); probability Adjusted R 2 (%) Dependent variables Model 1: DROA 0 .067 .095 .00 .00 .311*** .163** F 6,149 = 3.11; p b .0007 7.83 Model 2: DROI 0 .609*** .049 .00 .00 .248*** .057 F 6,143 = 21.28; p b .0001 45.97 Model 3: DROS 0 .024 .112 .00 .00 .193* .055 F 6,149 = 0.97; p b .448 0.01 Model 4: DOIA 0 .144* .142* .00 .00 .278*** .068 F 6,149 = 2.52; p b .024 5.78 Model 5: DOIS 0 .065 .245*** .00 .00 .157** .072 F 6,143 = 2.79; p b .014 6.99 Model 6: DCGSS 0 .706*** .064 .00 .00 .003 .033 F 6,149 = 24.31; p b .024 48.42 Model 7: DSGAS 0 .321*** .234*** .00 .00 .196** .022 F 6,117 = 3.77; p b .002 12.44 Model 8: DES 0 .583*** .132** .00 .00 .161** .083 F 6,129 = 14.10; p b .0001 37.86

Moderated regression model of average post-adoption differential performance on pre-adoption differential performance (period t 1), ERPS Use, ERPS Change, and ERPS Use/ Change Interaction. The variable used to calculate prior differential performance for the time period t 1 corresponds to the dependent variable tested in each model. Standardized coefficients are presented for all independent variables. The p-values for the ERP adopt event, ERP change event and interactions are one-sided, all others are two-sided. Probability levels are indicated as follows: *p b .10; **p b .05; ***p b .01.

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

29

ERP-based firm performance. As is seen in Table 4, the test includes main-effects for ERP Use, Early Add-ons, Late Add-ons, Early Abandonment, Late Abandonment, and Interactions between them. Similar to the focus in H1, ERP change firms are expected to witness significant changes to their differential performance as a result of changes completed after the implementation of the ERP system. Given the expected interaction between ERP adoption and the nature and timing of subsequent changes, we mainly focus on the interaction rows of Table 4. The results indicate a highly significant interaction between the adoption of ERP event and the bEarly Add-onQ change event for the ROA differential dependent variable (t = 5.33; p b .01). The same level of significance carries over to the interaction between the adoption of ERP and the bLate Add-onQ change event for the ROA differential dependent variable (t = 3.85; p b .01). It is important to note the signs (directions) of the interaction coefficients. The influence of the ERP adoption event on differential performance is positively moderated by the presence of early add-ons, while it is significantly negatively moderated by the presence of late add-ons. These effects are consistent across all differential performance measures where significant results are obtained (differential ROI, OIA, OIS, SGAS, and ES). The signs of the interaction coefficients would be expected to be reversed when the dependent variable measures costs or expenses. In a similar manner, the interactions between the ERP adoption event and the early/late abandonment events were also found to be significant. Significant results, albeit at a lower significance level for both the early and late abandonments are present for both the ROI differential and the OIA differential. Interestingly, only the ERP Abandonment interactions (for both early and late) are present for CGSS differential. Also noteworthy is the fact that the ERP Early Add-on interaction is significant for all performance differentials except the ROS and the CGSS differentials. The ERP Late Add-ons interaction is only significant for the balance-sheet returns measure but does not hold true for the temporary account differentials. Very similar patters also characterize both Early and Late Abandonment ERP interactions. As in the H1 results, the substitution of the 3-year average pre-adoption performance with the 1 year before adoption performance does not affect the results reported in Table 4. These results, in general, support the predictions made in H2. 5. Discussion Our research hypotheses specifically recognize that the time beyond initial implementation of an ERP system describes a period that is required by a company to adjust to the new system and train its users so that anticipated benefits may materialize. Such adjustments either take on the form of positive changes as upgrades or add-ons contingent on the time period elapsed since initial implementation; or assume the form of less-positive/ negative changes in the form of switches or outright abandonments again qualified by the elapsed time since implementation. The joint consideration of both the nature of the change and the timing of the change make it especially important that the measurements in this study are made during time periods over which an ERP-adopting firm would be expected to initiate changes to its installed ERP-base. Based on prior consideration of such periods (Nicolaou, 2004a), we limited our analysis to 4 years beyond the time of initial ERP implementation. Our first research hypothesis predicted that a firms differential performance as compared to a matched control firm, following changes made after the adoption of an ERP system, would be significantly higher than its differential performance prior to the adoption of the ERP system.

30 A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

Table 4 Results on testing research hypothesis H2 Independent variables Dependent variables Model 1: DROA Model 2: DROI Model 3: DROS Model 4: DOIA Model 5: DOIS Model 6: DCGSS Model 7: DSGAS Model 8: DES Intercept DPerformance T 1 ERP Adopt event Enhance Early Enhance Late Abandon Early Abandon Late ERP Enhance Early interaction ERP Enhance Late interaction ERP Abandon Early interaction ERP Abandon Late interaction Model F (df); probability Adjusted R 2 (%) 0 .014 .223** .770*** .561*** .310** .249** 1.285*** .898*** .445*** .354*** F10,149 = 4.14; p b .0001 17.41 0 .493*** .182** .799*** .640*** .108 .107 1.332*** 1.032*** .155* .153* F 10,149 = 19.42; p b .0001 56.30 0 .029 .201** .134 .057 .129 .102 .224 .091 .184 .145 F 10,149 = 0.66 p b .760 0.01 0 .199*** .274*** .674*** .484*** .204* .230* 1.125*** .775*** .292** .328*** F 10,149 = 3.18; p b .001 12.76 0 .065 .174* .296* .178 .064 .000 .500** .287 .092 .001 F 10,143 = 1.78; p b .07 5.17 0 .691*** .068 .087 .100 .130 .148 .145 .161 .187* .210** F 10,149 = 14.98; p b .0001 48.42 0 .322*** .336** .231 .058 .015 .010 .389* .093 .022 .015 F 10,117 = 2.20; p b .023 9.27 0 .576*** .200** .204 .061 .056 .030 .341* .099 .081 .043 F 10,129 = 8.22; p b .0001 35.89

Moderated regression model of average post-adoption differential performance on pre-adoption differential performance (period t 1), ERPS Use, Early Add-ons, Late Add-ons, Early Abandonment, Late Abandonment, and Interactions. The variable used to calculate prior differential performance for the time period t 1 corresponds to the dependent variable tested in each model. Standardized coefficients are presented for all independent variables. The p-values for the ERP adopt event, ERP change event and interactions are one-sided, all others are two-sided. Probability levels are indicated as follows: *p b .10; **p b .05; ***p b .01.

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

31

This hypothesis considered the nature of the transformation itself. We specifically considered system enhancements, which consist of add-ons and upgrades, and system abandonments, which consist of switches and abandonments. We used eight operational performance measures previously used by Nicolaou (2004a) and described as first-order performance measures in prior studies (Bhattacharya et al., 1997; Barua et al., 1995). These performance measures included return-ratios and expenses. As may be expected, positive coefficients on the return-ratios represented positive firm performance as a result of ERP changes while negative coefficients on the expense metrics represented positive performance derived from ERP changes as a result of the lowering of expense items for the change firms. The test of the first research hypothesis as laid out in Table 3 suggests that ERP-adoption related firm performance is indeed moderated by subsequent changes made to the firms ERP system. More specifically, we see that both enhancements (add-ons and upgrades) and abandonments (switches and abandonments) interact with the adoption event itself to present significant results on the DROA metric. The positive coefficients of the enhancement-related interactions suggest that ERP enhancements in the form of upgrades and add-ons improve firm performance and the negative coefficients on the abandonment-related interactions suggest that ERP abandonments negatively impact firm performance. While the enhancement interactions achieve significance at either the p b .01, the p b .05 or the p b .10 levels on all the Dreturn-ratios and all the Dexpense metrics other than the DCGSS metric, the abandonment interactions are only significant on the DROA metric. It is interesting to note, however, that despite the lack of significance, the Adopt Abandon interactions are characterized by negative coefficients for all the Dreturn-ratios and positive coefficients for all the Dexpense metrics other than the DSGAS metric. Data in Table 1 provide an explanation for the strong (weak) results obtained for Enhancement (Abandon) Adoption interactions in Table 3. As can be seen in Table 1, of the 182 total changes identified and used in this study, the majority of the changes were of the enhancement kind (148 add-ons plus fifteen upgrades totaling 163 enhancements). Similarly, of the 83 change-firms identified in the study, 89% (n = 74) were identified as enhancement firms. We suspect this lopsided distribution to be an artifact of reporting bias for both ERP-adopting and vendor firms. Both clients and vendors of ERP systems have a vested interest in focusing more on the successful ERP implementations and changes over the less successful ones. Such reporting bias for IS/IT success over failure is not limited to ERP alone but has been noted for other IS/IT endeavors as well (Bhattacharya and Wasson, 1997). We believe that the skewed distribution of the enhancement changes over the abandonment changes may explain the strong statistical significance of the enhancement results across almost all the performance metrics and the lack of statistical significance for the abandonment interactions for all but the DROA metric. However, despite the lack of statistical significance, the signs on the abandonment interactions are consistent with the development of the theory behind H1. This, in conjunction with the strong results obtained for the Enhancement Adoption interactions over all the performance metrics leads us to reject H1 in its null form to suggest that ERP-adoption-related firm performance effects are indeed moderated by subsequent changes made to ERP systems. Our second hypothesis examines the moderating effects of the nature and timing of postimplementation changes on differential ERP-based firm performance. This hypothesis was tested using data up to 4 years (t 1. . .t 4) after the completion of an ERP system with changes made during the year of (t 0) and year following implementation (t 1) considered to be bearlyQ and changes made 2 years and beyond from the implementation date (t 2. . .t 4) considered to be blateQ.

32

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

As in the tests for H1 above, we used the same eight differential performance metrics. Focusing again on the interaction terms of Table 4, the ERP Adoption Early Enhancement interaction term considered the moderating effects of early enhancements (add-ons and upgrades) on ERPadopting differential firm performance. Similarly, the ERP Adoption Late Enhancement term considered the effects of late enhancements (add-ons and upgrades) on ERP-adopting differential firm performance. In the same vein, the ERP Adoption Early Abandonment interaction term considered the effects of early abandonments (switches and abandonments) on ERP-adopting differential firm performance and the ERP Adoption Late Abandonment term considered the effects of late abandonments (switches and abandonments) on ERP-adopting differential firm performance. The strong statistical significance in the Adoption Early Enhancement interaction terms describe a pattern that is similar to the results obtained for the Adoption Enhancement interaction in H1. These results suggest that firms that undertake early enhancements to their ERPs enjoy statistically significant and positive differential firm performance benefits from such enhancement-related changes. Interestingly, the ERP Late Enhancement interactions almost describe the same levels of strong statistical significance as in Early Enhancement interactions but with the signs reversed. This suggests that firms making late enhancements to their ERP systems suffer statistically significant deterioration in relative firm performance. The latter pattern holds for ERP interactions with both early and late abandonments. These results help us reject H2 in its null form as well because they suggest that the timing and nature of postimplementation changes do indeed have a significant effect on adopting firms differential performance over the post-implementation period. Our findings add support to the conclusions of earlier studies such as Nicolaou (2004b) that emphasized the critical importance of managing firms post-implementation processes. These findings are also in keeping with studies such as Polymenakou and Serafeimidis (1997) that have suggested that systems evaluations should not only be restricted to just one set of ex ante and ex post practices because most of the value adding occurs in ongoing activities that closely follow developments in the construction, implementation and use of information systems. These results also provide empirical support for Nicolaous (2004b) qualitative findings that postimplementation factors are very significant for ERP-adopting firms ability to realize positive performance outcomes. The empirical analysis undertaken in this study also addresses what Nicolaou (2004c) described as ba dearth of theory-deepening empirical resultsQ supporting theoretical arguments in favor of post-implementation ERP management. Our results from the use of differential operating performance measures also support Nicolaous (2004a) contention that improved profitability in the shape of either cost reductions or sales growth are ultimately the raison detre of ERP-driven long-term firm performance improvement. Some of the limitations of this study may also be viewed as avenues for future research. Ranking chief among them is the use of only first-order performance metrics in this study. While first-order performance metrics such as the ones used here are good indicators of firm performance differentials, these findings may also be corroborated by higher-order metrics that test for firm performance in a more direct manner. Such higher-order metrics have previously been used in AIS literature in the form of Tobins Q (Bhattacharya et al., 1997) and their use in similar studies would help strengthen the results obtained from the performance metrics used in this study. Another avenue for future research would be to examine firm performance differentials discretely and separately over the post-implementation period. Such an analysis would avoid the averaging effects of the post-implementation period and would represent a more robust test of the effect of the post-implementation changes on the ERP-adopting firms. In a

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

33

similar manner, future studies may also explore post-implementation changes other than the four types we have examined here or may extend beyond the time periods examined here. Other studies may wish to test for differential firm performance between the ERP-adoption and subsequent transformation periods. Such studies would, of course, require a different data set and a different methodology than the one used in this study. Another limitation of this study relates to that the sample of ERP-adopting firms which voluntarily disclosed ERP revisions, whether enhancements or abandonments. This is especially an issue with regard to abandonments, as both users and vendors of ERP systems are prone to disavow knowledge of and publicity regarding ERP failures. As a result, the sample might be biased towards finding results that are representative for only the sub-set of firms that choose to disclose significant positive projects. Of the 182 total changes identified and used in this study, the majority of the changes were enhancements (148 add-ons plus 15 upgrades totaling 163 enhancements). Again, of the 83 change-firms identified in the study, 89% (n = 74) were identified as enhancement firms. This lopsided distribution was probably an artifact of reporting bias for positive outcomes by both ERP-adopting and vendor firms. The resulting skewed distribution of the enhancement changes over the abandonment changes may explain the strong statistical significance of the enhancement results across almost all the performance metrics and the lack of statistical significance for the abandonment interactions for all but the DROA metric. While we recognize the statistical bias introduced by such data incongruity, it is important to note that it results from the lopsided public reporting that is inherently biased towards positive outcomes. Therefore, this is a limitation that while readily apparent to us as researchers is nevertheless very difficult to overcome given the studys dependence on data sources such as Lexis/Nexis in the public domain. In conclusion, this study, despite the limitations described above, suggests that the nature and timing of ERP post-implementation changes represent significant events affecting an adopting firms financial performance. Such discrete changes occur over several periods beyond the initial implementation and often help surface implementation issues that affect subsequent use of and success from the use of ERP systems. In our study, we focused on four types of changes addons, upgrades, switches and abandonments. While the first two were described as enhancements, the last two were analyzed together as abandonments. Our findings indicate that ERP-adopting firms that initiate early enhancements in the form of either add-ons or upgrades enjoy superior differential performance benefits in comparison to their non-ERP peers. Late enhancements and both early and late abandonments lead to differential performance deterioration for the ERPadopting firms. Therefore, these changes may be viewed as resulting from implementations that had originally gone awry and such firms may be viewed as unable to recover from such blighted ERP implementations over several periods following their initial implementations. We believe that while our study provides some interesting insights on post-implementation changes undertaken by ERP-adopting firms, further research is needed to conclusively address alternative reasoning that was alluded to in this study. The existence of alternative explanations is the hallmark of early and novel research and this study contributes in such a way as to make the topic area of post-implementation ERP changes a fertile subject for such future research endeavors. Acknowledgments The authors acknowledge valuable comments received on earlier versions of this paper from the Senior Editor, Associate Editor, anonymous reviewers, and participants at the 2nd International Conference on Enterprise Systems and Accounting.

34

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

References
Barber BM, Lyon JD. Detecting abnormal operating performance: the empirical power and specification of test statistics. J Financ Econ 1996;41:359 99 [July]. Barua A, Kriebel C, Mukhopadhyay T. Information technology and business value: an analytical and empirical investigation. Inf Syst Res 1995;6(1):3 23. Bhattacharya S, Wasson W. Wherefore IT failures: a synthesis. Fail Lessons Learn Inf Technol Manag 1997;1: 191 203. Bhattacharya S, Braun GP, Traichal PA. Does IT pay off? Evidence from a multidimensional analysis. Adv Account Inf Syst 1997;5:35 66. Brown B, Perry S. Removing the financial performance halo from fortunes bMost AdmiredQ companies. Acad Manage J 1994;37(5):1347 59. Brown C, Vessey I. ERP implementation approaches: toward a contingency framework. Proceedings, International Conference on Information Systems; 1999. p. 411 6. Brynjolfsson E. The productivity paradox of information technology. Commun ACM 1993;36(12):66 77. Cohen J, Cohen P. Applied multiple regression and correlation analysis for the behavioralsciences. Erlbraum, Hillsdale, N.J.. Cooke DP, Peterson WJ. SAP implementation: strategies and results. New York, NY7 The Conference Board Inc; 1998. Davenport TH. Putting the enterprise into the enterprise system. Harvard Bus Rev 1998;76:121 31 [July/August]. Haartman FGH, Moers F. Testing contingency hypotheses in budgetary research: an evaluation of the use of moderated regression analysis. Account Organ Soc 1999;24:291 315. Hitt LM, Wu DJ, Zhou. Investment in enterprise resource planning: business impact and productivity measures. J Manage Inf Syst 2002;19:71 98 [Summer]. Holland, CP, Light, B. A stage maturity model for enterprise resource planning systems use. The DATA BASE for Advances in Information Systems 2001;32:3445 [Spring]. Hunton JE, Lippincott B, Reck JL. Enterprise resource planning systems: comparing firm performance of adopters and non-adopters. Int J Account Inf Syst 2003;4:165 84 [September]. Jaccard J, Turrisi R, Wan CK. Interaction effects in multiple regression. Newbury Park, CA7 Sage Publications; 1990. Kettinger WJ, Grover V, Guha S, Segars AH. Strategic information systems revisited: a study in sustainability and performance. MIS Quart 1994:31 58 [March]. Kremers M, Van Dissel H. ERP system migrations. Commun ACM 2000;43(4):53 6. Mata F, Fuerst W, Barney J. Information technology and sustainable competitive advantage: a resource-based analysis. MIS Quart 1995;19(4):487 505. Markus ML, Axline S, Petrie D, Tanis C. Learning from adopters experiences with ERP: problems encountered and success achieved. J Inf Technol 2000;15:245 65. McAfee A. Why Hershey had a scary Halloween: managers using cookie cutter strategies do so at their own peril. National Post, Monday National Edition, vol. BE 6. 2003 (January 27). Motwani J, Mirchandani D, Madan M, Gunasekaran A. Successful implementation of ERP projects: evidence from two case studies. Int J Prod Econ 2002;75:83 96. Murray MG, Coffin GW. A case study analysis of factors for success in ERP system implementations. Proceedings of the Seventh Americas Conference on Information Systems; 2001. p. 1012 8. Nicolaou AI. Firm performance effects in relation to the implementation and use of enterprise resource planning systems. J Inf Syst 2004a;18:79 105 [Fall]. Nicolaou AI. Quality of post-implementation review for enterprise resource planning systems. Int J Account Inf Syst 2004b;5:25 49 [May]. Nicolaou AI. Reply to discussion of: Firm performance effects in relation to the implementation and use of enterprise resource planning systems. J Inf Syst 2004c;18:111 2 [Fall]. OLeary D. Enterprise resource planning systems. Cambridge, UK7 Cambridge University Press; 2000. Peterson WJ, Gelman L, Cooke DP, 2001. ERP Trends. New York, NY: The Conference Board; Report 1292-01-RR. Polymenakou A, Serafeimidis V. The role of evaluation in dealing with information systems failure: conceptual explorations. Fail Lessons Learn Inf Technol Manag 1997;1:167 77. Poston R, Grabski S. The impact of enterprise resource planning systems on firm performance. International conference on information systems proceedings of the twenty-first International Conference on Information Systems, Brisbane, Queensland, Australia; 2000. p. 479 93.

A.I. Nicolaou, S. Bhattacharya / International Journal of Accounting Information Systems 7 (2006) 1835

35

Poston R, Grabski S. Financial impacts of enterprise resource planning implementations. Int J Account Inf Syst 2001; 2(4):271 94. Ross JW, Vitale MR. The ERP revolution: surviving vs thriving. Inf Syst Front 2000;2(2):233 41. Scott JE, Vessey I. Implementing enterprise resource planning systems: the role of learning from failure. Inf Syst Front 2000;2(2):213 32. Shang S, Seddon P. A comprehensive framework for classifying benefits of ERP systems. Proc AMCIS 2000; 2000:1005 14. Soh C, Kien SS, Tay-Yap J. Cultural fits and misfits: is ERP a universal solution? Commun ACM 2000;43:47 51 [April]. Stephanou CJ. The selection process of enterprise resource planning (ERP) systems. Proceedings of the Americas Conference on Information Systems (AMCIS); 2000. p. 988 91. Wah L. Give ERP a chance. Manage Rev 2000:20 4 [March]. Weill P. The relationship between investment in information technology and firm performance: a study of the valve manufacturing sector. Inf Syst Res 1992;3:307 33 [December].