Information Systems Research

Vol. 23, No. 2, June 2012, pp. 436–452

ISSN 1047-7047 (print) — ISSN 1526-5536 (online) http://dx.doi.org/10.1287/isre.1110.0362
© 2012 INFORMS

Reputation and Uncertainty in Online Markets:

An Experimental Study
Sarah C. Rice
School of Business, University of Connecticut, Storrs, Connecticut 06269, srice@business.uconn.edu

T his paper employs a modified investment game to study how online reputation ratings are assigned, and
thus how electronic reputations are formed in transactions where buyers and sellers interact anonymously.
Of particular interest are the important questions of how online reputations evolve and how specific reputation
information is interpreted by market participants. We vary the level of uncertainty in the transaction environ-
ment, and measure the effects of this manipulation on buyers’ trust and their subsequent rating behaviors. We
distinguish between a reputation mechanism and specific reputation information, finding the former has an
association with the overall decision of whether to transact in the marketplace, while the latter shows signif-
icance in purchase decisions regarding specific sellers. We also find that aggregate reputation information is
weighted differently than singular reputation information. Finally, we show that when reputations are increas-
ingly noisy, buyers are less likely to react negatively to poor ratings and are more likely to give sellers the
benefit of the doubt when seemingly uncooperative outcomes occur.
Key words: reputation systems; online markets; experimental economics
History: Alok Gupta, Senior Editor; Gautam Ray, Associate Editor. This paper was received on February 1,
2008, and was with the authors 14 months for 4 revisions. Published online in Articles in Advance
September 15, 2011.

1. Introduction delivering low-quality goods at high-quality prices, or

Reputation is in itself only a farthing candle of a by not shipping the good as promised.
wavering and uncertain flame, and easily blown out; To help address the information asymmetry in
but, it is the light by which the world looks for and online markets, many e-commerce sites feature pub-
finds merit. —James Russell Lowell lic online reputation systems. These systems allow
One of the definitive characteristics of electronic trading partners to leave ratings and comments about
markets is the information asymmetry that exists prior transactions. The result is an electronic reputa-
between buyers and sellers. Sellers have an informa- tion system that evolves from third-party reporting,
tion advantage over buyers, as only sellers know their where individuals can access information about spe-
type with certainty. In addition, buyers do not know cific community members online. The hope is these
the true quality of the goods and services offered, electronic reputation systems can substitute for repu-
so they must trust sellers’ descriptions when mak- tations that would otherwise be established through
ing a purchase decision. Economic theory shows that repeated interaction, providing incentives for buyers
the absence of an enforcement mechanism in markets and sellers to cooperate in spite of the existing moral
with information asymmetry can lead to moral haz- hazard problem.
ard problems and adverse selection. In extreme cases, Although there is evidence showing reputation sys-
this moral hazard and adverse selection can cause the tems can induce cooperation in markets with infor-
dissolution of the market such that no transactions mation asymmetry (Ba and Pavlou 2002, Resnick and
occur (Akerlof 1970). Zeckhauser 2002, Dellarocas 2005, Bolton et al. 2005),
A moral hazard problem arises when one party is the incidence of online fraud remains relatively high.
exposed to less risk than the rest of the market, pro- The most common instances of fraud occur in Internet-
viding an opportunity to capitalize on this advantage based auctions such as those hosted by eBay, Amazon
at the expense of others. The moral hazard problem in Auctions, and Yahoo! Auctions. Although the growth
online markets stems from the fact that buyers move in online auctions has been undeniable, the choice to
first by sending payment to sellers before goods are do business in this arena is not without risk. The Fed-
shipped. Because buyers are the first movers in the eral Trade Commission in 2000 noted an “explosion”
online transaction, they bear the vast majority of the of Internet auction fraud cases (http://www.fbi.gov/
risk, while sellers, in turn, bear much less risk. As a majcases/fraud/internetschemes.htm), and in 2007,
result, sellers may try to exploit buyers’ position by the Federal Bureau of Investigation statistics show the
436
Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS 437

highest sector of reported Internet fraud was Internet directional effects of uncertainty on reputation rat-
auction fraud. Furthermore, according to the National ings, and the subsequent effects of “noisy” reputation
Consumer League, in 2007, approximately 13% of information on transaction decisions.
all Internet fraud was because of nonshipment or Addressing the question of how reputation ratings
misrepresentation of goods (http://www.fraud.org/ are assigned, and subsequently, how reputation histo-
internet/2007internet.pdf). Though many online auc- ries evolve, we find that when sellers failed to meet
tion sites use electronic reputation systems, the rela- buyers’ expectations, poor ratings ensued. Good rat-
tively high incidence of fraud indicates there is still ings were more likely when buyers’ expectations were
much to be learned about how to effectively design exceeded. Exceptions to these findings were found
and implement these systems. in our treatments where additional uncertainty was
We use experimental economics to address the imposed. When uncertainty was greater, buyers were
important questions of how electronic reputations more willing to exhibit trust than in settings with-
evolve and how different reputation information is out the imposed uncertainty. Specifically, we find that
used in online transaction decisions. Specifically, we when subjects were uncertain about the intentions of
ask: (1) How are reputation ratings assigned, and sub- their transaction partner, a negative outcome was less
sequently, how are reputations formed in online mar- likely to be followed by a poor rating.
kets? Currently there is a body of literature discussing We distinguish between the reputation mechanism
various effects of reputation and reputation systems, and the specific reputation information disseminated
but this literature is silent regarding how reputation by the system, and find the presence of a reputation
ratings are assigned, and thus how online reputations mechanism had a significant association with buy-
are formed. (2) What are the economic and behavioral ers’ decisions to engage in the transaction. We also
effects of different reputation information? Empiri- find noteworthy differences regarding the interpreta-
cal work is inconclusive in answering this question, tion and weighing of each type of reputation infor-
perhaps because of confounding effects in field data. mation. Without additional uncertainty, the aggregate
A laboratory experiment provides additional control of a seller’s good ratings had a positive association
to allow further insights into the effects of reputation with the buyer’s investment decision, while a sin-
information. (3) How does uncertainty in the transac- gle good rating assigned in the prior round had no
tion environment affect the functionality of the repu- effect. We find that imposing uncertainty on the trans-
tation system? We know there is uncertainty in online action environment moderated the negative effects
transactions; however, to date there is no work that of poor ratings on investment amounts, resulting in
explores the effect of this uncertainty on reputation fewer poor ratings assigned to sellers when negative
formation and the use of reputation information in transaction outcomes ensued. This result suggests the
transaction decisions. possibility that buyers gave sellers the benefit of the
The market exchanges in this study are modeled doubt under conditions of greater uncertainty, and
after the investment game, which is a variation of the did not punish seemingly uncooperative outcomes
trust game (Berg et al. 1995). In our game, one player when intentions could not be perfectly inferred.
is given an endowment, which can be “invested” and The remainder of the paper proceeds as follows.
sent to another player. This investment is increased First, is a discussion of the background and moti-
by a designated amount, after which the second vation for this study, followed by an explanation of
player decides how much to send back to the first the game, and the experimental design. Next, the
player. In our reputation treatments, players are asked hypotheses and results are discussed. The paper con-
to rate their partner at the conclusion of the exchange. cludes with a summary of findings, limitations, and
We also design a treatment to investigate the direc- extensions.
tional effects of uncertainty in this type of market.
Specifically, we impose a setting where there is a 30% 2. Background and Motivation
chance the amount returned by the second player can
be intercepted and reduced to 0.1 Under these condi- 2.1. Reputation Systems
tions, when the first mover receives nothing in return, Reputation can be defined as the conditional proba-
it is not known whether the outcome was intended bility that an individual will behave in a certain man-
by the second mover or if the amount returned was ner (Kreps and Wilson 1982). One way for individuals
intercepted. This treatment allows us to assess the to develop a reputation is to interact with each other
repeatedly so that a transaction history evolves over
1
time. From this first-hand experience, a reputation is
We chose a high probability of interception for this treatment to
measure directional effect of uncertainty. Further extensions could established, by which potential transaction partners
include varying the levels of uncertainty to gain additional insight can derive a conditional expectation of an individ-
into this aspect of the transaction. ual’s future behavior. However, in online transactions,
 Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
438 Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS

it is likely that buyers and sellers may not interact While agent reputations in the era of the Maghribi
with enough frequency for reputations to be estab- traders were created and disseminated via word of
lished. In this type of setting, another way to facili- mouth, in the modern-day online business commu-
tate reputation formation is via third-party reporting, nity, reputations can be created and disseminated via
a community enforcement mechanism where buyers the Internet. An online reputation is a history of
and sellers report past actions of transaction partners reported evaluations left by prior transaction part-
(Ba 2001). The third-party reported information is ners, and this information is then disseminated to the
aggregated and disseminated in an online format and community. Sites such as eBay assign buyers and sell-
is what comprises an electronic reputation system. ers a reputation score tabulated from past ratings, and
While electronic reputation systems function in an individual’s reputation is a direct measure of the
a modern-day context, third-party reporting as an ratings assigned to him or her by other community
enforcement mechanism was also used in medieval members (Resnick and Zeckhauser 2002). By conduct-
communities, where word-of-mouth references helped ing a controlled study to gain understanding into how
facilitate economic and social activities. One relevant these ratings are assigned, we provide insight into
example is that of the Maghribi traders, a group how reputations are formed in online markets.
of Eleventh-Century Mediterranean traders governed Third-party reputation formation is becoming an
by a coalition structure. Traders sold their goods increasingly relevant factor in the functionality of
via overseas agents, allowing them to operate in a online markets, and as such, has emerged as an impor-
more diverse field and to use local expertise. These tant area of research. For example, an experiment by
overseas agents conducted business with assets they Bolton et al. (2004) investigates the effectiveness of
did not own but rather were “lent” to them by third-party reporting in establishing credible reputa-
the traders. In return, agents were responsible for tions. They construct three markets: (1) a strangers
sharing profits with the merchant who provided the market where players interact anonymously with no
assets. Although the information asymmetry between reputation building, (2) a feedback market where rep-
traders and agents left room for agents to exhibit utation is established based on the truthful report-
opportunistic behavior, only a few incidents of mis- ing of individuals’ past actions, and (3) a partners
conduct have been documented (Grief 1993). The market where the same two players are paired for the
control mechanism in place was one of reputation entire experiment. The study of Bolton et al. (2004)
built on trust and reciprocation; traders exhibited finds that efficiencies of trade, measured as the per-
trust by lending capital to agents, and this trust was centage of completed transactions, and social welfare
reciprocated when agents paid the trader a share of were significantly higher in the feedback market than
their earned profits. An agent who failed to recip- in the strangers market. The results of Bolton et al.
rocate the traders’ trust acquired a reputation for (2004) support prior work by Schwartz et al. (2000),
being a cheater and faced the likely prospect of future showing that although first-hand reputation building
unemployment. Grief (1993) notes that the actions of is preferred, reputations based on third-party report-
agents were imperfectly observable by traders, thus ing also have social and economic benefits.
there was a positive probability that at some point, Studies specifically focusing on electronic reputa-
an honest agent could be considered a cheater. For tion systems show these mechanisms to be effective in
example, if traders experienced unprofitable trade building trust and cooperation in online communities
outcomes, it was possible they could have blamed (Resnick et al. 2000, Dellarocas 2003). Ba and Pavlou
the agents, perhaps believing they were pocketing a (2002) conclude that trust can be induced through
larger share of the profits. Given that traders only eBay’s ratings system and show that when trust is
observed the outcomes of these transactions, blame enhanced, more profitable outcomes ensue. A field
could be placed regardless of whether it was truly the study by Lucking-Reiley (2000) investigates the deter-
fault of the agent, or the result of some exogenous minants of price in online transactions and finds that
influence. Although anecdotal, Grief’s (1993) point sellers with poor feedback ratings receive lower rents.
raises the question as to how imperfectly observable Resnick et al. (2006) conduct a series of field experi-
actions impact a reputation, and the subsequent eco- ments to investigate the value of reputation on eBay
nomic outcomes of the individual in question. One and find that sellers with more established reputa-
outcome might be, as Grief (1993) states, to assume tions receive higher profits than those with less estab-
the worst and punish the agent for what appears to lished reputations. Work by Dellarocas (2003) models
be cheating behavior. The other option is to give the a reputation mechanism in a moral hazard setting and
agent the benefit of the doubt and assume that the shows that ratings systems can induce cooperative
intent was not to cheat, thereby declining to punish outcomes between buyers and sellers. An empirical
what would otherwise be considered noncooperative study by Kauffman and Wood (2005) examines factors
behavior. that could explain individual-specific price variance
Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS 439

for the same good, one of which is reputation. Addi- under complete information. He shows that if there
tional empirical work shows that reputation systems is no uncertainty about the seller’s type, an eBay-
can have positive effects on prices (Ba and Pavlou type reputation mechanism can induce maximum effi-
2002, Melnick and Alm 2002, Dewan and Hsu 2004). ciency. However, if uncertainty about the seller’s type
For a more comprehensive list of empirical studies is imposed, these assumptions may not hold, imply-
testing the economic effects of eBay’s reputation sys- ing that discerning the effects of uncertainty on a
tem, we direct the reader to Resnick et al. (2006). reputation system is a topic worth exploring. While
the above literature shows that reputation systems
2.2. Reputation Formation and can have positive associations with price, cooperation,
Imperfect Information
trust, and strategic choice, there is still little guidance
Reputation formation has been studied in an offline
on how reputations evolve in an online context.
context, but remains for the most part unexplored in
the online medium. Marschak and Selten (1978) intro-
duced the notion of reputation formation, disputing 3. Experimental Design
the game-theoretic backward induction prediction The investment game in this experiment originates
that a repeated play setting is merely a repetition of from the ultimatum game and the trust game, and
what would occur if only one interaction took place. was initially used by Berg et al. (1995) to study the
They note that with repeated play, there might be effects of social history on trust and reciprocity. For
incentives to act differently in earlier rounds, estab- ease of exposition, when describing the game and
lishing a reputation for cooperation to reap eco- our experiment, players will be referred to as buy-
nomic benefits in subsequent rounds. Kreps et al. ers and sellers. Our design adds to the investment
(1982) and Milgrom and Roberts (1982) each formalize game by incorporating an earlier action to each round
game-theoretic models where reputation formation of play. At the start of each round, sellers represent
in finitely repeated games arises as a sequential their “products” to buyers by filling out a nonbind-
equilibrium, where players form beliefs at each node ing announcement, stating what would be returned
of the decision tree and these beliefs motivate a for each given level of investment. This nonbind-
strategic choice. Camerer and Weigelt (1988) study ing announcement serves as a baseline for measuring
reputation formation in an experimental setting and
trustworthy behavior and creates an ethical dilemma
find that observed play is consistent with a sequential
for sellers, forcing them to differentiate between what
equilibrium. Jung et al. (1994) also study reputa-
they claim they will return and their actual return
tion formation in an experimental setting, and estab-
decision. Next, buyers are given $1 at the start of
lish that reputation formation is a factor in strategic
the game, and after being shown the nonbinding
outcomes. An experimental study by King (1996)
announcement, they decide how much of this $1 to
explores how reputations for honesty in reporting
send to their paired seller. Buyers may send one of
can be established and sustained in a market setting.
five amounts; $0.00, $0.25, $0.50, $0.75, or $1.00. The
In his study, the buyer’s choice to trust the seller
depends on whether the seller’s actions over time amount sent is multiplied by 4, so the most sellers can
result in a reputation for honesty, or a reputation for receive is $4.00 and the least they can receive is $0.00.
cheating. Upon receipt of the investment, sellers decide how
In stage games with perfect public monitoring, a much to return to buyers. In the treatments without
steady-state outcome is the equilibrium where the reputation, the round ends when buyers are informed
player’s reputation converges to the Stackleberg type of their final returned amounts. In the treatments with
(Cripps et al. 2004, Dellarocas 2003). In reality, know- reputation, buyers are asked to rate the transaction
ing buyer and seller types with certainty in an online after its completion by assigning a rating of good,
setting is not realistic, as individual actions are not neutral, or poor.
perfectly observable. For example, good types might The experiment was run in a repeated play set-
behave as such and still receive a poor reputation ting, meaning the game was conducted over multi-
score, even though their actions were honorable. ple rounds. Rather than set a finite end point to the
Alternatively, bad types might receive a good repu- experiment, the choice was made to incorporate a
tation score in spite of the intent to cheat. Although stochastic ending so subjects did not know for cer-
online transactions are inherently noisy with respect tain which was the final round of play. The point of
to monitoring buyer and seller types, to date there this design choice was to mitigate possible end-game
is little work that specifically measures the effects of effects, where in the final round of the game, sellers
this uncertainty on the functionality of the reputation have no incentive to cooperate, as reputation con-
system, and on buyer and seller transaction decisions. cerns no longer matter. The result is that in the final
As previously stated, Dellarocas (2003) models a rep- round, sellers would be more likely to “burn” their
utation mechanism design with pure moral hazard reputation by keeping all of the buyer’s investment
 Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
440 Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS

and returning $0.00. Realizing this to be the likely experience a poor transaction outcome; they might
outcome, buyers would not invest in the final round be disappointed with the good, the shipping process,
and the second to last round would actually become communication with the seller or the final purchase
the concluding round of play. In theory, sellers would price. This unsatisfactory result might be because of
employ the same strategy and burn their reputation the seller’s true intentions but could also be a mis-
in this new final round, so buyers again would not take made by the buyer,3 the seller or an external
invest. If this pattern continued, eventually the entire circumstance such as shipping error or technology
game would unravel to the first round and no trans- failure. Understanding how uncertainty impacts the
actions would ever occur. Although this is an extreme decision making of buyers and sellers could be useful
outcome, we wanted to lessen the severity of an end in improving reputation mechanism design and is an
game effect, so the final round of the experiment was important objective of this research.
determined by rolling a pair of dice after the tenth While this is admittedly a highly structured experi-
round. Subjects were informed that if either a 7 or 10 ment and not a field study set in the context of actual
were rolled, the game ended, otherwise play contin- online transactions, the additional control provided
ued to another round. Implementing this ending rule, by the laboratory allows us to carefully draw infer-
where there was a 25% chance the game would ter- ences about specific behaviors and outcomes in trans-
minate each time the dice were rolled, made it more actions, where buyers and sellers are connected only
difficult for subjects to condition their strategy on the by the online medium. In an actual online environ-
final round.2 ment, there are many competing factors that could
Our experiment generalizes to online transactions confound measurement of trust and trustworthiness,
by incorporating the elements of trust, trustworthi- including different valuations for the same good, dif-
ness and reciprocity; all factors in an online trans- ferent perceptions of quality and interactions with
action setting. The seller’s announcement equates to other bidders. From our results, we are able to pro-
presenting a good or service to the buyer, to be deliv- vide insights into the behavioral and economic effects
ered upon the buyer’s payment of a predesignated of our manipulations, thus supplementing findings
amount. After seeing this “product,” buyers must from field studies that may not offer a high level of
then trust sellers by sending payment for the good internal validity.
before receiving the item. Because the announcement Figure 1 depicts the game sequence, indicating the
is nonbinding, there exists the possibility that our sell- specific manipulations for the treatments with repu-
ers can cheat buyers and return amounts (ship prod- tation and those with interception.
ucts) that do not align with what they initially rep- The experiment was computerized so that subjects
resented. Sellers have an opportunity to reciprocate never interacted face to face, and at no time were
trust, or they can choose to exploit buyers by either they told the identity of the subject with whom they
refusing to send the good (return $0.00), or sending were paired.4 Screen shots of what subjects saw in the
an inferior good (return less than promised). experiment are included in the appendix. All subjects
To investigate the effects of uncertainty on the participating in the pilot studies and the full experi-
transaction outcome and the functionality of the rep- ment were upper-level undergraduate and graduate-
utation system, we conducted a manipulation where level business students. Prior to implementing the full
there was a 30% chance the amount sellers chose design, a pilot study testing the effects of the elec-
to return to buyers was intercepted and reduced to tronic reputation system was conducted. A second
0. While the possibility of interception was common pilot study was run on a different subject group to
knowledge, none of the subjects ever knew whether test for comprehension of the experimental instruc-
their return was actually intercepted. We chose not tions. This pilot confirmed that subjects understood
to disclose this information so that subjects could not the subtleties of the manipulation and allowed us to
calculate probabilities going forward. focus on their responses to the experiment. None of
The practical significance of this manipulation lies the subjects who participated in the pilot studies par-
in the fact that online transactions exist in a setting ticipated in the main experiment.
where uncertainty is a factor. Intentions are inferred
from outcomes, but for the most part, intentions of 3
In an online auction, a buyer might succumb to the winners curse,
buyers and sellers are not perfectly known. For exam- experiencing regret on winning the auction because of the belief
that he or she overpaid for the good. Or the buyer might suffer
ple, there might be multiple reasons buyers would
from a pseudoendowment effect (Wolf et al. 2008), which could also
lead to initial overpayment and eventual regret.
2 4
There is literature showing that end-game effects are not as preva- During the training sessions, the experimenter placed emphasis
lent as theory would suggest (Hoffman et al. 1998), so it is possible on demonstrating that the computer did randomly rematch sub-
this element of the design was not necessary to prevent unraveling; jects and that no one had any way of knowing who they were
however, the experimenter chose to err on the side of caution. paired with.
Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS 441

Figure 1 Game Sequence

Return Uncertainty
intercepted treatments

Seller announces Buyer chooses

Seller chooses Buyer rates
intended return investment
return amount seller
amount

Reputation treatments
All treatments

A total of 90 students were recruited to partici- reputation. Alternatively, the information distributed
pate in the full-scale experiment. Subjects required by the reputation system is a potentially informative
no special skills or prior experiences to participate. input to buyer and seller decision-making processes,
Because the experiment tested decision making that as it can aid market participants in evaluating the
was not generalized to upper-level managerial set- probability of an individual’s future cooperation.
tings, the choice to use students was appropriate. The In markets where buyers and sellers are virtually
design was between subjects, thus each subject partic- anonymous, a reputation system establishes a control
ipated in only one of the four treatments. Subjects first mechanism, such that market participants are more
read a set of instructions detailing the implementation accountable for their actions than if no such system
of the experiment and then were given a short quiz
were implemented. Without a reputation mechanism
to test for comprehension. The experimenter led sub-
in place, cooperative outcomes are less likely, as there
jects through a series of training sessions, so that they
are no long-term economic benefits to cooperation
became familiar with the software interface. Subjects
were trained on both buyer and seller interfaces but that offset the short-term gains from opportunism.
were randomly assigned to one role at the start of the A significant body of literature notes the coopera-
experiment, and they maintained that role through- tive effects of a reputation mechanism on buyer and
out the experiment. Compensation for participation seller decisions, showing that reputation systems can
consisted of a $6 show-up fee, in addition to the accu- effectively mitigate moral hazard problems and pro-
mulated earnings from each round.5 mote cooperative behavior (Kreps et al. 1982, Milgrom
and Roberts 1982, Kreps 1990, Lucking-Reiley 2000,
Schwartz et al. 2000, Ba and Pavlou 2002, Resnick and
4. Hypotheses Development Zeckhauser 2002, Dellarocas 2003, Dewan and Hsu
Our hypotheses development is presented in three
sections, each relating to one of the three questions 2004, Bolton et al. 2005, Cripps et al. 2005, Dellarocas
addressed in this study: (1) the economic and behav- 2005). Using backward induction to derive buyer
ioral effects of online reputation systems, (2) the evo- and seller equilibrium behavior, we can gain further
lution of online reputations, and (3) the directional insight into the possible effects of a reputation mech-
impact of uncertainty on the functionality of online anism in an online transaction setting. Consider our
reputation systems. investment game: in a single-shot setting where only
one transaction occurs, sellers do not have an incen-
4.1. The Reputation Mechanism and Specific tive to cooperate, as there are no future economic ben-
Reputation Information efits to doing so. Based on this incentive structure, a
A primary objective of this work is to distinguish buyer’s optimal strategy is to choose not to invest.
between the effects of the reputation mechanism, or Simply put, theory predicts that in a single shot set-
the electronic reputation system, and the specific rep- ting where there is no control mechanism in place,
utation information provided by the system. The dif- sellers will expropriate all buyer investments. As a
ference is subtle, yet important to note. A reputation result, a buyer’s best response is to never invest, and
system is a control mechanism that can enforce
thus no transactions occur. In a repeated play set-
cooperation among market participants by providing
ting, where buyers and sellers interact anonymously
incentives for buyers and sellers to maintain a good
in each round, there is still no reputational incentive
5
for sellers to cooperate. For this reason, the theoretical
Buyer earnings for each round were calculated as: ($1) − (amount
result shows that repeated anonymous transactions
sent to seller) + (amount returned by seller). Earnings for sellers
were calculated as: (amount received from buyer ∗ 4) − (amount have the same equilibrium outcome as the single-shot
returned to buyer). game where no trade occurs.
 Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
442 Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS

In a repeated play setting where buyers and sell- Interestingly, Resnick and Zeckhauser (2002) find
ers remain in the same dyads over all rounds, repu- that neutral ratings are more likely to be used for
tations evolve as a result of first-hand experience. In slightly problematic exchanges, such as when a prod-
this setting, the Nash equilibrium is one where sellers uct is satisfactory but does not exactly match its online
cooperate to establish a good reputation to reap the description, or when shipping times are slow. More-
benefits of future play. Given that sellers have incen- over, in eBay’s calculation of reputation scores neutral
tives to cooperate when their reputations are at stake, ratings hold no value. Because the assignment of a
the buyers’ strategy is to invest a positive amount neutral rating is relatively ambiguous, we limit our
when they interact with the same seller repeatedly. hypotheses to only good and poor ratings.
When buyers and sellers do not interact repeatedly,
4.2. Online Reputation Formation
but a perfectly informed reputation system is in place,
An important aspect of this study is the investigation
the theoretical prediction should be the same; sellers
of how electronic reputations are formed in online
cooperate to maintain a good reputation, and subse-
market transactions. Specifically, we seek to under-
quently buyers invest. Accordingly, we expect buyer
stand what might prompt buyers to leave good versus
investments and seller return amounts to be higher in
poor ratings. Work by Bell (1985) on disappointment
markets where a reputation mechanism is used. theory provides insight into our predicted outcomes.
Hypothesis 1 (H1). The reputation system will have a Disappointment theory states that decision makers
positive association with buyers’ investment levels. under risk will form reference points and then evalu-
ate outcomes according to these anchors. Disappoint-
Hypothesis 2 (H2). The reputation system will have a ment is further defined as a psychological reaction to
positive association with sellers’ return decisions. an outcome that does not meet a decision maker’s a
priori expectation (Bell 1985), and is measured as the
In addition to hypothesizing the general effects distance between the average expected outcome and
of the reputation mechanism, we are also interested the actual payoff. Conversely, outcomes that exceed
in the effects of the specific reputation information. the decision maker’s expectations are viewed posi-
This specific reputation information pertains to the tively, and the individual may experience elation as
actual ratings assigned to a particular individual, a result.
and provides buyers a way to assess the likelihood Disappointment theory evolved from studies ex-
that a certain seller will cooperate. Work focusing ploring the effects of regret on risk attitude (Bell 1982,
on the effects of specific reputation ratings in eco- 1985; Loomes and Sugden 1982). Regret is defined as
nomic transactions includes a field study by Lucking- a psychological reaction to making a wrong decision,
Reiley (2000), exploring the determinants of pricing in where “wrong” is evaluated by comparing the real-
online auctions. He finds that seller reputations have ized outcome with the payoff one could have received
a measureable effect on pricing, and that a good seller if he or she had made a different decision. In our
reputation can generate a price premium. Resnick and experiment, the nonbinding announcement indicated
Zeckhauser (2002) show slight support for the associ- by the seller at the start of the transaction sets the
ation between positive ratings and higher sale prices, buyer’s reference point for a return amount. Accord-
as well as an association between negative ratings ing to the theory, any return less than the promised
and lower sale prices. In the same vein, work by amount could result in disappointment; therefore it is
Melnik and Alm (2002) also shows that a seller’s rep- likely that a poor rating would be assigned when a
utation can have a positive impact on prices. buyer experiences disappointment and/or regret fol-
To summarize our setting, in equilibrium, sellers lowing the seller’s return. Conversely, if the seller’s
should bear costs for a poor reputation that exceed the return amount exceeds the buyer’s reference point,
benefits they could gain from opportunistic behavior. this would be viewed as a good outcome, in which
Likewise, sellers’ should be rewarded for good repu- case a positive rating should be more likely.
tations, and these rewards should exceed the possible Hypothesis 4A (H4A). Return amounts higher than
gains from opportunism (Shapiro 1983). Therefore we promised will be associated with good ratings.
predict a negative association between poor ratings
and buyer investments, and predict a positive associ- Hypothesis 4B (H4B). Return amounts lower than
ation between good ratings and buyer investments. promised will be associated with poor ratings.

Hypothesis 3A (H3A). Sellers’ poor ratings will have 4.3. Uncertainty in the Transaction Environment
a negative association with buyers’ investment levels. To address our research questions regarding the direc-
tional effect of uncertainty on the provision and use
Hypothesis 3B (H3B). Sellers’ good ratings will have of reputation information, we incorporate a treat-
a positive association with buyers’ investment levels. ment, where additional noise is introduced into the
Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS 443

transaction environment. Recall that in the treatments address the effects of reputation information and
where additional uncertainty is imposed, the seller’s require tests that include specific reputation vari-
return choice may be intercepted by the experimenter ables (good and poor ratings). Hypotheses H4A and
and reduced to 0. This type of perturbation, where H4B address the buyers’ rating decisions and assess
players’ actions can also be shown as mistakes, is what constitutes a good versus poor reputation score.
often called a “trembling hand.” The trembling hand Finally, H5 and H6 relate to the marginal effect of
equilibrium was first specified analytically by Selten uncertainty on buyers’ investment choices and their
(1975) and assumes that players might play the off rating decisions.
equilibrium strategy because of a “slip of the hand.”
Prisoner’s dilemma games that incorporate trembling 5. Analysis and Results
treatments into the design show that when players The first section of analyses (5.1) focuses on the effects
cannot infer the intentions of others with certainty, of the reputation mechanism and specific reputation
they will sometimes give each other the benefit of the information on the transaction decisions of buyers
doubt when noncooperative outcomes occur (Bendor and sellers. The second part of the analysis (5.2) turns
et al. 1991, Sainty 1999). In the standard prisoner’s to the reputation formation aspect of our study, inves-
dilemma game, when one player knows the other has tigating how ratings are assigned with and without
defected, the equilibrium response is for the second the additional uncertainty imposed.
mover to also defect, following a tit-for-tat strategy.
However, in a noisy prisoner’s dilemma game, if the 5.1. Buyer and Seller Transaction Decisions
second mover is uncertain as to whether the defec- Table 1 shows a comparison of means, along with the
tion of the first mover was truly intended, that sec- proportion of transactions initiated and completed, as
ond mover may not always retaliate by defecting in a function of each of the four experimental treatments.
kind. In addition, Fudenberg and Maskin (1990) show Standard errors are reported in parentheses.
that an evolutionarily stable strategy of cooperation The comparisons in Table 1 show mean investments
can occur if players believe there is a small probabil- and mean returns were highest with a reputation
ity others can make mistakes. Their results indicate mechanism in place, and without additional uncer-
that uncertainty regarding the alignment of outcomes tainty imposed (RNI). Interestingly, this preliminary
versus intentions could impact cooperation and the comparison shows no statistical difference in mean
intent to punish. investment between the interception treatment with
If one views online reputation ratings as reward reputation (RI) and the interception treatment without
or punishment for past behavior, the findings of reputation (NRI). It generally appears that even with
Fudenberg and Maskin (1990) suggest that uncer- the reputation system in place, when interception was
tainty may have an effect on the rating choice. Specif- possible, mean investments were lower. Table 1 also
ically, if intentions are uncertain, it is reasonable to shows the mean proportions of initiated and com-
expect that a punishment strategy might not always pleted transactions were highest in the two reputation
follow a poor outcome. Also, if it is known that treatments (RI, RNI).6 This implies that buyers’ trust,
reporting could be less accurate, then the reliability exemplified by their willingness to transact, and sell-
of a poor rating is drawn into question, as the possi- ers’ reciprocation of that trust, were each higher when
bility that an honest seller is assigned a poor rating a reputation mechanism was in place. These are sim-
becomes viable. We therefore predict that additional ple univariate comparisons, however, which do not
uncertainty imposed on the reputation system will control for other associated factors. Accordingly, we
render poor ratings less reliable on the margin, and base our formal inferences on the subsequent empiri-
thus their detrimental effect on investment amounts cal tests.
will be lessened. We also predict that there will be a Model 1 below tests the relationship between buyer
lower incidence of poor ratings when interception is investments and the reputation mechanism (H1), as
possible and intentions are not known with certainty. well as seller-specific reputation information (H3A,
H3B). The model also tests whether the imposed
Hypothesis 5 (H5). The possibility of interception will uncertainty influenced buyers’ investment choices, as
moderate the negative relationship between poor ratings a function of sellers’ prior ratings (H5). Because of
and buyers’ investment amounts. the nonnormal distribution of the data, bootstrapping
was used to obtain the coefficient estimates, standard
Hypothesis 6 (H6). The possibility of interception will
errors, and p-values. We drew a sample of 596 obser-
have a negative effect on the number of poor ratings.
vations with replacement, and repeated this process
In summary, H1 and H2 relate to predictions about
the reputation mechanism, and involve testing for 6
A completed transaction is one where the seller made a nonzero
treatment specific effects. Hypotheses H3A and H3B return.
 Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
444 Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS

Table 1 Descriptive Statistics by Treatment

Reputation Reputation No reputation No reputation

no Intercptn with Intercptn no Intercptn with Intercptn
Variable (RNI) (RI) (NRNI) (NRI)

Mean invested $00690 $00560 $00460 $00570

4000335 4000245 4000355 4000425
Mean returned $00850 $10100 $00460 $00670
4000695 4000525 4000595 4000775
Mean buyer profit $10410 $10290 $10010 $10110
4000505 4000375 4000585 4000445
Mean seller profit $10670 $10370 $10360 $10590
4000905 4000665 4001125 4001335
Mean ratio of return 10390 10170 00530 00870
4000155 4000185 4000225 4000155
Proportion of trustworthy returns 00880 00830 00720 00610
4000205 4000305 4000405 4000405
Proportion of initiated transactions 00870 00850 00580 00740
4000305 4000305 4000405 4000405
Proportion completed transactions 00780 00640 00360 00440
4000305 4000205 4000305 4000405
Subject count 22 22 24 22
Rounds 13 24 14 11
Observations 143 264 168 121

1,000 times to estimate the regression parameters. variables in our model. The variables Total Good and
Because we structure each round of play as one unit Total Poor represent the “aggregate rating” variables,
of analysis, a random-effects model was specified to and each indicates the total number of good and poor
account for the likelihood of individual-specific corre- ratings in sellers’ reputation histories at the time of
lated errors resulting from this design choice (Casari buyers’ investment decisions. We also include binary
et al. 2007). In all models, subscript i indicates a spe- “single rating” variables, indicating whether the seller
cific seller, while subscript t indicates a specific round. received a good or poor rating in the prior round.
All analysis was done using Stata 9.0. These instances are represented by the variables Prior
Model 1: Bootstrapped Ordinary Least Squares Good and Prior Poor, respectively, and each is coded as
(OLS) Regression of Buyer Investment 1 if true, and 0 otherwise. We interact the reputation
variables with the interception treatment variable to
Investmenti1 t
test for moderating effects. The neutral rating is the
= ‚0 + ‚1 Intercptni1 t + ‚2 Reputationi1 t reference group.
+ ‚3 Reputation × Intercptni1 t + ‚4 Profit Promisei1 t The rationale for partitioning the reputation infor-
+ ‚5 Round_10 upi1 t + ‚6 Prior Return Amti1 t mation into aggregate versus single rating variables
is based on work in behavioral finance, showing
+ ‚7 Prior Goodi1 t + ‚8 Prior Poori1 t that investors may process and use information dif-
+ ‚9 Intercptn × Prior Goodi1 t ferently, depending on whether it is aggregated or
+ ‚10 Intercptn × Prior Poori1 t + ‚11 Total Poori1 t more finely partitioned (Hirshleifer and Teoh 2003).
This suggests that the reputation information in a
+ ‚12 Total Goodi1 t + ˜i1 t 0 (1)
seller’s rating history might be interpreted differently,
The Reputation variable is binary and indicates depending on whether it is processed as an aggre-
whether the reputation mechanism was present. It gate rating or a singular rating given in the most
is coded as 1 if the buyer was asked to rate the recent round. A reasonable distinction is that while
transaction on its completion, and 0 otherwise. The the aggregate ratings provide a way to measure a
Intercptn variable is also binary and indicates whether seller’s performance over time, the rating from the
the sellers’ returns could be intercepted (1), or not (0). prior round might provide information about a pos-
Reputation × Intercptn is the interaction term represent- sible trend.
ing treatments where buyers were asked to rate sellers We conjecture that although sellers’ initial return
and interception was also possible. announcements were nonbinding, this still may have
To tease out the effects of specific reputation infor- had an effect on transaction decisions and should be
mation, we include two different groups of reputation controlled for. In theory, buyers should not invest
Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS 445

if sellers promise a return equal to or less than the Table 2 shows that overall the Reputation vari-
invested amount, as this means the buyer bears all able has no significant effect on buyer investment
the risk with no expectation of additional profits. amounts, thus we do not find initial support for H1.
We identify those instances where sellers promised a We do see an association between buyer investments
profitable return to buyers, meaning they promised to and some of the singular reputation information (i.e.,
return amounts greater than what would be invested. Prior Poor and Interceptn × Prior Poor are significant),
If a profitable return is promised, we code these finding that when interception was possible, the neg-
instances as 1, and 0 otherwise. This comprises the ative impact of a prior poor rating was lessened. This
binary Profit Promise variable. result shows support for H5, which hypothesizes that
Although players are repaired after each round, the interception will moderate the negative relation-
thus making it unlikely the prior round outcome ship between poor ratings and buyer investments.
would be associated with a current partner, it is rea- Somewhat surprisingly, the overall effect of Prior Poor
sonable to assume that current round decisions would is positive (‚ˆ 8 + ‚ˆ 10 = 4036 cents), because the appar-
be influenced by prior experiences with other part- ent willingness of buyers in the interception treatment
ners. We control for this with the Prior Return Amt to invest more when sellers had a poor rating in the
variable, which is the amount returned to the buyer preceding round. Although this result is somewhat
in the prior round. Although our design addresses the counter intuitive, one possibility is that some buyers
possibility of end-game effects, we still control for the may have gambled on whether a seller would com-
possibility that investments might decrease markedly pensate for a poor rating received in the prior round.
in the final round because incentives to cooperate are It is reasonable to think that a seller with reputa-
less salient. Round_10up is binary, coded as 1 if the tion concerns might be more generous after receiv-
transaction occurs in round 10 or higher, and is 0 oth- ing a poor rating, particularly if that poor rating was
erwise. Results are shown in Table 2. undeserved, to ensure a good rating in the current
round. In contrast, the influence of Prior Good, with
and without interception, was not significant. It is
Table 2 Results of Bootstrapped OLS Regression for Buyers’
Investment possible buyers’ reaction to a prior poor rating was
accentuated by their concerns that the seller would
Variable E[sign] Coefficient P -value take advantage of their risk position, and cheat them
Constant ‚0 500990 00000 by returning less. Given that a prior poor rating could
4809705 have affirmed these fears, skeptical buyers would
Intercptn +/− ‚1 70920 00111 have been more likely to consider a poor rating than
41200405 a good rating in their investment decision.
Reputation + ‚2 110910 00221 Turning to the aggregate reputation information,
41102105
we show that the total number of good ratings (Total
Reputation × Intercptn +/− ‚3 −140020 00303
41500205
Good) had a positive effect on the amount invested,
Profit Promise − ‚4 250600 00001
while the total number of poor ratings (Total Poor)
4606105 had a negative association. One interesting result is
Round_10up +/− ‚5 −150100 00000 that the magnitude of the coefficient on the Total
4209305 Good variable is approximately twice that of the Total
Prior Return Amt + ‚6 000260 00227 Poor variable (4.70 versus −2.31). This could imply
4000245 that, unlike the single reputation information, buyers
Prior Good + ‚7 106780 00862 weighted the total number of good ratings more heav-
4500905
ily than the total number of poor ratings. This result
Prior Poor − ‚8 −140760 00069
4901005
has possible implications for reputation management
Intercptn × Prior Good +/− ‚9 20340 00654
strategies, implying that increasing the number of
4608105 good ratings could potentially mitigate the negative
Intercptn × Prior Poor + ‚10 190120 00048 effects of poor ratings.
4906105 To gain further insight into the interactions between
Total Poor − ‚11 −20310 00000 the specific reputation information and the inter-
4007205 ception treatment, Figure 2 illustrates the impact of
Total Good + ‚12 40700 00000 sellers’ reputation ratings from the prior round on
4009705
buyers’ investments in the current round. The figure
Observations 596
R2 /Adjusted 00260
is constructed from the raw data and shows the
average buyer investment (the y-axis) as a function of
Notes. Bootstrapped coefficients and p-values reported. Standard errors whether the seller received a poor rating in the prior
reported in parentheses.
 Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
446 Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS

Figure 2 Buyers’ Average Investment as a Function of Sellers’ Prior Figure 3 Average Proportion of Transactions as a Function of the
Poor Rating Reputation System, with and Without Interception

Average buyer investment conditional on seller Average proportion of initiated transactions as

receiving a poor rating in prior round a function of the reputation system
1.0 1.0

Proportion of initiated
0.8
Investment amount

0.8

transactions
0.6 0.6

0.4 0.4

0.2 Interception
0.2 Interception
No interception No interception
0 0
Reputation = 0 Reputation = 1
Prior poor = 0 Prior poor = 1
Indicates reputation treatment
Indicates whether seller received a poor rating
in prior round
levels, leaving H1 unsupported, Model 2 results
round. Included is a comparison with the intercep- indicate that the reputation system had a positive
tion treatment to show how buyer responses to a poor association with buyers’ decisions to engage in the
rating were influenced by the additional uncertainty transaction. Results show that overall the reputation
imposed (the x-axis). system increased the odds that buyers would initi-
The graph shows that when interception was pos- ate the transaction, suggesting that even a noisy rep-
sible, a seller’s poor rating did not diminish mean utation system can aid cooperation. This result does
investments as much as when the interception treat- provide support for H1, which again, hypothesizes
ment was absent. This suggests an interaction effect the reputation system will have a positive associa-
between the poor rating and the amount of noise tion with buyers’ investment levels. So while the rep-
in the transaction environment, a relationship that is utation system has no association with incremental
evident in Table 2. investment amounts (shown in Table 2 results), it does
Continuing the focus on buyers’ investment have a positive association with the choice to invest
choices, Model 2 tests buyers’ decisions to initiate the “something” versus “nothing.”
transaction, which is indicated by the choice to invest Figure 3 provides greater insight into the inter-
any amount greater than 0. If the buyer invested action between the interception treatment and the
$0.00, we interpret this to mean that he or she did reputation mechanism.7 The x-axis indicates whether
not want to engage the seller in that particular trans- the reputation system was in effect, and the y-axis
action. We specify the binary dependent variable as indicates the proportion of transactions initiated by
1 if the buyer chose to invest a nonzero amount, buyers.
and 0 otherwise. In contrast to Model 1, Model 2 The graph shows that the reputation system in-
includes the interaction variables Intercptn × Total Poor creased the likelihood that buyers would initiate a
and Intercptn × Total Good. The model does not include transaction, while the difference with and without
the singular reputation variables (Prior Good, Prior interception is negligible. Overall, the likelihood of
Poor), as we expect buyers’ transaction decisions to be buyers transacting was higher with the reputation
primarily based on sellers’ comprehensive reputation system than without, regardless of whether additional
histories. uncertainty was imposed.
Model 2: Logistic Regression Analysis of Buyers’ The results in Table 3, as well as the graphs con-
Decision to Transact structed from the data in Figures 4(a) and 4(b) below,
show that overall when interception was possible, the
Logit4Transacti1 t 5 total number of poor ratings did not have a nega-
= ‚0 +‚1 Intercptni1 t +‚2 Reputationi1 t tive effect on the proportion of initiated transactions.
Specifically, in treatments without additional uncer-
+‚3 Reputation×Intercptni1 t +‚4 Profit Promisei1 t
tainty, buyers were less likely to transact when a seller
+‚5 Round_10upi1 t +‚6 Total Poori1 t +‚7 Total Goodi1 t had more poor ratings, supporting H3B. However, in
+‚8 Intercptn×Total Poori1 t +‚9 Intercptn the interception treatment, buyers appear to be more
likely to give sellers the benefit of the doubt, and the
×Total Goodi1 t +˜i1 t 0 (2)
proportion of initiated transactions is higher. It seems
While Model 1 results showed the reputation sys-
tem (Reputation) had no association with investment 7
Calculated from the raw data.
Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS 447

Table 3 Results of Logistic Regression for Buyers’ Figure 4(a) Average Proportion of Transactions as a Function of Total
Decision to Transact Number of Good Ratings

Variable Odds P -value 1.0

Intercptn ‚1 1077 00312 0.8

Proportion of initiated
410005

transactions
Reputation ‚2 15057 00050 0.6
4210785
Reputation × Intercptn ‚3 0008 00105 0.4
400135
Profit Promise ‚4 0027 00002 0.2 Interception
400125 No interception
Round_10up ‚5 0038 00000 0
400085 Below average Above average
Total Poor ‚6 0062 00008 Indicates whether the proportion of total good
400115 ratings per transaction round are greater or
Total Good ‚7 0093 00770 less than the average
400215
Figure 4(b) Average Proportion of Initiated Transactions as a
Intercptn × Total Poor ‚8 1043 00051 Function of Total Number of Poor Ratings
400275
Intercptn × Total Good ‚9 1051 00072 1.0
400355

Proportion of initiated
0.8
Observations 596
Pseudo-R2 0017

transactions
0.6
Notes. Odds calculated as exp4‚ˆi 5 are reported. Standard
errors reported in parentheses. 0.4

buyers may not have perceived poor ratings to be 0.2 Interception

completely indicative of true behavior when it was No interception
unclear whether they were actually deserved. This 0
result shows additional support for H5, indicating the Below average Above average
interception did moderate the negative relationship Indicates whether the proportion of total poor
ratings per transaction round are greater
between poor ratings and the investment decision.
or less than the average
Results also show the total number of good ratings
increased the odds of buyers initiating a transaction, poor ratings the benefit of the doubt when it was less
indicating that buyers were generally more likely to clear whether those ratings were truly warranted.
trust those sellers who had a greater number of good Model 3 addresses whether sellers’ return choices
ratings. This result shows additional support for H3A. were influenced by the reputation mechanism, specif-
The x-axis of the graphs in Figures 4(a) and 4(b) ically testing H2. We omitted those instances where
indicate whether the total proportion of good or buyers sent nothing, as this means that they did not
poor ratings was below or above the mean propor- initiate the transaction and by design, return amounts
tion,8 while the y-axis is the proportion of initiated must be $0.00. After eliminating these data, our num-
transactions. ber of observations was 446. Because of nonnormal-
Figure 4(a) shows that a greater number of good ity of the data, we again estimated the empirical dis-
ratings (x-axis) resulted in a higher instance of initi- tributions for each of the OLS coefficients via boot-
ated transactions (y-axis), although this increase was strapping. We drew a sample of 446 observations with
less in cases where interception was possible. Fig- replacement and repeated this 1,000 times to derive
ure 4(b) shows that without interception, more poor the regression parameters.
ratings lead to fewer initiated transactions, but this Model 3: Bootstrapped OLS Regression of Seller
was not the case when interception was a factor. Returns
Specifically, Figure 4(b) shows that in the interception Returnedi1 t
treatment, buyers did not react as negatively to sellers
who had a greater number of poor ratings. Again, this = ‚0 + ‚1 Intercptni1 t + ‚2 Reputationi1 t
illustrates that buyers may have given sellers with + ‚3 Reputation × Intercptni1 t + ‚4 Round_10upi1 t
+ ‚5 Investedi1 t + ˜i1 t 0 (3)
8
Mean proportion of good or poor ratings were calculated per
round and then averaged. The overall mean proportion of good Our results in Table 4 show that returns were higher
ratings = 0040, and the mean proportion of poor = 0039. with the reputation system (Reputation) employed and
 Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
448 Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS

Table 4 Results of Bootstrapped OLS Regression for Sellers’ Return Figure 5 Average Proportion of Greater or Equal Returns as a Function
of the Reputation System, with and Without Interception
Variable Coefficient P -value
Average proportion of returns greater or
Constant ‚1 −13094 00000 equal to what was promised
480475 1.0

Proportion of greater or
Intercptn ‚2 4064 00713
0.8
4150505

equal returns
Reputation ‚3 30048 00001 0.6
4130005
Reputation × Intercptn ‚4 −8033 00844 0.4
4180845
0.2 Interception
Round_10up ‚4 −7008 00072
No interception
450555
0
Invested ‚5 1038 00000 Reputation = 0 Reputation = 1
400125 Indicates presence of the reputation treatment
Observations 446 446
R2 /Adjusted 0038
Results show the odds of a seller making a truthful
Notes. Bootstrapped coefficients and p-values reported. Standard errors
reported in parentheses.
return increased when the reputation system (Reputa-
tion) was present. This result provides additional sup-
port for H2, indicating the reputation system had a
no interception (i.e., ‚2 = 30048, p-value = 00001), positive impact on seller return decisions. Figure 5
showing support for H2. Results also show that the further illustrates the effect of the reputation system
amount invested by buyers (Invested) is significant on trustworthy returns.
and positive (i.e., ‚5 = 1038, p-value = 00000), indicat- The graph shows that in cases with and without
ing that reciprocation explains much of the sellers’ uncertainty, sellers were more likely to make trust-
return decisions. This finding implies that when buy- worthy returns when the reputation system was in
ers invested larger amounts, sellers were more likely place; however, the difference in trustworthy returns
to positively reciprocate. Positive reciprocity can be between the interception and no interception treat-
defined as the impulse or the desire to be kind to ments was negligible.
those who have been kind to us (Fehr et al. 1997) and Table 5 also shows the aggregate reputation vari-
has been observed in trust games and gift exchange ables, Total Poor and Total Good, are significant in the
games (Fehr et al. 1997, Berg et al. 1995). Finding predicted directions, suggesting aggregate reputation
that sellers’ return behaviors are driven, in part, by information may help buyers evaluate sellers’ propen-
the desire to reciprocate is consistent with prior stud- sity for a trustworthy return (or lack thereof). For
ies, showing when first movers send greater amounts, each unit increase in the amount invested (Invested),
they are rewarded with larger returns from second the odds the seller would return an amount greater
movers.
Finally, we test whether the reputation system had Table 5 Results Logistic Regression for Truthful Return
an influence on sellers’ choice to return truthfully,
in relation to the amounts they initially promised. Variable Odds P -value
We interpret a truthful return to mean the seller Intercptn ‚1 20450 00006
returned an amount equal to or greater than what 4008105
was promised. The dependent variable TRU Return is Reputation ‚2 50710 00001
binary, and is coded depending on whether the seller 4202905
returned an amount greater than or equal to what was Reputation × Intercptn ‚3 00430 00109
4002305
promised (1), or less than promised (0).
Round_10up ‚4 00730 00266
Model 4: Logistic Regression of Sellers’ Truthful
4002105
Return
Invested ‚5 00970 00000
4000035
Logit4TRU Returni1 t 5 Total Poor ‚6 00830 00001
4000405
= ‚0 +‚1 Intercptni1 t +‚2 Reputationi1 t
Total Good ‚7 10250 00001
+‚3 Reputation×Intercptni1 t +‚4 Round_10upi1 t 4000805
Pseudo-R2 00200
+‚5 Investedi1 t +‚6 Total Poori1 t
Notes. Odds calculated as exp4hat‚i 5 are reported. Standard
+‚7 Total Goodi1 t +˜i1 t0 (4) errors reported in parentheses.
Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS 449

than promised is approximately 1. A plausible expla- Figure 6 Likelihood of a Poor Rating When Return Is Less Than
nation is that often sellers promised buyers that if Promised
they invested the full amount ($1), they would split Probability of a poor rating conditional on seller
the total profit to achieve a distributionally fair return returning less than promised
1.0
($2). Therefore, in many cases when buyers invested
the maximum amount possible, they received return

Probability of a poor rating

0.8
amounts that were almost always equal to, but rarely
more than, the split that was promised.
0.6

5.2. Ratings and Reputation Formation

0.4
Model 5 addresses buyers rating decisions, and
specifically tests H4A, H4B, and H6. The dependent
0.2 Interception
variable Rate is specified at three levels depending on
No interception
whether the rating assigned to the seller was good (3),
0
neutral (2), or poor (1). The model is tested with Returnless = 0 Returnless = 1
an ordinal logistic regression. The neutral rating, and Indicates whether seller returned
the instances where return amounts equal what was less than promised
promised, are each specified as reference groups.
Model 5: Ordinal Logistic Regression of Buyers’ likelihood of good and poor rating decisions as a
Rating Decision function of seller returns.
Figure 6 suggests that buyers were less likely to
Ordinal Logit4Ratei1 t 5 punish sellers with a poor rating when interception
= ‚0 + ‚1 Intercptni1 t + ‚2 Return Lessi1 t was possible. Specifically, it seems that when addi-
tional uncertainty was imposed, some buyers who
+ ‚3 Return Greateri1 t + ‚4 Intercptn × Return Lessi1 t experienced low returns relative to the promise gave
+ ‚5 Intercptn×Return Greateri1 t + ˜i1 t 0 (5) that seller the benefit of the doubt and did not always
respond with a poor rating.
Additional variables for Model 5 are Return Less,
indicating whether the seller returned less than
promised, and Return Greater, indicating whether the
6. Discussion
This paper addresses three questions. The first asks
seller returned more than promised. Both are binary
how reputation ratings are assigned, and thus how
and are coded as 1 if true, and 0 otherwise. Results
reputations are formed, in online markets. We show
in Table 6 show that without interception, when sell-
that expectations and reference points may play a
ers made returns less than promised, they were more
prominent role in rating behaviors; when expectations
likely to receive a poor rating, relative to when inter-
are not met, poor ratings ensue. However, we also
ception was possible. To clarify, Figure 6 provides
show that in spite of unmet expectations, poor rat-
additional insight into the interaction effect between
ings may be less likely when buyers are not certain
the interception and sellers truthful returns. The fig-
ure is calculated from the raw data and shows the whether sellers intended the disappointing outcome.
Next, we investigate the economic and behavioral
Table 6 Results of Ordinal Logistic Regression for effects of different reputation information. We find
Ratings Assignment that the effects of the aggregate reputation ratings
Variable Odds P -value differ from the singular ratings. Aggregate ratings
impacted buyers’ willingness to engage in the transac-
Intercptn ‚1 2051 00000
tion, while single ratings were more relevant to buy-
400545
ers’ choices of how much to invest. We also find that
Return Less ‚2 0028 00000
400095
a single poor rating is weighted differently than a sin-
Return Greater ‚3 7048 00000
gle good rating, and poor ratings were less likely to
440185 be punished when intentions were not known with
Intercptn × Return Less ‚4 2046 00042 certainty. Third, we ask how uncertainty affects the
410095 functionality of the reputation system. We find an
Intercptn × Return Greater ‚5 0090 00896 interaction between the type of reputation rating and
400705 the interception treatment. In particular, poor ratings
Observations 350 had a less detrimental effect on buyers’ trust when
Pseudo-R2 0015 additional uncertainty was imposed.
Notes. Odds calculated as exp4‚ˆi 5 are reported. Standard We inform practitioners by showing that while a
errors reported in parentheses. less noisy reputation system is preferred, even with a
 Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
450 Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS

high level of noise, a reputation system can still pro- decisions. Understanding how reputation ratings are
vide benefits to cooperation. Noting the differences assigned provides insight into how buyer and seller
in how market participants perceive different types actions might impact online reputations. To date, little
of reputation has implications for reputation manage- attention has been given to the inherent uncertainty
ment. Finding that a single poor rating in some cases present in the online transaction environment, and its
is weighted more heavily than a single good rating, effect on the functionality of electronic reputation sys-
suggests a negative shock to a reputation may impact tems. Our study is one of the first to address this
the trust of future buyers or clients more than a posi- issue with a laboratory setting designed to approxi-
tive shock. Our findings also indicate that sellers may mate an online transaction. Finally, much of the work
want to be aware of buyer expectations to secure a on online reputation systems is either theoretical or
good rating. However, because this is a difficult if empirical, and game-theoretic experiments such as
not impossible task, another option is for sellers to this one provide additional measures that contribute
manage expectations by representing their product or to the existing literature. We believe our findings are
service as accurately as possible. It does not bene- important to online mechanism design, while ulti-
fit sellers to overstate the quality of their product, mately helping researchers and practitioners identify
as their reputation will likely suffer when the lower
more precise ways of measuring, and accounting for,
quality good is received.
the economic value of a reputation system.
One limitation of this study is that experimental
methods tend to impose a strictly controlled environ-
ment, and generalization of findings should be done Appendix. Experiment Screen Shots
with care. However, this limitation should also be
viewed as a strength of the methodology. Using the Screen 1
laboratory to strip away many of the contextual cues Shown to sellers
that could influence behavior in a field setting allows
us to isolate the individual’s decision-making behav-
ior. Another limitation of this work is that simplifying
assumptions were made about the transaction envi-
ronment, and the threat of litigation to enforce cooper-
ation is not considered. The experiment also does not
account for the possibility of explanatory text accom-
panying ratings scores, which is becoming more com-
mon in electronic reputation systems and is a logical
extension of this work.
Finally, if one’s goal were to make more compre-
hensive inferences about multiple levels of uncer-
tainty, it would be necessary to extend our uncertainty Note. The nonbinding announcement where sellers fill out their proposed
treatment to include additional interception probabil- return to buyers.
ities. Thus, one direct extension of this work is to fur-
ther investigate the effects of different levels of noise
Screen 2
on the reputation system. We impose a relatively high
Shown to buyers
probability of interception to establish a directional
effect of uncertainty. It could be beneficial to manip-
ulate this probability of interception to determine if
there is some acceptable level of noise where the ben-
efits of good ratings are not decreased, but the ben-
efits to cooperation remain. Such findings could be
of interest to future reputation system maintenance
and design.

7. Conclusion
Electronic commerce has become a mainstream busi-
ness practice, yet the information asymmetry that
exists between buyers and sellers in these markets
implies participation is not without risk. The ques-
tions addressed in this study are designed to expose
factors that impact the rating decisions of market Note. Buyer sees seller’s aggregate reputation score and round by round
participants, and the effects of uncertainty on these ratings then indicates how much they will invest.
Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS 451

References
Screen 3
Akerlof, G. 1970. The market for “lemons”: Quality uncertainty and
Shown to sellers the market mechanism. Quart. J. Econom. 84(3) 488–500.
Ba, S. 2001. Establishing online trust through a community respon-
sibility system. Decision Support Systems 31(3) 323–336.
Ba, S., P. Pavlou. 2002. Evidence of the effect of trust building
technology in electronic markets: Price premiums and buyer
behavior. MIS Quart. 26(3) 243–268.
Bell, D. 1982. Regret in decision making under uncertainty. Oper.
Res. 30(5) 961–981.
Bell, D. 1985. Putting a premium on regret. Management Sci.
31(1) 117–122.
Bendor, J., R. M. Kramer, S. Stout. 1991. When in doubt 0 0 0
cooperation in a noisy prisoner’s dilemma. J. Conflict Resolution
Note. Seller sees how much the buyer has invested and then decides how 35(4) 691–719.
much he would like to return to the buyer.
Berg, J., J. Dickhaut, K. McCabe. 1995. Trust, reciprocity, and social
history. Games Econom. Behavior 10(1) 122–142.
Bolton, G., E. Katok, A. Ockenfels. 2004. How effective are
electronic reputation mechanisms? Management Sci. 50(11)
1587–1602.
Screen 4
Bolton, G., E. Katok, A. Ockenfels. 2005. Cooperation among
Shown to buyers strangers with limited information about reputation. J. Public
Panel A Econom. 89(8) 1457–1468.
Camerer, C., K. Weigelt. 1988. Experimental tests of a sequential
equilibrium reputation model. Econometrica 56(1) 1–36.
Only shown in
interception Casari, M., J. C. Ham, J. H. Kagel. 2007. Selection bias, demographic
treatment effects, and ability effects in common value auction experi-
ments. Amer. Econom. Rev. 97(4) 1278–1304.
Cripps, M., E. Dekel, W. Pesendorfer. 2005. Reputation with equal
discounting in repeated games with strictly conflicting inter-
Panel B
ests. J. Econom. Theory 121(2) 259–272.
Cripps, M., G. Mailath, L. Samuelson. 2004. Imperfect monitoring
Only shown in and impermanent reputations. Econometrica 72(2) 407–432.
interception Dellarocas, C. 2003. The digitization of word-of-mouth: Promise
treatment and challenges of online reputation mechanisms. Management
Sci. 49(10) 1407–1424.
Dellarocas, C. 2005. Reputation mechanism design in online trad-
ing environments with pure moral hazard. Inform. Systems Res.
16(2) 209–230.
Note. Buyer sees the exchange history and their return from the seller. In Dewan, S., V. Hsu. 2004. Adverse selection in electronic markets:
both cases, the buyer is reminded that interception was possible when appli- Evidence from online stamp auctions. J. Indust. Econom. 52(4)
cable by displaying the circled text only in those instances. 497–516.
Fehr, E., S. Gächter, G. Kirchsteiger. 1997. Reciprocity as a contract
enforcement device: Experimental evidence. Econometrica 65(4)
833–860.
Fudenberg, D., E. Maskin. 1990. Evolution and cooperation in noisy
Screen 5 repeated games. Amer. Econom. Rev. 80(2) 274–279.
Shown to sellers Grief, A. 1993. Contract enforceability and economic institutions in
early trade: The Maghribi traders’ coalition. Amer. Econom. Rev.
83(3) 525–548.
Hirshleifer, D., S. Teoh. 2003. Limited attention, information dis-
closure, and financial reporting. J. Accounting Econom. 36(1–3)
337–386.
Hoffman, E., K. McCabe, V. Smith. 1998. Behavioral foundations
of reciprocity: Experimental economics and evolutionary psy-
chology. Econom. Inquiry 36(3) 335–352.
Jung, Y., J. Kagel, D. Levin. 1994. On the existence of predatory
pricing: An experimental study of reputation and entry deter-
rence in the chain store game. RAND J. Econom. 25(1) 72–93.
Kauffman, R., C. Wood. 2005. The effects of shilling on final bid
prices in online auctions. Electronic Commerce Res. Appl. 4(1)
21–34.
King, R. 1996. Reputation formation of reliable reporting: An exper-
imental investigation. Accounting Rev. 71(3) 375–396.
Note. Sellers see the transaction history, their most recent reputation score,
and their reputation history. Kreps, D., R. Wilson. 1982. Reputation and imperfect information.
J. Econom. Theory 27(2) 253–279.
 Rice: Reputation and Uncertainty in Online Markets: An Experimental Study
452 Information Systems Research 23(2), pp. 436–452, © 2012 INFORMS

Kreps, D., P. Milgrom, J. Roberts, R. Wilson. 1982. Rational coop- Resnick, P., K. Kuwabara, R. Zeckhauser, E. Friedman. 2000. Repu-
eration in the finitely repeated prisoners’ dilemma. J. Econom. tation systems. Comm. ACM 43(12) 45–48.
Theory 27(2) 245–252. Resnick, P., R. Zeckhauser, J. Swanson, K. Lockwood. 2006. The
Loomes, G., R. Sugden. 1982. Regret theory: An alternative theory value of reputation on eBay: A controlled experiment. Experi-
of rational choice under uncertainty. Econom. J. 92(368) 805–824. ment. Econom. 9(2) 79–101.
Lucking-Reiley, D. 2000. Auctions on the Internet: What’s being Sainty, B. 1999. Achieving greater cooperation in a noisy prisoner’s
auctioned, and how? J. Indust. Econom. 48(3) 227–252. dilemma: An experimental investigation. J. Econom. Behavior
Marschak, T., R. Selten. 1978. Restabilizing responses, inertia Organ. 39(4) 421–435.
supergames and oligopolistic equilibria. Quart. J. Econom. 92(1) Schwartz, S. T., R. A. Young, K. Zvinakis. 2000. Reputation with-
71–93. out repeated interaction: A role for public disclosures. Rev.
Melnik, M., J. Alm. 2002. Does a seller’s ecommerce reputation Accounting Stud. 5(4) 351–375.
matter? Evidence from eBay auctions. J. Indust. Econom. 50(3) Selten, R. 1975. A reexamination of the perfectness concept for equi-
337–349. librium points in extensive games. Internat. J. Game Theory 4(1)
Milgrom, P., J. Roberts. 1982. Predation, reputation and entry deter- 25–55.
rence. J. Econom. Theory 27(2) 280–312. Shapiro, C. 1983. Premiums for high quality products as returns to
Resnick, P., R. Zeckhauser. 2002. Trust among strangers in Internet reputations. Quart. J. Econom. 98(4) 659–680.
transactions: Empirical analysis of eBay’s reputation system. Wolf, J., H. Arkes, W. Muhanna. 2008. The power of touch: An
The Economics of the Internet and E-Commerce. JAI Press, Green- examination of the effect of duration of physical contact on the
wich, CT, 127–157. valuation of objects. Judgment Decision Making 3(6) 253–279.