Holiday 2000 E-Commerce

_______________________________________________

Avoiding $14 Billion in “Silent Losses”

by Aamir Rehman

October 2000

good@creativegood.com
www.creativegood.com
 Acknowledgements
Thanks to all who helped create the Holiday 2000 E-Commerce report:

• Nicole Rubin, Creative Good’s director of research, managed the report overall.
She managed the project, directed the research and edited drafts of the report.

• Mark Hurst, Creative Good’s founder and president, edited the final version of the
report.

• Susan Larson, Creative Good consultant, contributed to the research and

production of the report.

• Creative Good analysts Allison Jacoby, Jeniece Primus, Jennifer Thomson and
Lauren Greenberg contributed to the research and analyzed the customer test
results.

• Boston-based interactive agency Red98 produced an HTML version of Creative

Good’s prototype to test for the case study. Committed to creating “great user
experiences,” Red98 clients span a wide range of industries, from Fortune 100
companies to startups. You can learn more about them at www.red98.com.

About the author

Aamir Rehman (aamir@creativegood.com), senior analyst at Creative Good, has

advised e-commerce clients in the grocery, automotive, gift and apparel industries.
He holds a bachelor’s degree from Harvard College and a master’s degree from
Harvard’s Graduate School of Arts and Sciences. Aamir also manages
Creative Good’s best practices research.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 2
 Contents
Executive Summary 5
1. Introduction 7
2. Business Case 11
3. Methodology 16
4. Checkout 19
5. Promotions and Merchandising 34
6. Case Study: BestBuy 45

All company names, brand names and product names in this report are trademarks of their
respective owners and companies. v1018.1

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 3
 About Creative Good’s
E-Commerce Consulting Services
Creative Good (www.creativegood.com) is a 35-person consulting firm based in
New York, San Francisco, and Boston. We’re on a mission to improve the online
customer experience — on e-commerce websites, content and entertainment sites,
intranets, b2b sites, and wireless Internet services. Clients include Gateway,
Travelocity, MGM.com, kinkos.com, Rogers, and Wineshopper.com.

We have generated 40% to 150% improvements in all of our major projects. Please
contact us if Creative Good can help improve your e-business.

Phil Terry, CEO

pterry@creativegood.com

P.S. You can find our other free resources at www.goodexperience.com. To get our
free customer experience newsletter, e-mail update@goodexperience.com.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 4
 Executive Summary

About this Report

The purpose of this report is to help consumer e-commerce websites prepare for the
Holiday 2000 season. To get a representative view of online holiday shopping, we
conducted more than 50 consumer tests on eight major e-commerce websites across
four key categories:

• Apparel: Gap.com, LandsEnd.com

• Books and music: Amazon.com, bn.com (Barnes & Noble)

• Electronics: BestBuy.com, Buy.com

• Toys: eToys.com, KBKids.com

To further demonstrate the impact of improving the customer experience, we chose

one of the sites we tested — BestBuy.com — and prototyped an improved website
based on what we learned in tests. When we then tested the new prototype, the
results were dramatic: customers were four times more successful on our prototype
than they were on the original BestBuy.com (see Chapter 6).

The BestBuy case study suggests that many e-commerce sites can dramatically
increase their revenues with simple improvements to the customer experience.

Summary of Findings
The customer tests and the BestBuy case study produced the following key findings:

• In our tests, 43% of purchase attempts failed. As we explain in the Business Case
(see Chapter 2), these failures may represent $14 billion in lost sales worldwide this
holiday season.
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 5
 • Problems with checkout were the greatest cause of failure. Over 40% of failed
purchase attempts were due to difficulty during the checkout process. Poor
checkout experience alone may cost the worldwide e-commerce industry over
$6.5 billion in holiday sales (including over $3 billion in North America alone).

• Promotions and merchandising are an area of opportunity this holiday season. Gift
shoppers were open to suggestions and promotions: in 30% of tests, customers
clicked on at least one featured product.

• If sites prepare for the holidays by improving the customer experience, the industry
stands to gain $14 billion in sales.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 6
 CHAPTER 1

Introduction

This report shows how to improve the customer experience on any e-commerce site
as the 2000 holiday season approaches. It’s a timely subject, given how hard this
year has been on the e-commerce industry. Investors now want profits, not just
clever TV ads and a dotcom name. Many sites are looking to Holiday 2000 to prove
their viability as real businesses.

The single most important factor in any e-commerce site’s survival and success is its
customer experience. This report shows exactly what e-commerce sites must do to
improve the customer experience for Holiday 2000. It is worth thousands or millions
of dollars to companies that follow its suggestions.

We know these suggestions work because we tested them. In our BestBuy case study
(see Chapter 6), we tested the customer experience of the current BestBuy site, and
then created our own redesigned version — according to the customer experience
suggestions in this report — and tested the improved prototype. Result: The
conversion rate quadrupled. Our tests indicate that BestBuy.com could make the
simple changes we suggest and instantly double its revenues, if not triple or
quadruple them. (Indeed, it’s not unusual for Creative Good clients to measure a
similar rise in their own key metrics.)

The lesson here is striking:

Making simple improvements to the customer experience can easily double the sales
of many top e-commerce sites.
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 7
 Why focus on customer experience?

It’s a fair question: why focus on on-site customer experience during Holiday 2000,
when last year’s holiday season showed so many other problems on e-commerce
sites? Fulfillment, for example: the New York Times reported just after Holiday ’99
that “the primary bugaboo was failing to deliver packages on time1.” And a recent
Business 2.0 cover story remembered last year’s “flooded customer service desks,”
as angry customers called and e-mailed about undelivered packages. In fact, as a
result of fulfillment troubles, seven e-commerce sites were fined a total of
$1.5 million by the Federal Trade Commission2 .

Yet this report maintains that the on-site customer experience is the single most
important factor in e-commerce success. Regardless of the dozens of articles about
fulfillment and overloaded call centers that were written last year, and will probably
be written this year, poor customer experience is still the biggest problem in the
e-commerce industry.

Fulfillment is naturally a high-profile issue. Complaints to the customer service

department inevitably raise attention in the company, and in the press, because
complaints are loud. The customer completed the order, paid the money, and now the
package is late; only complaining loudly, the customer reasons, will get the problem
solved.

But what about the customers who didn’t complete their orders on the website? The
site never hears from them. Customers who get confused on the site, then leave
forever without ordering or paying, aren’t interviewed by the press. They don’t
overload the call center with complaints. They just get frustrated, leave the site
quietly, and never return. The orders that customers fail to complete, because of the
poor customer experience, are what we call silent losses.

No matter how many customers complain about fulfillment problems, many more
customers leave the site without complaining, because the site is too hard to use.

1
The New York Times, “A Very Big E-Commerce Christmas,” Jan. 2, 2000.
http://www.nytimes.com/library/tech/00/01/biztech/articles/02webb.html
2
Business 2.0, “Hope for the Holidays,” October 10, 2000.
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 8
 “Silent losses”

Silent losses are hard to measure, since frustrated customers don’t call or e-mail the
site to say they’re leaving. The press doesn’t usually write stories about silent losses,
since it’s hard to measure the size of the problem, and it’s difficult to find people who
have quietly left a site in frustration.

But the silent losses are there. We know this because we observed, face-to-face,
average e-commerce customers attempting to place orders on e-commerce sites. Our
more than 50 tests of eight major e-commerce sites yielded a startling finding:

In our tests, 43% of purchase attempts ended in failure.

Forty-three percent! Can you imagine 43% of shoppers physically walking out of a
Wal-Mart store because they couldn’t figure out how to pay for their order? What if
43% of visitors to the Starbucks on the corner left the store in frustration, grumbling
about buying coffee somewhere else? No brick-and-mortar store would ever stand for
such an insane failure rate. But here is the e-commerce industry, letting 43% of its
revenues slip away. In our Business Case (see Chapter 2) we show that this amounts
to a loss of $14 billion industry-wide this holiday season.

This report is our plea to e-commerce sites to improve their customer experience so
that they, and the industry as a whole, can survive into next year — and so that
customers will no longer have thoughts like these, comments spoken by customers in
our tests:

• “Screw it, I’m just going to go to the store and buy it. I don’t know what I want to
do at this point.”

• “I’d probably get off right now.”

• “That’s it, I’m out of here.”

Simple solutions

Fortunately, there are simple improvements that e-commerce sites can make to
substantially raise holiday sales. If an e-commerce site can improve just two things
before the holidays, we’d suggest the following:

• Make it easy for holiday shoppers to find gifts. In Chapter 5 we show how
promotions and featured items were especially effective in tests.
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 9
 • Make it easy to check out. More customers had trouble with checkout than any
other feature on the sites we tested. In Chapter 4 we show some easy ways to raise
revenues by improving the checkout process.

We should also note that these are not the only improvements e-commerce sites
should make. Successful e-businesses make customer experience a constant focus,
continually monitoring and improving it month after month. And that means more
than simply prototyping and redesigning the tactics of the website interface. Instead,
our client projects have shown that to improve the e-business, we have to change the
organization behind the interface. We’ve found the Web interface to operate like a
roadmap of the organization, describing its strategy and culture.

So as you read the report, do note the tactical changes we suggest to help prepare for
Holiday 2000. But more importantly, consider making a strategic commitment to
customer experience in your e-business. And let us know if we can help.

Mark Hurst
Founder and President, Creative Good
mark@creativegood.com

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 10
 CHAPTER 2

Business Case
$14 Billion in Silent Losses

The year 2000 has been tough for online retailers. Changes in the market have driven
many sites out of business, and many others are only barely surviving. The holidays,
promising greatly increased sales, offer hope for many e-commerce sites.

The good news for retailers is that the holiday market will grow; this year’s holiday
sales are predicted to be 66% higher than last year’s.3 In a year when over half the
U.S. population has access to the Internet,4 e-commerce sites can expect dramatic
increases in site traffic before the holidays. Forrester Research predicts $10 billion in
holiday revenues, and Jupiter forecasts the U.S. holiday market as $12 billion.
Gartner Group predicts $10.7 billion in North American holiday sales and worldwide
sales of $19.5 billion.5 While revenue estimates vary, it’s clear that Holiday 2000 will
be big for e-commerce.

Customers will shop online this holiday season for convenience; by shopping online,
customers avoid crowds, transportation hassles, and inconvenient store hours. One
customer commented in our tests that she will shop online “to avoid the hassle of
going to different stores.” Another shopper said that buying gifts online is “a lot
easier because of the lines [in the stores].”

3
Cyberatlas - Jupiter, September 18, 2000
4
Nielsen/NetRatings, August 17, 2000
5
eMarketer - Gartner Group, September 6, 2000
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 11
 Despite the eager shoppers and the predictions of a big holiday season, there is bad
news for online retailers: the e-commerce industry is losing billions of dollars in
potential sales. Shoppers want to purchase online, but many sites make it too hard to
buy.

In our tests, we observed that 43% of purchase attempts ended in failure. Frustrated
customers made comments like “if it took this long, I would just go to the [off-line]
store.” As one shopper put it, “[I might] buy something this holiday season online,
but not on these sites, because they are so slow and annoying.” Sites’ actual revenues
this season will represent only a fraction of what they could be if sites made it easier
to shop and buy.

To gain the most from the holiday season, sites must pursue a simple strategy:
improve the customer experience. Improving the customer experience can yield an
additional $8 billion in North American sales, and $14 billion in worldwide online
sales this holiday season.

Poor customer experience may cost the worldwide e-commerce industry

$14 billion this holiday season.

$14 Billion in Lost Sales

According to Gartner Group projections, worldwide online revenues for the holiday
season will be $19.5 billion.6 In our consumer tests of major sites across key holiday
categories, we found that 43% of buying attempts failed. These findings suggest that
the $19.5 billion represent only 57% of potential sales this holiday season:

• 100% (potential market) - 43% (due to buying failures) = 57% (of sales realized).

If $19.5 billion represents only 57% of potential sales, total worldwide potential sales
for Holiday 2000 are over $34 billion:

• $19.5 billion / 0.57 = $34.2 billion.

Lost sales due to poor customer experience thus total more than $14 billion:

• $34.2 billion - $19.5 billion = $14.7 billion

Of course, these figures are based on several assumptions about how the market will
6
eMarketer - Gartner Group, September 6, 2000
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 12
 perform. They are not meant to be scientifically accurate but rather to give a sense of
the possible loss of revenue this holiday season. If our calculations are anywhere near
correct, sites will leave a massive amount of money unrealized — unless they
improve the customer experience.

It is worth noting that our $14 billion does not include future losses due to poor
experience this holiday season. Millions of consumers will shop online for the first
time this holiday season. If they have a good experience, they are likely to continue
shopping on their favorite sites in the future. If they have a bad experience, they will
likely never return to the sites where they were unable to buy (if indeed they return to
shop online at all). The actual losses due to poor customer experience may thus be
significantly higher than our $14 billion estimate.

Problems with Checkout

In our tests, problems with the checkout process were the greatest cause of failed
purchase attempts. More than 40% of failures were due to difficulty with checkout.
Customers had found a product they wanted, added it to their cart and decided to
place an order. They were ready to buy and wanted to pay. Unfortunately, the
checkout process was too difficult and made it impossible for them to buy.

Customers want to buy, but many sites won’t let them pay.

As we discuss in Chapter 4, there were several different types of problems that arose
in the checkout process. The most important lesson is that poor checkout experience
will cost the industry billions in holiday sales. Causing over 40% of all failures,
checkout problems amount to over $6.5 billion.

Improving checkout alone could add over $6.5 billion to worldwide holiday sales.

The Impact of Improved Conversion

The best measure of a site’s effectiveness is the site’s “conversion rate.” This metric
refers to a site’s ability to “convert” visitors to buyers. A site’s overall (“blended”)
conversion rate is the percentage of visits that result in a purchase. For example, a site
with three purchases for every hundred visits will have a blended conversion rate of
3%. Other metrics like “hits” and unique visitors might, at best, measure a site’s
popularity — but the conversion rate measures the overall effectiveness of a site’s

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 13
 business. The conversion rate is the most important metric to observe, because it
measures a site’s ability to produce revenues and operate profitably.

According to Shop.org and the Boston Consulting Group, the average conversion
rate for e-commerce sites is 1.8%.7 This means that for every hundred visits, roughly
two result in a sale. 98% of visits do not end in a purchase. E-commerce leaders,
however, enjoy much higher conversion rates. For example, Amazon.com’s
conversion rate is reported to be above 9%, over five times the industry average.8

The best e-commerce sites constantly monitor and improve their conversion rate by
continually working to improve the customer experience. They recognize that even
slight increases in the conversion rate result in significant incremental revenue for the
business.

Small increases in the conversion rate lead to large increases in revenue.

Consider, for example, a hypothetical site with a 1.8% conversion rate and annual
revenues of $100 million. Assume that the site’s customer base is growing 25%
annually as more customers come online. Even small increases in the conversion rate
yield huge gains in revenue:

• With a constant conversion rate of 1.8%, revenues will be $100 million this year,
$125 million next year, and $156 million in two years.

• With a (modest) 10% increase in conversion (raising the conversion rate to 2%) ,
revenues will be $110 million this year, $137.5 million next year, and $172 million
in two years.

• With a 30% increase in conversion (raising the conversion rate to 2.3%), revenues
will be $130 million this year, $162.5 million next year, and $203 million in two
years.

• With a 50% increase in conversion (raising the conversion rate to 2.7%), revenues
will be $150 million this year, $187.5 million next year, and $234 million in two
years.

Raising the conversion rate to 2.7% is a modest goal, far below the conversion rates
of industry leaders. The impact, however, is tremendous: a difference of $50 million
this year, $62.5 million next year, and $78 million in two years. Over a three-year
7
Shop.org and BCG, The State of Online Retailing, April 2000.
8
PC Data Online, 2000
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 14
 period, the cumulative incremental revenue is $190.5 million.

Online retailers have a choice where to invest this holiday season. They can spend
more money on advertisements and marketing campaigns while continuing to convert
only 1.8% of shoppers. They can also devote months of development time and
millions of dollars to create advanced recommendation engines and gift-wrap features
that only a few of their customers will use. Or, they can invest in raising the
conversion rate by improving the customer experience.

Focusing on conversion has the greatest revenue impact for any e-commerce site.

The Three Most Important Improvements

The best investment a site can make to prepare for Holiday 2000 is to improve the
customer experience. Based on our tests, here are the three most important
improvements e-commerce sites should make before the holiday season arrives:

• Improve the checkout process to keep sales from slipping away.

• Develop a promotion and merchandising strategy to help holiday shoppers find

gifts. This will raise the order size and conversion rate.

• Commit to a long-term strategy for continually raising the conversion rate.

The following chapters will elaborate on these improvements. We hope that this
report will help the industry earn some of the $14 billion it now stands to lose due to
poor customer experience.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 15
 CHAPTER 3

Testing Methodology

Creative Good conducted more than 50 consumer tests to generate the data for this
report. The tests were conducted at an independent testing facility in a suburb outside
New York City. We conducted the first round of tests in August 2000 and the
BestBuy prototype tests in early September.

We tested eight major e-commerce sites in four categories important to holiday

shoppers:

• Apparel: Gap.com, LandsEnd.com

• Books and Music: Amazon.com, bn.com (Barnes & Noble)

• Electronics: BestBuy.com, Buy.com

• Toys: eToys.com, KBKids.com

The consumers recruited for testing were representative of customers who will shop
online this holiday season. The consumers we tested met the following criteria:

• All had used the Internet for at least 6 months.

• All had successfully made an online purchase in the past.

• All had either bought gifts online last holiday season or were seriously considering
buying gifts online for this year’s holidays.

• None were Web developers, Web designers or Internet professionals.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 16
 • None worked for any of the companies whose sites we tested.

To simulate the actual shopping experience, we conducted three-quarters of the tests

on dial-up connections and one-quarter of the tests on an ISDN line. We tested on
PCs running Microsoft Windows and the Internet Explorer Web browser.

About “Listening Labs”

The Holiday 2000 tests employed Creative Good’s “listening lab” methodology.
Though based primarily on traditional usability tests, listening labs are less task-
oriented and more open-ended than usability tests. In particular, listening labs are
one-on-one sessions in which the customer — not the moderator — sets the context.
The labs are non-directed in an attempt to recreate the environment at home or work,
where customers actually use the site — and where there are no pre-defined tasks or
scripted moderators sitting beside them.

Listening labs overcome some key shortcomings of usability and focus groups:

• Listening labs are non-task-oriented, since pre-defined tasks tend to neglect what
each individual customer wants to do on the site, and often miss larger strategic
findings. In our Holiday 2000 listening labs, we merely asked customers to use the
site to shop for a holiday gift for someone on their gift-buying list.

• Listening labs rely less on quantitative measurement than some usability tests. For
example, we did not use “eye-tracking” or any other detailed measurement during
the tests. We did, however, quantify the most important metrics: how often, and
where, customers succeeded and failed on the sites.

• Focus groups, in which a scripted moderator asks questions to a group of

customers sitting in the testing room, measure what customers say they like.
Listening labs observe what customers do, not what they say they do.

• Traditional usability and focus groups are controlled by a moderator, following a

script and specifying the user’s task. Listening labs are controlled by the customer,
following whatever path he or she wants through the site.

For more information on the listening lab methodology, refer to Creative Good’s
Dotcom Survival Guide (http://www.creativegood.com/survival).

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 17
 Timing of Screenshots
The screenshots featured in this report were taken in August and September 2000.
Because sites are constantly changing, some of the sites may have redesigned since
we took the screenshots. In fact, in most cases we hope that the site has redesigned
by the time you read the report, so that the site improves the experience for its
customers. The screenshots in the report depict what customers saw in our tests.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 18
 CHAPTER 4

Checkout

In our tests, checkout was the most common cause for failure. To be exact, in all the
tests in which a customer failed to complete an order online, checkout was
responsible for the failure over 40% of the time. This is a huge failure rate for one
feature — consistently poor performance across multiple sites in multiple categories.
As discussed in the Business Case (see Chapter 2), the worldwide e-commerce
industry could gain over $6.5 billion, including $3 billion in North America alone, if
every site made simple improvements only to the checkout process.

While checkout was by far the most common problem, customers in our tests did
encounter other problems on the sites:

• Some customers were unable to find what they were looking for (even though the
site carried it) because of poor categorization and search.

• Others failed to buy because they could not order the quantity of products they
wanted.

• Some were frustrated by slow-loading pages and decided to leave the site.

Still, problems with checkout were the greatest cause of failure in the buying attempts
we saw. Many times during the tests, we saw customers successfully find a product,
decide to buy it and successfully add it to their cart — all in vain. Checkout was too
difficult to complete. Frustrated, many customers gave up and left the site. To use an
off-line metaphor, it was as if these customers made it to the cash register but the
store would not let them pay.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 19
 Failures during checkout were due to three main causes:

• Confusion over “Return” and “New” customer paths

• Difficulty creating an account

• Ineffective error messages

In all these cases, sites failed to make it clear what customers needed to do and how
they could advance in the checkout process. Shoppers failed to buy because the
customer experience was too difficult. We show examples below and propose
solutions to the problems we saw.

“Return” vs. “New” Customer Paths

Several of the sites we tested offered separate checkout paths for new and return
customers. Sites offer separate paths to make checkout easier for return shoppers (so
that on the second visit, for example, return shoppers don’t have to type in all of their
information again).

The problem is that separate checkout paths tend to confuse customers. In our tests,
many new customers mistakenly selected the checkout path for return customers.

Below is the first checkout screen on BestBuy.com:

New customers
immediately began the
“Registered Customer”
checkout.

Most customers ignored all the text instructions and immediately entered their email
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 20
 address in the field under the “Registered Customers” header, possibly because
(scanning the page from left to right) it was the first field customers noticed. Some
were then confused as to what password BestBuy wanted, and several entered the
password they use to check email. The BestBuy site then displayed an error message
that only added to their confusion.

Gap.com also offers two options at the start of checkout:

First-time shoppers
mistakenly filled in
these fields.

New customers were drawn to the empty email and password fields and filled them
out, even though the text directly above reads “I am a registered user.” Customers
then got an error message because, of course, there was no Gap.com account with
that email and password. (Some readers might notice that the “new user” radio button
is selected; on Gap.com, the filled-out fields overrode that setting and the server
considered the customer a registered user.)

Online shoppers rarely read instructional text; instead, they look for an action to take
that will bring them closer to their goal. In this case, shoppers saw the empty fields
and acted on them. In general, we found that two checkout paths were problematic
when both paths were prominently displayed to the customer.

Shoppers are confused by sites that offer two equally prominent checkout paths.

Solutions from B&N and Amazon

Some sites, however, were able to offer separate checkout paths without confusing
shoppers. The websites of both Barnes & Noble and Amazon effectively served both
new and return customers by making only the new customer path prominent.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 21
 Every new shopper
successfully began
checkout.

On Barnes & Noble, no new shopper clicked on the “sign in” link for return
customers. Notice that the “Checkout Now” button, which is the beginning of the
new customer checkout path, is much more prominent. Barnes & Noble made the
right decision by making the new customer checkout path more prominent than that of
return customers. (This particularly makes sense for return customers, since by
definition they are more familiar with the site and more likely to see a less prominent
link.)

Amazon’s email field

applies to both new
and return customers.

In our tests, every shopper successfully checked out at Amazon. As with other sites
we tested, customers filled in their email address right away. But in this case, filling
in the empty field was the correct way to begin checkout. The Amazon form is
successful for several reasons:

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 22
 • The email address field is first, motivating customers to (correctly) begin filling out
the form.

• The radio buttons, to choose between new and returning customers, appear
between the email and password forms, distracting customers to read the radio
button text before immediately filling out the next field.

• The radio buttons’ text is well-worded. They mention “new customer” and
“returning customer,” much clearer than Gap’s “new or unregistered user” and
“registered user” fields.

• The “new customer” option states clearly that “You’ll create a password later,”
helping new customers avoid mistakenly typing their email password into the
password field.
• The password field is clearly described as being only for the “returning customer.”

• The checkout form changes slightly for a return customer (see below).

Amazon uses “cookies” to recognize its return customers9 and present them with a
slightly modified checkout page. Here’s how the first page of checkout looks to a
return customer:

The email field is

filled in for return
customers.

The page defaults to

the return customer
option if the cookie is
set.

For return customers, Amazon has already filled in the first field with the email
address, and the radio button defaults to the “returning customer” option. All the
return customer has to do is complete the only empty field — the password — and
click “Sign in” to continue.

While Amazon does many things right with the beginning of its checkout, the most
important takeaway is that the page is designed primarily to help new customers get
9
For more information on cookies, see http://www.cookiecentral.com/cm002.htm
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 23
 through the page correctly. Of secondary importance is that return customers are
served by the clever cookie-based change.

Design checkout for new customers. Use cookies to remember return customers and
present them with a different checkout path.

Creating an Account
Several sites confused customers by making parts of their sign-in or registration
process too prominent to new customers. For example, some sites required
customers to register before they could make a purchase. The registration process
prevented many customers from buying the products they had added to the shopping
cart.

In Gap.com’s case, the site confused customers by reminding them that they had not
yet signed in to their account. The Gap.com home page, shown below, contains this
text at the top of the page: “You are not signed in.”

Customers were
perplexed when
told they were
“not signed in.”

Several shoppers noticed this text because it loaded first. Because the rest of the page,
consisting mostly of graphics, loaded slowly over the dialup modem, “You are not
signed in” appeared by itself for several seconds while the other page elements
loaded. While no customers actually left the Gap.com site because of the text, it
certainly didn’t enhance the customer experience, as shown in these comments:

• One shopper on Gap.com said the message “means I can’t get access to the site.”

• Another commented that when the message “pops up, I think the page isn’t going to
show up.”

Another problem occurs on Buy.com’s homepage, which urges customers to “create

an account to get started”:

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 24
 Buy.com urges
customers to register
before they even
view products.

Buy.com seems more concerned with pushing registration than with showing
customers products. Why would a shopper sign up before knowing what the site
offers? As on Gap.com, it’s not a good idea to distract new customers with
prominent reminders about accounts and signing in.

A much worse problem on Buy.com occurs when a new customer finishes adding
products to the shopping cart and clicks “Check Out”: the site immediately requires
new customers to create an account before they can proceed. New customers, credit
card in hand, hardly want to “create an account” just for the privilege of giving the
site their money. They want to proceed in the checkout process.

To make matters even worse, Buy.com’s first Create My Account page includes
“Account Type,” a required field:

Shoppers cannot buy

without stating what
type of user they are.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 25
 Buy.com should remove the Account Type field, since there is no benefit to
customers by completing the field. And there is no benefit to Buy.com, either; if
customers get confused by the field or otherwise leave Account Type incomplete, the
Buy.com website prevents them from going any further in the checkout process. (At
this point on Buy.com, one exasperated shopper said, “Where do you just buy it?”)

Barnes & Noble also includes in its checkout process a required field that shouldn’t
be there. The checkout page below asks customers, “For future reference, please
name this address”. Unlike the Buy.com Account Type field, the “name this address”
field could be helpful to shoppers on a return visit to bn.com. But like the Account
Type field, there’s no reason for it to be a required field. Again, any customer who is
confused by the field and leaves it empty is prevented from going further in the
checkout process.

On BN.com, customers
need to “name” their
address.

In tests, the idea of naming an address was unclear to customers, despite

Barnes & Noble’s example (“e.g. Home”). One customer asked, “Please name this
address? What does that mean?”

BestBuy.com confused several customers by asking them to create a password hint.

This was not a required field, but it did confuse customers with its lengthy
instructions:

Select a Password Hint (optional)

Enter a personal question that will help
you remember your password (e.g.
What is my mother's maiden name?).
If you forget your password, we'll ask
you your personal question.
[form]
Enter the answer to your personal
question, which should not be your
password.
[form]

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 26
 Customers had difficulty
understanding BestBuy’s
“personal question.”

Here’s how the process is supposed to work:

• Customers select a personal question to be asked in case they forget their password,
and tell BestBuy the answer to the question.

• The answer to that question should be something other than their BestBuy
password.

• If a return customer clicks “Forgot your password? Get help here”, the site will ask
him his personal question. A correct answer will permit the customer to log on.

Some customers simply did not understand what BestBuy was asking of them. One
said, “I am thinking maybe they mean to put in a fake password.”

Solution: “passive registration”

The problems above show that new customers can get confused if certain parts of the
registration process is too prominent during checkout. Amazon.com avoids this
problem by “registering” new customers automatically when they make their first
purchase. Amazon quietly creates the account during the regular checkout, without
explicitly asking customers to create an account. We call this process “passive
registration.”

For new customers, Amazon’s first-time checkout process involves the following
steps:

• Enter an email address

• Enter a shipping address

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 27
 • Choose a shipping method

• Choose a password

• Enter a credit card number

• Confirm a billing address

• Review order and submit

The Amazon checkout process is successful for several reasons:

• Nowhere in the Amazon checkout process are new customers required to “create an
account,” choose an “account type,” or even asked to choose a “personal question.”

• The process is front-loaded with steps that make sense: entering an e-mail address
(as discussed above) and the shipping address and method.

• Only in the fourth step are new customers asked to choose a password. By that time
customers are halfway through with the checkout process and less likely to abandon
the site.

Overall, Amazon gives customers the benefit of an account — an easier checkout

process on the return visit — without subjecting customers to unnecessary obstacles
in creating the account. This focus on the customer experience makes Amazon’s
checkout a success.

Checkout Error Messages

Some customers will inevitably make mistakes entering information no matter how
easy checkout is. In these cases, it’s important to have clear, simple, and polite error
messages to help customers proceed.

In our tests, we observed how customers reacted to several different error messages
in checkout. The quality of the error message often meant the difference between a
successful purchase and a failed attempt.

Barnes & Noble customers see the following error message if they fail to complete
the “name this address” field:

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 28
 Shoppers did not
realize this message
applied to the bottom
field.

The error message, which reads “Please name this address for future reference,”
appears on the top of the page, far away from the field it refers to. The “name this
address” field is halfway down the page with no indication to the customer that it
needs to be filled in. Unsurprisingly, customers did not find this error message very
helpful.

Problems with email address entry

Many shoppers, especially AOL users, had difficulty entering their email address
when creating an account during checkout. Some entered a username (e.g. JohnDoe)
but no domain name. Others entered only part of the domain name (JohnDoe@aol)
and were unable to buy because they did not understand to enter their full email
address (JohnDoe@aol.com).

On the sites we tested, error messages did not always help customers with email
problems. For example, Buy.com shows the page below if a shopper’s email address
is invalid:

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 29
Buy.com asks for a
“valid email” but
provides no example
to help shoppers.

Buy.com’s error message reads, “Please provide a valid email address.” It tells
customers they’ve made a mistake but does not show them how to fix it. Customers
may try again, only to be baffled when their address is repeatedly rejected. Buy.com
should include an example of a complete, valid email address to help customers
through checkout. Throughout all our tests, over 10% of customers were never able
to correctly enter their e-mail address on the site they were on, despite repeated tries.

Note what happens if a new BestBuy customer chooses the return customer checkout
path:

New customers
think there’s a
problem with their
email , unaware
that they’ve taken
the wrong
checkout path.

BestBuy’s error message reads, “The email address and password you entered do not
match” — suggesting that the shopper entered the email or password incorrectly. The

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 30
 real problem, however, is that as a new customer, the customer has no password at
all on the system and should have taken the new customer checkout path instead.

Solution: clear, simple, polite error messages

eToys, by contrast, helps customers succeed with its clear and prominent checkout
error messages. For example, below eToys asks customers to “Please enter the credit
card’s month of expiration.”

All shoppers noticed

and understood eToys’
error messages.

eToys’ error message succeeds for three reasons:

• The error message is written politely: “Oops! Before we can go on, please see if
you can fix this...”

• It’s displayed clearly : The message is displayed in bright red text, beside a large
graphic, to indicate that an error occurred. (clearly displayed)

• It’s worded simply: The message clearly explains how to fix the error and continue
to make a purchase. (worded simply)

eToys could improve the message further by moving the credit card number field
closer to the error message, making it even easier for customers to correct the mistake
and proceed in checkout.

KBKids also displays its checkout error messages clearly, helping customers see
where they need to make a correction:
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 31
 KBKids uses bright red,
blinking triangles to
indicate which fields
need to be filled in.

KBKids places blinking red triangles next to the fields that customers need to fill in.
While this practice would be annoying on practically any other page on the site, it’s
helpful in checkout, since it’s then especially easy for customers to see where to fix
the problem and move on.

Amazon’s error messages go a step further to explain its checkout process:

Amazon tells
shoppers why it
needs their
phone number.

When customers leave a required field blank, Amazon explains says why the
information is necessary. Doing so helps shoppers through the checkout process, and
in this case — explaining the need for a phone number — reflects sensitivity to
customers’ privacy concerns.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 32
 What to Learn About Checkout

• In our tests, over 40% of failures in buying attempts were due to checkout.

• Fixing checkout alone would save the industry billions of dollars in lost sales and
raise conversion rates by 20%.

• Shoppers are confused by sites that offer two equally prominent checkout paths.

• Design checkout for new customers. Use cookies to remember return customers
and present them with a different checkout path.

• Do not require shoppers to register before they can buy.

• If a form is incomplete, prominently highlight for the customer whatever needs to

be changed, and word the error message simply and politely.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 33
 CHAPTER 5

Promotions and Merchandising

Promotions and merchandising are opportunities for retailers to boost revenues

during the holiday season. Customers in our tests were highly responsive to
promotions and featured products, especially so because they, like most holiday
shoppers, were shopping for a gift for someone else:

• In 30% of tests, customers clicked on at least one featured product.

• In 19% of tests, customers clicked on a promotion or special offer.

• Some promotions were so effective that 50% of customers who saw the promotion
clicked on it.

Promotions and featured products appeal to holiday shoppers.

Effective promotions and merchandising can generate substantial revenue gains. For
many retailers, effective promotions and merchandising may mean the difference
between profitability and continued losses. According to Shop.org and BCG:

• The average conversion rate for apparel sites is 1.8%.

• The average order size for apparel sites is $81.10

On an average apparel site, increasing the average order size by 10% would add $8
per purchase. Increasing the conversion rate by 10% (a modest increase) would
increase conversion to nearly 2%. For a site with $100 million in sales, the impact of
10
Shop.org and BCG, The State of Online Retailing, April 2000
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 34
 these two changes would amount to $21 million in incremental revenue.

$100 million
x 1.1 (due to increased order size)
x 1.1 (due to increased conversion)
= $121 million.

If the overall e-commerce industry raised average order size by a conservative 10%
and raised conversion rates by a modest 10%, the increase in holiday revenue would
total over $4 billion.

$19.5 billion
x 1.1 (due to increased order size)
x 1.1 (due to increased conversion)
= $23.6 billion

Effective promotions and merchandising can significantly raise holiday revenues for
many e-commerce sites.

In this section, we present promotion and merchandising tactics that our tests showed
to be most effective. We also describe several lost opportunities — how sites may be
losing sales by hiding good offers or by making them hard to use.

Effective Promotions
We discovered that promotions, like other areas of e-commerce sites, are more
effective when they are presented clearly and are easy for customers to use. (For the
purposes of this report, we define “promotions” as monetary offers or discounts that
are not tied to specific products — for example, offering 10% discount off a
customer’s first purchase.)

The most effective promotion we saw was on the homepage of Gap.com, and it
enjoyed a 50% clickthrough rate in tests:

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 35
 Half of all test
customers clicked
on this homepage
promotion.

Multiple customers
clicked immediately
on GapMaternity.

Gap.com’s prominent and straightforward promotion, “$100 gets you $20,” appealed
strongly to shoppers. Consumers immediately understood the premise of the
promotion — a $100 purchase would “get a $20 Gap ShopCard good towards your
next purchase.” One shopper said, “I would do this. A hundred dollars gets you
twenty. I would definitely check that out.” Another noted that it would be “easy to
spend a hundred dollars” shopping at the Gap, and thus the promotion was attractive
to her. Half of the customers who saw the promotion clicked to see the details.

Another element on Gap.com’s homepage showed good merchandising. Multiple

shoppers immediately clicked on the large GapMaternity graphic. By prominently
displaying this section of the store, Gap.com communicated its breadth of product
offering to customers who might not have known otherwise. One shopper remarked,
“I would never even have thought that they had maternity clothes. This is great.”

Although the “$100 gets you $20” promotion drew a high clickthrough rate, more
than one customer had trouble taking advantage of the promotion. The page took so
long to load that one customer left the site entirely. Frustrated, she said “If it’s going
to take so long, what’s the point? It’s just a special offer.” Another customer
commented that it might be easier to drive to a nearby Gap store than to go through
the online promotion.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 36
 One customer left
Gap.com because
the promotion
page loaded too
slowly.

From studying Gap.com, we offer two lessons about promotions:

• Display promotions prominently and explain them clearly.

• When customers click into the promotion, make it quick and easy to take part in the
promotion.

Admittedly, these are basic, common-sense guidelines — but as evidenced in our

tests, many top e-commerce sites aren’t following them. Just following these simple
rules could improve revenues substantially.

Good promotions, poorly presented

LandsEnd.com offered a good promotion but missed the opportunity to present it to

customers:

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 37
 LandsEnd doesn’t
mention free hemming
until customers begin
the buying process.

During the checkout process above, LandsEnd asked customers if they wanted the
wool pants to be hemmed — at no extra charge. Customers were happy to learn of
the offer of free hemming, but multiple customers commented that they would have
liked to know about the offer earlier in the shopping process. LandsEnd does not
promote its free hemming anywhere else, such as on the product page, and it may be
losing potential sales by not making the free hemming offer more prominent.

Similarly, Buy.com offers free shipping on office supply orders over $50 — but
doesn’t explicitly tell customers about the offer:

Buy.com customers
were surprised
to see shipping
was free.

Just as with LandsEnd, customers on Buy.com said that they would have liked the
site to mention the free shipping earlier in the buying process. Buy.com makes no
mention of the offer on its office supply product pages, and it doesn’t remind
customers of the offer when their order is close to $50. The only other way that

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 38
 customers can find out about the offer is in the shipping section of Buy.com’s
Frequently Asked Questions — not the best place for promotional announcements.

Strangely enough, even when customers are in checkout and their order exceeds $50,
they still are not told of the offer. They can only assume that shipping is free because
the shipping dropdown defaults to “Standard Delivery $0.00” (but it doesn’t mention
the $50 minimum). One shopper on Buy.com said, “If there’s no shipping charge,
hey great... tell us that up front on the site.”

Communicate special offers clearly and early in the buying process.

Merchandising
By “merchandised” products, we mean featured products and other items presented in
a special manner distinct from the site’s standard product listings.

Many customers appreciated seeing featured products on sites’ home or category

pages. In 30% of tests, customers clicked on at least one featured product, showing
the importance of merchandising.

LandsEnd, for example, featured two products on its Women’s category page,
causing a third of customers to click to get more information on one of the products:

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 39
 LandsEnd’s category
page prominently
displays two featured
products.

Once again these highlighted products caught customers’ attention because they were
presented prominently and clearly. The interested customers considered buying these
items even if they came to the site for something else.

BestBuy, by contrast, buries its featured products on a homepage cluttered with

graphics and links to pages without products:

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 40
 The center of
BestBuy’s
homepage is
devoted to a
sweepstakes.

None of the
links in this
column lead to
products.

Featured products
only appear near
the bottom of the
page.

BestBuy devotes most of its above-fold homepage space to a sweepstakes, which

doesn’t lead to any products. All of the featured products appear below the fold at
800 x 600 browser resolution. While there’s nothing wrong with promoting
sweepstakes, our tests suggest that holiday shoppers are more interested in getting to
products that they can buy.

Place featured items above the fold on the homepage.

Featured products were particularly important on the toy sites we tested (eToys and
KBKids). Adult customers, shopping for children, were especially eager to get gift
suggestions. This was particularly true when customers shopped for other people’s
children and had little idea of the children’s preferences.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 41
 Notice how KBKids effectively uses its homepage to display featured products:

KBKids’ homepage
features products from
various categories.

The “Surprise of
the Day” is a
discounted
product.

KBKids shows products from several categories alongside links to the entire
category. The site also displays a “Surprise of the Day” in the left-hand column
(though near the bottom of the page), a featured product at a discounted price.

Both KBKids and eToys let customers shop by age as well as by category. In our
tests, almost all customers shopped both by age and by category within a single
session. Providing multiple ways to browse was helpful to shoppers eager to find
ideas for gifts to buy.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 42
 eToys customers who shop by age for a five-year-old see this well-rounded page:

The center of the

page provides
category
recommendations.

The right-hand
column features
characters and
brands.

The page offers customers over a dozen product or brand suggestions. The
suggestions range from brands (Lego and Barbie) to categories (“books they can
handle”) to situations (“Plan a Stress-free Playdate”). Because the product
suggestions were prominently and clearly displayed, several customers in tests
clicked into various products from this page. eToys’ shop-by-age page is an example
of thoughtful and varied merchandising.

Use brands, categories and situational merchandising to suggest products to

customers.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 43
 What to Learn About Promotions and Merchandising

• Promotions and featured products appeal to holiday shoppers.

• Effective promotions and merchandising can significantly raise holiday revenues for
many e-commerce sites.

• Communicate special offers clearly and early in the buying process.

• Place featured items above the fold on the homepage.

• Use brands, categories and situational merchandising to suggest products to

customers.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 44
 CHAPTER 6

Case Study: BestBuy

The BestBuy case study shows the benefits of improving the customer experience.
After testing the BestBuy site with customers, we applied the Creative Good
consulting methodology to prototype our own “new” customer experience for
BestBuy.com. When we tested the prototype, the results were dramatic: shoppers
were four times more successful on the prototype than they were on the original site.

Why a Case Study?

In our tests of eight top e-commerce sites we observed that customers failed to
complete an order over 40% of the time. Ironically, this large failure rate was caused
by a handful of problems in the customer experience of the sites we tested. We
hypothesized that fixing that small set of problems could yield significant gains for
any e-commerce site.

To test our hypothesis, we chose one of the eight sites we had tested originally —
BestBuy.com — and created a “new” version of the site. “New” is in quotes because
(a) we created and tested new versions of only a few pages from the site, and (b)
BestBuy wasn’t aware of the case study process and therefore wasn’t a participant.11

We significantly improved the BestBuy customer experience with a relatively small

11
Copyrights, trademarks and all proprietary rights included on or comprising the BestBuy.com website are
property of BestBuy.com™ Inc., a wholly owned subsidiary of Best Buy Co. Inc. Use of these materials by
Creative Good, in connection with this Case Study, is for illustrative and analysis purposes only. The
trademark “BestBuy.com™” is property of BestBuy.com Inc.
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 45
 investment in time and effort. Instead of prototyping the entire site, we just improved
those areas of the site that confused customers in the first set of tests. Many
e-commerce sites also could improve significantly with a small investment in the
customer experience.

This case study will show the following:

• Where customers had difficulty on BestBuy.com during the initial tests (on the live
BestBuy.com website)

• How we designed our prototype to address those problems

• How customers reacted to our prototype during the tests of the prototype

Summary of Results
The new BestBuy prototype shown in this chapter represents a sizable improvement
in the BestBuy customer experience, as shown in before-and-after tests:

• In our first set of tests (that is, the tests on the live BestBuy.com website), over
70% of customers failed to complete a purchase.

• In the second set of tests (the tests of the prototype), over 80% of customers
succeeded in completing a purchase. The BestBuy conversion rate quadrupled
between the first and second set of tests.

• Also, three times as many customers clicked on featured products in the prototype
as they did on the live site.

Customers in both tests met the same criteria, got the same instruction (to shop for a
holiday gift), and of course, no customer appeared in both sets of tests.

Developing the Prototype

To develop the BestBuy prototype, Creative Good first created “wireframes,” block
diagrams with the location and description of each page element. Below is an example
of a wireframe that we created; notice the descriptions of Customer Goals,
Prominence, and Notes on the right side of the page.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 46
 The wireframe
shows the position
of page elements
but not their graphic
design treatment.

The wireframe
explicitly lists the
customer goals
on the page and
the relative
prominence of
page elements.

It took three days for Creative Good to create wireframes for eleven new BestBuy
pages. Once they were done, interactive agency Red98 (a Boston-based interactive
agency committed to creating “great user experiences”) turned the wireframes into the
working HTML prototype that Creative Good used in the second set of tests. It took
one week for Red98 to create the HTML prototype. Thus, in less than two weeks, we
transformed the customer experience on BestBuy — at least in prototype form. An
actual e-commerce site would take longer to implement and integrate such changes,
but even doubling the time period is still a tiny investment of time to get higher
revenues in return:

With a few simple, fast changes to the customer experience, many e-commerce sites
can substantially increase their revenues.

We designed the new BestBuy prototype with new customers in mind, rather than on
return customers. More specifically, the prototype design was guided by four pieces
of data:

• Customer objectives, as observed in our initial tests

• BestBuy’s business objectives, based on some research on the company (see

below) and some basic assumptions about the objectives of any e-commerce site

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 47
 • Best practices of online retailing, based on past Creative Good research and client
work

• An informal competitive analysis of five other e-commerce sites that sell electronics

BestBuy’s business objectives

As stated above, we conducted some informal research on BestBuy to better inform

the decisions we made in creating the prototype. From reading BestBuy’s public
statements, we conclude the following about its Web strategy:

• BestBuy is investing heavily in its website.

• BestBuy wants customers to buy from its website.

• BestBuy values customer service and explicitly tries to focus on customers.

Best Buy, the leading off-line electronics retailer, has 360 stores spread across 40
states.12 The company’s projected revenues for the year 2000 are over $1.2 billion, an
increase of nearly 25% over 1999.13 Best Buy is successful and growing rapidly.

In May 2000, BestBuy.com (a wholly-owned subsidiary company of Best Buy Co.,

Inc.) announced it was spending $50 million on an online and off-line marketing
initiative.14 BestBuy.com is investing heavily to drive traffic to the site. As part of a
multi-channel strategy, the site aims to “give customers access to all three of Best
Buy’s shopping channels: in-store, online and catalog.”15

Shoppers can buy products from the BestBuy.com website and pick them up at a
BestBuy store. Consumers can also browse the off-line store and place their order
through the website.

BestBuy.com’s senior management has cited customer focus as a top concern.

Company president John Walden said “Best Buy seeks to pioneer the clicks-and-
mortar revolution by creating a better shopping experience for our customers at every
point we interact with them.”16 BestBuy recognizes that customers drive any
company’s success.
12
Hoovers.com
13
Twice magazine: Twice 2000 Retail Registry
14
BestBuy.com: press release, May 1, 2000.
15
BestBuy.com: press release, June 12, 2000.
16
BestBuy.com: press release, June 12, 2000.
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 48
 We drew several conclusions from this information:

• We concluded that BestBuy primarily wants to sell products on the site (hence the
prototype focused on making it easier to buy products).

• We confirmed that store pick-up was important (hence the prototype featured store
pick-up on a homepage banner).

• We knew that since BestBuy has a strategic commitment to the performance of its
website, and tries to focus on the customer, it would reasonably want to make such
customer experience improvements as we made in the prototype.

Comparing BestBuy and the Prototype

Below we describe the problems customers had on BestBuy.com in the first set of
tests, and how our prototype attempted to fix those problems. To help keep the two
sites straight, “BestBuy” or “BestBuy.com” refers to the live site; “the prototype”
refers to the prototype that we created and used in the second set of tests.

BestBuy.com: Homepage

For some shoppers, problems with BestBuy.com began on the homepage:

Shoppers must wait

as the background
image — a man’s
face — slowly loads.

One customer wanted

a DVD movie but did not
see the Movies link.

At 800 x 600 browser

resolution, there are no
featured products
visible above the fold.

Instead of a plain background, BestBuy.com uses the faded image of a man’s face,
thereby lengthening the load time of the homepage without helping customers shop.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 49
 When the homepage does finish loading, there is nothing visible above the fold that
customers can buy. The most prominent element of this page is the home
entertainment sweepstakes.

One shopper came to the site looking for a movie on DVD. She was unable to find the
Movie category link, since it was located below the main list of departments and
separated from them by a white line.

Prototype: Homepage

In our prototype homepage, we tried to make it easier for customers to begin

shopping right away:

On the prototype, all

categories were
presented together.

Almost every
customer clicked on a
featured product.

The prototype highlighted the product categories and listed them in blue underlined
text links. In tests, customers more easily saw the range of products the site offered.
We also renamed and created categories so shoppers could find items more easily.
For example, we added links to “DVD Players” and “Camcorders” categories, so that
customers could find popular products right away. (The BestBuy.com site currently
groups these products under the “Home Audio and Video” category.) The prototype
homepage also displayed four featured products.

One note on the prototype’s revised product categories: the categorization scheme
isn’t perfect. It was merely based on what we heard from customers in our tests and

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 50
 on our informal review of competitor websites. In an actual client project, Creative
Good would also use site data and the client’s expertise to create a more accurate set
of categories.

BestBuy: Results Page

Aware that BestBuy sells DVDs in its off-line stores, one customer searched by
“DVD” from the BestBuy homepage and received the following results page:

Finding no DVD movies

above the fold, one
shopper thought
BestBuy.com did not
sell any DVDs.

BestBuy.com presented its search results by category. In this case, results from the
Movie category appeared below the fold. The customer assumed BestBuy.com did
not sell DVD movies and thus gave up trying to buy one.

BestBuy.com’s search was, however, helpful for some customers. Note what the site
returns if a shopper searches by “cell phones.” Shown below, BestBuy.com
effectively maps a “cell phones” search to a page explaining that wireless products are
not yet sold on the site. Further, the page includes a prominent link to the store
locator. Shoppers can thus easily find a BestBuy store that sells what they are looking
for.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 51
 Shoppers who
search by “cell
phones” see that
wireless products
are not yet offered.

How do home
office products
relate to a search
by “cell phones”?

BestBuy: Category Page

On the BestBuy category page (see below), several customers were disappointed by
what they perceived to be a limited product selection. BestBuy.com does offer a fairly
broad selection, but the site’s categorization gave shoppers an impression of a narrow
selection. One problem was that the site’s category pages featured very few products
and made subcategories hard to find:

Links to product
listings are hidden
in the left- hand
navigation bar.

Only two images

of camcorders
are visible on the
camcorder page.

Customers had difficulty, for example, realizing that BestBuy carried more than two
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 52
 camcorders, since only two camcorders are displayed on the Camcorders page
(shown above). Other camcorders were available through the subcategory links in the
left-hand navigation, but customers didn’t know to go to the navigation elements to
find more products.

Customers familiar with the broad selection in BestBuy’s off-line stores were
disappointed. One customer commented: “Not a big selection. It would be nice if they
had more selection on here... When you go to the store, there is more selection.”
What the customer didn’t realize is that BestBuy.com offered far more products than
she could see.

Prototype: Category Page

Our prototype made featured products and subcategory links much more prominent:

Subcategories were
highlighted at the top
of a contextual
navigation bar.

Subcategory links were

also displayed in the
center of the page.

At 800 x 600 browser

resolution, four
products are visible
above the fold.

If customers wanted more camcorders than the four displayed on the page, the links
to subcategories (containing more camcorders) were located prominently in the top
center of the page.

BestBuy: Product List

BestBuy.com’s product lists — the standard way that BestBuy displays its products
within their subcategories — may frustrate customers by failing to show items’
prices:
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 53
 For some items,
customers must
click on See our
Price.. .

Inexplicably, BestBuy’s product list doesn’t display prices for each product. In the
screenshot above, the top three products show crossed-out prices, presumably
because they are discounted. Customers who want to know the price of those items
might reasonably click on the “See Our Price” link — but even that doesn’t display
the price. Instead, customers get a paragraph of text saying that customers must first
click the Buy button in order to see the price.

... and find they

must add to cart
to really see the
price.

Clearly, this is a poor customer experience. How many customers will begin buying
a product without knowing its price?

It’s particularly surprising that BestBuy would make some prices hard to find,
considering how expensive some of the products are. One customer commented that
she would compare prices between different retailers: “A big purchase like that I
would not impulse buy... I would have printed [the page] out to compare.”

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 54
 Prototype: Product List

The prototype displayed a price whenever a product was displayed. Here is the
prototype’s Digital Camcorders page:

The price of
every item is
clearly visible.

Customers on the prototype could clearly see the price of each item. This also made it
easier to compare the prices of listed items.

BestBuy: Begin Checkout

On BestBuy.com, several new customers mistakenly chose the checkout path for
return customers (a common problem in many sites’ checkout, as described in
Chapter 4):

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 55
 New customers
were drawn to the
empty email and
password fields.

Customers naturally gravitated toward the empty email and password fields on the
left-hand side of the page, without reading the “Registered Customers” header above
them. After clicking “Continue Secure Checkout,” those customers saw an error
message, since they were new customers and didn’t have a BestBuy password.

Prototype: Begin Checkout

The protoype’s checkout process focused on the new customer checkout path, as
suggested in Chapter 4:

The prototype leads

new customers down
the appropriate path.

Return customers
can click into a
shorter checkout.

To the right of the default checkout path is a small text link allowing return customers
to enter into a shorter checkout. Since return customers are more familiar with the
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 56
 site, it makes sense to provide them with a less prominent entry to checkout. Every
customer who began checkout on the prototype successfully completed the
transaction.

An improvement to the prototype, had we had the means to implement it, would have
been to use cookies to identify return customers. Then, when they began checkout,
the site would know to place them automatically in the shorter checkout path for
return customers.

A Prototype Weakness

Even though our prototype performed much better than the live BestBuy site, the
customer tests of the prototype revealed an improvement we could have made. In one
test, the prototype’s shortcoming prevented a customer from being able to buy from
the product page:

One shopper missed

the Buy button and
instead clicked
Shopping Cart to buy.

One customer tried to add a camcorder to his cart by clicking Shopping Cart instead
of Buy. While we intended to make the Buy button prominent, the customer’s
difficulty showed us that we had not made the Buy button prominent enough.

In an actual client project, it’s common — even desirable — to find problems in the
prototype during listening labs. By continually improving the prototype based on
customer feedback, the site in development becomes that much more effective.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 57
 Rough Calculation of Results
When we tested the current BestBuy site, only 22% of customers who wanted to buy
were able to do so. When we tested our prototype of the BestBuy site, 88% of
buying attempts succeeded. If our case study is anywhere near accurate, these
findings suggest that BestBuy.com’s sales could be several times what they are today
if simple improvements were made to the customer experience on the site.

We made a rough calculation of the revenue gains that such improvements would
bring to BestBuy:

Although we don’t know BestBuy.com’s current conversion rate, the average

conversion rate for electronics sites is 0.8%17 . Assuming BestBuy’s conversion rate
to be the average, quadrupling it (a 300% increase) would raise conversion to 3.2%.
But to keep our revenue estimate conservative, we assume an increase of half that size
— 150% — raising conversion to 2.0%. This also seems like a modest enough goal,
since 2% is still less than the average conversion rate on toy sites (2.7%), on books
and music sites (2.1%), and just above the overall e-commerce average
of 1.8%.

Since we don’t know BestBuy.com’s annual sales, we assume a modest estimate of

$100 million. This figure is less than 10% of its off-line sales ($1.2 billion) and only
twice the amount that BestBuy.com is spending to market the site ($50 million).

If BestBuy.com’s annual sales are $100 million, raising conversion 150% would
increase sales to $250 million.

$100M x 2.5 (due to increase in conversion rate) = $250 million

Given our assumptions, simple improvements to BestBuy’s customer experience

could bring in $150 million more in annual revenue than the site makes today. This is
an enormous revenue gain, translating to several million extra dollars per month. Any
investment in improving the customer experience would pay for itself in a matter of
days, with hundreds of thousands of dollars in daily incremental revenue. It’s hard to
see why a site would invest in anything other than the customer experience when
improving the experience can have such profound impact on the business.

17
Shop.org and BCG, The State of Online Retailing, April 2000.
_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 58
 Again, given our assumptions, two other results are interesting to note:

• With $250 million in online sales, BestBuy’s website would generate revenue equal
to about 20% of its off-line revenues. The change would bring BestBuy closer to its
goal of “pioneering the clicks-and-mortar revolution.”

• Increased revenues would also dramatically raise the effectiveness of BestBuy’s

marketing campaign. If BestBuy.com now grosses $2 for every dollar it spent
marketing the site, a better experience could bring in $5 per dollar spent on
marketing. Improved marketing efficiency would justify further investment in
BestBuy’s customer experience.

The lesson of our BestBuy case study is clear: it pays to invest in the customer
experience.

What to Learn from BestBuy

While the BestBuy case study provides valuable tactical lessons, the most important
lessons are strategic:

• Even well-intentioned companies, committed to focusing to the customer, can

greatly improve their online customer experience.

• With a few simple, fast changes to the customer experience, many e-commerce sites
can substantially increase their revenues.

• When testing a website in front of customers, do it in a non-directive way. Instead

of directing customers, watch and listen to how they experience the site.

• When creating prototypes, keep in mind the business objectives while applying the
lessons from customer tests.

• Test the prototype in front of customers to assess the changes and identify new
problems.

• It pays to invest in the customer experience.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 59
 Join our free newsletter

If you found this report useful, you’ll like our Good Experience Update, a free e-mail
newsletter that goes out every week to more than 30,000 subscribers worldwide.
Written by Creative Good founder Mark Hurst, Good Experience contains news and
commentary about customer experience in e-commerce, wireless, and other Internet
services.

To sign up, just send an e-mail to update@goodexperience.com.

_____________________________________________________________________________________________
Copyright 2000, Creative Good, Inc. More free resources: goodexperience.com 60