BA 4: WEB AND SOCIAL MEDIA ANALYTICS

MODULE III

Leveraging Qualitative Data:

 Lab usability studies,

 Usability alternatives,

 Surveys & Web Enabled User Research Options,

 Power of Testing & Experimentation.

LAB USABILITY STUDIES:
• Usability studies, remote testing, and surveys are powerful techniques that
help you unlock the mystery of WHY.
• User research is the science of observing and monitoring how we interact
with everyday things, such as websites, software, or hardware, and then
drawing conclusions about how to improve those things.
• Lab usability tests measure a user’s ability to complete tasks.
• In a typical usability test, a user attempts to complete a task or set of tasks
using a website.
• Each of these tasks has a specified goal for effectiveness, efficiency, and
satisfaction in a specified context of use.
• A typical study will have 8 to 12 participants. During the test patterns will
begin to emerge that highlight which parts of the customer experience or
process work well and which cause problems.
• Lab tests are conducted by a user-centric design expert, who is typically
supported by someone taking notes.
• Key stakeholders such as business owners, engineers, developers,
analysts, and product managers participate as observers.
• Tests can be conducted with a live version of the website, beta versions,
onscreen HTML or PowerPoint prototypes, or even with paper printouts.
• These paper prototypes, sometimes called wireframes, approximate what
a user might otherwise see on a computer screen but save the
development team from having to produce an on-screen product.
• In a lab environment, usability tests are typically held in a specially
designed room called a usability lab.
• The lab is split into two rooms that are divided by a one-way mirrored
window that allows observers to watch the test without being seen by the
test subject.
• As the test subjects work on their tasks, a test moderator observes, takes
notes on user actions, and records outcomes.
• While the participant is working at their task, the moderator limits their
own interactions to providing initial task instructions and occasionally
prompting the participant to further explain their comments.
• Often lab usability tests are also recorded on video for later review and to
present to a larger audience in a company.
• Usability tests are best for optimizing user interface (UI ) designs,
optimizing workflows, understanding the Voice of Customer, and
understanding what customers really do.
PROCESS FOR CONDUCTING LAB
USABILITY STUDIES:
Conducting a successful lab usability test involves four stages: preparing
the test, conducting the test, analyzing the results, and following up.

Part 1 - Preparing: In this stage you define scope, identify participants, and
ensure all things are lined up right.
The main steps in the preparation phase:
1. Identify the critical tasks for which you are testing. Example: how easily
can customers return a product or request a replacement?
2. Create scenarios for the test participant of each task. Example: A lens is
missing in a camera ordered by the customer from a website.
3. Identify successful scenario. Example: Customer is able to submit a
complaint.
4. Identify who your test participants should be. new users, existing users,
people who shop at competitors’ sites, etc.
5. Determine a compensation structure for the participants.
6. Contact a recruiter to recruit the right people for you.
7. Before conducting the test with live participants, do dry runs with
internal staff to make sure your scripts and other elements work.

Part 2 - Conducting the Test. The main steps in the test phase are as
follows:
8. Welcome your participants, and orient them to the environment.
9. Understand what the participants are thinking about the process.
10. Ask participants to read out the tasks to be performed. This will make
sure they read all instructions and hence understand the task or scenario.
11. All company observers pay attention to what participants are doing.
Look at verbal and non verbal signals. Note down where the participants
are doing right things and where they are failing.
5. The moderator can ask the participants follow-up questions to get more
clarity.
6. Thank the participants, and pay them right away (if you can).

Part 3 – Analyzing the Data.

7. Share the thoughts & observations of all the observers.
8. Note down trends and patterns.
9. The moderator notes down success and failures by each participant for
each task.
10. Analyze in-depth to identify the root causes for the failures based on
actual observations.
11. Make recommendations to fix the problems that were identified.
Categorize the recommendations into Urgent, Important, and Nice to
Have, to help business decision makers prioritize.
Following Up, Retesting, and Measuring Success:

• The role of testing experts doesn’t end after presenting the results.
• They should help fix the problems identified by the test, offering their
services and partner with website developers and designers to improve
the site experience.
• Finally, don’t forget to measure success post-implementation. what was
the outcome of new website? Did the new website make more money? Are
customers satisfied? Do you have lower Abandonment Rates?
Benefits of Lab Usability Studies:

• Lab testing allows you meet the customers, observe them and interact with
them.
• Lab tests can be a great way to get customer feedback early in the process
to identify big problems.
• These tests really help identify what works and what does not.
• Testing is especially helpful if you are unable to make sense of your
Clickstream data.
• Finally, usability tests are a great mechanism to generate ideas to solve
customer problems.
USABILITY ALTERNATIVES:
There can be two alternatives to Lab Usability Studies:

Alternative 1 - Live Recruiting and Remote User Research.

• Problem with Lab Usability study is to get proper participants and get
them to do the work like actual customers.
• To solve this problem You can recruit actual people who visit your site to
complete a task, and you can conduct the study remotely.

Live recruiting and remote studies work as follows:

1. Create a few questions for visitors to your website. Based on the answers
you can identify the eligible participants for your study.
2. Create a small window with some questions which wont take more than
15-20 mins. Offer a reward for answering questions. Some visitors may
take the survey and others may close the window.
3. Details of visitors who fill up the
survey and submit it will show in
your live database.
4. Visitors who meet your criteria, call
them back right away or set up a
convenient time to contact them.
5. These visitors can become part of
your study through screen sharing
applications.
6. You use nearly the same process as
described in lab usability studies:
observe tasks, record video, do
analysis.
Alternative 2 - Outsourced Online Usability.
• Outsourced online usability is great for those companies who don’t have
numerous resources. They don’t have researchers but they just want to
know what people think of their web experience.
• Many companies allow you to completely outsource usability testing.
• UserTesting.com is one such company. It has a large panel of people who
have expressed an interest in participating in usability tests.
• You can commission usability studies through them for an affordable
price.
• Each participant goes through your test, and you get a video with the
participant’s experience as well as a short written summary of the
participant’s thoughts.
• The process is simple and cheap.
Disadvantages of Outsourced Online Usability:

• You don’t have a lot of flexibility in getting the precise kind of test
participants.
• As with lab usability studies, there is an artificiality about the situation,
specifically because panel members may just want the $10 per study that
UserTesting.com pays.
• You sacrifice the ability to test unstructured experiences, such as telling a
user, “Just do what you would normally do on our site.”
SURVEYS:
surveys are an optimal tool for gaining more data about your visitors.

• Surveys are usually very affordable, and good free options are available.
• You can use surveys as a continuous listening methodology.
• They provide a mix of data that is suited for quantitative and qualitative
analysis.
• Surveys are timely; we can detect problems and opportunities right away.
• Sample sizes can be small, so you don’t need every visitor on your site to
talk.
• Surveys can now use cookies, integrated with web analytics Clickstream
data, and can be customized for specific jobs.
Page-Level Surveys:

• Page-level surveys typically

use a passive invitation
model and collect micro-
level data for a focused
purpose or task.
• If the user is interested in
giving feedback, then he or
she will initiate the survey
by clicking the invite.
Benefits of Page-Level Surveys:
• Page-level surveys are always available on the web page.
• They focus on one task, such as getting user ratings on features, reporting
bugs or questions about the site, or learning whether the page was helpful.
• Many free or affordable survey options are available. Also, it is fairly easy
to create your own page-level survey.
• You hear from your most upset or engaged customers.
Challenges of Page-Level Surveys:
• The invitation model for page-level surveys is passive; hence, only the
people who really want to talk to you will talk to you.
• You get feedback only about the specific page where the user clicks the
invite or feedback about a specific task.
• These surveys are not very good for collecting feedback about intent or
complete site experience.
Site-Level Surveys:
• They collect macrolevel data about
customer intent, behavior, and customer
experience.
• In first case, as soon as the visitor enters
your site, or at some point during their
visit, a window pops open with the
survey.
• The second method is to use a
permission-based model for the survey
invitation. In this case, when the visitor
enters your website, they are politely
asked for permission to participate in a
survey.
Benefits of Site-Level Surveys:
• Good at capturing the macro experience, intent, and Outcome-level
information about the visit.
• These surveys provide a great deal of control over who sees the survey.
You can set it up so only 5 percent of your site traffic gets the invitation.
You can leverage cookies so the same person will see the survey only once
every three months.
• you can use technologies such as Ajax to make the experience much easier
and use conditional logic to focus your questions (for example, if the user
already said she was a student, so you don’t ask her the size of her
business).
• The invitation model is proactive, which means you’ll get a significantly
more diverse sample of participants. You’ll get people who are on your
site for multiple purposes, and you’ll get people who are happy or sad or
just OK. The feedback is much more representative.
Challenges of Site-Level Surveys:

• Typically, the sophisticated site-level surveys are not free.

• You will need someone at your company—or a consultant from your
survey provider—to help you sift through a good amount of rich data.
• Although you can easily identify big problems, such as the shopping cart
is broken, it is harder to identify micro problems using site-level surveys
(for example, “What did you think of page xyz?”).
• Site-level surveys are fantastic for measuring the effectiveness of your
website, the macro experience of your customers, and the projected impact
on your company, brand, and offline existence.
Three Greatest Survey Questions Ever:

• Q1: What is the purpose of your visit to our website today?

• Q2: Were you able to complete your task?
• Q3: If you were not able to complete your task today, why not?

These questions will tell you the purpose of site visit and which sections or
pages of your site are not working properly. This is Voice of Customer
(VOC).
Clickstream data can not provide you these valuable inputs of VOC.
WEB-ENABLED USER RESEARCH
OPTIONS:
In the field of web user research many innovative options are available on
the internet. Some of these options are as follows:
Competitive Benchmarking Studies:
• Competitive benchmarking studies compare your website's performance
with your competitors'
• In the past, it was difficult to conduct such studies due to budget
constraints and difficulty in finding participants
• Web-based companies like UserZoom now provide an easy and scalable
way to conduct these studies
• These companies collect data from users who execute tasks and provide an
analysis
• Web-based studies are powerful and cost-effective compared to traditional
studies
• Data from these studies can be used to improve customer experience and
convince senior management to take action
Rapid Usability Tests:

• Rapid usability tests can be conducted through websites like

www.fivesecondtest.com
• The user uploads an image of a webpage and invites participants to view
it for 5 seconds
• Participants are asked to recall the things they remember from the
image/webpage
• The test is fast, free, and easy to execute
• Open text summaries of what people recall from the image are provided
as deliverables
• The test is useful for testing the clarity of big calls to action, identifying
standout structural elements on webpages, and identifying what people
react to in the text.
Online Card-Sorting Studies:
• Card-sorting exercises are a great way to improve website information
architectures (IA)
• Traditional card-sorting studies involved writing names on index cards
and recruiting test participants
• This approach was expensive and suffered from recruitment challenges
• Online card-sorting vendors like OptimalSort and WebSort let you quickly
and efficiently create card-sorting studies
• Users can be invited to go through sorting exercises quickly and efficiently
online and then answer a few questions
• Open and closed card sorts can be done, with open sorts allowing users to
create their categories and closed sorts fitting things into predefined
categories.
Artificially Intelligent Visual Heat Maps:

• Eye-tracking studies help understand website visitor behavior

• Eye-tracking studies can be expensive and difficult to conduct
• Feng-GUI is a program that creates heat maps based on an algorithm that
predicts what a human would look at
• Feng-GUI's heat maps show the intensity of visitor interest, areas of
interest, and eye movement
• Feng-GUI is a low-cost alternative to eye-tracking studies
• Feng-GUI uses sophisticated algorithms that are improving over time
• Results from Feng-GUI are not as accurate as actual eye-tracking studies,
but still provide valuable insights.
 T h a n k Yo u ! !
Important Questions
• Describe the process for conducting a Lab Usability Study and discuss its
importance in improving user experience in web analytics 2.0.
• Explain different Usability Alternatives and their impact on improving the user
experience.
• Surveys are an important tool in web analytics 2.0 for collecting user feedback.
Discuss the types of surveys used, their benefits and drawbacks and explain
how to create an effective survey to obtain valuable insights.
• Web-enabled user research options are gaining popularity in web analytics 2.0.
Explain different web-enabled user research options and their advantages