See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/339632118

A usability evaluation of Web user interface scrolling types

Article in First Monday · February 2020

DOI: 10.5210/fm.v25i3.10309

CITATIONS READS

3 763

2 authors, including:

Pietro Murano
Oslo Metropolitan University
38 PUBLICATIONS 207 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

A Universal System for Auto-Detecting Cognitive Impairments View project

Anthropomorphic User Interfaces View project

All content following this page was uploaded by Pietro Murano on 03 March 2020.

The user has requested enhancement of the downloaded file.

 A Usability Evaluation of Web User Interface Scrolling
Types
Sushil Sharma* s310225@oslomet.no, Pietro Murano* pietro.murano@oslomet.no
*OsloMet – Oslo Metropolitan University, Oslo, Norway.

Abstract
This paper details a usability evaluation of scrolling techniques on Web sites. The scrolling methods
evaluated were normal scrolling (with default pagination), infinite scrolling, infinite scrolling with a
load more button and infinite scrolling with pagination. The four scrolling types were evaluated in
the context of tasks that involved either serendipitous type tasks or goal-oriented type tasks. The
evaluation was principally about the ‘raw’ performance and participant perceptions. This is
because it was felt that the greatest gap in knowledge concerned these aspects. The evaluation
was done by means of an experiment and the data collected was statistically analysed. The results
were mixed in nature, where no single scrolling method stood out as being the most usable.

Keywords
Usability, Web interface, scrolling, infinite scrolling, infinite scrolling with pagination

1. Introduction
Navigating a Web site efficiently and pleasurably is an important aspect that all Web designers and
businesses should be concerned about. Some research has been carried out on Web navigation
and menu structures (e.g., Murano and Sander, 2016; Murano and Khan, 2015; Murano and
Lomas, 2015; Murano and Oenga, 2012). However, another aspect of navigation that requires
more attention is navigating within a Web page by means of scrolling.

Scrolling is typically performed with a mouse, often with a built-in scroll wheel on a desktop
computer and a touch pad on a laptop computer. A finger or a stylus are used for scrolling on
mobile devices. With the development of Web page design in the last few years, there have been
a number of enhancements in scrolling design.

This study is focused on the evaluation of scrolling techniques on Web sites. Infinite scrolling along
with normal scrolling techniques were investigated in this study. Before investigating the common
issues found in infinite scrolling, it is important to define infinite scrolling and normal scrolling.
According to Loranger (2014), ‘Infinite scrolling is a Web-design technique that loads content
continuously as the user scrolls down the page, eliminating the need for pagination.’ ‘Normal’
scrolling is simply a slide movement, which can be done either horizontally or vertically.

In this study we were interested in following up on the discussion in Loranger (2014) and in other
online examinations (e.g., Ahuvia, 2013) concerning scrolling types and their suitability for certain
kinds of Web pages. However, we were particularly interested in the ‘raw’ performance and
participant perceptions. We feel that the greatest gap in knowledge concerned these aspects,
while some work has already been done in ascertaining the accessibility of infinite scrolling (See
section 2.4 below).
We chose to adopt an experimental approach to evaluate four different scrolling types using a
prototype Web site with four versions of scrolling types. The types of scrolling methods we
evaluated were normal scrolling with default pagination, infinite scrolling, infinite scrolling with a
load more button and infinite scrolling with pagination. The four scrolling types were evaluated in
relation to a series of tasks that involved either serendipitous type tasks or goal-oriented type
tasks [See Note 1]. A serendipitous task tends to be of the kind that is engaged in by users
essentially simply having a look around a web site with no specific or detailed aim in mind.
However, a goal-oriented task is one where a user has a specific aim in mind, e.g. buying a red silk
tie that costs less than a certain amount of money.

The rest of this paper will contain a section reviewing some of the research on scrolling. Then, an
empirical experiment will be presented in detail along with the main results of a statistical analysis
on data collected from participants. Finally, the paper will conclude with a discussion and analysis.

2. Literature Review
This section summarises related recent work supporting this study.

In this research, the main aim is to evaluate the usability of infinite scrolling that is used in some
Web sites, in terms of ‘raw’ performance and participant perceptions.

2.1 Overview of Scrolling Technique in Web Sites

As any Web user knows, scrolling is among the most frequently used interactions in interactive
applications such as word processors, spreadsheets and Web applications (Neervoort, 2004).
Scrolling is common because the full content of applications is typically larger than a display screen.
To see all of the content, users scroll in order to examine details in documents and other digital
materials.
Nielsen (2010) found that first-time users of a Web page did not like scrolling. Among the studied
sample only 23 percent of users used scrolling while visiting a given Web site [See Note 2].
Scrolling is one of the most common techniques for moving within a Web page. A study conducted
by Chartbeat analysed data from two billion visits and found that 66 percent of visits look at
information below the ‘fold’, i.e., scrolling (Haile, 2014). A study by Fessenden (2018) using eye
tracking technology showed that people use scrolling to move down, especially if the page is
designed to encourage scrolling [See Note 3].
Hinckley, et al. (2002) conducted an experiment with 27 people, 12 of whom were male, 22–50
years of age, to evaluate interactive scrolling techniques. They used Fitts’ paradigm as a tool to
study scrolling, an IBM ScrollPoint Pro mouse with an isometric joystick (“ScrollPoint”) and the
Microsoft IntelliMouse Explorer with a scrolling wheel were used to vary the scrolling distance as
well as the required tolerance of scrolling. They concluded that the efficiency of the wheel could
be considerably enhanced by means of an acceleration algorithm. Their findings resulted in a
practical and rigorous technique for evaluating scrolling methods. In addition, there were other
performance and user acceptance aspects. These included device procurement times, visual
diversion needed to use a graphical scroll bar, and the incorporation of scrolling into complicated
functions, such as navigation and choice of goals with the mouse.
Frederick, et al. (2015) performed a study on the Parallax scrolling technique that created an
illusion of depth on a Web page by making background images move slower than foreground
images. In addition to parallax scrolling engaging users with a Web site, it was claimed that it
improved the user experience. They hypothesised that parallax scrolling positively influenced each
of five variables: usability, satisfaction, enjoyment, fun and visual appeal, and subsequently the
overall user experience. They performed an experiment with 86 participants, 43 of which
performed a test with a parallax Web site and the others with a non-parallax site. The results of the
study found the parallax scrolling was significantly more fun. However, there was no significant
difference with respect to perceived usability, enjoyment, satisfaction and visual appeal. Two
participants suffered motion sickness and experienced significant usability issues while interacting
with parallax scrolling. Their study suggested parallax scrolling could be useful in ‘play’.
Lasch and Kujala (2012) investigated the comparative impact on driver distraction when searching
music albums using user interface features of a touch-based mobile music player. They designed a
driving simulator experiment involving nine male and nine female volunteers between the ages of
21 to 38 years old. As the study focused on the comparison of three scrolling methods: buttons,
swipe and kinetic, participants were asked to search for music tracks in a grid-style menu using
these methods. Swipe refers to a technique that enables page-by-page scrolling by using a swiping
gesture. Kinetic scrolling utilizes the same swiping gesture, but it accelerates the menu, which then
stops automatically after deceleration. Tapping on the menu can also stop the movement, whereas
the number of music tracks presented in a list-style format varied between three, five and seven
items per page. Nine of the participants were told to use the music player in portrait mode and the
remainder of the sample were told to use it in landscape mode. The study assumed that the swipe
technique should incur fewer distraction effects than kinetic or button techniques due to
systematic page by page scrolling and low levels of pointing accuracy needed for browsing. They
expected that three items would enable more efficient visual sampling efficiency per page.
However, visual demands were increased since more scrolling was required. Screen orientation
was assumed to have no distraction effects. The research findings suggested that visual searching
was better with a list-style menu structure with a reduced number of items per page than by a
grid-style menu with 12 items; this even applied to a list with kinetic scrolling. The swipe scrolling
method could be better than the kinetic or small buttons options for in-car menu browsing
activities because lower visual precision was needed.
Wherry (2003) performed a study to compare a touchpad scroll ring to a mouse scroll wheel and
touchpad scrolling zone using a Fitts’ Law testing approach. Twelve right-handed participants with
prior scrolling experience, seven of whom were male, evaluated time, error, and subjective result.
The scroll ring achieved the lowest error rates and performance times. Participants also preferred
the scroll ring. It was suggested that future studies explore ‘how the touchpad scrolling zone
performs when not integrated within the touchpad and how these scrolling devices perform in
compound tasks when users are typing or navigating while also scrolling in an application.’ (Wherry,
2003).
2.2 Pagination in Web Sites
Pagination is the process of splitting content into discrete pages. Most Web sites use pagination to
present a set number of results in Web search results or demonstrating a set number of posts
while seeing a discussion string. Pagination can be used as a part of some structure on each Web
application to isolate returned information and show it on different pages (Furche, et al., 2012).
Pagination additionally incorporates a rationale of planning and showing connections to different
pages.
Scrolling, along with pagination, is a technique to view more content which cannot fit on the first
page. Pagination enables users to view a part of the result on the current page, and then the user
has to click on a ‘next’ button to see a new page with another part of the result. Also, scrolling is a
method of traversing a Web page, wherein users either roll the scroll wheel on their mouse, or
manually move the scroll bar located on the right side of the browser screen (U.S. Department of
Health and Human Services, 2003).
Kim, et al. (2016) conducted a user experiment to investigate the effects of horizontal and vertical
control types (pagination versus scrolling) on Web searches with mobile devices. They recruited 24
participants, 14 of whom were male, aged 22–41 years. Despite participants having greater
familiarity with vertical scrolling during a search, their findings suggested that ‘... participants using
pagination were more likely to find relevant documents, especially those over the fold; spent more
time attending to relevant results; and were faster to click while spending less time on the search
result pages overall’ (Kim, et al., 2016). They also found that scrolling action affected speed for
searching. Considering the limitations regarding users’ individual differences and the lab conditions
of their study, the research could not conclusively say that horizontal control was better than
vertical scrolling. Rather, they suggested that it is worthwhile for search engines to provide both
scrolling types to enhance user experiences on touch-enabled mobile devices.
Thung, et al. (2010) examined and compared a number of design patterns for general features in
Web applications focusing on architectural and navigational design patterns. The case study
focused on the web site of a school and proposed several designs to improve the site’s usability.
They added a pagination pattern that was used to organize search results in a simplified and more
structured manner for performance reasons. Pagination was suitable when users needed to view a
long list of items that could not fit on one page. In terms of efficiency, the pagination option
worked if it only displayed a few results on a page rather than showing all results on a single page.
It supported unstructured information, such as Google search results, but it failed to support user
preferences for structured information.
2.3 Why Infinite Scrolling is not Easy for Every Web Site?
Infinite or continuous scrolling works well for content that streams constantly and has a relatively
flat structure, where each unit of content belongs at the same level and has similar chances of
being interesting to users (Loranger, 2014). However, endless scrolling is not recommended for
goal-oriented tasks, such as those requiring users to locate specific content or compare options.
Users who need specific types of information expect content to be grouped and layered according
to relevance, by pages. Users do not mind clicking links if each click is meaningful and leads them
closer to a desired goal (Loranger, 2014). In addition, the way text in which is presented can
interact with learners and affect their goals (Sanchez and Wiley, 2009).
Infinite scrolling has been widely used on Web sites with content of equal importance, such as
social media (Leeds, 2014). However, on a site where users need to locate desired information
quickly, such as in an e-commerce site, infinite scrolling is appropriate because it might overwhelm
users with content. When loading seemingly endless content, at some point information needs be
removed, otherwise the browser will eventually crash (Karlsson and Larsson, 2016).
Infinite scrolling is a type of Web interaction where data is fetched asynchronously from a data
store and inserted into a page as the user consumes information (Leeds, 2014). This method
results in an illusion that information on a page is for all appearances infinite.
Since this technique asynchronously fetches data while the user is consuming content, it depends
solely on JavaScript (Infinite Scroll, 2019). A point relative to the bottom of the page is usually set
by the Web site. When the user reaches this point, new data is loaded automatically and inserted
into the Document Object Model (DOM). This method is employed by social networks such as
Facebook, Twitter, Pinterest and others. As such, this technique is much more suited to Web sites
that produce large amounts of data, especially sites that contain user generated content.
Since the technique is suited to sites with large sets of data, it is not suitable for every design.
Infinite scrolling has its advantages as well as its shortcomings. Some of the advantages associated
with infinite scrolling are that it is suited for touch devices and better for visual content as it
captures the interest of users and keeps them captivated (Akin, 2018; Andrew, 2018; Babich, 2016;
Schenker, 2014). According to Ahuvia (2013), while viewing Google searches, only six percent
advanced to the next page. This suggests that infinite scrolling had better content exposure.
Further, it requires fewer clicks from the user’s side and novice users can use a site seamlessly as
content is automatically updated.
However, infinite scrolling has also some negative aspects. Users need to stay focused on their
desired information (Akin, 2018; Andrew, 2018; Karlsson and Larsson, 2016; Leeds, 2014; Loranger,
2014; Schenker, 2014). In returning back from a link in the page, the user’s position is lost. Also,
when returning to specific content, users cannot skip previous content. Users cannot reach footer
content as the page keeps loading automatically. With infinite scrolling, search engine optimization
(SEO) is affected and Web developers have to consider special measures.
Karlsson and Larsson (2016) conducted a quantitative study on how user behaviour differed
between a Web site with pagination and infinite scrolling. The study indicated that infinite scrolling
could be useful on an intranet when adapted according to usability guidelines. The researchers
evaluated user behaviour on an intranet of a health care company regarding which pagination
technique to choose (pagination or infinite scrolling). Three versions of a prototype Web site were
evaluated simultaneously; an infinite scrolling site with a load more button, a version with
pagination and a specially designed version with infinite scrolling adapted to the specific intranet.
They designed an evaluation task that made the site partly goal-driven, by finding specific
information, and another part not so goal-driven when scrolling through the feed. Finally, the
following conclusions and recommendations were drawn from their study:
• Implement the back button functionality together with automatic loading;
• Infinite scrolling can be valuable on a goal-driven site. Infinite scrolling and the paginated
version performed similarly. The only area where differences were seen were between the
number of articles loaded per session;
• Use infinite scrolling, and implement support to retrieve the user’s position when returning to
the feed.
Etsy has used infinite scrolling features on their Web site, at https://www.etsy.com/. However, they
observed the feature had certain undesirable aspects. Some of these were fewer clicks on result
items and less items being classed as ‘favourite’. Nguyen (2013) conducted a study looking into the
reasons for the failure of infinite scrolling, presenting two assumptions behind the use of infinite
scrolling. Users wanted more results per page as well as faster results.
Both assumptions were tested with particular users but their engagement level was not statistically
significant. Nguyen did not know why infinite scrolling did not succeed for Etsy. There was not, as
far as they could tell, a technical fault, such as infinite scrolling breaking on a specific browser. The
most annoying issue for infinite scrolling was a lack of a footer, which probably the average Etsy
user does not need when making purchasing or viewing decisions (Nguyen, 2013).
Holst (2016) conducted a usability study which tested three design patterns against usability
factors in the context of loading products in an e-commerce Web site. The design patterns used on
desktop and mobile devices were infinite scrolling, pagination and load more options. The results
suggested that load more buttons combined with lazy loading were superior than other methods
that provided a seamless user experience. Infinite scrolling was not favorable for mobile in terms
of usability for search results. Infinite scrolling was ideal for quickly showing the breadth of an
entire category. However, because users were not naturally halted when scrolling, they tended to
scan more and focus less on individual products on a list. The study concluded that no single
method was perfect for all instances.
2.4 Infinite Scrolling and Accessibility
Web sites with infinite scrolling can have accessibility issues. A number of authors (Lembree, 2015;
Langmo, 2017; Leeds, 2014; Loranger, 2014; Parker, et al., 2017; Peri, 2018; Roselli, 2014) have
found a series of accessibility problems with infinite scrolling. Some of these are:

• With infinite scrolling, it is not possible to skip chunks of irrelevant content;

• Focus is lost when navigating back. Keyboard-only users, including screen reader users, are
forced to re-navigate through all of the content to locate an element activated when the focus
is lost. The user has to reorient themselves on the page;
• With infinite scrolling, finding content that one viewed previously can be very difficult once a
page is more than a few screens long;
• Footer content is inaccessible. With infinite scrolling, assistive technology users have an issue in
navigating to the footer. The footer usually contains copyright and contact information;
• Quick navigation is difficult to achieve. If the Web site has a lot of headings and links within
each feed, navigation becomes very difficult when it comes to loading feeds infinitely even
though the site has proper heading elements and structure. Many contain news feeds with
additional links, forcing users to navigate through the active elements for each feed;
• Hard to locate the content. Sighted keyboard-only users and screen reader users may have
difficulty locating content using search feature or navigation techniques;
• With infinite scrolling it is difficult to see updates to the URL bar as additional pages are
loaded;
• Not able to go to the end of the page. Keyboard-only users are not able to use Ctrl+End
keyboard command to navigate to the end of the page.

Although the use of infinite scrolling seems to be on the increase, there is a lack of a larger body of
evidence concerning scrolling types and their usability and accessibility. There appears to be some
consensus that infinite scrolling is not the best option in many circumstances. This research aims to
build on the current research on scrolling types. To that end, the next section will detail an
empirical experiment which evaluated four scrolling types with real users.

3. Experiment
Four identical prototype Web sites were developed. Each was implemented with one of four
scrolling methods: normal scrolling with default pagination; infinite scrolling; infinite scrolling with
a load more button; and infinite scrolling with pagination.
Each site had two pages, home and shop pages. The home page had news and blog information in
different categories while the shop page had different kinds of e-commerce products and
electronic gadgets. A screenshot of each scrolling method used in the prototype is shown below.

Figure 1. Screenshots of each scrolling method.

3.1 Participants

A total of 16 participants, 12 males and four females, were included in this experiment. The
participants were university students pursuing bachelor’s or Master’s degrees; some were
employed in software development, design and quality assurance roles. Most of the participants
were 26–35 years old with the other age ranges represented in a minor way. To make the
experiment more efficient, a choice of locations were made available. These were either at a
university project room or the participant’s own home. The experiment date and time were set up
in advance by means of social media, text message and/or telephone call. Only the experimenter
and participant were present in the quiet and private experiment location.
The experiment was carried out ethically, in line with Norwegian research expectations. All
participants gave informed consent and no identifiable data was collected from them. No
participant was exposed to any dangerous or unpleasant circumstances. All participants had the
opportunity to ask questions related to their participation and the research and all had the option
of backing out of the experiment at any time.
3.2 Experimental Design

A within users experimental design was used in this experiment. The main reason this approach
was chosen was so that each participant could make subjective preference comparisons on all four
scrolling methods used.
3.3 Variables

The independent variables were four types of scrolling methods and specific tasks (see section 3.5
below). The four scrolling methods consisted of normal scrolling with default pagination; infinite
scrolling; infinite scrolling with a load more button; and infinite scrolling with pagination. The tasks
involved shopping activities on prototype Web pages.
The defined dependent variables were participant performance and subjective opinions. The
dependent measures were task completion time and number of errors.
Errors were categorized in two ways. The first consisted of a user clicking on something that they
thought was a link when it was not a link. The second involved users being asked to not click on the
search box. If they did click on the search box, this was categorized as an error. All the errors were
combined into a single score used in statistical analysis.
Participants’ opinions regarding the four scrolling types in relation to task types (see Task design
section for a detailed description of tasks) were obtained by using a post-experiment questionnaire.
A Likert (1932) type scale ranging from one to five was used for all questions, where for all
questions a score of five was the highest possible positive score.
3.4 Apparatus and Materials

For this experiment the following systems and materials were used to conduct the experiment:

• MacBook Pro with Mac OS Mojave, Intel Core i5 processor, 4GB RAM and 13” screen;
• HP laptop with Windows 10 OS, 4GB RAM, Intel Core i3 processor and 17” screen;
• A stopwatch;
• Chrome Web browser (Firefox/Safari depending on participant’s choice);
• Chrome browser extension screen recorder;
• Consent form;
• Pre-experiment questionnaire;
• Tasks document for the experiment;
• Post experiment questionnaire.

3.5 Task Design

Four task groups were designed for this experiment. All the tasks were related to serendipitous
discovery, exploration and goal-driven finding tasks. The tasks were designed around the concept
found in the study by Loranger (2014), where users’ activities tend to be either time-killing with
serendipitous discovery, goal-driven finding or both. A serendipitous task tends to be of the kind
that is engaged in by users simply having a look around a Web site with no specific or detailed aim
in mind. However, a goal-oriented task is one where a user has a specific aim in mind, finding some
specific piece of information on some topic. With these explanations in mind, four task groups with
four tasks were designed. These are detailed in Table 1:
Tasks
No. Task Type Task Description
1 Serendipitous Go to Home page, discover sports news that you find really
interesting or informative.
2 Serendipitous Go to shop page, explore an electronics product which is
Task latest or fascinating.
Group A 3 Goal-Driven Go to Home page, find music news with title “10 long-
awaited albums that will be the soundtrack of 2019”.
4 Goal-Driven Go to shop page, Find a book with name “The climb: Tragic
ambitions on Everest” and price “200kr”.
1 Serendipitous Go to Home page, discover cars related news that you find
really interesting or informative.
2 Serendipitous Go to shop page, explore shoes product which is latest or
Task fascinating in Shop page.
Group B 3 Goal-Driven Go to Home page, find a health news with title “Two
compounds in coffee may team up to fight Parkston’s”.
4 Goal-Driven Go to shop page, find a kitchenware product with Title
“‘Richardson 5pcs knife block magnet” which is on “sale”.
1 Serendipitous Go to Home page, discover music news that you find really
interesting.
2 Serendipitous Go to shop page, explore a book which is interesting or
Task informative.
Group C 3 Goal-Driven Go to home page, find sports news with title “Women-only
motorsport series launched to find potential F-1 Stars”.
4 Goal-Driven Go to shop page, find an electronics product with Title “‘JBL
Xtreme2’ and price “1200kr”.
1 Serendipitous Go to Home page, discover a Health news that you find really
beneficial or informative.
2 Serendipitous Go to shop page, explore a kitchenware product which is
Task interesting and useful in Shop page.
Group D 3 Goal-Driven Go to Home page, find cars related news with title “Porsche
Taycan Electric Car Teased, to debut in September”.
4 Goal-Driven Go to shop page, Find shoes with Title “JORDAN 1 MID
RETRO BASKETBALL SHOES “with price “700kr”.

Table 1: Specific Tasks.

In order to minimise learning effects, tasks were administered to participants in a ‘randomised’

manner as described in Table 2:
Participant Experiment Experiment Experiment Experiment
Part 1 Part 2 Part 3 Part 4
Web Site Task Web Site Task Web Site Task Web Site Task
Group Group Group Group
1 W1 A W2 B W3 C W4 D
2 W1 B W2 C W3 D W4 A
3 W1 C W2 D W3 A W4 B
4 W1 D W2 A W3 B W4 C
5 W2 A W3 B W4 C W3 D
6 W2 B W3 C W4 D W3 A
7 W2 C W3 D W4 A W3 B
8 W2 D W3 A W4 B W3 C
9 W3 A W4 B W1 C W2 D
10 W3 B W4 C W1 D W2 A
11 W3 C W4 D W1 A W2 B
12 W3 D W4 A W1 B W2 C
13 W4 A W1 B W2 C W1 D
14 W4 B W1 C W2 D W1 A
15 W4 C W1 D W2 A W1 B
16 W4 D W1 A W2 B W1 C
W1: Prototype web site1 with “Normal Scrolling”
W2: Prototype web site2 with “Infinite Scrolling”
W3: Prototype web site3 with “Infinite Scrolling with Load More”
W4: Prototype web site4 with “Infinite Scrolling with Pagination”
Task group (A, B, C, D): Tasks (2 serendipitous discovery tasks 2 goal-driven finding tasks)
Table 2: Task Randomization Procedure
3.6 Procedure

The experiment was carried out in a series of systematic stages. Before the start of the actual
experiment with selected participants, pilot testing was conducted with two different participants.
The goal of the pilot study was to identify possible problems in advance of the actual experiment.
After completion of the pilot experiment, the following results were obtained and changes were
made in the actual experiment:
• Participants took longer than expected. There were some difficulties in using scrolling
techniques at the beginning. To overcome this issue, a training component was added before
the start of the real experiment.
• Participants appeared to learn from task experiences and performed better in the last two
tasks. To minimize this issue, randomization was used in the real experiment (see Table 2).
The experiment was divided into four sections. Participants were initially greeted and brief
information about the experimenter was provided, consisting of personal name and course of
study. The experiment then was conducted in four stages comprising of the pre-experiment,
training, actual experiment and post-experiment.
The pre-experiment stage involved participants being seated and given a consent form. They were
asked to read it carefully and, if in agreement, sign it. The consent form had details about the
research study, purpose of the study, experimental procedures and confidentiality information.
Then participants were given a pre-experiment questionnaire. The questionnaire gathered
background information about participants and also information regarding their experiences with
computers, Web sites and different scrolling techniques use.
The training stage provided participants with a short session about different scrolling techniques
used in the prototype Web sites. This session lasted between three and seven minutes, where
basic functionality of scrolling techniques was covered. Then some trial tasks were given to each
participant.
The actual experiment stage was carried out next with the prototype Web sites and tasks as shown
noted in Table 1. A screen recorder was used to record screen interaction and a stopwatch was
utilized to time tasks. If a participant asked any questions regarding issues while performing tasks,
the experimenter provided some hints to complete tasks. Those hints were the same for every
participant so as to reduce potential bias.
The post-experiment stage involved completing a post-experiment questionnaire which dealt with
aspects of participant preferences and opinions about the four scrolling techniques, task design
and Web site design. Some of the areas covered in the questionnaire were aspects of simplicity
and ease of use, pleasantness, responsiveness, feelings of confidence in use and recommendations.
At the end of the experiment, the participants were thanked for their participation.
3.7 Results

The data collected for performance and participant opinions were analysed by means of a one-way
repeated measure ANOVA using SPSS (SPSS, 2019). As discussed above, there were two kinds of
tasks in this experiment. These were time-killing activities with serendipitous discovery and goal-
driven finding tasks. Therefore, the analyses were done based on each type of task individually.
The data collected met all the required assumptions (Mayers, 2013) for a one-way repeated
measure ANOVA and was based on N = 16.
Task time and errors were measured for all tasks. The tasks were either serendipitous or goal-
driven in nature. For these, no statistically significant results were observed. Therefore, with the
aim of keeping this paper as brief as possible, they will not be discussed any further in this paper.
The post-experiment questionnaire asked several questions using a Likert-type scale for the
responses. The questions used a Likert-type scale ranging from 1 to 5. For each question a 5
response was the most positive score one could allocate. For the serendipitous tasks, in relation to
the four scrolling types, the questions asked whether: it was simple and easy to use the scrolling
techniques, the scrolling techniques were very responsive, the scrolling techniques were pleasant
to use and confidence was felt whilst using the scrolling techniques. For these four questions no
statistically significant results were observed.
The fifth question concerned whether participants felt that on completion of the serendipitous
tasks and using the four scrolling techniques they would recommend use of the technique in a web
site. This question yielded a statistically significantly different result across the four scrolling types:
infinite scrolling web site (Mean = 4.44; SD = 0.512), Infinite load more scrolling (Mean =3.94; SD =
0.443), normal scrolling (Mean = 3.62; SD = 0.500) and infinite pagination scrolling (Mean = 3.50;
SD = 0.966), F (3, 45 this number looks incorrect) =7.336, p < 0.001, ηp2 =0.328.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly
different from each other. It was found that the pairs of normal scrolling and infinite scrolling were
more significant (p=0.001) followed by infinite scrolling and infinite scrolling with pagination
(p=0.003), and infinite scrolling and infinite scrolling with load more (p=0.039). Overall the infinite
scrolling technique was scored highest by participants.
The same questions and scales described above were also asked in relation to the goal-driven tasks
that participants carried out. The first question asked whether it was simple and easy to use the
scrolling techniques. This question yielded a statistically significantly different result across the four
scrolling types: normal scrolling (Mean = 4.19; SD = 0.655), infinite load more scrolling (Mean
=4.06; SD = 0.574), infinite pagination scrolling (Mean = 3.56; SD = 0.892) and infinite scrolling
(Mean = 3.44; SD = 0.512), F (3, 45) =5.909, p = 0.002, ηp2 =0.283.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly
different from each other. There was one significant difference between the pairs, i.e. normal
scrolling and Infinite scrolling (p=0.009). Overall the normal scrolling technique was scored highest
by participants for ease and simplicity of use.
The second question asked whether the scrolling techniques were responsive. This question
yielded a statistically significantly different result across the four scrolling types: normal scrolling
(Mean = 4.13; SD = 0.500), infinite load more scrolling (Mean =3.75; SD = 0.577), infinite pagination
scrolling (Mean = 3.69; SD = 0.479) and infinite scrolling (Mean = 3.13; SD = 0.619), F (3, 45) =8.408,
p = 0.000, ηp2 =0.359.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly
different from each other. There was one significant difference between the pairs, i.e. normal
scrolling and infinite scrolling (p<0.001). Overall the normal scrolling technique was scored highest
by participants for responsiveness.
The third question asked whether the scrolling techniques were pleasant to use. This question
yielded a statistically significantly different result across the four scrolling types: infinite load more
scrolling (Mean =4.06; SD = 0.574), infinite pagination scrolling (Mean = 3.88; SD = 0.619), Normal
scrolling (Mean = 3.56; SD = 0.892) and infinite scrolling (Mean = 3.38; SD = 0.500), F (3, 45)
=4.041, p = 0.013, ηp2 =0.212.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly
different from each other. Two pairs were significantly different, i.e. Infinite scrolling and infinite
load more scrolling (p<0.001) and Infinite scrolling and infinite pagination scrolling (p=0.009).
Overall the infinite load more scrolling type was scored highest by participants for being pleasant
to use.
The fourth question asked whether the participants felt very confident in using the four scrolling
techniques. This question yielded a statistically significantly different result across the four scrolling
types: infinite pagination scrolling (Mean = 4.31; SD = 0.602), normal scrolling (Mean = 4.13; SD =
0.619), infinite load more scrolling (Mean =3.69; SD = 0.946) and infinite scrolling (Mean = 3.38; SD
= 0.806), F (3, 45) =5.000, p = 0.004, ηp2 =0.250.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly
different from each other. There was one significant difference between the pairs, i.e. Infinite
pagination scrolling and Infinite scrolling (p=0.011). Overall the infinite pagination scrolling was
scored highest by participants for eliciting feelings of confidence whilst scrolling.
The fifth question asked participants to score their recommendations for each scrolling type. This
question yielded a statistically significantly different result across the four scrolling types: Infinite
load more scrolling web site (Mean = 4.25; SD = 0.775), normal scrolling (Mean =4.19; SD = 0.574),
infinite pagination scrolling (Mean = 3.88; SD = 0.719) and infinite scrolling (Mean = 3.06; SD =
0.574), F (3, 45) =9.899, p = 0.000, ηp2 =0.398.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly
different from each other. It was found that the pairs of infinite scrolling and infinite load more
scrolling(P=0.000001) were more significant. These were followed by normal scrolling and Infinite
scrolling (p=0.000315) and infinite scrolling and infinite pagination scrolling(P=0.017). Overall the
infinite load more scrolling type was scored highest by participants in terms of their
recommendations.
The participants were also asked some general questions concerning the experiment and the
prototypes used in the experiment. The questions used a Likert-type scale ranging from 1 to 5. For
each question a 5 response was the most positive score one could allocate. The questions aimed to
elicit attitudes concerning whether the tasks were understandable, interesting and involving
practicality. Further, the questions asked about the functionality of the prototype web site design
and personal satisfaction over the design and content of the prototypes.
For all questions, participants scored these either in the 4 or 5 region, indicating an overall
unanimous series of very positive opinions concerning the experiment and prototypes used in the
experiment.
This section has presented the statistically significant results for the data that was collected. In the
next section the results will be discussed along with some conclusions.

4 Discussion and Conclusions

This study aimed to add to the current knowledge about different scrolling types. Namely, this
study investigated normal scrolling with default pagination, infinite scrolling, infinite scrolling with
a load more button and infinite scrolling with pagination. The study was closely linked with the
work of Loranger (2014) and the use of tasks that were either serendipitous or goal-driven in
nature.

Our work suggests that when it comes to performance in using the different scrolling methods
tested, task times and errors were not statistically significantly different from each other. This was
found for tasks that were either serendipitous or goal-driven in nature.

These results suggest that it may not matter which scrolling method might be used. However, it is a
strong part of usability engineering to be concerned with how a user experiences and perceives an
interface or some element of it.

This research primarily dealt with user perceptions by means of a post-experiment questionnaire.
The first four questions asked participants to rank their opinions on whether: it was simple and
easy to use the scrolling techniques; scrolling techniques were very responsive; scrolling
techniques were pleasant to use; and confidence was felt whilst using the scrolling techniques. For
these four questions, in relation to the serendipitous tasks, no statistically significant results were
obtained, suggesting that perceptions about each scrolling method and these four topics were not
strongly affected. This series of results could be expected as the whole approach to a serendipitous
task is that it involves something that is potentially a ‘time-killing’ activity.

The fifth question concerned participants ranking which scrolling method they would recommend.
This question had the infinite scrolling method ranked the highest. This result is within normal
expectations, given that the tasks were serendipitous in nature.

The same questions were then asked in relation to goal-driven tasks. For questions one (ease and
simplicity) and two (responsiveness) the normal scrolling method was ranked highest. For question
three (pleasantness) the infinite load more scrolling method was ranked highest. For question four
(confidence in use) the infinite pagination scrolling method was ranked highest. For question five
(recommendation) participants ranked the infinite load more scrolling method highest.
Overall, the picture concerning which scrolling method a designer should use is not categorically
clear in terms of ‘raw’ performance and participant perceptions. Previous work seem to overall
suggest that for ease of use and accessibility, the normal scrolling method is better than infinite
scrolling. This may be the case, but further work needs to be done to achieve more categorical
results for ‘raw’ performance and participant perceptions. We did not achieve overall categorical
results pointing to the normal scrolling method as being the way forward for ‘raw’ performance
and participant perceptions. However, we did not specifically test in the experiment for the
negative accessibility issues found by other authors and presented earlier, as these seemed already
clear and unequivocal.

In future work we would like to have a larger participant sample. Future work should also address
the possibility that the prototypes and subsequent tasks used perhaps were too simple, thus
potentially making it more difficult to achieve categorical results. Although we did try to have a
prototype and tasks that had validity, they could have been improved by having more complex
information and in turn more complex tasks that could have highlighted differences. Also,
serendipitous tasks had to be controlled for being valid in an experiment and this in turn could
have affected their realism. A final thought is the questionnaire itself. This could have been
perhaps followed up with a semi-structured interview of the participants to get more in-depth
information about their perceptions on what they had experienced.

Until more work is done in the context of ‘raw’ performance and participant perceptions, normal
scrolling is still recommended for designers as being the more usable option. It still appears to be
the more accessible option when compared to infinite scrolling. If a different scrolling method such
as infinite scrolling is used the designers should ensure that best practice for accessibility is
followed.

Given the current body of knowledge, infinite scrolling appears to create accessibility problems.
Any interaction that increases the time or number of steps required to accomplish tasks should be
avoided. An interaction that makes it more difficult for certain kinds of users, such as users using a
screen reader, should always be avoided. Lastly, navigation must always be designed to be efficient,
pleasant to use and usable by all users, irrespective of their skills or personal requirements.
requirements.

About the Authors

Sushil Sharma is a Master of Universal Design of ICT and a former Master student at Oslo
Metropolitan University, Norway.

Dr Pietro Murano is an Associate Professor of Computer Science at Oslo Metropolitan University,

Norway. He is a Computer Scientist specializing in interaction, HCI, Usability and Universal Design
of computer systems.

Notes
1. In this paper we will be using the same kind of expressions as found in Loranger (2014) referring
to tasks types, i.e., serendipitous and goal-oriented.

2. See also the follow-up study by Fessenden (2018).

3. See also Nielsen (2010).

References
Y. Ahuvia, 2013. “Infinite scrolling: Let’s get to the bottom of this,” Smashing Magazine (3 May),
at https://www.smashingmagazine.com/2013/05/infinite-scrolling-lets-get-to-the-bottom-of-this,
accessed 18 February 2020.

F.K. Akin, 2018. “Stop building Web sites with infinite scroll!” LogRocket (18 October),
at https://logrocket.com/blog/infinite-scroll/, accessed 18 February 2020.

P. Andrew, 2018. “Infinite scrolling: Is it helping or hurting your business?” speckyboy (5 March),
at https://speckyboy.com/infinite-scrolling/, accessed 18 February 2020.

N. Babich, 2016. “UX: Infinite scrolling vs. pagination,” UX Planet (1 May),

at https://uxplanet.org/ux-infinite-scrolling-vs-pagination-1030d29376f1, accessed 18 February
2020.

D. Lembree, 2015. “Infinite scrolling and accessibility (It’s usually bad),” Web Axe (25 August),
at http://www.webaxe.org/infinite-scrolling-and-accessibility/, accessed 18 February 2020.

T. Fessenden, 2018. “Scrolling and attention,” Nielsen Norman Group (15 April),
at https://www.nngroup.com/articles/scrolling-and-attention/, accessed 18 February 2020.

D. Frederick, J. Mohler, M. Vorvoreanu and R. Glotzbach, 2015. “The effects of parallax scrolling on
user experience in Web design,” Journal of Usability Studies, volume 10, number 2, pp. 87–95, and
at https://uxpajournal.org/the-effects-of-parallax-scrolling-on-user-experience-in-web-design/,
accessed 18 February 2020.

T. Furche, G. Grasso, A. Kravchenko and C. Schallhart, 2012. “Turn the page: Automated traversal
of paginated Web sites,” In: M. Brambilla, T. Tokuda and R. Tolksdorf (editors). Web
engineering>. Lecture Notes in Computer Science, volume 7387. Berlin: Springer, pp. 332–346.
doi: https://doi.org/10.1007/978-3-642-31753-8_27, accessed 18 February 2020.

T. Haile, 2014. “What you think you know about the Web is wrong,” Time (9 March),
at https://time.com/12933/what-you-think-you-know-about-the-web-is-wrong/, accessed 18
February 2020.

K. Hinckley, E. Cutrell, S. Bathiche and T. Muss, 2002. “Quantitative analysis of scrolling

techniques,” CHI ’02: Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems, pp. 65–72.
doi: https://doi.org/10.1145/503376.503389, accessed 18 February 2020.

C. Holst, 2016. “Infinite scrolling, pagination or ‘load more’ buttons? Usability findings in
ecommerce,” Smashing Magazine (1 March),
at https://www.smashingmagazine.com/2016/03/pagination-infinite-scrolling-load-more-
buttons/, accessed 18 February 2020.

Infinite Scroll, 2019. “Infinite scroll — Automatically add next page,” at https://infinite-scroll.com/,
accessed 18 February 2020.
J. Karlsson and M. Larsson, 2016. “Adapting infinite-scroll with the user experience in mind,”
thesis, Linköping University (Sweden), at http://liu.diva-portal.org/, accessed 18 February 2020.

J. Kim, P. Thomas, R. Sankaranarayana, T. Gedeon and H.-J. Yoon, 2016. “Pagination versus
scrolling in mobile Web search,” >CIKM ’16: Proceedings of the 25th ACM International on
Conference on Information and Knowledge Management, pp. 751–760.
doi: https://doi.org/10.1145/2983323.2983720, accessed 18 February 2020.

O. Langmo, 2017. “Accessibility problems with infinite scrolling,” Accessible360 (11 May),
at https://accessible360.com/accessible360-blog/accessibility-problems-infinite-scrolling/,
accessed 18 February 2020.

A. Lasch and T. Kujala, 2012. “Designing browsing for in-car music player: Effects of touch screen
scrolling techniques, items per page and screen orientation on driver distraction,” AutomotiveUI
’12: Proceedings of the Fourth International Conference on Automotive User Interfaces and
Interactive Vehicular Applications, pp. 41–48.
doi: https://doi.org/10.1145/2390256.2390262, accessed 18 February 2020.

C. Leeds, 2014. “The UX of infinite scroll: The good, the bad, and the maybe,” SitePoint (4
November), at https://www.sitepoint.com/ux-infinite-scroll-good-bad-maybe/, accessed 18
February 2020.

D. Lembree, 2015. “Infinite scrolling and accessibility (It’s usually bad),” Web Axe (25 August),
at http://www.webaxe.org/infinite-scrolling-and-accessibility/, accessed 18 February 2020.

R. Likert, 1932. A technique for the measurement of attitudes. Archives of psychology, number 140.
New York: Science Press.

H. Loranger, 2014. “Infinite scrolling is not for every Web site,” Nielsen Norman Group (2
February), http://www.nngroup.com/articles/infinite-scrolling, accessed 18 February 2020.

A. Mayers, 2013. Introduction to statistics and SPSS in psychology. Harlow: Pearson.

P. Murano and M. Sander, 2016. “User interface menu design performance and user preferences:
A review and ways forward,” International Journal of Advanced Computer Science and
Applications, volume 7, number 4.
doi: https://doi.org/10.14569/IJACSA.2016.070447, accessed 18 February 2020.

P. Murano and I.N. Khan, 2015. “Pie menus or linear menus, which is better?” Journal of Emerging
Trends in Computing and Information Sciences, volume 6, number 9, pp. 476–481; version
at http://pietromurano.org/Papers/Murano-Khan-Published-Version.pdf, accessed 18 February
2020.

P. Murano and T.J. Lomas, 2015. “Menu positioning on Web pages. Does it matter?” International
Journal of Advanced Computer Science and Applications, volume 6, number 4.
doi: https://doi.org/10.14569/IJACSA.2015.060419, accessed 18 February 2020.

P. Murano and K.K. Oenga, 2012. “The impact on effectiveness and user satisfaction of menu
positioning on Web pages,” International Journal of Advanced Computer Science and Applications,
 volume 3, number 9.
doi: https://doi.org/10.14569/IJACSA.2012.030931, accessed 18 February 2020.

P.C. Neervoort, 2004. “Method of scrolling through a document,” Google Patents,

at https://patents.google.com/patent/US7689934B2/en, accessed 18 February 2020.

D. Nguyen, 2013. “Why did infinite scroll fail at Etsy?” danwin.com (6 January),
at http://danwin.com/2013/01/infinite-scroll-fail-etsy/, accessed 18 February 2020.

J. Nielsen, 2010. “Scrolling and attention (Original research study),” Nielsen Norman Group (21
March), at https://www.nngroup.com/articles/scrolling-and-attention-original-research/, accessed
18 February 2020.

S. Parker, S. Odio and A. Mosseri, 2017. “Infinite scrolling,” Google Patents,

at https://patents.google.com/patent/US20120011430, accessed 18 February 2020.

R.S. Peri, 2018. “Infinite scroll & accessibility! Is it any good?” (25 April),
at https://www.digitala11y.com/infinite-scroll-accessibility-is-it-any-good/, accessed 18 February
2020.

A. Roselli, 2014. “So you think you’ve built a good infinite scroll” (31 May),
at http://blog.adrianroselli.com/2014/05/so-you-think-you-built-good-infinite.html, accessed 18
February 2020.

C.A. Sanchez and J. Wiley, 2009. “To scroll or not to scroll: Scrolling, working memory capacity, and
comprehending complex texts,” Human Factors, volume 51, number 5, pp. 730–738.
doi: https://doi.org/10.1177/0018720809352788, accessed 18 February 2020.

M. Schenker, 2014. “The advantages of an infinite scrolling Web site” (3 February),

at https://www.webdesign.org/the-advantages-of-an-infinite-scrolling-website.22396.html,
accessed 18 February 2020.

SPSS, 2019. “IBM SPSS statistics,” at https://www.ibm.com/no-en/marketplace/spss-statistics,

accessed 18 February 2020.

P.L. Thung, C.J. Ng, S.J. Thung and S. Sulaiman, 2010. “Improving a Web application using design
patterns: A case study,” 2010 International Symposium on Information Technology.
doi: https://doi.org/10.1109/ITSIM.2010.5561301, accessed 18 February 2020.

U.S. Department of Health and Human Services, 2003. “Research-based Web design and usability
guidelines,” at https://www.usability.gov/sites/default/files/documents/guidelines_book.pdf,
accessed 18 February 2020.

E. Wherry, 2003. “Scroll ring performance evaluation,” CHI EA ’03: CHI ’03 Extended Abstracts on
Human Factors in Computing Systems, pp. 758–759.
doi: https://doi.org/10.1145/765891.765973, accessed 18 February 2020.

View publication stats