The Association of System

Performance Professionals

The Computer Measurement Group, commonly called CMG, is a not for profit, worldwide organization of data processing professionals committed to the
measurement and management of computer systems. CMG members are primarily concerned with performance evaluation of existing systems to maximize
performance (eg. response time, throughput, etc.) and with capacity management where planned enhancements to existing systems or the design of new
systems are evaluated to find the necessary resources required to provide adequate performance at a reasonable cost.

This paper was originally published in the Proceedings of the Computer Measurement Group’s 2001 International Conference.

For more information on CMG please visit http://www.cmg.org

Copyright Notice and License

Copyright 2001 by The Computer Measurement Group, Inc. All Rights Reserved. Published by The Computer Measurement Group, Inc. (CMG), a non-profit
Illinois membership corporation. Permission to reprint in whole or in any part may be granted for educational and scientific purposes upon written application to
the Editor, CMG Headquarters, 151 Fries Mill Road, Suite 104, Turnersville , NJ 08012.

BY DOWNLOADING THIS PUBLICATION, YOU ACKNOWLEDGE THAT YOU HAVE READ, UNDERSTOOD AND AGREE TO BE BOUND BY THE
FOLLOWING TERMS AND CONDITIONS:

License: CMG hereby grants you a nonexclusive, nontransferable right to download this publication from the CMG Web site for personal use on a single
computer owned, leased or otherwise controlled by you. In the event that the computer becomes dysfunctional, such that you are unable to access the
publication, you may transfer the publication to another single computer, provided that it is removed from the computer from which it is transferred and its use
on the replacement computer otherwise complies with the terms of this Copyright Notice and License.

Concurrent use on two or more computers or on a network is not allowed.

Copyright: No part of this publication or electronic file may be reproduced or transmitted in any form to anyone else, including transmittal by e-mail, by file
transfer protocol (FTP), or by being made part of a network-accessible system, without the prior written permission of CMG. You may not merge, adapt,
translate, modify, rent, lease, sell, sublicense, assign or otherwise transfer the publication, or remove any proprietary notice or label appearing on the
publication.

Disclaimer; Limitation of Liability: The ideas and concepts set forth in this publication are solely those of the respective authors, and not of CMG, and CMG
does not endorse, approve, guarantee or otherwise certify any such ideas or concepts in any application or usage. CMG assumes no responsibility or liability
in connection with the use or misuse of the publication or electronic file. CMG makes no warranty or representation that the electronic file will be free from
errors, viruses, worms or other elements or codes that manifest contaminating or destructive properties, and it expressly disclaims liability arising from such
errors, elements or codes.

General: CMG reserves the right to terminate this Agreement immediately upon discovery of violation of any of its terms.
 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

Performance Modeling for Web Application Optimization

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
Richard L. Gimarc Amy Spellmann
HyPerformix, Inc. HyPerformix, Inc.
rgimarc@hyperformix.com amy@hyperformix.com
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

This paper describes how to quickly build a performance model of a Web

application that can be used to evaluate optimization alternatives. Using
readily available measurement data and a simulation model, any Web-
based application can be represented from the end user (browser)
perspective. The methodology described in this paper illustrates and
validates the premise that performance modeling is a technique that can be
efficiently used to evaluate performance optimization alternatives for Web
applications.

1 Introduction 1. eBusiness Site Architecture. Determine the

As companies fully implement eBusiness hardware, software, and network architecture for
technologies, performance becomes increasingly the Web site. The intent is to develop a
comprehensive inventory of the components that
critical. Not only does the performance of these
systems affect the efficiency of the business; it also comprise the site.
defines the customer experience. It is critical for 2. Measure the eBusiness Site. The performance
companies to quickly deliver responsive application of the site must be measured from different
features and functions that provide a positive perspectives. Measurements are needed from
customer experience. Ensuring Web application both internal (system and application) and
performance requires that attention be focused not external (customer) sources.
only on the application, but also on the customer’s 3. Characterize Customer Behavior. Develop
connectivity to the application through the Internet. profiles of the types of customers that interact
Analyzing the interaction of customers, Web with the site. Different customer classes place
applications, and the Internet requires detailed different demands on site resources (e.g.,
knowledge and understanding of each. Information shoppers versus buyers).
Technology departments are challenged with the 4. Characterize Site Workload. The workload
complex task of gathering, organizing, and placed on the site by the different customer
evaluating measurement data to monitor and predict classes needs to be quantified. This requires a
system performance and, more importantly, the precise description of the resource demands, visit
customer experience. Monitoring requires quality frequency, site navigation, and usage scenarios
measurement and reporting tools; performance placed on the site by each customer class.
prediction requires measurement and modeling.
5. Develop Performance Models. A performance
In [MENA2000], Menascé describes an 8-step model of the site is developed. The model will be
Quantitative Analysis Cycle of an eBusiness site. used to predict site performance for a variety of
Before applying this methodology, the business changes to configuration, application, workload
must have a clearly documented understanding of level, and network connectivity.
the measurable goals and objectives of their site and 6. Obtain Performance Parameters. The
application(s). Once formulated, the following performance model’s input parameters are
Quantitative Analysis Cycle can be applied to ensure determined from direct measurement or
that the goals are achieved: estimation. Some parameters (e.g., disk I/O
time) may be measured directly. Other

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

parameters (e.g., bandwidth between a remote § Modeling Techniques characterizes the

customer and the site) may need to be estimated. simulation model and key capabilities that make
7. Forecast Workload Evolution. The forecast is it generally applicable.
§ Measurement Methodology identifies

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
an estimate of the workload level at some future
time or event. For example, a brokerage site will measurement and collection techniques for
need to understand the customer experience gathering performance metrics.
during a market storm; an online bookstore will § Validation of the Approach demonstrates the
be interested in site availability following the accuracy of the model results and the efficient
release of a new novel by a best-selling author. collection of the model inputs.
§ Optimization Alternatives outlines the potential
8. Predict eBusiness Site Performance. The
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

topology and site design scenarios that can be

performance model is used to predict site
evaluated with the model to quantify the
performance for the workload forecast(s).
improvement in the customer experience.
Changes to the site’s architecture and
applications can be evaluated to determine the
most cost-effective way to satisfy future 2 The Approach
workload(s). Our goal is to demonstrate that a performance
model of a Web application can be built quickly and
For this paper, we focus on steps 5 and 6, efficiently. Achieving this goal requires a well-
developing a performance model and obtaining defined process that is robust, repeatable, and
parameters for the model. generally applicable to a large number of Web
applications. To accomplish this, we need the
While measurement tools abound, few can capture following key ingredients:
all the detailed metrics needed for predictive
modeling. Usually, data will come from many § A standard framework for describing the
diverse sources, and it is often a challenge to
performance of a Web application.
correlate the collected data. We describe a
methodology for collecting the data needed to § A reusable, robust technology that supports the
characterize the performance of a Web application automatic construction of a predictive model.
for a model. Th e model will serve as the framework § A repeatable data collection methodology
for correlating and organizing these metrics as well based on generally available metrics.
as the vehicle for predicting the response time of the
application. We will use Strategizer®, a discrete Reference Model
event simulation and modeling tool, to mimic the
The Response Time Reference Model described in
behavior of Web applications for predictive analysis
[LOOS2000] will be used as the framework for
[HYPR2001]. The process we describe can be used
describing the performance of a Web application.
to represent the performance of any Web application
The Reference Model describes a business function
from the end user’s perspective.
as a sequence of Web pages. Each page is then
decomposed into a set of eight stages that specify
Our goal is to construct a model that represents the
how it is produced on an end user’s browser. The
customer experience; we will be evaluating end user
duration of each stage represents a response time
response times based on Web page design, Internet
component, and the end user’s response time is the
connectivity, and service times for the Web server.
sum of the components across all pages in the
More detailed modeling and analysis of server-
application. The Reference Model’s response time
based application performance, although an
components are:
important topic, is beyond the scope of this paper.
DNS Lookup. The browser begins by sending a
To illustrate how to model the customer experience,
message to the local Internet Service Provider’s
we will use three typical Web sites that provide static
Domain Name System (DNS) server to translate the
content. We will use a single model and common
domain name in the target URL
measurement methodology to analyze and predict
(www.hyperformix.com) into its actual Internet
performance of all three sites to demonstrate the
Protocol (IP) address (208.44.216.93).
feasibility, efficiency and accuracy of this approach.
The remainder of this paper is organized as follows: TCP Connection. After DNS lookup completes, the
browser sends a TCP connection request to the
§ The Approach describes the approach for Web server’s IP address. A TCP connection will be
modeling and measuring any Web site. opened using a 3-way handshake between the
browser and Web server [COME1995].

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

Server Processing. With the open TCP For ease of explanation, we make the following
connection, the browser sends a GET request to the assumptions about the complexity of the Web sites
server for a particular Web page. If the request is for and user interaction that we will be studying: the
a static page, the server will prepare to send the user will request a single page with static content,

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
fixed HTML file representing the page back to the page requests will not be redirected, and the
browser. In the case of a dynamic page, additional browser time required to produce a page is
Web server or other backend processing may be negligible. Thus, we will not be including the
required before the page is ready to be returned to following Reference Model components:
the browser. Redirection, Page Rendering, and User Interaction.
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

Redirection. Instead of returning the requested

Web Application Topology Components
page, the Web server may redirect the browser to
fetch a replacement URL. In this case, the browser Figure 1 depicts the standard topological
will again perform a DNS lookup and TCP components we will consider in this study. The
connection with the replacement URL’s Web server. browser has a direct connection to the Internet.
This sequence may repeat several times until the There is a DNS server at the browser’s local ISP that
server is reached that contains the requested page. will translate domain names into IP addresses. The
target Web server is located behind a firewall and
Base Page Download. After the Web server has contains local copies of all requested pages and
fetched and prepared the requested page, it is sent content elements.
back to the browser.
Annotations in the figure identify the measurements
Content Download. The browser will examine the required to model application performance from the
received page to determine if additional elements end user perspective:
are needed for its production. These content
elements generally take the form of embedded Browser. Total response time o f r a page request
graphic objects or applets. The browser will submit (as seen by the end user) is the primary metric we
requests to retrieve each embedded element. will be predicting with the model. A measured value
will serve as the validation target to judge the
Page Rendering. The production (display) of the “goodness” of our model’s predictive capability.
complete page may require additional processing by
the browser. Examples include dynamic page DNS Server. The DNS server is assumed to be
reformatting and applet execution. located at the browser’s local ISP. The DNS Lookup
latency, although small, is the first component of
User Interaction. After the browser has produced total response time.
the completed page, the user will need time to
absorb what was displayed and decide how to Internet. The Internet serves as the communication
proceed. channel between the browser and Web server.
Since our focus is on Web application performance,

DNS Lookup Time

DNS Server
Page Name & Size
Object Name & Size

Web
Firewall
Browser Server Page
Content
i M ac Internet

Response Time Latency Page Request Service Time

Bandwidth Object Request Service Time

Figure 1. Standard Web Application Topology

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

we will be viewing the Internet as a single abstract exist, or c) many interdependent concurrent
“cloud” characterized by latency and bandwidth. For transactions compete within a heterogeneous
browsers at different geographic locations, their environment such as modern client-server
respective Internet characterization will be systems.

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
dependent on their connectivity with the Web server.
Simulation is a problem-solving approach that
Firewall. A firewall is generally used to shield a is proactive, more flexible and adaptive than
Web site from the global Internet population. As an are rules of thumb, more predictive than is
abstraction, a Web site’s firewall will not be directly measurement, more efficient and practical
represented in our model. We assume that the than is benchmarking, and better suited to
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

performance characteristics of the firewall are complex new technology than is analytic
captured in the Web server and/or Internet modeling.
measurement data.
Domanski points out a potential drawback of
Web Server. The single Web server will receive simulation modeling in [DOMA1997]:
and service three types of requests: open a TCP
connection, request a page, and request a content Discrete event simulation, while very
element. Since the server time required to open a accurate, is time-consuming in terms of both
TCP connection is negligible, it will not be directly the time required to build the model as well as
represented in the model. Web server service times the time required to execute the model.
are dependent on the server itself and not the
location of the requesting browser. In this study, we develop a reusable Web application
model using Strategizer that has just enough detail
Page Content. The final set of data required by the to accurately predict end user response times. We
model is a description of the pages that are being demonstrate that simulation modeling is an effective
requested. This data must describe the size of each technique for analyzing Web application
HTML page and embedded content element (e.g., performance.
graphic object) requested by the browser. Page
content is an example of static data, that is, data that In addition, a simulation model is the appropriate
is independent of architectural, geographic, or tool for staged development. The high-level model
loading factors. described in this paper is appropriate for
representing the customer experience. The next
step is to refine the representation of the site’s
3 Modeling Techniques components behind the firewall to develop a more
detailed model appropriate for studying site
Analytic Versus Simulation Modeling scalability and architecture.
A simulation model was used in this study, rather
than an analytic model, due to the dynamic nature of A Reusable Web Application Model
Internet applications and the need to vary the level As stated earlier, we are using the Reference Model
of detail in the model. Analytic modeling cannot, as the framework for describing the performance of
without great effort, represent systems at varying a Web application from the end user’s perspective.
levels of detail. Additionally, Internet applications In order to be generally reusable, a performance
tend to have transient loads (bursty and varying by model must represent the same stages as the
time of day); analytic models cannot easily capture Reference Model. The model developed for this
these types of behaviors. Modeling a Web study includes the same components illustrated
application requires a technology that is robust earlier in Figure 1: browsers with varying network
enough to handle these key characteristics. connectivity, the DNS server, the Internet, and the
Web server. Additionally, we include descriptions of
In [NEUS1998], Neuse argues that simulation the Web pages and content elements that are
modeling is the appropriate tool for analyzing requested by the user.
complex systems, especially those with rapidly
changing components and architectures: Strategizer is the general-purpose modeling tool we
used to develop our reusable Web application
Analytic models are poorly suited to projects model. Strategizer contains a number of built-in
in which a) new architectures are being components for representing the basic hardware
studied and little time is available to develop and network elements. This section describes the
and validate the models (e.g., the alternatives additional browser and application logic that was
must be evaluated soon after they are modeled to represent the workflow and resource
proposed), b) new technology is being used requirements of our Web application.
for which existing, validated models do not

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

Figure 2. Web Application Model

Figure 2 shows the Strategizer model of the Web The model’s input parameters are shown in Figure
site. Client_1 and Client_2 represent browsers at 3. Note that these are exactly the same parameters
two different locations. Each browser has identified earlier in Figure 1. Each browser is
connectivity to a Local_ISP for DNS lookup and to parameterized with switches to enable local browser
the Internet via a network link. The Web site is caching (CLIENT_ENABLE_BROWSER_CACHE)
represented at a high-level by the components and the use of TCP persistent connections to the
connected to the switched Ethernet (HPFX_LAN): a Web server
Web server and an application server. For this initial (CLIENT_PERSISTENT_TCP_CONNECTION). In
study, the application server is a placeholder order to represent different types of browsers, there
included as a starting point for future analysis. is an additional parameter describing the maximum
number of parallel threads that may be used to

Parameter Value Description

Client 1 Link - Average bandwidth in terms of
CLIENT_1_LINK_BANDWIDTH 592.85 kilobits per second
Client 1 Link - Average 1-way link latency in
CLIENT_1_LINK_LATENCY 50 milliseconds
CLIENT_1_NAME "Seattle_Genuity" Client 1 - Client name
Client 2 Link - Average bandwidth in terms of
CLIENT_2_LINK_BANDWIDTH 1466.35
kilobits per second
Client 2 Link - Average 1-way link latency in
CLIENT_2_LINK_LATENCY 10 milliseconds
CLIENT_2_NAME "Dallas_Level3" Client 2 - Client name
Switch: 1=enable browser cache, 0=disable
CLIENT_ENABLE_BROWSER_CACHE 0
browser cache.
Switch: 1=client uses persistent TCP
CLIENT_PERSISTENT_TCP_CONNECTION 0 connection, 0=use nonpersistent TCP
connection
DNS Server - DNS lookup request service
DNS_LOOKUP_SERVICETIME 0.005
time (in seconds).
Maximum number of Web browser TCP
MAX_BROWSER_CONNECTIONS 4
connections.
Web Server – Service time to process an
WEBSVR_OBJECT_SERVICETIME 0.005
object request (in seconds).
Web Server - Service time to process a base
WEBSVR_PAGE_SERVICETIME 0.030
page request (in seconds).

Figure 3. Model Input Parameters

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

// Page_Name Page_Size_Bytes Page_Source

PAGE HIRES_HOME_PAGE 44709 HPFX_Web_Services
//

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
// Object_Name Object_Size_Bytes Object_Source
OBJ /images/background.jpg 5624 HPFX_Web_Services
OBJ /images/back.gif 283 HPFX_Web_Services
OBJ /images/top-bar.gif 101 HPFX_Web_Services
OBJ /images/top-bar-fade.jpg 640 HPFX_Web_Services
OBJ /images/below -logo.jpg 2851 HPFX_Web_Services
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

Figure 4. Page and Object Description Input File

retrieve embedded content elements order of the content elements on the page so that
(MAX_BROWSER_CONNECTIONS). For the model can accurately mimic the behavior of
simplicity, DNS Lookup time is described with a the browser’s download of content.
single parameter (DNS_LOOKUP_SERVICETIME). 2. Browser Connectivity. Data is required that
Each browser location is connected to the Web site characterizes the browser’s connectivity with the
using its own parameterized view of the Internet Web site: latency and bandwidth.
(e.g., Client_1_Link is the network link for Client_1).
3. Web Application Performance. The Web
Web site connectivity for Client 1 is parameterized
server is the primary component of the
with latency (CLIENT_1_LINK_LATENCY) and
application that we are representing in our initial
bandwidth (CLIENT_1_LINK_BANDWIDTH). There
model. Data must be collected that characterizes
are two final parameters that describe the
the service time for page and content element
performance of the Web server: service time for
requests.
processing a page request
(WEBSVR_PAGE_SERVICETIME) and service time
This section describes the measurement data
to process a request for a content element
sources for the performance model. We then
(WEBSVR_OBJECT_SERVICETIME).
describe how we used the collected data to develop
The static data describing Web page content is values for the model’s input parameters.
supplied from a spreadsheet. The spreadsheet
contains a description of the base HTML page and Page Content
the page’s embedded content elements. Figure 4 The following data is required to characterize a Web
lists the initial portion of the spreadsheet. The figure page:
shows a Web page named HIRES_HOME_PAGE.
The size of the base page is 44,709 bytes, and is § Size of the base HTML page. We assume that
located at HPFX_Web_Services. The page’s the base page is stored locally at the Web site.
content elements are listed, in order, on the OBJ § Size and location of each embedded content
lines. element. Note that there may be static content
elements stored remotely on a Content Delivery
The key to quickly building a model of a Web
Network.
application lies in collecting measurement data and
developing appropriate values for the model’s input § Order of the content elements on the page. In
parameters. Automating this process requires a general, the browser will request content
well-defined and repeatable methodology. This elements in the order they appear in the base
methodology is described in the following section. page’s HTML. This may have a performance
impact depending on the number of parallel
4 Measurement Methodology threads a browser uses to retrieve embedded
content.
The measurement data required to build our
performance model can be classified into the Microsoft Internet Explorer can be used to directly
following three categories: determine the size of a Web page.

1. Page Content. Data that characterizes the § The File menu contains a Properties selection
content of each requested page: the size of the that will display the number of bytes in the
base page and the size of each embedded displayed page.
content element. It is also important to know the

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

§ The View menu contains a Source selection that lists graphic images; if the Web page contains other
displays the HTML source in Notepad. This file downloadable content (e.g., a Macromedia flash file)
may be saved and examined for size they will not be shown in the image list.
information.

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
Internet Explorer can also be used to determine the
§ The File menu also has a Save As selection that
graphic content of a Web page by examining the
allows you to save a copy of the displayed page
subdirectory created with the File menu Save As
and embedded graphics. Note that this
selection. The subdirectory will contain a copy of
technique will modify the base page’s HTML
each graphic object contained on the saved page.
since relative directory names based on the
The problem with this technique is that it does not
page’s title are inserted into the IMG tags. For
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

provide the order of the objects on the page.

example, a test page was correctly sized at
44,709 bytes. The HTML page created by Save
As expanded to 51,891 bytes. Measuring Browser Connectivity
Our model uses two input parameters to
Microsoft provides a handy utility for Internet characterize the browser’s connectivity to the Web
Explorer that will display the images contained on a site: latency and bandwidth. Latency is a measure
page: Images List Utility [MICR2001]. A sample of the time is takes a packet to travel from the
screen capture is shown in Figure 5. The order of browser to the Web server (and vice versa). Two
the images listed by Images List is almost the same tools for estimating network latency are described in
as the order that the browser will request them from this section. Bandwidth is more difficult to measure
the server. The one exception we have observed and will be discussed in the next section.
concerns the page background that may be
specified with the BODY HTML tag; Images List will The t wo generally available tools for measuring
list the background as a special image at the end of network latency are “ping” and “traceroute”. Both
the list. It is important to note that Images List only tools measure the round-trip latency between the
issuing station and an arbitrary destination.

Figure 5. Images List Utility

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

Pinging www.hyperformix.com [208.44.216.93] with 32 bytes of data:

Reply from 208.44.216.93: bytes=32 time=90ms TTL=112

Reply from 208.44.216.93: bytes=32 time=90ms TTL=112

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
Reply from 208.44.216.93: bytes=32 time=100ms TTL=112
Reply from 208.44.216.93: bytes=32 time=90ms TTL=112
Reply from 208.44.216.93: bytes=32 time=90ms TTL=112
Reply from 208.44.216.93: bytes=32 time=90ms TTL=112
Reply from 208.44.216.93: bytes=32 time=90ms TTL=112
Reply from 208.44.216.93: bytes=32 time=91ms TTL=112
Reply from 208.44.216.93: bytes=32 time=90ms TTL=112
Reply from 208.44.216.93: bytes=32 time=100ms TTL=112
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

Ping statistics for 208.44.216.93:

Packets: Sent = 10, Received = 10, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 90ms, Maximum = 100ms, Average = 92ms

Figure 6. Sample ping Output

the path is required to decrement the TTL on a

Ping is a useful network debugging tool found on packet by at least 1 before forwarding it, so the TTL
most platforms that can be used to estimate the is effectively a hop count. When the TTL on a
latency between the issuing station and an arbitrary packet reaches 0, the router is supposed to send
IP address. Ping sends a short Internet Control back an ICMP Time Exceeded message to the
Message Protocol (ICMP) echo packet to the target source system. Traceroute determines the route by
IP address and listens for a single packet reply. sending the first echo packet with a TTL of 1 and
Ping places a time stamp in each packet it sends incrementing the TTL by 1 on each subsequent
that is echoed back and can be used to compute the transmission until the target responds or the
round trip time for the packet exchange. maximum TTL is reached.
Figure 6 shows sample output from the Windows NT Three probes are sent at each TTL (hop limit)
ping command. setting, and a line is printed showing the TTL (hop
limit), the hostname and the address of the gateway,
The traceroute diagnostic utility traces the route that and the TTL (round trip time) of each probe.
an IP packet follows to another Internet host by
sending ICMP echo packets with varying time-to-live Figure 7 shows output from the Windows NT
(TTL) values to the destination. Each router along traceroute command (tracert).

Tracing route to www.hyperformix.com [208.44.216.93]

over a maximum of 30 hops:
1 <10 ms <10 ms <10 ms 64.85.91.1
2 <10 ms <10 ms <10 ms 64.85.91.129
3 <10 ms <10 ms <10 ms dcr01-g1-0.tkwl01.exodus.net [216.34.64.33]
4 <10 ms <10 ms <10 ms bbr02-g4-0.tkwl01.exodus.net [216.34.64.164]
5 <10 ms <10 ms <10 ms bbr02-p2-0.sttl01.exodus.net [216.32.132.121]
6 40 ms 50 ms 51 ms bbr01-p0-0.okbr01.exodus.net [216.32.132.90]
7 40 ms 50 ms 50 ms ibr02-g4-0.okbr01.exodus.net [216.34.183.67]
8 41 ms 50 ms 50 ms chi-brdr-03.inet.qwest.net [205.171.4.49]
9 40 ms 50 ms 50 ms chi-core-02.inet.qwest.net [205.171.20.137]
10 60 ms 160 ms 60 ms kcm-core-01.inet.qwest.net [205.171.5.211]
11 60 ms 61 ms 60 ms kcm-core-02.inet.qwest.net [205.171.29.126]
12 70 ms 70 ms 80 ms dal-core-01.inet.qwest.net [205.171.5.203]
13 70 ms 70 ms 70 ms dal-core-02.inet.qwest.net [205.171.25.2]
14 70 ms 81 ms 80 ms hou-core-02.inet.qwest.net [205.171.5.170]
15 70 ms 80 ms 80 ms hou-edge-01.inet.qwest.net [205.171.23.50]
16 90 ms 100 ms 90 ms 205.171.40.162
17 * 90 ms 100 ms www.hyperformix.com [208.44.216.93]

Trace complete.

Figure 7. Sample tracert Output

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

Sample
Component Description
Value
Total Time 2.98 Response time for the single page request.

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
Time spent translating the domain name of the requested URL into an IP
DNS Time 0.00 address.
The time required for the agent to open a TCP connection with the Web
Connection Setup Time 0.10 server. This is the 3-way protocol handshake done by TCP.
The elapsed time from when the agent sends the request for a page until it
Time to First Byte 0.23 sees the first byte back from the Web server. This is a single round trip
across the Internet.
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

The amount of time required to download the base page (usually

Index Time 0.59 index.html). This measurement starts when the first byte arrives and ends
when the last byte of the file arrives.
The time required to parse the received base page and download all the
Content Time 2.06 embedded images.

Figure 8. Keynote Perspective Metrics

There is one potential drawback when using either between the browser and the Web site. We used
ping or traceroute; many Web site administrators the Index Time metric reported by the measurement
configure their routers to ignore echo packets, so the agents to develop an estimate for bandwidth.
commands timeout.
Index Time is the elapsed time between the arrival
Web Application Performance of the first HTML packet at the browser and the
complete arrival of the last packet in the base page.
Keynote Perspective (see [KEYN2001]) was used to Bandwidth is estimated by dividing the page size by
collect measurements for the time it takes to access Index Time.
and download the total elements of a page. These
measurements are taken from a number of CLIENT_1_LINK_LATENCY. A latency value is
geographically distributed measurement computers required for each browser connected to the Web
(“agents”). The four key operational characteristics site. This value can come from three sources: ping,
of these agents are: traceroute, or Keynote Perspective.

1. The agent’s local cache is cleared before each Ping and traceroute are direct measures of the
page request. round trip time between the issuing station and a
target IP address. The Connection Setup Time
2. The agents use HTTP 1.0. This means that a
new TCP connection is opened for each agent metric can also be used as an estimate. In
[MILL2001] Mills’ states that TCP Connection time
request to the Web server (nonpersistent
(from the Reference Model) is an excellent estimate
connections).
of the round-trip latency between a browser and
3. Page content is downloaded using up to four Web server. The reason for this is that (a) the SYN
parallel threads. and SYN/ACK packets used in the 3-way handshake
4. Agents connect to the Internet using high-speed are relatively small, (b) under normal load, very little
networks (T1 or T3 lines). delay occurs within the server's TCP software, and
(c) no server applications software is executed.
Figure 8 summarizes the measurement data used in Figure 9 compares the three data sources for
this study. The sample values are those measured network latency.
by the Seattle Exodus agent accessing
www.hyperformix.com/index.html. Average Minimum Maximum
ping 93 90 110
Developing Model Input Parameters traceroute 96 90 100
Connection Setup Time 101.4 90 190
The measurement source described in the previous
three sections can be used to develop values for the Figure 9. Network Latency (milliseconds)
performance model’s input parameters. This section
describes the techniques used to determine values CLIENT_ENABLE_BROWSER_CACHE. The
for the parameters listed in Figure 3. modeled browser has a switch that controls caching
of objects within the browser. This option was
CLIENT_1_BANDWIDTH. For each browser, the disabled because (a) we were representing single
model requires a value for the average bandwidth page requests, and (b) no duplicate content

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

elements were contained in the page description file

(see Figure 4). WEBSVR_PAGE_SERVICETIME. Page service
time is the average amount of time required by the
CLIENT_PERSISTENT_TCP_CONNECTION. Web server to fetch and prepare the base page for

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
Another browser parameter controls whether it is return to the browser. We initially assume equal
using HTTP 1.0 (non-persistent connection) or HTTP values for page and object service time.
1.1 (persistent connection) to communicate with the
Web server. The measured response times for our The page and object service times are verified and
validation targets were taken from agents using non- refined as part of the model calibration. Model
persistent connections; thus we set the modeled calibration will be performed as the first step of
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

browsers to use non-persistent connections. model validation, which is discussed in the next
section.
DNS_LOOKUP_SERVICETIME. DNS Lookup is a
difficult metric to measure without an instrumented
browser or a network packet analyzer. For our
5 Validation of the Approach
model we used DNS Time as reported by the Proving that we have met our goal to develop a
measurement agent. As an alternative, we suggest reusable and efficient modeling and measurement
using industry standard values for DNS Lookup process requires that we demonstrate the following:
since it is such a small rf action (on average, less
than 1%) of the overall end user response time. In 1. Model inputs can be developed quickly using a
[MILL2001] Mills summarizes measurements from repeatable methodology.
40 leading eBusiness sites during January 2001. 2. The performance model accurately predicts Web
Industry average times are shown in Figure 10. site response times.
Average Median Standard We validate our approach by first measuring and
Deviation
modeling the HyPerformix, Inc. home page and
DNS Lookup 0.02 0.01 0.03
validating the accuracy of the model by comparing
Response Time 2.28 1.81 1.78
predicted response times to measured times. Then,
DNS Lookup Pct 0.87% 0.55%
we repeat the process on two other Web sites (that
offer static content) to show that we can quickly build
Figure 10. KB40 DNS Lookup Times
models to accurately predict the response times of
MAX_BROWSER_CONNECTIONS. This those sites.
parameter limits the number of parallel threads the
The baseline model described in Section 3 was used
browser may use for retrieval of embedded page
to predict the response times for all three Web sites.
content. The agents used to take response time
For each site we collected (remote) measurements
validation measurements emulated a browser with
describing the Web page content, user connectivity,
four threads; thus we set the thread limit of our
and the performance of the application. These
modeled browsers to four.
measurements were developed into model input
WEBSVR_OBJECT_SERVICETIME. Object parameters as described earlier. Then, we used
service time is the average amount of time required response time measurements from four remote
by the Web server to fetch and prepare an object for measurement locations to validate each model.
return to the browser. Measurements collected
directly from the Web server are the best data HyPerformix Home Page
source. In this study we used a technique to The first validation test was performed against the
estimate service time for both a page and object home page for HyPerformix, Inc. Approximately 900
request that proved to work quite well. measurements were collected from each remote
measurement agent. The home page’s base HTML
The measurement agents collect and report a metric
is approximately 44Kbytes and contains 33 unique
called Time to First Byte. This value represents the graphic images totaling 39Kbytes. The Web server
time between the completion of TCP Connection
service times were refined during the initial
and the browser’s receipt of the first HTML packet
calibration of the model. These refined values were
containing the requested page. Service time can be then used for the three remaining validation tests for
estimated from this metric by subtracting out the
this site. The results of the validation are shown in
average round-trip latency between the browser and
Figure 11.
Web server.

Since we are using an indirect estimate of service

time, this value may need to be refined during model
validation.

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
Amsterdam UUNET Boston UUNET
Measured Modeled Delta Measured Modeled Delta
Base Page Download 1.06 1.03 3.1% 0.60 0.63 5.3%
Content Download 2.73 2.90 6.1% 1.52 1.61 5.9%
Response Time 4.02 4.16 3.5% 2.19 2.31 5.4%
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

Dallas Level3 Seattle Exodus

Measured Modeled Delta Measured Modeled Delta
Base Page Download 0.32 0.39 20.0% 0.82 0.81 0.7%
Content Download 0.74 0.65 12.2% 2.06 2.26 9.8%
Response Time 1.09 1.07 1.8% 2.98 3.17 6.5%

Figure 11. Validation Results - HyPerformix Home Page

Dallas MH1 Seattle Exodus

Measured Modeled Delta Measured Modeled Delta
Base Page Download 0.64 0.66 2.6% 0.84 0.86 2.5%
Content Download 1.61 1.64 1.8% 2.11 2.18 3.2%
Response Time 2.32 2.37 1.9% 3.05 3.13 2.6%

Seattle Genuity Washington D.C. Qwest

Measured Modeled Delta Measured Modeled Delta
Base Page Download 1.08 1.11 2.8% 0.44 0.46 2.7%
Content Download 2.44 2.71 11.1% 0.99 1.00 0.3%
Response Time 3.65 3.93 7.6% 1.46 1.48 1.2%

Figure 12. Validation Results - Technology Company Home Page

Boston Exodus Denver MH1

Measured Modeled Delta Measured Modeled Delta
Base Page Download 0.42 0.40 6.0% 0.30 0.33 8.1%
Content Download 0.88 0.82 7.0% 0.63 0.60 5.0%
Response Time 1.50 1.42 5.5% 1.06 1.05 0.7%

Houston MH1 Los Angeles Qwest

Measured Modeled Delta Measured Modeled Delta
Base Page Download 0.29 0.32 7.9% 0.19 0.24 26.8%
Content Download 0.59 0.56 5.5% 0.31 0.27 13.4%
Response Time 1.02 1.01 1.0% 0.54 0.55 1.8%

Figure 13. Validation Results - TPC Home Page

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

Technology Company Home Page 2. Browser Configurations. The baseline model

The second validation test was performed using the was defined to mimic the behavior of a browser

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
home page of a technology company. Remote capable of using 4 parallel threads for retrieval of
agents were again used as the measurement embedded content. The model parameter
source. The company’s base page is approximately MAX_BROWSER_CONNECTIONS may be
28Kbytes and contains about 30 unique graphic varied to represent Internet Explorer (2 threads)
images totaling 42Kbytes. The measurements or Netscape Navigator (up to 6 threads) (see
collected from this site were used directly in the [WANG1998]).
model; no refinement of service times was required.
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

The results of the validation are shown in Figure 12. 3. Web Server Configurations. The model uses
non-persistent TCP connections for retrieval of
TPC Home Page Web pages and content elements. Persistent
The final validation test used the home page for the connections are supported in HTTP 1.1; however
Transaction Processing Performance Council (TPC) there are Web sites that still use HTTP 1.0 due to
at www.tpc.org. Approximately 600 measurements concerns about concurrent connection resource
were collected from each remote measurement usage. The model parameter
agent. The TPC home page’s base HTML is CLIENT_PERSISTENT_TCP_CONNECTION
approximately 14Kbytes with 12 unique graphic can be used to study user response time
images totaling 20Kbytes. The Web server service improvement with persistent connections.
times were refined during the initial validation test.
These refined values were then used for the 4. Page Design. The model supports the ability to
remaining three tests for this site. Figure 13 shows vary page composition (the number and size of
the results of the validation from the four remote the page’s content elements) to predict response
measurement agents. time improvements expected at each remote user
location. Page design changes would be made
Validation Summary to the page description file shown in Figure 4.
The modeled results from each validation test
compared favorably with measured values: modeled 5. Baseline Model. Most importantly, the model is
response times were within 10% of measured. a starting point for analyzing changes to the
Furthermore, the same methodology was used to application(s) behind the firewall. Drilling down to
develop model input parameters for each validation add detail to the model to represent more
test. complex backend processing requires additional
data collection and workload characterization.
These results demonstrate that our model is The high-level model presented in this paper is a
sufficiently accurate to proceed with the evaluation starting point; it contains sufficient detail to study
of optimization alternatives. a number of design and connectivity questions.

6 Optimization Alternatives 7 Summary

The simulation model described above is This paper has demonstrated modeling techniques
appropriate for analyzing Web page response time and a measurement methodology that can quickly
from the browser’s point of view. There is sufficient be applied to any Web site that delivers static
detail in the model to predict response times for content. The resulting model is a tool that can be
potential changes to network connectivity, page used to analyze a set of optimization alternatives,
design and browser configurations. and it serves as the baseline for a more detailed
performance analysis of Web applications.
1. Content Delivery Networks. The source of
static page elements can be moved from the The primary value of this approach is a reusable
baseline Web site (HPFX_Web_Services) to a methodology that provides the capability to model
candidate CDN that provides better network any Web application in just a few minutes. The key
connectivity to the requesting users. To do this, factors for the rapid application of this approach
the new CDN site would have to be added to the include a common language for specifying the
model with the appropriate network connectivity performance of a Web application, a reusable
parameters (latency and bandwidth). The page modeling technique, and a repeatable measurement
description file shown in Figure 4 would then methodology. The Reference Model guides the
have to be modified to identify the new source of structure of the simulation model and identifies the
the relocated static content. performance metrics required.

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

We developed a simulation model that characterizes 9 References

Web site performance using the same components
specified by the Reference Model. The model is an [COME1995] Douglas E. Comer, “Internetworking
implementation of this natural framework for with TCP/IP Volume 1”, Prentice Hall, 1995.

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
representing response times and therefore can be
easily parameterized to represent any Web [DOMA1997] Bernard Domanski and James J.
application. McGalliard, “Sizing the Server(s) but Not Knowing
Enough: Open System Performance Modeling”,
The greatest challenge comes in collecting the CMG 1997 Proceedings.
metrics needed to develop the model’s input
[HYPR2001] For information about Strategizer from
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

parameters. Static page and object definitions can

be gathered easily. Network connectivity of a single HyPerformix, see
browser can be determined with commonly available http://www.hyperformix.com/products/products.htm.
utilities, but collecting this information from
numerous sites is physically difficult. Browser [KEYN2001] For information about the Keynote
response times needed for validation must also be Perspective service, see
collected from a number of remote sites; and again, http://www.keynote.com/services/index.html.
this is physically difficult without a geographically
distributed measurement service. Estimation of [LOOS2000] Chris Loosley, Richard L. Gimarc, and
server processing times has been simplified. Before Amy C. Spellmann, “E -Commerce Response Time:
any scalability analysis can be performed, these A Reference Model”, CMG 2000 Proceedings.
service times must be decomposed into CPU, disk,
memory, and network components. [MENA2000] Daniel A. Menascé and Virgilio A. F.
Almeida, “Scaling for E-Business: Technologies,
This paper demonstrated how to quickly model a Models, Performance, and Capacity Planning”,
Web application from the end user (browser) Prentice Hall PTR 2000.
perspective. The described methodology is
appropriate for evaluating optimization alternatives [MICR2001] For information about the Images List
for Web page design and Internet connectivity. The Utility which is a part of Microsoft’s Web Accessories
resulting model serves as a starting point to evaluate for Internet Explorer 5, see
server-based optimization alternatives. http://www.microsoft.com/windows/Ie/default.htm.

[MILL2001] Patrick Mills and Chris Loosley, “A

8 Acknowledgements Performance Analysis of 40 E-business Sites”, CMG
The authors would like to acknowledge Chris Journal of Computer Resource Management, Issue
Loosley for his advise, support, and encouragement #102, 2001.
during the preparation of this paper.
[NEUS1998] Douglas M. Neuse, “Why Simulate?”,
Capacity Management Review, February 1998.

[WANG1998] Zhe Wang and Pei Cao, “Persistent

Connection Behavior of Popular Browsers,”
http://www.cs.wisc.edu/~cao/papers/persistent-
connection.html (1998).

Find a CMG regional meeting near you at www.cmg.org/regions