How Web Servers Work

by Marshall Brain

 Print
 Cite
 Feedback

Share

 Recommend

Cite This!
Close 
Please copy/paste the following text to properly cite this HowStuffWorks article:

Brain, Marshall.  "How Web Servers Work"  01 April 2000.  HowStuffWorks.com.

<http://computer.howstuffworks.com/web-server.htm>  18 September 2010.

Inside this Article

1. Introduction to How Web Servers Work

2. The Basic Process
3. Behind the Scenes

4. The Internet
5. Clients and Servers
6. IP Addresses
7. See more »
HTTP

 More Tech Videos »

The Internet
So what is "the Internet"? The Internet is a gigantic collection of millions of computers, all
linked together on a computer network. The network allows all of the computers to
communicate with one another. A home computer may be linked to the Internet using a
phone-line modem, DSL or cable modem that talks to an Internet service provider (ISP). A
computer in a business or university will usually have a network interface card (NIC) that
directly connects it to a local area network (LAN) inside the business. The business can then
connect its LAN to an ISP using a high-speed phone line like a T1 line. A T1 line can handle
approximately 1.5 million bits per second, while a normal phone line using a modem can
typically handle 30,000 to 50,000 bits per second.

ISPs then connect to larger ISPs, and the largest ISPs maintain fiber-optic "backbones" for an
entire nation or region. Backbones around the world are connected through fiber-optic lines,
undersea cables or satellite links (see An Atlas of Cyberspaces for some interesting backbone
maps). In this way, every computer on the Internet is connected to every other computer on
the Internet.

Clients and Servers

In general, all of the machines on the Internet can be categorized as two types: servers and
clients. Those machines that provide services (like Web servers or FTP servers) to other
machines are servers. And the machines that are used to connect to those services are clients.
When you connect to Yahoo! at www.yahoo.com to read a page, Yahoo! is providing a
machine (probably a cluster of very large machines), for use on the Internet, to service your
request. Yahoo! is providing a server. Your machine, on the other hand, is probably providing
no services to anyone else on the Internet. Therefore, it is a user machine, also known as a
client. It is possible and common for a machine to be both a server and a client, but for our
purposes here you can think of most machines as one or the other.

A server machine may provide one or more services on the Internet. For example, a server
machine might have software running on it that allows it to act as a Web server, an e-mail
server and an FTP server. Clients that come to a server machine do so with a specific intent,
so clients direct their requests to a specific software server running on the overall server
machine. For example, if you are running a Web browser on your machine, it will most likely
want to talk to the Web server on the server machine. Your Telnet application will want to
talk to the Telnet server, your e-mail application will talk to the e-mail server, and so on...

IP Addresses
To keep all of these machines straight, each machine on the Internet is assigned a unique
address called an IP address. IP stands for Internet protocol, and these addresses are 32-bit
numbers, normally expressed as four "octets" in a "dotted decimal number." A typical IP
address looks like this:

216.27.61.137

The four numbers in an IP address are called octets because they can have values between 0
and 255, which is 2 possibilities per octet.
8

Every machine on the Internet has a unique IP address. A server has a static IP address that
does not change very often. A home machine that is dialing up through a modem often has an
IP address that is assigned by the ISP when the machine dials in. That IP address is unique
for that session -- it may be different the next time the machine dials in. This way, an ISP
only needs one IP address for each modem it supports, rather than for each customer.

If you are working on a Windows machine, you can view a lot of the Internet information for
your machine, including your current IP address and hostname, with the command
WINIPCFG.EXE (IPCONFIG.EXE for Windows 2000/XP). On a UNIX machine, type
nslookup at the command prompt, along with a machine name, like
www.howstuffworks.com -- e.g. "nslookup www.howstuffworks.com" -- to display the IP
address of the machine, and you can use the command hostname to learn the name of your
machine. (For more information on IP addresses, see IANA.)

As far as the Internet's machines are concerned, an IP address is all you need to talk to a
server. For example, in your browser, you can type the URL http://209.116.69.66 and arrive
at the machine that contains the Web server for HowStuffWorks. On some servers, the IP
address alone is not sufficient, but on most large servers it is -- keep reading for details.

Domain Names
Because most people have trouble remembering the strings of numbers that make up IP
addresses, and because IP addresses sometimes need to change, all servers on the Internet
also have human-readable names, called domain names. For example,
www.howstuffworks.com is a permanent, human-readable name. It is easier for most of us to
remember www.howstuffworks.com than it is to remember 209.116.69.66.

The name www.howstuffworks.com actually has three parts:

1. The host name ("www")

2. The domain name ("howstuffworks")
3. The top-level domain name ("com")
Domain names within the ".com" domain are managed by the registrar called VeriSign.
VeriSign also manages ".net" domain names. Other registrars (like RegistryPro, NeuLevel
and Public Interest Registry) manage the other domains (like .pro, .biz and .org). VeriSign
creates the top-level domain names and guarantees that all names within a top-level domain
are unique. VeriSign also maintains contact information for each site and runs the "whois"
database. The host name is created by the company hosting the domain. "www" is a very
common host name, but many places now either omit it or replace it with a different host
name that indicates a specific area of the site. For example, in encarta.msn.com, the domain
name for Microsoft's Encarta encyclopedia, "encarta" is designated as the host name instead
of "www."

Name Servers
The whois Command
On a UNIX machine, you can use the whois command to look up information about a domain name.
You can do the same thing using the whois form at VeriSign. If you type in a domain name, like
"howstuffworks.com," it will return to you the registration information for that domain, including its IP
address.

A set of servers called domain name servers (DNS) maps the human-readable names to the IP
addresses. These servers are simple databases that map names to IP addresses, and they are
distributed all over the Internet. Most individual companies, ISPs and universities maintain
small name servers to map host names to IP addresses. There are also central name servers
that use data supplied by VeriSign to map domain names to IP addresses.

If you type the URL "http://www.howstuffworks.com/web-server.htm" into your browser,

your browser extracts the name "www.howstuffworks.com," passes it to a domain name
server, and the domain name server returns the correct IP address for
www.howstuffworks.com. A number of name servers may be involved to get the right IP
address. For example, in the case of www.howstuffworks.com, the name server for the "com"
top-level domain will know the IP address for the name server that knows host names, and a
separate query to that name server, operated by the HowStuffWorks ISP, may deliver the
actual IP address for the HowStuffWorks server machine.

On a UNIX machine, you can access the same service using the nslookup command. Simply
type a name like "www.howstuffworks.com" into the command line, and the command will
query the name servers and deliver the corresponding IP address to you.

So here it is: The Internet is made up of millions of machines, each with a unique IP address.
Many of these machines are server machines, meaning that they provide services to other
machines on the Internet. You have heard of many of these servers: e-mail servers, Web
servers, FTP servers, Gopher servers and Telnet servers, to name a few. All of these are
provided by server machines.

Ports
Any server machine makes its services available to the Internet using numbered ports, one
for each service that is available on the server. For example, if a server machine is running a
Web server and an FTP server, the Web server would typically be available on port 80, and
the FTP server would be available on port 21. Clients connect to a service at a specific IP
address and on a specific port.

Each of the most well-known services is available at a well-known port number. Here are
some common port numbers:

 echo 7
 daytime 13
 qotd 17 (Quote of the Day)
 ftp 21
 telnet 23
 smtp 25 (Simple Mail Transfer, meaning e-mail)
 time 37
 nameserver 53
 nicname 43 (Who Is)
 gopher 70
 finger 79
 WWW 80

If the server machine accepts connections on a port from the outside world, and if a firewall
is not protecting the port, you can connect to the port from anywhere on the Internet and use
the service. Note that there is nothing that forces, for example, a Web server to be on port 80.
If you were to set up your own machine and load Web server software on it, you could put
the Web server on port 918, or any other unused port, if you wanted to. Then, if your machine
were known as xxx.yyy.com, someone on the Internet could connect to your server with the
URL http://xxx.yyy.com:918. The ":918" explicitly specifies the port number, and would
have to be included for someone to reach your server. When no port is specified, the browser
simply assumes that the server is using the well-known port 80.

Protocols
Once a client has connected to a service on a particular port, it accesses the service using a
specific protocol. The protocol is the pre-defined way that someone who wants to use a
service talks with that service. The "someone" could be a person, but more often it is a
computer program like a Web browser. Protocols are often text, and simply describe how the
client and server will have their conversation.

Perhaps the simplest protocol is the daytime protocol. If you connect to port 13 on a
machine that supports a daytime server, the server will send you its impression of the current
date and time and then close the connection. The protocol is, "If you connect to me, I will
send you the date and time and then disconnect." Most UNIX machines support this server. If
you would like to try it out, you can connect to one with the Telnet application. In UNIX, the
session would look like this:

%telnet web67.ntx.net 13
Trying 216.27.61.137...
Connected to web67.ntx.net.
Escape character is '^]'.
Sun Oct 25 08:34:06 1998
Connection closed by foreign host.

On a Windows machine, you can access this server by typing "telnet web67.ntx.net 13" at the
MSDOS prompt.

In this example, web67.ntx.net is the server's UNIX machine, and 13 is the port number for
the daytime service. The Telnet application connects to port 13 (telnet naturally connects to
port 23, but you can direct it to connect to any port), then the server sends the date and time
and disconnects. Most versions of Telnet allow you to specify a port number, so you can try
this using whatever version of Telnet you have available on your machine.

Most protocols are more involved than daytime and are specified in Request for Comment
(RFC) documents that are publicly available (see http://sunsite.auc.dk/RFC/ for a nice archive
of all RFCs). Every Web server on the Internet conforms to the HTTP protocol, summarized
nicely in The Original HTTP as defined in 1991. The most basic form of the protocol
understood by an HTTP server involves just one command: GET. If you connect to a server
that understands the HTTP protocol and tell it to "GET filename," the server will respond by
sending you the contents of the named file and then disconnecting. Here's a typical session:

%telnet www.howstuffworks.com 80
Trying 216.27.61.137...
Connected to howstuffworks.com.
Escape character is '^]'.
GET http://www.howstuffworks.com/

...
Connection closed by foreign host.

In the original HTTP protocol, all you would have sent was the actual filename, such as "/" or
"/web-server.htm." The protocol was later modified to handle the sending of the complete
URL. This has allowed companies that host virtual domains, where many domains live on a
single machine, to use one IP address for all of the domains they host. It turns out that
hundreds of domains are hosted on 209.116.69.66 -- the HowStuffWorks IP address.

Putting It All Together

Now you know a tremendous amount about the Internet. You know that when you type a
URL into a browser, the following steps occur:

 The browser breaks the URL into three parts:

1. The protocol ("http")
2. The server name ("www.howstuffworks.com")
3. The file name ("web-server.htm")
 The browser communicates with a name server to translate the server name,
"www.howstuffworks.com," into an IP address, which it uses to connect to that
server machine.
 The browser then forms a connection to the Web server at that IP address on port 80.
  Following the HTTP protocol, the browser sends a GET request to the server, asking
for the file "http://www.howstuffworks.com/web-server.htm." (Note that cookies may
be sent from browser to server with the GET request -- see How Internet Cookies
Work for details.)
 The server sends the HTML text for the Web page to the browser. (Cookies may also
be sent from server to browser in the header for the page.)
 The browser reads the HTML tags and formats the page onto your screen.

The Origin of Modems

The word "modem" is a contraction of the words modulator-demodulator. A modem is
typically used to send digital data over a phone line.

The sending modem modulates the data into a signal that is compatible with the phone line,
and the receiving modem demodulates the signal back into digital data. Wireless modems
convert digital data into radio signals and back.

Modems came into existence in the 1960s as a way to allow terminals to connect to
computers over the phone lines. A typical arrangement is shown below:

In a configuration like this, a dumb terminal at an off-site office or store could "dial in" to a
large, central computer. The 1960s were the age of time-shared computers, so a business
would often buy computer time from a time-share facility and connect to it via a 300-bit-per-
second (bps) modem.

A dumb terminal is simply a keyboard and a screen. A very common dumb terminal at the
time was called the DEC VT-100, and it became a standard of the day (now memorialized in
terminal emulators worldwide). The VT-100 could display 25 lines of 80 characters each.
When the user typed a character on the terminal, the modem sent the ASCII code for the
character to the computer. The computer then sent the character back to the computer so it
would appear on the screen.

When personal computers started appearing in the late 1970s, bulletin board systems (BBS)
became the rage. A person would set up a computer with a modem or two and some BBS
software, and other people would dial in to connect to the bulletin board. The users would run
terminal emulators on their computers to emulate a dumb terminal.

People got along at 300 bps for quite a while. The reason this speed was tolerable was
because 300 bps represents about 30 characters per second, which is a lot more characters per
second than a person can type or read. Once people started transferring large programs and
images to and from bulletin board systems, however, 300 bps became intolerable. Modem
speeds went through a series of steps at approximately two-year intervals:

 300 bps - 1960s through 1983 or so

 1200 bps - Gained popularity in 1984 and 1985
 2400 bps
 9600 bps - First appeared in late 1990 and early 1991
 19.2 kilobits per second (Kbps)
 28.8 Kbps
 33.6 Kbps
 56 Kbps - Became the standard in 1998
 ADSL, with theoretical maximum of up to 8 megabits per second (Mbps) - Gained
popularity in 1999

300-bps Modems
We'll use 300-bps modems as a starting point because they are extremely easy to understand.
A 300-bps modem is a device that uses frequency shift keying (FSK) to transmit digital
information over a telephone line. In frequency shift keying, a different tone (frequency) is
used for the different bits (see How Guitars Work for a discussion of tones and frequencies).

When a terminal's modem dials a computer's modem, the terminal's modem is called the
originate modem. It transmits a 1,070-hertz tone for a 0 and a 1,270-hertz tone for a 1. The
computer's modem is called the answer modem, and it transmits a 2,025-hertz tone for a 0
and a 2,225-hertz tone for a 1. Because the originate and answer modems transmit different
tones, they can use the line simultaneously. This is known as full-duplex operation. Modems
that can transmit in only one direction at a time are known as half-duplex modems, and they
are rare.

Let's say that two 300-bps modems are connected, and the user at the terminal types the letter
"a." The ASCII code for this letter is 97 decimal or 01100001 binary (see How Bits and Bytes
Work for details on binary). A device inside the terminal called a UART (universal
asynchronous receiver/transmitter) converts the byte into its bits and sends them out one at a
time through the terminal's RS-232 port (also known as a serial port). The terminal's modem
is connected to the RS-232 port, so it receives the bits one at a time and its job is to send them
over the phone line.

Faster Modems
In order to create faster modems, modem designers had to use techniques far more
sophisticated than frequency-shift keying. First they moved to phase-shift keying (PSK), and
then quadrature amplitude modulation (QAM). These techniques allow an incredible
amount of information to be crammed into the 3,000 hertz of bandwidth available on a
normal voice-grade phone line. 56K modems, which actually connect at something like 48
Kbps on anything but absolutely perfect lines, are about the limit of these techniques (see the
links at the end of this article for more information).
Here's a look inside a typical 56K modem:

All of these high-speed modems incorporate a concept of gradual degradation, meaning

they can test the phone line and fall back to slower speeds if the line cannot handle the
modem's fastest speed.

The next step in the evolution of the modem was asymmetric digital subscriber line
(ADSL) modems. The word asymmetric is used because these modems send data faster in
one direction than they do in another. An ADSL modem takes advantage of the fact that any
normal home, apartment or office has a dedicated copper wire running between it and phone
company's nearest mux or central office. This dedicated copper wire can carry far more data
than the 3,000-hertz signal needed for your phone's voice channel. If both the phone
company's central office and your house are equipped with an ADSL modem on your line,
then the section of copper wire between your house and the phone company can act as a
purely digital high-speed transmission channel. The capacity is something like 1 million bits
per second (Mbps) between the home and the phone company (upstream) and 8 Mbps
between the phone company and the home (downstream) under ideal conditions. The same
line can transmit both a phone conversation and the digital data.

The approach an ADSL modem takes is very simple in principle. The phone line's bandwidth
between 24,000 hertz and 1,100,000 hertz is divided into 4,000-hertz bands, and a virtual
modem is assigned to each band. Each of these 249 virtual modems tests its band and does
the best it can with the slice of bandwidth it is allocated. The aggregate of the 249 virtual
modems is the total speed of the pipe.

Point-to-Point Protocol
Today, no one uses dumb terminals or terminal emulators to connect to an individual
computer. Instead, we use our modems to connect to an Internet service provider (ISP), and
the ISP connects us into the Internet. The Internet lets us connect to any machine in the world
(see How Web Servers and the Internet Work for details). Because of the relationship
between your computer, the ISP and the Internet, it is no longer appropriate to send
individual characters. Instead, your modem is routing TCP/IP packets between you and your
ISP.

The standard technique for routing these packets through your modem is called the Point-to-
Point Protocol (PPP). The basic idea is simple -- your computer's TCP/IP stack forms its
TCP/IP datagrams normally, but then the datagrams are handed to the modem for
transmission. The ISP receives each datagram and routes it appropriately onto the Internet.
The same process occurs to get data from the ISP to your computer. See this page for
additional information on PPP.

If you want to know more about modems, protocols, and especially if you wish to delve into
things like PSK and QAM in more detail, check out the links on the next page!

Extras: Security
You can see from this description that a Web server can be a pretty simple piece of software.
It takes the file name sent in with the GET command, retrieves that file and sends it down the
wire to the browser. Even if you take into account all of the code to handle the ports and port
connections, you could easily create a C program that implements a simple Web server in less
than 500 lines of code. Obviously, a full-blown enterprise-level Web server is more involved,
but the basics are very simple.

Most servers add some level of security to the serving process. For example, if you have ever
gone to a Web page and had the browser pop up a dialog box asking for your name and
password, you have encountered a password-protected page. The server lets the owner of the
page maintain a list of names and passwords for those people who are allowed to access the
page; the server lets only those people who know the proper password see the page. More
advanced servers add further security to allow an encrypted connection between server and
browser, so that sensitive information like credit card numbers can be sent on the Internet.

That's really all there is to a Web server that delivers standard, static pages. Static pages are
those that do not change unless the creator edits the page.

Extras: Dynamic Pages

But what about the Web pages that are dynamic? For example:

 Any guest book allows you to enter a message in an HTML form, and the next time
the guest book is viewed, the page will contain the new entry.
 The whois form at Network Solutions allows you to enter a domain name on a form,
and the page returned is different depending on the domain name entered.
 Any search engine lets you enter keywords on an HTML form, and then it
dynamically creates a page based on the keywords you enter.

In all of these cases, the Web server is not simply "looking up a file." It is actually processing
information and generating a page based on the specifics of the query. In almost all cases, the
Web server is using something called CGI scripts to accomplish this feat. CGI scripts are a
topic unto themselves, and are described in the HowStuffWorks article How CGI Scripting
Work.

For more information on Web servers and related topics, check out the links on the next page.