1.

SCALABLE COMPUTING OVER THE INTERNET

Scalable Computing over the Internet

 1.1 The Age of Internet Computing

 1.2 Scalable Computing Trends and New Paradigms
 1.3 The Internet of Things and Cyber-Physical Systems

2
SCALABLE COMPUTING OVER THE INTERNET

 Scalability is the ability of a computer application or product

(hardware or software) to continue to function well when it is changed
in size or volume in order to meet a user need.

 Instead of using a centralized computer to solve computational

problems, a parallel and distributed computing system uses
multiple computers to solve large-scale problems over the Internet.

 Thus, distributed computing becomes data-intensive and network-

centric.
SCALABLE COMPUTING OVER THE INTERNET

1.1. The Age of Internet Computing

HTC Example:
Distributed Computing

High-performance computing (HPC) For example, if you have to process

 Applications is no longer optimal for 100 video clips, and each one
takes ~1 hr, then you would
 measuring system performance need ~100 hrs of computing
time on your laptop
High-throughput computing (HTC) systems
 Built with parallel and distributed computing However, if you had 100 laptops, you
could theoretically do the task
technologies in 1 hr assuming that you
could instantly command each
 Upgrade data centers using fast servers, one to begin the processing
task
storage systems, and high-bandwidth
networks.

3/112
SCALABLE COMPUTING OVER THE INTERNET

Computer technology has gone through five generations of development

1950 to 1970: Mainframes, including the IBM 360 and CDC 6400, were built to satisfy the demands
of large businesses and government organizations.

1960 to 1980: lower-cost mini computers such as the DEC PDP 11 and VAX Series became popular
among small businesses and on college campuses

1970 to 1990: widespread use of personal computers built with VLSI microprocessors

1980 to 2000: Massive numbers of portable computers and pervasive devices appeared in
both wired and wireless applications

Since1990: The use of both HPC and HTC systems hidden in clusters, grids, or Internet clouds has
proliferated.HTC and HPC are Computational Grids

4/112
 On the HPC side
 supercomputers (massively parallel processors or MPPs) are gradually replaced by clusters of
cooperative computers to share computing resources.
 The cluster is often a collection of homogeneous compute nodes that are physically connected in
close range to one another.

 On the HTC side:

Peer-to-peer (P2P) networks are
formed for distributed file sharing and
content delivery applications.

 A P2P system is built over many

client machines. Peer machines are
globally distributed in nature.

 P2P, cloud computing, and web

service platforms are more focused
on HTC applications than on HPC
applications.

 Clustering and P2P technologies

lead to the development of
computational grids or data grids

Figure: Evolutionary trend toward parallel, distributed, and cloud

5/112
1.2 Scalable Computing Trends and New Paradigms

 Radio-frequency identification (RFID)

Global Positioning System (GPS)

 Sensor technologies has triggered the development of the

Internet of Things (IoT)

6/112
 WHAT ARE COMPUTING PARADIGMS?
Distributed Computing is the opposite of centralized computing.
Parallel computing overlaps with distributed computing to a great extent.
Cloud computing overlaps with distributed, centralized, and parallel computing

Centralized computing

This is a computing paradigm by which all

computer resources are centralized in
one physical system.

All resources (processors, memory, and

storage) are fully shared and tightly
coupled within one integrated OS.
Parallel computing

Many data centers and supercomputers are In parallel computing, all processors are either tightly
centralized systems, but they are used in coupled with centralized shared memory or loosely
parallel, distributed, and cloud computing coupled with distributed memory. Inter processor
applications communication is accomplished through shared memory
or via message

7/112
 Distributed Computing is the opposite of centralized computing.
Parallel computing overlaps with distributed computing to a great extent.
Cloud computing overlaps with distributed, centralized, and parallel computing

Distributed Computing:
Group of computers working together as to
appear as a single computer to the end users

A distributed system consists of multiple

autonomous computers, each having its own private
memory, communicating through a computer
network. Information exchange in a distributed
system is accomplished through message passing

Example: ATM Machines, Wireless Ad-hoc Networks

(WANET), Internet, Intranet Fault Tolerance means When one or
some nodes fails the whole system
can still work fine except performance
Properties: Fault Tolerance, Load Sharing, Resource
sharing

8/112
1.2 Scalable Computing Trends and New Paradigms

CLOUD COMPUTING:
 Cloud Computing o An Internet cloud of resources can be either
a centralized or a distributed computing
 The cloud applies parallel or distributed computing, or both.

 Clouds can be built with physical or virtualized resources over

large data centers that are centralized or distributed.
 It is a form of utility computing or service computing. stributed
computing system.

9/112
Distributed Vs Cluster Computing

 1. Distributed refers to splitting a business into different sub-

services and distributing them on different machines.

 2. Cluster refers to a group of servers that are grouped

together to achieve the same business and can be considered
as one computer .

10/112
SCALABLE COMPUTING TRENDS AND NEW PARADIGMS
1.2.1.Innovative Applications of HTC and HPC Systems

11/112
1.2.2.TREND TOWARDS UTILITY COMPUTING

Utility Computing is Better than Cloud Computing

The vision of computer utilities in modern distributed computing

systems.
12/112
The Hype Cycle of New
Technologies

13/112
1.3.THE INTERNET OF THINGS AND CYBER-PHYSICAL
SYSTEMS

 Internet of Things is about connecting "Things"

(Objects and Machines) to the internet and eventually to each
other;

 The Internet of Things (IoT) is a networked connection of

everyday objects including computers, sensors, humans, etc.

 Cyber Physical Systems (CPS) are integration of computation,

networking and physical process.

14/112
2.TECHNOLOGIES FOR NETWORK-BASED SYSTEMS
Technologies for Network-Based Systems

 2.1 Multicore CPUs and Multithreading Technologies

 2.2 GPU Computing to Exascale and Beyond
 2.3 Memory, Storage, and Wide-Area Networking
 2.4 Virtual Machines and Virtualization Middleware
 2.5Data Center Virtualization for Cloud Computing

15/112
MULTICORE CPU Vs MULTITHREADING

 Multicore refers to a computer or processor that has more

than one logical CPU core, and that can physically execute
multiple instructions at the same time.

 Multithreading refers to a program that can running more

than one core at the same time.

16/112
2.1 MULTICORE CPUS AND MULTITHREADING TECHNOLOGIES

MULTITHREADING
TECHNOLOGY

Five independent threads of

instructions to four pipelined data
paths in each of the following five
processor categories

Multicore refers to a computer

or processor that has more than
one logical CPU core, and that can
physically execute multiple
instructions at the same time.

17/112
2.2 GPU COMPUTING TO EXASCALE AND BEYOND

 Graphics Processing Unit-GPU was marketed by NVIDIA in 1999

 It is a graphics coprocessor or accelerator mounted on a computer’s

graphics card or video card.

 Unlike CPUs, GPUs have a throughput architecture that exploits massive

parallelism by executing many concurrent threads slowly, instead of
executing a single long thread in a conventional microprocessor very
quickly.

 For example, the Xeon X5670 CPU has six cores. However, a modern
GPU chip can be built with hundreds of processing cores.
18/112
GPU COMPUTING

A CPU consists of four to eight

CPU cores, while the GPU consists
of hundreds of smaller cores.

The NVIDIA, CUDA, Tesla is used in

GPU clusters or in High Processing
Systems(HPS) for parallel processing
of massive floating-pointing data.

19/112
GPU Programming Model

Four challenges are identified for exascale computing:

(1) Energy and power,

(2) Memory and storage,
(3) Concurrency and locality
(4) System resiliency (system elasticity)

20/112
USE OF GPU WITH CPU

 The CPU is the conventional multicore processor with limited parallelism

to exploit.

 The GPU has a many-core architecture that has hundreds of simple

processing cores organized as multiprocessors.

 Each core can have one or more threads.

 The interaction between a CPU and

GPU in performing parallel execution
of floating-point operations
concurrently

21/112
2.3.MEMORY STORAGE AND WIDE AREA NETWORKS

LAN typically is used to connect client hosts

to big servers

A storage area network (SAN) connects

servers to network storage such as disk
arrays

Network attached storage (NAS) connects

client hosts directly to the disk arrays.

22/112
2.4 VIRTUAL MACHINES AND VIRTUALIZATION MIDDLEWARE

How Virtual Machines Work:

Hypervisor (Virtual Machine Monitor - VMM)
The hypervisor is the software that creates and manages VMs.
It allocates resources like CPU, memory, and storage to each VM.
There are two types of hypervisors:
Type 1 (Bare-Metal Hypervisor): Runs directly on hardware (e.g., VMware ESXi, Microsoft Hyper-V, Xen).
Type 2 (Hosted Hypervisor): Runs on an existing OS (e.g., VMware Workstation, VirtualBox).
Guest Operating System
Each VM has its own OS (Windows, Linux, etc.), independent of the host machine.
Virtual Hardware
The VM emulates physical hardware components (CPU, RAM, disk, network interfaces).
Common Uses of Virtual Machines:
Server Virtualization – Running multiple servers on a single physical machine to optimize resources.
Software Testing – Running different OS environments without affecting the host system.
Cloud Computing – Used by platforms like AWS, Azure, and Google Cloud.
Security & Sandboxing – Isolating applications from the main system for security testing.
Legacy System Support – Running old software that is incompatible with modern OS.

23/112
2.4 VIRTUAL MACHINES AND VIRTUALIZATION MIDDLEWARE
 Virtual machines (VMs) offer novel solutions to underutilized resources, application inflexibility,
software manageability, and security concerns in existing physical machines.

 To build large clusters, grids, and clouds, we need to access large amounts of computing,
storage, and networking resources in a virtualized manner

Multiple OS environments can exist

simultaneously on the
same machine is called Virtual
Machine

If you want to run multiple

operating systems on one machine,
or multiple copies of the same
operating system, then you only
have two ways to do it: dual
Compare Physical Machine(a) with Three Virtual Machines (b,c,d) boot or virtual machine
24/112
 VIRTUAL MACHINES AND VIRTUALIZATION MIDDLEWARE
 a native VM installed with the use of a VMM called a hypervisor in privileged mode The guest
OS could be a Linux system and the hypervisor is the server system

HOSTED VIRTUAL MACHINE NATIVE VIRTUAL MACHINE

25/112
2.5.DATA CENTER VIRTUALIZATION FOR CLOUD COMPUTING

 A large data center may be built with thousands of servers.

 Smaller data centers are typically built with hundreds of
servers.
 High-end switches or routers may be too cost-prohibitive for
building data centers
 Currently, nearly all cloud computing data centers use Ethernet
as their fundamental network technology
 30% of data center cost: IT Equipment Servers/Disks
 33% of data center cost for chillers
 18% of data center cost for UPS
 9% of data center cost for A.C
 7% of data center cost for lighting in room
26/112
BENEFITS OF DATA CENTER VIRTUALIZATION

Key benefits of Data Center Virtualization

 Reduced Hardware Vendor Lock-in
 Improved Disaster Recovery
 Smooth Migration to Cloud
 Reduced Data Center Footprint
 Faster Server Provisioning.

27/112
 Classification of Distributed Computing Systems
• These can be classified into 4 groups: clusters, peer-to-peer networks, grids, and
clouds.
• A computing cluster consists of interconnected stand-alone computers which work
cooperatively as a single integrated computing resource. The network of compute
nodes are connected by LAN/SAN and are typically homogeneous with distributed
control running Unix/Linux. They are suited to HPC.
 Peer-to-peer (P2P) Networks
• In a P2P network, every node (peer) acts as both a client and server. Peers act
autonomously to join or leave the network. No central coordination or central database is
needed. No peer machine has a global view of the entire P2P system. The system is self-
organizing with distributed control.

• Unlike the cluster or grid, a P2P network does not use dedicated interconnection network.

• P2P Networks are classified into different groups:

Distributed File Sharing: content distribution of MP3 music, video, etc. E.g. Gnutella,
Napster, BitTorrent.

Collaboration P2P networks: Skype chatting, instant messaging, gaming etc.

Distributed P2P computing: specific application computing such as SETI@home provides

25 Tflops of distributed computing power over 3 million Internet host machines.
•
 Computational and Data Grids
• Grids are heterogeneous clusters interconnected by high-speed networks. They
have centralized control, are server-oriented with authenticated security. They are
suited to distributed supercomputing. E.g. TeraGrid.

• Like an electric utility power grid, a computing grid offers an infrastructure that
couples computers, software/middleware, people, and sensors together.

• The grid is constructed across LANs, WANs, or Internet backbones at a regional,

national, or global scale.

• The computers used in a grid include servers, clusters, and supercomputers. PCs,
laptops, and mobile devices can be used to access a grid system.
 Clouds
• A Cloud is a pool of virtualized computer resources. A cloud can host a variety of
different workloads, including batch-style backend jobs and interactive and user-
facing applications.

• Workloads can be deployed and scaled out quickly through rapid provisioning of
VMs. Virtualization of server resources has enabled cost effectiveness and allowed
cloud systems to leverage low costs to benefit both users and providers.

• Cloud system should be able to monitor resource usage in real time to enable
rebalancing of allocations when needed.

• Cloud computing applies a virtualized platform with elastic resources on demand by

provisioning hardware, software, and data sets dynamically. Desktop computing is
moved to a service-oriented platform using server clusters and huge databases at
datacenters.
 Advantage of Clouds over Traditional Distributed
Systems
• Traditional distributed computing systems provided for on-premise
computing and were owned and operated by autonomous administrative
domains (e.g. a company).

• These traditional systems encountered performance bottlenecks, constant

system maintenance, poor server (and other resource) utilization, and
increasing costs associated with hardware/software upgrades.

• Cloud computing as an on-demand computing paradigm resolves or

relieves many of these problems.
Software Environments for Distributed Systems and Clouds:

Service-Oriented Architecture (SOA) Layered Architecture

• In web services, Java RMI, and CORBA, an
entity is, respectively, a service, a Java CORBA Stack RMI Stack Web Services Stack
remote object, and a CORBA object. These IDL Java WSDL
build on the TCP/IP network stack. On top interface
of the network stack we have a base
CORBA Services RMI UDDI
software environment, which would Registry
be .NET/Apache Axis for web services, the
JVM for Java, and the ORB network for CORBA RMI SOAP Message
CORBA. On top of this base environment, a Stubs/Skeletons Stubs/Skelet
higher level environment with features ons
specific to the distributed computing CDR binary Java native XML Unicode
environment is built. encoding encoding - encoding
serialization

• Loose coupling and support of IIOP JRMP HTTP

heterogeneous implementations make RPC or Message Oriented Middleware (Websphere MQ or
services more attractive than distributed JMS)
objects. ORB JVM .NET/Apache Axis
TCP/IP/DataLink/Physical
 Performance Metrics and Scalability Analysis
• Performance Metrics:
• CPU speed: MHz or GHz, SPEC benchmarks like SPECINT
• Network Bandwidth: Mbps or Gbps
• System throughput: MIPS, TFlops (tera floating-point operations per second), TPS
(transactions per second), IOPS (IO operations per second)
• Other metrics: Response time, network latency, system availability

• Scalability:
• Scalability is the ability of a system to handle growing amount of work in a
capable/efficient manner or its ability to be enlarged to accommodate that growth.
• For example, it can refer to the capability of a system to increase total throughput
under an increased load when resources (typically hardware) are added.
 Scalability
Scale Vertically
To scale vertically (or scale up) means to add resources to a single node in a system,
typically involving the addition of CPUs or memory to a single computer.

Tradeoffs
There are tradeoffs between the two models. Larger numbers of computers means
increased management complexity, as well as a more complex programming model
and issues such as throughput and latency between nodes.
Also, some applications do not lend themselves to a distributed computing model.
In the past, the price difference between the two models has favored "scale up"
computing for those applications that fit its paradigm, but recent advances in
virtualization technology have blurred that advantage, since deploying a new virtual
system/server over a hypervisor is almost always less expensive than actually
buying and installing a real one.
 Scalability
• One form of scalability for parallel and distributed systems is:
• Size Scalability
This refers to achieving higher performance or more functionality by increasing the
machine size. Size in this case refers to adding processors, cache, memory, storage,
or I/O channels.
• Scale Horizontally and Vertically
Methods of adding more resources for a particular application fall into two broad
categories:

Scale Horizontally
To scale horizontally (or scale out) means to add more nodes to a system, such as
adding a new computer to a distributed software application. An example might be
scaling out from one Web server system to three.

The scale-out model has created an increased demand for shared data storage with
very high I/O performance, especially where processing of large amounts of data is
required.
 Amdahl’s Law
It is typically cheaper to add a new node to a system in order to achieve improved
performance than to perform performance tuning to improve the capacity that each
node can handle. But this approach can have diminishing returns as indicated by
Amdahl’s Law.

Consider the execution of a given program on a uniprocessor workstation with a

total execution time of T minutes. Now, let’s say that the program has been
parallelized or partitioned for parallel execution on a cluster of many processing
nodes.

Assume that a fraction α of the code must be executed sequentially, called the
sequential block. Therefore, (1 - α ) of the code can be compiled for parallel
execution by n processors. The total execution time of program is calculated by:
α T + (1 - α ) T / n

where the first term is the sequential execution time on a single processor and the
second term is the parallel execution time on n processing nodes.

All system or communication overhead is ignored here. The I/O and exception
handling time is also not included in the speedup analysis.
 Amdahl’s Law
Amdahl’s Law states that the Speedup Factor of using the n-processor system over the
use of a single processor is expressed by
Speedup = S = T / [α T + (1 - α ) T / n]
= 1 / [α + (1 - α ) / n]

The maximum speedup of n is achievable only when α = 0, i.e. the entire program is
parallelizable.

As the cluster becomes sufficiently large, i.e. n  ∞, then S  1 / α, an upper bound on

the speedup S. This upper bound is independent of the cluster size, n. The sequential
bottleneck is the portion of the code that cannot be parallelized.

Example, α = 0.25 and so (1 – 0.25) = 0.75 then the maximum speedup, S = 4 even if one
uses hundreds of processors.

Amdahl’s Law teaches us that we should make the sequential bottleneck as small as
possible. Increasing the cluster size alone may not result in a good speedup in this case.
 Amdahl’s Law
• Example: suppose 70% of a program can be sped up if parallelized and run on
multiple CPUs instead of one CPU.
• N = 4 processors

S = 1 / [0.3 + (1 – 0.3) / 4] = 2.105

• Doubling the number of processors to N = 8 processors

S = 1 / [0.3 + (1 – 0.3) / 8] = 2.581

Double the processing power has only improved the speedup by roughly one-fifth.
Therefore, throwing in more hardware is not necessarily the optimal approach.
 System Efficiency
• To execute a fixed workload on n processors, parallel processing may lead to a system
efficiency defined as:

System Efficiency, E = S / n = 1 / [α n + (1 - α ) ]

System efficiency can be rather low if the cluster size is very large.

Example: To execute a program on a cluster with n = 4, α = 0.25 and so

(1 – 0.25) = 0.75,

E = 1 / [0.25 * 4 + 0.75] = 0.57 or 57%

Now if we have 256 nodes (i.e. n = 256)

E = 1 / [0.25 * 256 + 0.75] = 0.015 or 1.5%

This is because only a few processors (4, as in the previous case) are kept busy, while the
majority of the processors (or nodes) are left idling.
 Fault Tolerance and System Availability
• High availability (HA) is desired in all clusters, grids, P2P networks, and cloud
systems. A system is highly available if it has a long Mean Time to Failure (MTTF)
and a short Mean Time to Repair (MTTR).

• System Availability = MTTF / (MTTF + MTTR)

• All hardware, software, and network components may fail. Single points of failure
that bring down the entire system must be avoided when designing distributed
systems.

• Adding hardware redundancy, increasing component reliability, designing for

testability all help to enhance system availability and dependability.

• In general, as a distributed system increases in size, availability decreases due to a

higher chance of failure and a difficulty in isolating failures.