Cloud Computing and Big Data

Subject Code: CS71 (Credits: 4:0:0)

Textbook:

1. Cloud Computing Theory and Practice – DAN C. Marinescu – Morgan Kaufmann

Elsevier.
2. Cloud Computing A hands - on approach – Arshdeep Bahga & Vijay madisetti
Universities press
3. Big Data Analytics, Seema Acharya and Subhashini Chellappan. 2nd edition, Wiley
India Pvt. Ltd. 2019

NOTE: I declare that the PPT content is picked up from the prescribed course text
books or reference material prescribed in the syllabus book and Online Portals.
 Unit II
Cloud Resource Virtualization:
• Layering and virtualization, Virtual machine monitors, Virtual machines,
• VM Performance and security isolation,
• Virtualization types, Hardware support for virtualization,
• A performance comparison of virtual machines,
• The darker side of virtualization, Software fault isolation.

Cloud Resource Management and Scheduling:

• Policies and mechanisms for resource management,
• Resource bundling, combinatorial auctions for cloud
• Scheduling algorithms for computing clouds,
• Fair queuing, start time fair queuing,
• Borrowed virtual time
• Resource management and application scaling
 Motivation
Three fundamental abstractions are necessary to describe the operation of a computing
systems:
(1) processors, (2) memory, (3) communications links and Storage

• As the scale of a system and the size of its users grows, it becomes very challenging to
manage its recourses
• Resource management issues:
• provision for peak demands  overprovisioning
• heterogeneity of hardware and software
• machine failures

• Virtualization is a basic enabler of Cloud Computing, it simplifies the

management of physical resources for the three abstractions

• For example, the state of a virtual machine (VM) running under a virtual
machine monitor (VMM) can de saved and migrated to another server to
balance the load
• For example, virtualization allows users to operate in environments they
are familiar with, rather than forcing them to specific ones
 Virtualization
• “Virtualization, in computing, refers to the act of creating a virtual (rather than
actual) version of something, including but not limited to a virtual computer hardware
platform, operating system (OS), storage device, or computer network resources.”

• Virtualization abstracts the underlying resources; simplifies their use; isolates users
from one another; and supports replication which increases the elasticity of a system

• In other words, Virtualization is a technique, which allows to share a single physical

instance of a resource or an application among multiple customers and organizations.

• Virtualization is the process of converting a physical IT resource into a

virtual IT resource.

• Virtualization is a partitioning of single physical server into multiple logical

servers.
• Once the physical server is divided, each logical server behaves like
a physical server and can run an operating system and
applications independently.
Benefits of Virtualization Technology :

• Efficient Usage of Physical Computing

Resources
• Increased Availability
• Easy System and Application Testing
• Simple Administration and Less Power
Consumption
• Faster Failure Recovery

Issues in Virtualization :

• Server Sharing Performance Issues

• Increased Networking Complexity
 Different types of Virtualization

There are many different types of virtualization, including hardware, software, desktop,
memory, storage, data and network virtualization.

• Hardware virtualization is one of the most common types of virtualization.

It is synonymous with platform virtualization, which occurs through the
creation of a virtual machine that behaves like a real computer or computer
operating system.

• Desktop virtualization is also another widely used type of virtualization that

creates and stores a client’s desktop on a server that can be remotely
accessed by the client over a network.

• Software virtualization is yet another commonly used type of virtualization

that allows different versions of an operating system to coexist and run on the
same physical machine, providing the ability to run applications in different
environments without the need to invest in additional hardware.
• Network Virtualization: It is a method of combining the available resources in
a network by splitting up the available bandwidth into channels, each of
which is independent from the others and each channel is independent of others
and can be assigned to a specific server or device in real time.

• Storage Virtualization: It is the pooling of physical storage from multiple

network storage devices into what appears to be a single storage device that is
managed from a central console. Storage virtualization is commonly used in
storage area networks (SANs).

• Server Virtualization: Server virtualization is the masking of server resources

like processors, RAM, operating system etc, from server users. The intention of
server virtualization is to increase the resource sharing and reduce the
burden and complexity of computation from users.
Resource management for a community of users with a wide range of applications
running under different operating systems is a very difficult problem.

• Resource management becomes even more complex when resources are

oversubscribed and users are uncooperative.

• In addition to external factors, resource management is affected by internal factors,

such as the heterogeneity of the hardware and software systems,

• The ability to approximate the global state of the system and to redistribute the
load, the failure rates of different components, and many other factors.

• The traditional solution for a data center is to install standard operating systems on
individual systems and rely on conventional OS techniques to ensure resource
sharing, application protection, and performance isolation.

• The alternative is Resource Virtualization, a technique analyzed in this chapter.

 Virtualization

Virtualization is a basic tenet of cloud computing – that simplifies some of the resource
management tasks.

For example,
• The state of a virtual machine (VM) running under a virtual machine
monitor (VMM) can be saved and migrated to another server to balance the
load.

• At the same time, virtualization allows users to operate in environments

with which they are familiar rather than forcing them to work in
idiosyncratic environments.
• Resource sharing in a virtual machine environment requires not only ample hardware
support and, in particular, powerful processors but also architectural support for
multilevel control.

• Indeed, resources such as CPU cycles, memory, secondary storage, and I/O and
communication bandwidth are shared among several virtual machines; for each VM,
resources must be shared among multiple instances of an application.
Virtualization Simulates the interface to a physical object by:

• Multiplexing: creates multiple virtual objects from one instance of a

physical object. Example - a processor is multiplexed among a number of
processes or threads.
• Aggregation: creates one virtual object from multiple physical objects.
Example - a number of physical disks are aggregated into a RAID disk.

• Emulation: constructs a virtual object from a different type of a physical

object. Example - a physical disk emulates a Random Access Memory
(RAM).

• Multiplexing and emulation. Examples - virtual memory with paging

multiplexes real memory and disk; a virtual address emulates a real address.
Cloud resource virtualization is important for:
• System security, as it allows isolation of services running on the same
hardware.
• Performance and reliability, as it allows applications to migrate from one
platform to another.
 Layering & Virtualization

• A common approach to manage system complexity is to identify a set of Layers with

well defined interfaces among them.
• Interfaces separate different levels of abstraction

Layering – a common approach to manage system complexity.

• Minimizes the interactions among the subsystems of a complex system.

• Simplifies the description of the subsystems; each subsystem is abstracted
through its interfaces with the other subsystems.
• We are able to design, implement, and modify the individual subsystems
independently.

Layering in a computer system.

• Hardware.
• Software.
• Operating system.
• Libraries.
• Applications.
Layering and Interfaces
between layers in Computer System

A1 Applications

API

Libraries A2

ABI
System calls

Operating System A3

ISA
System ISA User ISA
Hardware
Figure 5.1, which shows the interfaces among the software components and the
hardware.

• The hardware consists of one or more multicore processors, a system

interconnect (e.g., one or more buses), a memory translation unit, the main
memory, and I/O devices, including one or more networking interfaces.

• Applications written mostly in high-level languages (HLL) often call library

modules and are compiled into object code.

• Privileged operations, such as I/O requests, cannot be executed in user mode;

instead, application and library modules issue system calls and

• The operating system determines whether the privileged operations required by

the application do not violate system security or integrity and, if they don’t,
executes them on behalf of the user.

• The binaries resulting from the translation of HLL programs are targeted
to a specific hardware architecture.
Instruction Set Architecture (ISA) – at the boundary between hardware and software.
It defines a processors set of instructions. Ex: Intel architecture is represented by x86-32 and
x86-64 instruction sets for systems supporting 32-bit addressing and 64-bit addressing.
• The Hardware supports two execution mode:
• Privileged or Kernel mode and
• a User Mode.
• The Instruction Set consist of two sets:
• Sensitive Instructions,
• Non Privileged Instructions.

Application Binary Interface (ABI) – which allows the group consisting of the application
and the library modules to access the hardware; the ABI does not include privileged system
instructions, instead it invokes system calls.

Application Program Interface (API) - defines the set of instructions the hardware was
designed to execute and gives the application access to the ISA; it includes HLL library
calls which often invoke system calls.
 Code Portability

• A process is the abstraction for the code of an application at execution time;

• A thread is a lightweight process.
• The ABI is the projection of the computer system seen by the process, and
• The API is the projection of the system from the perspective of the HLL
program.

• The binaries created by a compiler for a specific ISA and a specific operating
system are not portable. Such code cannot run on a computer with a different ISA or
on computers with the same ISA but different operating systems.

• However, it is possible to compile an HLL program for a VM environment, as

shown in Figure 5.2, where portable code is produced and distributed and then
converted by binary translators to the ISA of the host system.

• A dynamic binary translation converts blocks of guest instructions

from the portable code to the host instruction and leads to a significant
performance improvement as such blocks are cached and reused.
 Code Portability
HLL code

Compiler front-end Compiler

Intermediate Portable
code code

Compiler back-end VM loader

Object code VM image

VM compiler/ VM compiler/
Loader
interpreter interpreter

Memory Memory Memory

image image ISA-1 image ISA-2
 (VMM / hypervisor)
VMM is the software that securely partitions the resources of a computer system into
one or more virtual machines (VMs).

• A guest OS is an OS that runs under the control of a VMM rather than directly
on the hardware.
• VMM runs in kernel mode, a guest OS runs in user mode.
• Allows several operating systems to run concurrently on a single hardware
platform; at the same time.
• VMM controls how the guest OS uses the hardware resources.
• Events occurring in one VM do not affect other VM running under same
VMM.
• At the same time the VMM enables
• Multiple services to share the same platform.
• Live migration - the movement of a server from one platform to another.
• System modification while maintaining backward compatibility with the
original system.
• Enforces isolation among the systems, thus security.
 Virtualization Approaches

[Reference]https://www.vmware.com/pdf/virtualization.pdf
 Virtualization Approaches

[Reference]https://www.vmware.com/pdf/virtualization.pdf
 Virtualization Approaches

[Reference]https://www.vmware.com/pdf/virtualization.pdf
VMM virtualizes the CPU and the memory

• Traps the privileged instructions executed by a guest OS and enforces the

correctness and safety of the operation.

• Traps interrupts and dispatches them to the individual guest operating systems.

VMM Controls the virtual memory management.

• Maintains a shadow page table for each guest OS and replicates any modification
made by the guest OS in its own shadow page table.

• This shadow page table points to the actual page frame and it is used by the Memory
Management Unit (MMU) for dynamic address translation.

• Monitors the system performance and takes corrective actions to avoid

performance degradation. For example, the VMM may swap out a Virtual
Machine to avoid thrashing.
 Virtual machines (VMs)

VM – is an isolated environment that appears to be a whole computer, but actually only

has access to a portion of the computer resources.

• Each VM appears to be running on the bare (native)hardware, giving the

appearance of multiple instances of the same computer, though all are supported
by a single physical system.

A virtual machine (VM) is a software program or operating system that not only exhibits
the behavior of a separate computer, but is also capable of performing tasks such as
running applications and programs like a separate computer.

• A virtual machine, usually known as a guest is created within another computing

environment referred as a "host."

• Multiple virtual machines can exist within a single host at one time.

• A virtual machine is also known as a guest.

 Two types of VM:
Process & System VMs.
Process Virtual Machine: A process VM is an virtual platform created for an individual
process and destroyed once the process terminates.

• Virtually all operating systems provide a process VM for each one of the
applications running, but the more interesting process VMs are those that support
binaries compiled on a different instruction set.

System Virtual Machines: A system VM supports an operating system together with many
user processes. When the VM runs under the control of a normal OS and provides a
platform-independent host for a single application, we have an application virtual machine
(e.g., Java Virtual Machine [JVM]).

• A system virtual machine provides a complete system; each VM can run its own
OS, which in turn can run multiple applications.

• Operating system-level virtualization allows a physical server to run multiple

isolated operating system instances, subject to several constraints; the instances are
known as containers, virtual private servers (VPSs), or virtual environments (VEs).
Two types of VM: process & system VMs.

System Virtual Machines:

• A system platform that supports the sharing of the host computer's physical
resources between multiple virtual machines, each running with its own
copy of the operating system.

• The virtualization technique is provided by a software layer known as a

hypervisor, which can run either on bare hardware or on top of an operating
system.

Process Virtual Machine:

• Designed to provide a platform-independent programming environment that

masks the information of the underlying hardware or operating system and
allows program execution to take place in the same way on any given platform
 Process VMs System VMs

Same ISA Different ISA Same ISA Different ISA

Application Application

Multi Dynamic Traditional Whole Guest Guest

program translators VM system VM OS -1 OS -n

Binary
optimizers HLL VMs
Hybrid VM
Codesigned VM-1 VM-n
VM

Virtual Machine Monitor

Hosted VM

Hardware

(a) (b)

Application Application
Application
Application

Guest OS -n
Application

Guest OS -1
Application

VM-1 VM-n
Guest OS
Virtual Machine Monitor

Host OS VMM Host OS

Hardware Hardware

(c) (d)
 Performance and security isolation

• Performance isolation - a critical condition for QoS guarantees in shared

computing environments.

• Process virtualization presents multiple copies of the same process or multicore

systems. The code is executed directly by the hardware. Whereas, process
emulation presents a model of another hardware system in which instructions
are “emulated “ in software more slowly than virtualization.

• Processor emulation is a virtualization technology that allows software compiled

for one processor/operating system to run on a system with a different
processor/operating system, without any source code or binary changes.
 Performance and security isolation

The run-time behavior of an application is affected by other applications running

concurrently on the same platform and competing for CPU cycles, cache, main
memory, disk and network access. Thus, it is difficult to predict the completion
time!

• A VMM is a much simpler and better specified system than a traditional

operating system.

• Example - Xen has approximately 60,000 lines of code; Denali has only
about half: 30,000

• The security vulnerability of VMMs is considerably reduced as the systems

expose a much smaller number of privileged functions.

• For example, Xen VMM has 28 hypercalls while Linux has 100s of
system calls
 Full virtualization and
Par-virtualization
• Full virtualization ,in which each
virtual machine runs on an exact Guest OS Guest OS
copy of the actual hardware. Hardware Hardware
• (Guest OS is unaware that its in abstraction abstraction
a virtualized environment) layer layer
• Example: Vmware
Hypervisor Hypervisor
• Paravirtualization , in which each
virtual machine runs on a slightly
Hardware Hardware
modified copy of the actual
hardware.
• (Guest OS is already aware that (a) Full virtualization (b) Paravirtualization
they are shared hardware)
• Example: Xen
An equivalent formulation of the conditions for efficient virtualization can be based on this
classification of machine instructions.
• A VMM for a third-generation (or later) computer can be constructed if the set of
sensitive instructions is a subset of the privileged instructions of that machine. To handle
non virtualizable instructions, one could resort to two strategies:
• Binary translation. The VMM monitors the execution of guest operating
systems; non virtualizable instructions executed by a guest operating system
are replaced with other instructions.
• Para virtualization. The guest operating system is modified to use only
instructions that can be virtualized.

There are two basic approaches to processor virtualization:

• Full virtualization, in which each virtual machine runs on an exact copy of the
actual hardware, and
• Paravirtualization, in which each virtual machine runs on a slightly modified
copy of the actual hardware (see Figure 5.4).
The reasons that paravirtualization is often adopted are
• Some aspects of the hardware cannot be virtualized;
• To improve performance; and
• To present a simpler interface.
 Full virtualization

• Guest operating systems are unaware of each other

• Provide support for unmodified guest operating system.
• Hypervisor directly interact with the hardware such as CPU, disks.
• Hypervisor allow to run multiple OS simultaneously on host computer.
• Each guest server run on its own operating system

Few implementations: Oracle's Virtaulbox , VMware server, Microsoft Virtual PC

Advantages:
• This type of virtualization provide best isolation and security for Virtual machine.
• Truly isolated multiple guest OS can run simultaneously on same hardware.
• It's only option that requires no hardware assist or OS assist to virtualize sensitive
and privileged instructions.
Limitations:
• Full virtualization is usually bit slower ,because of all emulation.
• Hypervisor contain the device driver and it might be difficult for new device
drivers to be installer by users
 Para virtualization
• Unlike full virtualization ,guest servers are aware of one another.
• Hypervisor does not need large amounts of processing power to manage guest OS.
• The entire system work as a cohesive unit.

Advantages:
• As a guest OS can directly communicate with hypervisor

• This is efficient virtualization.

• Allow users to make use of new or modified device drivers.

Limitations:
• Para virtualization requires the guest OS to be modified in order to interact
with para virtualization interfaces.

• It requires significant support and maintaibilty issues in production

environment.
 Hardware virtualization
Hardware virtualization is the virtualization of computers as complete hardware
platforms, certain logical abstractions of their componentry, or only the functionality
required to run various operating systems.

In computing, hardware-assisted virtualization is a platform virtualization approach

that enables efficient full virtualization using help from hardware capabilities,
primarily from the host processors.

• A full virtualization is used to emulate a complete hardware environment, or

virtual machine, in which an unmodified guest operating system (using the
same instruction set as the host machine) effectively executes in complete
isolation.

• Hardware-assisted virtualization was added to x86 processors (Intel VT-x or

AMD-V) in 2005 and 2006 (respectively).
 x86 virtualization
• x86 virtualization is the use of hardware-assisted virtualization capabilities on an
x86/x86-64 CPU.
• Intel virtualization (VT-x)
• In 2005 Intel released two Pentium 4 models supporting VT-x. VT-x supports two modes
of operations (Figure (a)):
• VMX root - for VMM operations.
• VMX non-root - support a VM.
• a new data structure called the Virtual Machine Control Structure(VMCS) including
host-state and guest-state areas (Figure (b)).
• VM entry - the processor state is loaded from the guest-state of the VM scheduled
to run; then the control is transferred from VMM to the VM.
• VM exit - saves the processor state in the guest-state area of the running VM; then it
loads the processor state from the host-state area, finally transfers control to the
VMM.
Dual-mode operation allows OS to protect itself and other system components

• User mode and kernel mode

• Mode bit provided by hardware
• Ability to distinguish when system is running user or kernel code
• Some instructions are privileged, only executable in kernel mode
• System call changes mode to kernel, return resets it to user
Three classes of machine instructions:

• Privileged instructions can be executed in kernel mode. When attempted to

be executed in user mode, they cause a trap and so executed in kernel mode.

• Nonprivileged instructions the ones that can be executed in user mode

• Sensitive instructions can be executed in either kernel or user but they

behave differently.
 Challenges/Problems faced by
x86 CPU Virtualization
• Ring de-privileging –

• a VMMs forces the guest software, operating system and the applications to run at a
privilege level greater than 0.
• The x86 architecture provides Four layers of privilege executionrings(level 0-3)

Two solutions are then possible:

• The (0/1/3) mode, in which the VMM, the

OS, and the application run at privilege
levels 0, 1, and 3, respectively; or

• The (0,3,3) mode, in which the VMM, a

guest OS, and applications run at privilege
levels 0, 3, and 3, respectively.
 Problem faced by virtualization
of the x86 architecture:
• Ring aliasing - a guest OS is forced to run at a privilege level other than that it
was originally designed for.

• Address space compression - a VMM uses parts of the guest address space to
store several system data structures.

• Non-faulting access to privileged state - several store instructions can only be

executed at privileged level 0 because they operate on data structures that control
the CPU operation. They fail silently when executed at a privilege level other than
0.

• Guest system calls which cause transitions to/from privilege level 0 must be
emulated by the VMM.

• Interrupt virtualization - in response to a physical interrupt, the VMM generates

a ``virtual interrupt'' and delivers it later to the target guest OS which can mask
interrupts.
 Problem faced by virtualization
of the x86 architecture:

• Access to hidden state - elements of the system state, e.g., descriptor caches for
segment registers, are hidden; there is no mechanism for saving and restoring the
hidden components when there is a context switch from one VM to another.

• Ring compression - paging and segmentation protect VMM code from being
overwritten by guest OS and applications. Systems running in 64-bit mode can
only use paging, but paging does not distinguish between privilege levels 0, 1, and
2, thus the guest OS must run at privilege level 3, the so called (0/3/3) mode.
Privilege levels 1 and 2 cannot be used thus, the name ring compression.

• Frequent access to privileged resources increases VMM overhead: The task-

priority register is frequently used by a guest OS; the VMM must protect the
access to this register and trap all attempts to access it. This can cause a significant
performance degradation.
 Overview of x86 Virtualization

• virtualization layer is added between

the hardware and operating system as
seen in Figure 2.

• This virtualization layer allows

multiple operating system instances to
run concurrently within virtual
machines on a single computer,

• dynamically partitioning and

sharing the available physical
resources such as CPU, storage,
memory and I/O devices.
• For industry standard x86 systems, virtualization approaches use either a hosted or a
hypervisor architecture.

• A hosted architecture installs and runs the virtualization layer as an application on

top of an operating system and supports the broadest range of hardware
configurations.

• In contrast, a hypervisor (bare-metal) architecture installs the virtualization layer

directly on a clean x86-based system.

• Since it has direct access to the hardware resources rather than going through an
operating system, a hypervisor is more efficient than a hosted architecture and
delivers greater scalability, robustness and performance.

Figure 3, the virtualization layer is

a hypervisor running directly on the
hardware.. Each VMM running on the
hypervisor implements the virtual
machine hardware abstraction and
is responsible for running a guest OS.
Each VMM has to partition and
share the CPU, memory and I/O
devices to successfully virtualize the
system
The Challenges of x86 Hardware Virtualization

X86 operating systems are designed to run directly on the bare-metal hardware, so they
naturally assume they fully ‘own’ the computer hardware. As shown in Figure 4,

the x86 architecture offers four levels of privilege known as Ring 0, 1, 2 and 3 to operating
systems and applications to manage access to the computer hardware.
• While user level applications typically run in Ring 3,

• the operating system needs to have direct access to the memory and hardware and
must execute its privileged instructions in Ring 0.

Virtualizing the x86 architecture requires placing a

virtualization layer under the operating system (which
expects to be in the most privileged Ring 0) to create and
manage the virtual machines that deliver shared resources.
Further complicating the situation, some sensitive
instructions can’t effectively be virtualized as they have
different semantics when they are not executed in Ring
0. The difficulty in trapping and translating these sensitive
and privileged instruction requests at runtime was the
challenge that originally made x86 architecture
virtualization look impossible.
VMware resolved the challenge in 1998, developing binary translation techniques that
allow the VMM to run in Ring 0 for isolation and performance,

while moving the operating system to a user level ring with greater privilege than
applications in Ring 3 but less privilege than the virtual machine monitor in Ring 0.

Three alternative techniques now exist for handling sensitive and privileged instructions
to virtualize the CPU on the x86 architecture:

• Full virtualization using binary translation

• OS assisted virtualization or paravirtualization

• Hardware assisted virtualization (first generation)

Technique 1 – Full Virtualization using Binary
Translation
The guest OS is not aware it is being virtualized
and requires no modification.

Full virtualization is the only option that requires

no hardware assist or operating system assist to
virtualize sensitive and privileged instructions.

The hypervisor translates all operating system

instructions on the fly and caches the results for
future use, while user level instructions run
unmodified at native speed.

Full virtualization offers the best isolation and

security for virtual machines, and simplifies
migration and portability as the same guest OS
instance can run virtualized or on native hardware.
Technique 2 – OS Assisted Virtualization or
Paravirtualization
Paravirtualization is different from full
virtualization, where the unmodified OS does
not know it is virtualized and sensitive OS calls
are trapped using binary translation.

As paravirtualization cannot support

unmodified operating systems (e.g. Windows
2000/XP), its compatibility and portability is
poor.

Paravirtualization can also introduce significant

support and maintainability issues in
production environments as it requires deep OS
kernel modifications.

The open source Xen project is an example of

paravirtualization that virtualizes the processor
and memory using a modified Linux kernel and
virtualizes the I/O using custom guest OS
device drivers
 Technique 3 – Hardware Assisted
Virtualization Hardware vendors are rapidly embracing
virtualization and developing new
features to simplify virtualization
techniques.

First generation enhancements include

Intel Virtualization Technology (VT-x)
and AMD’s AMD-V which both target
privileged instructions with a new CPU
execution mode feature that allows the
VMM to run in a new root mode below
ring 0.

As depicted in Figure 7, privileged and

sensitive calls are set to automatically
The guest state is stored in Virtual Machine trap to the hypervisor, removing the need
Control Structures (VT-x) or Virtual Machine for either binary translation or
Control Blocks (AMD-V). paravirtualization.
• Processors with Intel VT and AMD-V became available in 2006, so only newer systems
contain these hardware assist features.
Management
OS Application Application Application

Guest OS Guest OS Guest OS

Xen-aware
device drivers
Xen-aware Xen-aware Xen-aware
device drivers device drivers device drivers

Xen
Domain0 control Virtual x86 Virtual physical Virtual block
interface Virtual network
CPU memory devices

X86 hardware
Performance comparison of virtual machines
We have seen that a VMM such as Xen introduces additional overhead and negatively
affects performance…..Will Compare the performance of Xen and OpenVZ

• The questions examined are:

• How the performance scales up with the load?
• What is the impact of a mix of applications?
• What are the implications of the load assignment on individual servers?

• The main conclusions:

• The virtualization overhead of Xen is considerably higher than that of OpenVZ

and that this is due primarily to L2-cache misses.

• The performance degradation when the workload increases is also noticeable

for Xen.

• Hosting multiple tiers of the same application on the same server is not an optimal
solution.
The experimental setups for three different experiments are shown in Figure 5.9 .

• In the first group of experiments the two tiers of the application, the Web and the
DB, run on a single server for the Linux, the OpenVZ, and the Xen systems.

• When the workload increases from 500 to 800 threads, The throughput
increases linearly with the workload.

• The response time increases only slightly for the base system and for the
OpenVZ system, whereas it increases 600% for the Xen system.

• For 800 threads the response time of the Xen system is four times longer than
the time for OpenVZ.

• The CPU consumption grows linearly with the load in all three systems; the DB
consumption represents only 1–4% of it.

For a given workload, the Web-tier CPU consumption for the OpenVZ system is close to that
of the base system and is about half of that for the Xen system.

• The performance analysis tool shows that the OpenVZ execution has two times more L2-
cache misses than the base system,
• whereas the Xen Dom0 has 2.5 times more and the Xen application domain has 9 times
more.
• The second group of experiments uses two servers, one for the Web and the other for
the DB application, for each one of the three systems.

• When the load increases from 500 to 800 threads the throughput increases linearly
with the workload.
• The response time of the Xen system increases only 114%, compared with 600%
reported for the first experiments. The CPU time of the base system,
• The OpenVZ system, the Xen Dom0, and the User Domain are similar for the Web
application.

• The third group of experiments uses two servers, one for the Web and the other for
the DB application,

• for each one of the three systems but runs four instances of the Web and the DB
application on the two servers.
• The throughput increases linearly with the workload for the range used in the
previous two experiments, from 500 to 800 threads.
• The response time remains relatively constant for OpenVZ and increases 5 times
for Xen
The main conclusion drawn from these experiments is that

• the virtualization overhead of Xen is considerably higher than that of

OpenVZ and that this is due primarily to L2-cache misses.

• The performance degradation when the workload increases is also noticeable

for Xen.

• Another important conclusion is that hosting multiple tiers of the same

application on the same server is not an optimal solution.
The darker side of virtualization
In a layered structure, a defense mechanism at some layer can be disabled by malware
running at a layer below it. a VMM allows a guest operating system to run on virtual
hardware. The VMM offers to the guest operating systems a hardware abstraction and
mediates its access to the physical hardware

• It is feasible to insert a rogue VMM, a Virtual-Machine Based Rootkit (VMBR)

between the physical hardware and an operating system.

• Rootkit - malware with a privileged access to a system.

• The VMBR can enable a separate malicious OS to run surreptitiously and make this
malicious OS invisible to the guest OS and to the application running under it.

• Under the protection of the VMBR, the malicious OS could:

• observe the data, the events, or the state of the target system.
• run services, such as spam relays or distributed denial-of-service attacks.
• interfere with the application.
 Application

Application
Malicious Guest OS
OS
Operating
Malicious system (OS)
OS Virtual machine monitor

Virtual machine based rootkit Virtual machine based rootkit

Hardware Hardware

(a) (b)

The insertion of a Virtual-Machine Based Rootkit (VMBR) as the lowest layer of the
software stack running on the physical hardware;

(a) below an operating system; (b) below a legitimate virtual machine monitor. The
VMBR enables a malicious OS to run surreptitiously and makes it invisible to the
genuine or the guest OS and to the application.
How such an insertion is possible

The malware runs either inside a VMM or with the support of a VMM;

But a VMM is a very potent engine for the malware. It prevents the software of the
guest operating system or the application from detecting malicious activities.

A VMBR can record key strokes, system state, data buffers sent to or received from
the network, and data to be written to or read from the disk with impunity; moreover, it
can change any data at will.

The only way for a VMBR to take control of a system is to modify the boot sequence
and to first load the malware and only then load the legitimate VMM or the operating
system. This is only possible if the attacker has root privileges.
Software fault isolation
• Software fault isolation (SFI) offers a technical solution for sandboxing binary code of
questionable provenance that can affect security in cloud computing.

• Insecure and tampered VM images are one of the security threats because binary codes
of questionable provenance for native plug-ins to a Web browser can pose a security
threat when Web browsers are used to access cloud services

• The application of the sandboxing technology for two modern CPU architectures,
ARM and 64-bit x86. ARM is a load/store architecture with 32-bit instruction and 16
general-purpose registers
Cloud Resource Management and Scheduling

Cloud Resource Management and Scheduling:

• Policies and mechanisms for resource management,

• Resource bundling, combinatorial auctions for cloud
• Scheduling algorithms for computing clouds,
• Fair queuing, start time fair queuing,
• Borrowed virtual time
• Resource management and application scaling
• Resource management is the practice of planning, scheduling, and allocating people,
money, and technology to a project or program.

• Resource management is a core function required for any cloud system or man-
made system. and inefficient resource management has a direct negative effect
on performance and cost, while it can also indirectly affect system functionality,
becoming too expensive or ineffective due to poor performance.

• It affects the three basic criteria for the evaluation of a system:

• Functionality. (indirect )
• Performance.(direct negative effect)
• Cost. (direct negative effect)

• Cloud resource management requires complex policies and decisions for multi-
objective optimization.

• The Strategies for resource management is associated with the three cloud delivery
models. IaaS, PaaS,SaaS differ from one another.

• Policies and mechanisms for resource allocation.

• Policy  principles guiding decisions.

• Mechanisms  the means to implement policies.
Cloud resource management polices can be loosely grouped into five classes:

Admission control  The goal is to prevent the system from accepting workload in
violation of high-level system policies. Example:
• workload requires some a system may not accept an additional workload that
would prevent it from completing work already in progress or contracted.
• Limiting the knowledge of the global state of the system.

Capacity allocation  It means allocate resources for individual instances (instance is an

activations of a service).
Load balancing  distribute the workload evenly among the servers.
• Consider 4 identical server A,B,C,D whose relative loads are 80%,60%.40%
and 20% respectively of their capacity.
• As a result of perfect load balancing all servers would end with the same load
– 50% of each server capacity. In cloud computing critical goal is
minimizing the cost of providing the service and minimizing the energy
consumption.
• Hence load from D is shifted to A and C is shifted to B. Thus A and B will be
loaded at full capacity, whereas C and D will be switched to standby
mode.
Energy optimization  minimization of energy consumption.
Quality of service (QoS) guarantees  ability to satisfy timing or other conditions
specified by a Service LevelAgreement.
The four Mechanisms for the implementation of resource management policies

• Control theory  uses the feedback to guarantee system stability and predict
transient behavior. It can be used only to predict local rather than global
behavior.

• Machine learning  A major advantage is that they do not need a performance

model of the system. This technique could be applied to coordination of several
autonomous system managers.

• Utility-based  It require a performance model and a mechanism to correlate

user-level performance with cost.

• Market-oriented/economic  do not require a model of the system, e.g.,

combinatorial auctions (auction: It is a process of buying and selling goods or
services)for bundles of resources.
 disturbance

r s  (k )
u* (k)
Predictive Optimal Queuing
filter controller dynamics
external forecast  (k )
traffic

state feedback q(k)

Control Theory: The controller uses the feedback regarding the current state and
the estimation of the future disturbance due to environment to compute the optimal
inputs over a finite horizon.

r and s are the weighting factors of the performance index.

 Stability of a Two-level cloud controller Architecture:

• The actions of the control system should Application 1 Application n

be carried out in a rhythm(strong) that Application 1 SLA 1 …. SLA n Application n

VM VM
…. VM VM

does not lead to instability.

Application …. Application
controller controller
Monitor Monitor
• Adjustments should only be carried out
after the performance of the system has Decision …. Decision
stabilized. Cloud Controller Actuator Actuator

• The actions consist of

allocation/deallocation of one or more Cloud Platform
virtual machines. Sometimes
allocation/dealocation of a single VM
required by one of the threshold may Feed back control based on
cause crossing of the other, another source dynamic thresholds:
of instability.
The elements involved in control systems are
-Sensors ( measures the parameters of interest, then transmit the measured values to monitor.)
-Monitors( determines whether the system behavior should be changed or not)
-Actuators (if changes required, actuators carry out the necessary actions)
Feedback control based on : Dynamic Thresholding
Algorithm: Proportional Thresholding

• Compute the integral value of the high and the low threshold as averages of the
maximum and, respectively, the minimum of the processor utilization over the process
history.
• Request additional VMs when the average value of the CPU utilization over
the current time slice exceeds the high threshold.
• Release a VM when the average value of the CPU utilization over the current
time slice falls below the low threshold.

Conclusions

• Dynamic thresholds perform better than the static ones.

• Two thresholds are better than one.

Resource bundling: Combinatorial auctions for cloud resources

• Resources in a cloud are allocated in bundles.(pakages)

• Users get maximum benefit from a specific combination of resources: CPU

cycles, main memory, disk space, network bandwidth, and so on.

• Resource bundling complicates traditional resource allocation models and has

generated an interest in economic models and, in particular, in auction
algorithms.

• In the context of cloud computing, an auction is the allocation of resources to

the highest bidder.
• Auctions in which participants can bid on combinations of items, or packages
are called combinatorial auctions.

• Such auctions provide a relatively simple , scalable and tractable solution to cloud
resource allocation.

• Combinatorial auction algorithms:

• Simultaneous clock auction ,

• clock proxy auction,
• Ascending clock auction (ASCA)

In all these algorithms , the current price for each resource is represented by a
“clock” seen by all participants at the auction.
Combinatorial auctions for cloud resources
• Prices and allocation are set as a result of an auction.
• Users provide bids for desirable bundles and the price they are willing to pay.

Assume a population of users U, u={ 1,2,…,U} and

resources R, r={1,2,…,R}.

The bid of user u is Bu = { Qu , ∏u } ,Qi= { qu1,qu2,qu3… }

an R-Component Vector.

-qu1- bundle of resource user u would accept

-∏u - total price to be paid for the resource

qui - is a positive quantity – resource desired

- is a negative quantity – resource offered
User desire is Indifferent set I = (q 1 XORqu2XORqu3 )
u

The bidding process aims to optimize an objective function f(x,p).

Constraints for Combinatorial auction algorithm

1. The user either gets one of the bundles it has opted for or nothing. No partial
allocation is acceptable.
2. The system awards only available resources ; only offered can be allocated.
3. The bid of winners exceeds the final price
4. The winners get the least expensive bundles in their indifference set.
5. Losers bid below the final price.
6. All prices are positive numbers.
Pricing and allocation algorithms
A pricing and allocation algorithm partitions the set of users in two disjoint sets, winners
and losers.
Desirable properties of a pricing algorithm:

• Be computationally tractable; traditional combinatorial auction algorithms e.g.,

Vickey-Clarke-Groves (VLG) are not computationally tractable.

• Scale well - given the scale of the system and the number of requests for service,
scalability is a necessary condition.

• Be objective - partitioning in winners and losers should only be based on the price
of a user's bid; if the price exceeds the threshold then the user is a winner, otherwise
the user is a loser.

• Be fair - make sure that the prices are uniform, all winners within a given resource
pool pay the same price.

• Indicate clearly at the end of the auction the unit prices for each resource pool.

• Indicate clearly to all participants the relationship between the supply and the
demand in the system.
 u1 Proxy x1(t)

u2 Proxy x2(t)

x3(t)
u3 Proxy Auctioneer

 x (t )  0
u
u

uU Proxy xU(t)

p(t+1)

Ascending Clock Auction, (ASCA)  the current price for each resource is represented by
a “clock” seen by all participants at the auction. The algorithm involves user bidding in
multiple rounds; to address this problem the user proxies automatically adjust their demands
on behalf of the actual bidders.
Scheduling algorithms for computing clouds
Scheduling It is a critical component of cloud resource management. It is responsible for
resource sharing/multiplexing at several levels:
• A server can be shared among several virtual machines.
• A virtual machine could support several applications.
• An application may consist of multiple threads.

A scheduling algorithm should be efficient, fair, and starvation-free. The objectives of a

scheduler :
• Batch system 
• maximize throughput(the number of jobs completed in one unit of
time) and
• minimize turnaround time(the time between job submission and its
completion).
• Real-time system  meet the deadlines and be predictable.

Two different dimensions of resource management must be addressed by a scheduling

policy:
• The amount or quality of the resources allocated
• The timing when access to resource is granted
 Best-effort policies  do not impose(force)
requirements regarding either the amount of
resources allocated to an application, or the
timing when an application is scheduled.

• Soft-requirements policies 
require statistically guaranteed
amounts and timing constraints

• Hard-requirements policies  are

the most challenging because they
demand strict timing and precise
amounts of resources.

Figure 6.7 identifies several broad classes of resource allocation requirements in the
space defined by these two dimensions: best-effort, soft requirements, and hard
requirements.
Fair Scheduling Algorithms:
Max-Min fairness criterion:

• Consider a resource with Bandwidth B shared among n users who have equal rights.
• Each user requests an amount bi and receives Bi.
• Then according to max-min criterion, following conditions for fair allocation:
C1:Bi<=bi (The amount received by any user is not larger than the amount
requested.)

C2:Bmin should not be higher (if the minimum allocation of any user is Bmin no
allocation satisfying C1 has a higher Bmin than the current allocation.)

C3:B-Bmin (When we remove the user receiving the minimum allocation Bmin
and then reduce the total amount of resources available from B to( B-Bmin ) ,the
condition C2 remains recursively true.)
Fairness criterion for CPU scheduling

• A fairness criterion for CPU scheduling requires that the amount of work in the
time interval t1 to t2 of two runnable threads a and b ,

• Ω a (t1,t2) and Ω b (t1,t2) , minimize the expression

• Where ω a and ω b are the weights of the thread a and b respectively

The quality-of-service (QoS) requirements differ for different classes of cloud

applications and demand different scheduling policies.

• Best-effort applications such as batch applications and analytics do not require

QoS guarantees.

• Round-robin, FCFS, shortest-job-first (SJF), and priority algorithms are

among the most common scheduling algorithms for best-effort applications
Fair queuing
• Fair scheduling algorithm can be used for scheduling packet transmission , as well as
threads.

• Interconnection networks allow cloud servers to communicate with one another and
with users.

• These n/w consists of communication links of limited bandwidth and

switches/routers/gateways of limited capacity.

• When the load exceeds its capacity , a switch starts dropping packets,
because it has limited buffers for the switching fabric and for the outgoing links
, as well as CPU cycles.

• A switch must handle flows and pairs of source destination endpoints of the traffic. Thus
a scheduling algorithm has to manage several quantities at the same time.

• bandwidth: amount of data each flow is allowed to transport.

• timing: when the packets of individual flows are transmitted
• Buffer space: space allocated to each flow.
The first strategy to avoid network congestion is to use FCFS

Solution_1: first strategy : FCFS.

• To avoid network congestion – FCFS. But does not guarantees fairness;

Solution_2: Fair Queuing(FQ) proposes that separate queues, one per flow , be
maintained by a switch and the queues be serviced in a round robin (RR) manner.

• manages buffer space management, not bandwidth allocation. Indeed a flow

transporting large packets will benefit from a larger bandwidth.

Solution_3: bit by bit RR (BR)

• impractical scheme , a single bit from each queue is transmitted and the queues are
visited in a round robin fashion.

• R(t) – number of rounds aof the BR algorithm up to time t.

• Nactive(t) – number of active flows through the switch.
The transmission of packet i of a
flow can only start after the packet
is available and the transmission of
the previous packet has finished.

• (a)The new packet arrives

after the previous has finished.

• (b)The new packet arrives

before the previous one was
finished.
Start-time fair queuing
The basic idea of the start-time fair queuing (SFQ) algorithm is to organize the
consumers of the CPU bandwidth in a tree structure; the root node is the processor and
the leaves of this tree are the threads of each application.

• A scheduler acts at each level of the hierarchy. The fraction of the processor
bandwidth, B, allocated to the intermediate node i is

• When a virtual machine is not active, its bandwidth is reallocated to the

other VMs active at the time.

• When one of the applications of a virtual machine is not active, its

allocation is transferred to the other applications running on the same VM.

• Similarly, if one of the threads of an application is not runnable, its

allocation is transferred to the other threads of the applications.
• va(t) and vb(t) the virtual time of threads a and b, respectively, at real time t.

• The virtual time of the scheduler at time t is denoted by v(t).

• Call q the time quantum of the scheduler in milliseconds.
• The threads a and b have their time quanta, qa and qb,
• weighted by wa and wb, respectively

• Thus, in the example, the time quanta of the two threads are q/wa and
q/wb, respectively

• The i -th activation of thread a will start at the virtual time Sia and will
finish at virtual time Fia

• We call τ j the real time of the j -th invocation of the scheduler.

An SFQ scheduler follows several rules:

R1. The threads are serviced in the order of their virtual start-up time; ties are
broken arbitrarily.

R2. The virtual startup time of the i -th activation of thread x is

R3. The virtual finish time of the i -th activation of thread x is

R4. The virtual time of all threads is initially zero, v0x = 0. The virtual time v(t) at real
time t is computed as follows:
Borrowed virtual time
Objective - support low-latency dispatching of real-time applications, and weighted
sharing of CPU among several classes of applications.

• Like SFQ , BVT supports scheduling of a mix of applications- hard, soft real
time constraints ,best effort.

A thread i has
• an effective virtual time, Ei.
• an actual virtual time, Ai.
• a virtual time warp, Wi.

• The scheduler thread maintains its own scheduler virtual time (SVT) defined as
the minimum actual virtual time Aj of any thread.

• The threads are dispatched in the order of their effective virtual time Ei, a policy
called the Earliest Virtual Time (EVT).

• The virtual warp time allows a thread to acquire an earlier effective time ie., to
borrow virtual time from its future CPU allocation.

• The virtual warp time is enabled when the variable warpBack is set.
In this case a latency-sensitive thread gains dispatching preference as

• The algorithm measures time in minimum charging units(mcu) and uses time
quantum called Context switch allowance(C)
• which measures the real time a thread is allowed to run when competing with
other threads, measured in multiples of mcu.
• example: mcu=100μsec C= 100 msec

Context switches are triggered by events such as:

• the running thread is blocked waiting for an event to occur.
• the time quantum expires and an interrupt occurs.
• when a thread becomes runnable after sleeping.
• When the thread becomes runnable after sleeping , its actual virtual time is
updated as follows:
Resource management and application scaling

• The demand for computing resources, such as CPU cycles, primary and secondary
storage, and network bandwidth, depends heavily on the volume of data processed by
an application.

• The demand for resources can be a function of the time of day, can monotonically
increase or decrease in time, or can experience predictable or unpredictable peaks.

For example,

• a new Web service will experience a low request rate when the service is
first introduced and the load will exponentially increase if the service is
successful.

• A service for income tax processing will experience a peak around the tax
filling deadline, whereas access to a service provided by Federal Emergency
Management Agency (FEMA)will increase dramatically after a natural
disaster.
The question we address is: How scaling can actually be implemented in a cloud when a
very large number of applications exhibit this often unpredictable behavior.

We distinguish two scaling modes: vertical and horizontal.

Vertical scaling keeps the number of VMs of an application constant, but increases the
amount of resources allocated to each one of them.

• This can be done either by migrating the VMs to more powerful servers or by
keeping the VMs on the same servers but increasing their share of the CPU time.

• The first alternative involves additional overhead; the VMis stopped, a snapshot
of it is taken, the file is transported to a more powerful server, and, finally, the
VM is restated at the new site.
Horizontal scaling is the most common mode of scaling on a cloud; it is done by
increasing the number of VMs as the load increases and reducing the number of VMs
when the load decreases.

• Often, this leads to an increase in communication bandwidth consumed by the

application. Load balancing among the running VMs is critical to this mode of
operation.

• For a very large application, multiple load balancers may need to cooperate with
one another. In some instances the load balancing is done by a front-end server
that distributes incoming requests of a transaction-oriented system to back-end
servers.

In the case of an arbitrarily divisible application the workload can be partitioned

dynamically; as the load increases, the system can allocate additional VMs to process the
additional workload.

Most cloud applications belong to this class, which justifies our statement that horizontal
scaling is the most common scaling mode.
There are several strategies to support scaling.

• Automatic VM scaling uses predefined metrics, e.g., CPU utilization, to make

scaling decisions.

• Automatic scaling requires sensors to monitor the state of VMs and servers;
controllers make decisions based on the information about the state of the cloud,
often using a statemachine model for decision making. Amazon and Rightscale
(www.rightscale.com) offer automatic scaling.

• In the case of AWS the CloudWatch service supports applications monitoring and
allows a user to set up conditions for automatic migrations.

• Nonscalable or single-load balancers are also used for horizontal scaling.

• The Elastic Load Balancing service from Amazon automatically distributes

incoming application traffic across multiple EC2 instances. Another service, the
Elastic Beanstalk, allows dynamic scaling between a low and a high number of
instances specified by the user (see Section 3.1). T