Cloud Computing

CP 424

Lecture 2
Virtualization

1
1. Introuduction
2. Virtualization
1.1 CPU virtualization
1.2 Memory,
1.3 I/O virtualization
3. Storage Virtualization
4. Network Virtualization

2
1. Introduction
• Virtualization is the abstraction of the underlying physical
resources to logical resources to increase system elasticity.
• It is a critical aspect of cloud computing, equally important to the
providers and consumers of cloud services.
• It plays an important role in:
System security because it allows isolation of services running on the
same hardware.
Performance and reliability because it allows applications to migrate
from one platform to another.
The development and management of services offered by a provider.
Performance isolation.
3
1. Introduction
Before virtualization After virtualization

APP APP APP APP APP APP APP APP APP APP APP APP
APP APP APP
Win Linux Win Linux Win Linux Win Linux Win Linux Win Linux

Windows Linux LinuxVirtualization

Resource pool

Virtualization layer Virtualization layer

Server 1 Server 2 Server 3

 Shared resource pool.
 Scattered IT resources.
 OS decoupled from hardware and resource
 Tightly-coupled OS and hardware.
allocation from the shared resource pool.

4
1.1 Virtual Machine Monitor (VMM)
• A virtual machine monitor (VMM), also called a hypervisor, is the
software that securely partitions the resources of a computer
system into one or more virtual machines.
• A guest operating system is an operating system that runs under
the control of a VMM rather than directly on the hardware.
• VMM allows several operating systems to run concurrently on a
single hardware platform; at the same time.
• Also, enforces isolation among these systems, thus enhancing
security.
5
1.1 Virtual Machine Monitor (VMM)
• The VMM enables:
Multiple services to share the same platform.
The movement of a server from one platform to another, the so-
called live migration.
System modification while maintaining backward compatibility
with the original system
• VMM traps the guest OS privileged instruction and enforces its
execution.
• It guarantees the isolation of the individual VMs, and thus
ensures security and encapsulation, a major concern in cloud
computing.

6
1.1 Virtual Machine Monitor (VMM)
Physical machine VM

Guest OS Guest OS

Guest machine Guest machine

Host OS Hypervisor

Host machine Host machine

7
1.2 Virtual machines
• A virtual machine (VM) is an isolated environment that appears
to be a whole computer but actually only has access to a portion
of the computer resources.
• Virtual machines have been around since the early 1970s, when
IBM released its VM/370 operating system.
• There are two types of VMs;
i. Process VMs
ii. System VMs

8
1.2 Virtual machines
• A process VM is a virtual platform created for an individual
process and destroyed once the process terminates.
• Virtually all operating systems provide a process VM for each
one of the applications running.
• More interesting process VMs are those that support binaries
compiled on a different instruction set.
• When VM runs under the control of a normal OS and provides a
platform-independent host for a single application, we have an
application virtual machine (e.g., Java Virtual Machine [JVM]).
9
1.3 Categories of system virtualization
• A system virtual machine provides a complete system; each VM
can run its own OS, which in turn can run multiple applications.
• There are four categories of system virtualization;
i. Bare metal virtualization
ii. Hybrid virtualization
iii. Operating system-level virtualization
iv. Hosted virtualization

10
1.3 Categories of system virtualization
Bare metal Virtualization
• The hypervisor supports hardware-level virtualization on bare
metal devices like CPU, memory, disk and network interfaces.
• The hypervisor software sits directly between the physical
hardware and its OS.
• This virtualization layer is referred to as either the VMM or the
hypervisor.

11
• The hypervisor provides hypercalls for the guest OSes and
applications. Example architecture of Microsoft Hyper-V, and a
monolithic hypervisor architecture of the VMware ESX for
server virtualization

12
• VMware puts the VMM at Ring 0
and the guest OS at Ring 1.
• The VMM scans the instruction
stream and identifies the
privileged, control- and behavior-
sensitive instructions.
• When these instructions are
identified, they are trapped into
the VMM, which emulates the
behavior of these instructions.
13
• The method used in this emulation is called binary translation.
Therefore, full virtualization combines binary translation and
direct execution.
• The guest OS is completely decoupled from the underlying
hardware. Consequently, the guest OS is unaware that it is
being virtualized.

14
1.3 Categories of system virtualization
Hybrid virtualization
The VMM shares the hardware with the existing OS. Example:
VMWare Workstation.
Hosted virtualization
The VM runs on top of an existing OS . The main advantage of this
approach is that the VM is easier to build and install
Performance as well as the challenges to support complete
isolation of VMs make this solution less attractive for servers in a
cloud computing environment. Example: User-mode Linux.

15
Compared to the hypervisor/VMM architecture, the
performance of the host-based architecture may also be low.
When an application requests hardware access, it involves
four layers of mapping which downgrades performance
significantly.

16
• Operating system-level virtualization
Allows a physical server to run multiple isolated operating system
instances, subject to several constraints; the instances are known as
containers, virtual private servers (VPSs), or virtual environments
(VEs).

17
1.2 Virtual machines
Bare-metal Hybrid
Hosted virtualization OS-level virtualization
virtualization virtualization
App App App App App App
App App App
App OS System System
Service template librarytemplate library
OS OS console OS OS OS
Virtualization layer Virtual container
Virtual container
Virtual hardware
Host OS Virtualization layer Host OS Host OS management

Hosted Virtualization Bare-metal Virtualization OS-level Virtualization Hybrid Virtualization

• OS independent VMs • Easy to implement • High performance due
Advantage • Easy to implement • Wide compatibility with • Low management to no redundancy
OSs and applications overhead • Multiple OSs supported
• Application deployment
feasibility determined by • Requiring the
• Poor isolation because
whether the host OS supports the • Difficult to develop the underlying hardware to
Disadvantage containers share the
device virtualization layer support virtualization
same OS
• High management overhead extension
and performance losses
• WMware ESXServer
• VMware Workstation • Citrix XenServer
Vendor • Virtuozzo • Red Hat KVM
• Virtual PC • Microsoft Hyper-V
• HUAWEI FusionSphere
18
Bare-mental virtualization and hybrid virtualization are the major trends.
2.0 Aspects of Virtualization
1 CPU virtualization

Computing
virtualization
2 Memory virtualization

3 I/O virtualization

1 Raw devices and logical volumes

Storage
2 Storage device virtualization
virtualization
Host storage virtualization and file
3
systems

1 VMDq

Network
2 SR-IOV
virtualization

3…

20
2.1 Computing virtualization

• It involves virtualization of the following components

1. CPU virtualization
2. Memory virtualization
3. I/O virtualization

21
2.1.1 CPU virtualization
• VMs share the same CPU.
Using timer interrupts, a mechanism similar to that used on raw
OSs, instructions are trapped to the VMM upon triggering
"interrupt" and are then scheduled based on the scheduling
policy. OS

Instruction 1

VMM

Instruction (1)

Hardware CPU

22
• There are two classes of machine instructions:
Innocuous instructions, which are not sensitive
Sensitive instructions, which require special precautions at
execution time, such as;
Control sensitive, which are instructions that attempt to change either the
memory allocation or the privileged mode.
Mode sensitive, which are instructions whose behavior is different in the
privileged mode.

23
• There are three basic approaches to processor
virtualization:
i. Full virtualization, in which each virtual machine runs on an
exact copy of the actual hardware, and
ii. Para-virtualization, in which each virtual machine runs on a
slightly modified copy of the actual hardware
iii. Hardware-assisted virtualization

24
Full virtualization
• Full virtualization requires the hardware
abstraction layer of the guest OS to have
some knowledge about the hardware
• In full virtualization architecture; the
hardware is fully exposed to the guest OS,
which runs unchanged, and this ensures
that this direct execution mode is efficient.
• VMware VMMs are examples of full
virtualization

25
Paravirtualization
• Paravirtualization demands that the
guest OS be modified to run under the
VMM.
• This abstraction is meant to place many
I/O operations outside the virtual
environment, which makes it both
programmatically easier and more
efficient to execute device I/O than it
would be inside a virtual environment
• Xen and Microsoft's Hyper-V are
based on paravirtualization.

26
Hardware assisted virtualization
• The virtual machine simulates
hardware, so it can be
independent of the underlying
system hardware.
• A guest operating system using
emulation does not need to be
modified in any way.
• Hardware solutions improve not
only performance but also security
and, at the same time, simplify the
software systems.

27
2.1.2 Memory virtualization
• The physical memory
of a physical machine Guest physical memory
is managed in a unified VM1
1
VM2 VM3 VM4
manner and is divided 2
3
4
into several parts for 5

individual VMs. Virtual Machine Monitor (VMM)

5
1

2
3 4

Machine physical memory

28
• Physical machine OSs' comprehension and management of
memory:
• Memory begins with physical address 0.
• Memory is sequential.
• Virtualization brings about the following problems:
• Beginning with physical address 0: There is only one physical address
0, making it impossible for all clients to begin at 0.
• Sequential address: Although sequential physical addresses can be
allocated, the memory usage is low and rigid (unable to share memory).

29
• The key point of memory virtualization lies in introducing a new
layer of address space, that is, the physical address space of
clients.
• Clients think that they run in the actual physical address space;
however, they access physical addresses through the VMM in
fact.
• The VMM saves the mapping table that records relationships
between clients' address spaces and physical machines'
address spaces.

30
2.1.3 I/O virtualization

• I/O virtualization involves managing the routing of I/O requests

between virtual devices and the shared physical hardware.

• There are three ways to implement I/O virtualization:

Full device emulation,
Para-virtualization, and
Direct I/O

31
• Peripherals
• I/O virtualization needs to resolve the following two problems:
• Device discovery
• Controls the devices accessible to VMs.
• Access interception
• Processes devices' access requests through I/O ports.

32
Full device emulation:
• The functions of a device or bus
infrastructure, such as device
enumeration, identification,
interrupts, and DMA, are
replicated in software.
• The I/O access requests of the
guest OS are trapped in the
VMM which interacts with the I/O
devices.
33
• A single hardware device can be shared by multiple VMs that
run concurrently.
• However, software emulation runs much slower than the
hardware it emulates.
Para-I/O virtualization
• The para-virtualization method of I/O virtualization is typically
used in Xen.
• It is also known as the split driver model consisting of a
frontend driver and a backend driver

34
• The frontend driver is running in Domain U and the backend
driver is running in Domain 0.
• They interact with each other via a block of shared memory.
• The frontend driver manages the I/O requests of the guest
OSes and the backend driver is responsible for managing the
real I/O devices and multiplexing the I/O data of different VMs.
• Although para-I/O-virtualization achieves better device
performance than full device emulation, it comes with a higher
CPU overhead

35
• The front-end driver (blkfront) forwards data to the back-end driver
(blkback) through the interfaces provided by the VMM.
• Using the back-end driver, VMs' data is processed in a time-division
multi-channel manner.
Domain 0 Domain U

User Control
panel User mode
mode
Blkfront

Kernel Kernel
Device Blkback Blkfront
driver

VMM

Physical hardware (CPU, memory and I/O device) 36

• The front-end and back-end driver models require the re-
enablement of drivers.
• How to discover a device?
• The device information about all VMs is saved on the XenStore of
Domain0.
• The XenBus (a paravirtualization driver designed for Xen) on VMs
communicates with the XenStore of Domain0 to obtain device
information.
• The front-end driver for the device is loaded.

37
• How to intercept device data?
• The front-end device driver forwards data completely to the back-end
driver through the interfaces provided by the VMM.
• The back-end driver processes VM data on a time and channel basis.

38
Direct I/O virtualization
• It lets the VM access devices directly.
• Also can achieve close-to-native performance without high CPU
costs.
• Currently, the direct I/O virtualization implementations focus on
networking for mainframes.
• However, when a physical device is reclaimed (required by workload
migration) for later reassignment, it may have been set to an
arbitrary state (e.g., DMA to some arbitrary memory locations) that
can function incorrectly or even crash the whole system

39
• Since software-based I/O virtualization requires a very high
overhead of device emulation, hardware-assisted I/O
virtualization is critical.
• Intel VT-d supports the remapping of I/O DMA transfers and
device-generated interrupts.
• The architecture of VT-d provides the flexibility to support
multiple usage models that may run unmodified, special-
purpose, or “virtualization-aware” guest OSes

40
Self-virtualized I/O (SV-IO)
• Another way to help I/O virtualization is via self-virtualized I/O
(SV-IO)
• The key idea of SV-IO is to harness the rich resources of a
multicore processor.
• All tasks associated with virtualizing an I/O device are
encapsulated in SV-IO.
• It provides virtual devices and an associated access API to VMs
and a management API to the VMM.

41
3. What is storage virtualization?
• Definition
• Storage virtualization creates a logical layer of storage from
physical storage devices so that the storage resources are
available through the logical layer.
• Administrators can easily adjust storage resources to improve
storage utilization.
• End users can enjoy better storage performance and usability
due to the centralization of storage devices.

42
• Implementation of storage virtualization
• Raw Device + Logical Volume
• Storage Device Virtualization
• Host Storage Virtualization + File System

43
Principle of storage virtualization
Raw Device + Logical Storage Device Host Storage Virtualization
Volume Virtualization + File System

Storage devices (SAN or Storage devices support Storage devices (SAN or

local disks) are mounted the ability to create and local disks) are mounted
on hosts. manage storage units on hosts.
Physical volumes are using interfaces. Hosts create file systems
created at the generic Storage units divided on storage devices.
block layer and are then from storage devices are All VM disks are saved
divided into logical mounted on hosts to as files in file systems.
volumes to be managed. carry services.

44
3.1 Raw device + Logical volume
• "Raw device + Logical volume" is the most direct storage
controller method.
• Raw devices are divided into blocks in the unit of 1 GB at the
generic block layer.
• The system manages such blocks to maintain volumes.
• Storage device virtualization leverages storage devices
capabilities to maintain volumes.

45
• With this type of virtualization, storage devices can offer some
advanced storage functions such as thin provisioning, snapshot,
and linked clone.
• "Host storage virtualization + File system" allows hosts to
manage VM disk files through file systems.
• This type of virtualization allows the virtualization layer to
provide advanced services that are independent of storage
devices.

46
Raw device + Logical volume
Guest OS Guest OS
Blkfront Blkfront

Dom-0 kernel Back-end volume mounting driver

space

Logical volume Logical volume

Generic block layer

Device driver layer

Storage device

47
• "Raw device + logical volume" is the most direct storage
controller method.
• Raw devices are divided into blocks with 1GB unit at the
generic block layer.
• The system manages such blocks to maintain volumes.
• Simple I/O paths and highest read & write performance.
• No support for advanced services.

48
3.2 Storage device virtualization
Mgmt.
Storage mgmt. Storage mgmt.
node

Guest OS Guest OS
Blkfront Blkfront

Dom-0 kernel Back-end volume mounting driver

space

Logical volume Logical volume

Computing node
Generic block layer

Device driver layer

S5500T FusionStorage

49
• Storage device virtualization leverages the capabilities of
storage devices to maintain volumes.
• With this type of virtualization, storage devices can offer some
advanced storage functions such as thin provisioning, snapshot,
and linked clone.
• Currently, storage device virtualization applies only to Huawei
Advanced SAN and FusionStorage and provides a small
number of advanced functions.

50
3.3 Storage virtualization + File system
Guest OS Guest OS
Blkfront Blkfront

Dom-0 kernel User mode

Back-end volume mounting driver
space mounting driver

Image file

NFS File system

Generic block layer

Device driver layer

Host user space

NAS Local disk SAN

51
• "Host Storage Virtualization + File System" allows hosts to
manage VM disk files through file systems. This type of
virtualization allows the virtualization layer to provide advanced
services. Currently, this type of virtualization is widely applied.
• Support for heterogeneous storage devices and heterogeneous
servers.
• Rich advanced functions, independent of hardware devices.
• Performance loss due to long I/O paths.

52
Types of storage virtualization
Type Raw Device + Storage Device Host Storage
Logical Volume Virtualization Virtualization + File
System
Supported •Thin provisioning •Thin provisioning •Thin provisioning
service (only for IP SAN •Differential snapshot •Differential snapshot
3900) •Linked clone •Live and offline
•Full snapshot storage migration
•Disk expansion
•Reclamation of thin
disk space
•Linked clone
Supported •IP SAN •HUAWEI 5500T •IP SAN
storage •FC SAN •HUAWEI •FC SAN
•Local storage FusionStorage •NAS
•Local storage

53
4. Network virtualization
Traditional data center Cloud data center
VM Migrate
… … …

Server
Server

Computing
virtualization

Switch
Switch

54
• With computing virtualization, a server can be virtualized into
multiple hosts, meeting the inter-VM communication
requirements that traditional networks are unable to handle.
• A cloud computing data center allows dynamic migration of VMs,
which is hardly impossible in traditional data centers.

55
Large Layer 2 Network


Multipathing layer 2 network
Large-scale VM migration Network requirements

One physical server virtualized

into multiple VMs Virtual awareness

Multiple servers virtualized Large amount of horizontal

traffic Network requirements
into one virtual application

Cloud Network Ethernet carrying Network requirements

convergence
computing FCoE and DCB
Logical network isolation

Multi-tenant separation Network requirements

High reliability

Link redundancy Network requirements

Cloud computing service requirements introduce the

following new challenges for the DC network model:
Layer 2 network: basic requirements for large-scale VM
migration and storage convergence.
Multipathing: basic requirements for a large amount of
horizontal traffic and high reliability.

56
• VM：Virtual Machine
• ETH：Ethernet
• Fiber Channel over Ethernet (FCoE) maps fiber channels (FCs)
to Ethernet and inserts FC information to Ethernet packets so
that FC requests and data can be transmitted from servers to
FC SAN storage devices over Ethernet without dedicated FCs.
FCoE allows the transmission of LAN and FC SAN data over
the same communication cable.

57
• A converged network supports both LAN and SAN data
transmission, thereby reducing the number of data center
devices and cables, reducing power consumption, cooling loads,
and the number of network nodes, and simplifying management.
• It provides I/O integration solutions based on the FC storage
protocol to transmit SAN data over the Ethernet and maximizes
return on investment (ROI) in the existing FC SAN, such as FC-
SAN tools, employee training, established FC-SAN devices,
and the corresponding management architecture.

58
• Data Center Bridging (DCB) is defined to enhance traditional
Ethernet and build a lossless Ethernet network that can prevent
packet loss.
• The FCoE technology is the FC protocol that can be used on
Ethernet networks and does not allow packet loss, and both
Ethernet and the FCoE protocol can run properly over Ethernet
networks.

59
Virtual switching
Server Description: Enables virtual switching within the CPU
of a server.
CPU
1 Advantage: flexible extension of functionality.
1
vSwitch Disadvantage: performance deterioration due to
intensive workloads on the CPU.

PCI Description: Employs complete virtual switching onto

network interface cards (NICs).
NIC
2 2 Advantage: high performance due to less occupation
eSwitch of CPU resources.

Disadvantage: dedicated NICs required.

Network cable Description: Employs virtual switching onto physical

switches.
Physical switch
3 Advantage: Layer 2 feature inherited.
3
Disadvantage: low specifications, poor extensibility,
QBG
and Hypervisor incompatibility.

60
• vSwitch: is short for virtual switch. vSwitch implements Layer 2
Ethernet switching functions on servers CPUs, including VM
switching, QoS control, and security isolation.
• eSwitch: is short for embedded switch. eSwitch implements
Layer 2 Ethernet switching functions on servers network
adapters, including VM switching, QoS control, and security
isolation.

61
Large Layer 2 Network Technology
Tenan TRILL
SPB/AQ Tenan
CVLAN=10 mapped CVLAN=2
to I-SID=200
tB tB
Tena
CVLAN=1
I-SID 200 Tenant Tenant Tena
logical CVLAN=1 CVLAN=1
nt Ato I-
mapped
SID=100
broadcasting A
mapped to I-
SID=100
A nt A
domain
Logical broadcasting
domain shared by
I-SID 100
CVLAN 1 and CVLAN 2
logical
Tenant
CVLAN=1
broadcasting
domain
Tenan Tenan Tenan
CVLAN=1 CVLAN=2
BSID=200
mapped to I-
t Bto I-
mapped
SID=200
tB CVLAN=2
tB
Tenan
CVLAN=10 mapped
to I-SID=100 Tena
tA CVLAN=1

Feature SPB TRILL

nt A
Camp Huawei, Avaya, HP, Ericsson, and AL Cisco, Huawei, Brocade, HP, and IBM

VM live migration Supported Supported

Multi-tenant 16 million (24-bit I-SID identifier) 4000 (12-bit VLAN ID identifier)

Convergence Slow (integrated source computing Fast (distributed computing paths based on per-
speed paths and heavy workload) hop behavior (PHB) and light workload)
Not supported (non-compatible with
Compatibility Supported (compatible with existing switches)
existing switches)

62
• SPB: Shortest Path Bridging
• TRILL: Transparent Interconnection of Lots of Links
• The two technologies belong to the larger layer 2 network
technology based on both the Ethernet layer 2 network and the
dynamic routing intermediate system to intermediate system
(IS-IS).
• Difference: SPB supports a maximum of 16 million isolation
domain labels but is not compatible with existing switches.

63
• SV-IO defines one virtual interface (VIF) for every kind of
virtualized I/O device, such as virtual network interfaces, virtual
block devices (disk), virtual camera device and others.
• The guest OS interacts with the VIFs via VIF device drivers.
Each VIF consists of two message queues.
• One is for outgoing messages to the devices and the other is for
incoming messages from the devices.
• In addition, each VIF has a unique ID for identifying it in SV-IO.

64