EECatalog SPECIAL FEATURE

Type Zero Hypervisor – the New

Frontier in Embedded Virtualization
The hypervisor’s full control over the hardware platform and ability to virtualize
hardware platforms are beneficial in environments that face high security threats
and demand high reliability.
By Will Keegan and Arun Subbarao, LynuxWorks, Inc.

Virtualization is a thriving technology proven to be suc- Hypervisors for IT Infrastructure

cessful in enterprise IT such as data centers and cloud The “hypervisor” is software that creates an abstraction
computing. However, technology vendors have only layer between hardware and operating systems, serving
scratched the surface on providing virtualization-based as the underlying technology of computer virtualization.
solutions, leaving untapped opportunities in industries Hypervisors achieve this layer of abstraction by taking
beyond IT, specifically in the security-critical and safety- full control over the physical computing platform to create
critical markets. A major tech producing industry that has software “virtual” hardware platforms that emulate the
yet to fully seize the expansive opportunities of virtual- underlying hardware (Figure 1). These emulated platforms
ization is the embedded computing world, which serves a then allow operating systems, referred to as guest OSs, to
wide set of markets from defense systems to biomedical run on the emulated platform instead of on the physical
devices. This slower adoption is due to the underlying hardware. The emulated platforms can be replicated
technology of virtualization – the hypervisor. Up until multiple times to support multiple guest OSs on the same
now, hypervisors were primarily designed to serve the machine, and can also be transferred to other hypervisor
popular demands of enterprise IT, focused to run in enabled machines.
IT server and desktop environments. As a result, these
enterprise IT hypervisors do not support the strict prop- Today, hypervisors are most commonly deployed on IT
erties commonly needed in embedded designs such as low servers and PCs to take advantage of multi-guest OS
power, small size, and determinism. However, as security operation, which reduces the cost of maintaining multiple
in these embedded devices becomes a significant concern, platforms and combines the capabilities offered by mul-
the possibly of using virtualization to achieve security in tiple flavors of OSs on a single platform. Hypervisors used
embedded devices is gaining momentum in the embedded in IT fit into two commonly designated architectures, type
market. 2 and type 1:

This article identifies unique security and reliability Type 2 hypervisors run as applications on top of a gen-
capabilities hypervisors have to offer to the embedded eral purpose OSs such as Windows or Mac OS. Type 2
community and how the new Type Zero Hypervisor is able hypervisors are commonly deployed to run user programs
to deliver these capabilities with its unique architecture. designed for OSs on a machine running a different OS; for
example, running Windows applications on a Mac.

Type 1, also referred to as bare metal, is a single software

hypervisor package that runs directly on
hardware. The software packages in today’s IT
type 1 hypervisors include a hypervisor inte-
grated, or paired, with a special purpose host
operating system and additional applications
to support features needed by the enterprise
IT market.

Existing type 2 and type 1 hypervisors are

unsuited for use in embedded systems because
they include a significant amount of unnec-
essary functionality that can greatly impact
the size, security, and performance of an
embedded system design.

Figure 1 - Hypervisor Embedded Hypervisors – Going

24 Engineers’ Guide to Embedded Linux and Android 2013

 EECatalog SPECIAL FEATURE

Beyond IT maintaining separation between the security domains

Hypervisors, if designed correctly, can offer benefits for (Figure 2).
embedded devices, and provide capabilities that are not
offered by today’s enterprise hypervisors. The hypervi- • Independent Measurement - In safety-critical envi-
sor’s full control over the hardware platform and ability ronments, systems are commonly built with redundant
to virtualize hardware platforms can be used to build components and system health monitors to detect the
advanced solutions to solve major problems in environ- event of a component failure and recover operation
ments that face a high security threat and demand high with redundant components. Hypervisors can create
reliability. independent computing environments that allow mis-
sion-critical functions to run without the interference
Some of the major security and reliability use cases offered of co-existing applications or complex dependencies of
by hypervisors are listed below:

• Security Domain Isolation - The hypervisor’s full

control over the hardware platform has the ability to
isolate access to hardware resources to create separate
computing environments for guest OSs that prohibit
unauthorized information flow between security
domains. Security domain isolation is extremely useful
in tactical defense systems deployed on size, weight, and
power (SWaP) restricted platforms, such as Humvees

Figure 4 - Hypervisor Reference Monitor

full operating systems. Using a hypervisor, a single com-

puting node can run a system application in one virtual
environment and an independent health monitor in a
separate environment to measure the status of the appli-
cation (Figure 3). In the event of an application error the
Figure 2 - Hypervisor Security Domain Isolation health monitor has the opportunity to locally reset the
application or direct a failover procedure for quicker
response time and smarter fault-tolerant designs.

• Reference Monitoring - Both safety-critical and

security-critical system computing nodes rely on data
channel interfaces for either local storage or inter-
system communication. A compromise in the integrity
or authenticity of data transferred over communication
channels can compromise the security and availability
of the entire system. Hypervisors can provide the ability
to independently mediate access and monitor infor-
mation flow between applications and data channel
interfaces to insure all information flow is un-tampered
Figure 3- Hypervisor Independent Measurement and always authorized to maintain operation.

and aircraft, that currently require multiple computing These hypervisor security and reliability use cases face
platforms to process separate levels of classified data. two major technical challenges: 1) Having a security
With a hypervisor a single computing platform can be foundation that hosts independent computing domains
used to process multiple levels of classified data while and controls information flow between guest OSs, critical

www.eecatalog.com/embeddedlinux 25
 EECatalog SPECIAL FEATURE

Figure 5 - Hypervisor Size Comparison Chart

functions, and system resources. 2) Availability of a hyper- embedded, security, and reliability requirements are rec-
visor that addresses the needs of embedded platforms. ognized and accommodated. The following properties are
These challenges by themselves are hard to satisfy with identified as key hypervisor architecture requirements for
today’s existing solutions. Trying to satisfy both requires embedded virtualization systems for use in safety-critical
a new design. and security-critical environments:

The “Type Zero” hypervisor architecture, designed by • Minimal Size - Embedded systems are commonly
LynuxWorks from the ground-up to operate in safety- faced with limiting storage and memory restrictions.
critical and security-critical environments while meeting Embedded solutions utilizing virtualization technology
the stringent demands of embedded computing platforms, must consider both the footprint of the guest OS and
fully satisfies the requirements of these and many other the foot print of the supporting hypervisor. Typical
use cases. embedded hypervisors consume less than 512 KB of
storage and less than 4MB of system RAM. In contrast,
Introducing “Type Zero” today’s available type 1 hypervisors require storage
“Type Zero” is a new bare-metal architecture, designed by footprints from hundreds of megabytes to several
LynuxWorks, that differentiates from type 1 by removing gigabytes before adding guest OS images, and consume
the all un-needed functionality from the “security sensi- several hundreds of megabytes to nearly a gigabyte of
tive” hypervisor mode yet virtualizes guest operating RAM. The base storage and memory footprint of type 1
systems in a tiny stand-alone package. By shedding the hypervisors range from tens to thousands times larger
need of support by a full operating system, the Type Zero than the demands of traditional embedded OSs which
hypervisor drastically reduces the size and computational may well exceed the size restrictions on an embedded
overhead imposed on target embedded systems. Figure 5 platform.
shows a comparison in size between type 2, type 1, and
Type Zero architectures, indicating that the majority of • Maximum Efficiency - Efficiency is very important for
code size in the type 2 and type 1 hypervisors is attributed embedded solutions that have demanding throughput
to the underlying host or helper OS. specifications or must operate in power-conscious
devices with very limited processing capabilities. In
Small size is one of many hypervisor design aspects needed order to maximize efficiency, hypervisors must only
by embedded systems. In order for hypervisors to operate contain the functionality that is necessary and suf-
in embedded mission critical systems, major architectural ficient to serve the guest OS & its applications. Type
design considerations must be addressed to ensure key 1 hypervisors, for example, depend on the underlying

26 Engineers’ Guide to Embedded Linux and Android 2013

 EECatalog SPECIAL FEATURE

support of a closed operating system, which may con- • Flexibility - Any foundational technology used in
sume unnecessary CPU cycles outside the control of the embedded systems requires flexibility for architects to
embedded system architect. mold the technology to fit their specific system designs.
Although hypervisors are mainly marketed for their
• Determinism - Embedded systems often rely on the ability to host multiple OSs, the hypervisor’s control
ability to guarantee the time of execution for all system over the physical hardware can provide capabilities that
operations. Having control over the timeliness of system go beyond emulating computer platforms. Type 1 hyper-
operations allows architects to construct solutions that visors provide a limiting user model that conforms to
ensure the proper behavior of mission-critical func- enterprise IT use cases.
tions and overall system availability. The biggest impact
hypervisors have on determinism is the scheduler used LynuxWorks’ LynxSecure Type Zero hypervisor exempli-
to assign guest OSs CPU processing cycles. In order fies these architectural principles to ensure that key
to perform any function that requires deterministic embedded mission-critical requirements can be realized
behavior in a virtualized environment, architects must using virtualization, as discussed in detail in the next sec-
have full control over the hypervisor scheduler to guar- tion.
antee that critical functions are scheduled to execute on
time, and to ensure that other low priority operations LynxSecure - Type Zero Hypervisor Archi-
do not interfere with critical processes. Type 1 hypervi- tecture
sors utilize a dynamic CPU scheduler that determines The design goal of the LynxSecure Type Zero hypervisor
the order of execution of guest OSs on CPU based on architecture is to provide a secure and reliable founda-
guest OS throughput demand. Dynamic CPU schedulers tion for virtualization platforms to serve a broad array
take control of execution from the system architect and of computing environments from embedded to enterprise
pass it to the guest applications, which invariably get systems. This objective of providing a secure foundation
exploited by rogue applications for DDoS attacks. with the features to serve an expansive market poses a
common paradox found in architecture design. A secure
• Security - Security is the most important property of and reliable foundation demands a small and simple
a hypervisor running in high threat environments. The code base, but offering broad functionality increases
hypervisor is privileged software responsible for orches- complexity which can compromise size and security. Lynx-
trating the simultaneous execution of guest OSs while Secure’s Type Zero architecture solves this problem of by
protecting each guest OS’s integrity, confidentiality, establishing a foundational core needed by all virtualiza-
and availability. All code running in the hypervisor tion markets while providing an external configuration
has a direct impact the on overall security, reliability, framework that allows for many unique virtualization
and determinism of a hypervisor-enabled platform. solutions to be constructed, without imposing unneces-
Any unauthorized access or control over the hypervisor sary code bloat in the hypervisor core.
can be devastating for embedded solutions targeted for
operation in safety or security-critical environments. LynxSecure - Type Zero Hypervisor Core
The best way to strengthen the security of a hypervisor, The core foundation of the Type Zero hypervisor
or any system, is to limit the access components have establishes a baseline set of functionality to support a vir-
over privileged resources and to reduce the complexity tualization framework that will enable system architects
of the design. Type 1 hypervisors that rely on host OSs to build virtualization solutions for any market. The key
include complex privileged components like device to supporting this framework is selecting the minimal set
drivers, and I/O stacks. This creates a situation which of components needed maintain a secure, reliable, and
makes it very difficult to verify that the code in these efficient foundation for all forms Type Zero hypervisor
components do not possess an exploitable flaw to gain deployments. The following set of functional components
unauthorized access to the hypervisor. is implemented to comprise the LynxSecure Type Zero
hypervisor core foundation (Figure 6):
• Reliability - Reliability is the most important property
for safety-critical systems. Many factors contribute to • Real-time Virtual CPU (RTvCPU) Scheduler - The real-
the reliability of a hypervisor, including, design com- time virtual CPU scheduler orchestrates the execution
plexity, determinism, and foundational security. Type of general guest OSs, real-time guest OS, and bare-metal
1 hypervisors are heavily tested to maintain opera- applications) on the hardware CPU cores. The real-time
tion, but the reliance on a full operating system does scheduler gives system architects the flexibility to
introduce significant risk through complexities in core control execution scheduling on multiple, dedicated, or
components such as: dynamic process scheduling, full shared CPU cores with clock-tick precision to host real-
process model, dynamic memory management, file time OSs and applications. The virtual CPU scheduler
systems, I/O stacks, and third party device drivers. Any utilizes Intel VT-x to allow guest OSs to run directly on
flaw in these components can cause system failure. the CPU cores, reducing significant software complexity

www.eecatalog.com/embeddedlinux 27
 EECatalog SPECIAL FEATURE

Figure 6 - LynxSecure Type Zero Hypervisor Core

and computational overhead. Without VT-x, hypervisors interrupt signal routing for efficient asymmetric com-
require additional software support to emulate the CPU munication channels between guest OSs, bare-metal
for proper guest OS execution. applications, virtual devices, para-virtual devices, and
physical devices.
• Memory Manager - The memory manager allocates
the memory for each guest OS and is responsible for • Exception Handler - The exception handler manages
protecting the integrity and confidentiality of the infor- illegal or privileged guest OS operations to ensure all
mation stored and processed by each of the co-existing system operations do not subvert the availability, integ-
guests. Protecting the integrity and confidentiality rity, and confidentiality protections provided by the
of each guest OS is extremely important for solutions hypervisor.
that require security domain separation between guest
OSs. The memory manager also controls shared memory • Security Monitor - The security monitor is responsible
structures for intercommunication between guest OSs, for bringing the hypervisor into a secure state and con-
bare-metal applications, virtual devices, para-virtual tinuously monitors security critical hardware resources
devices, and physical devices. The memory manager’s to maintain a secure operational state. The security
role in fully protecting guest OS memory from unau- monitor relies on the Intel TXT feature set during the
thorized access is broken into two categories: protecting startup initialization process. Prior to loading the
unauthorized access to guest OS memory from co- hypervisor, the hardware trusted platform module
existing guest OSs, and protecting guest OS memory (TPM) is controlled via Intel’s TXT instruction set to
from external I/O devices. validate the Type Zero hypervisor is not compromised
and is ready to enter full operational state.
The memory manager is able to protect against unauthor-
ized access requests originated from guest OSs, however • System Audit - The system audit component is an
the memory manager must rely on Intel’s hardware VT-d advanced service for recording major security, safety,
to explicitly control the boundaries of memory read and or user defined system events that can be passed up to
write requests originating from external devices. In addi- guest OSs or bare-metal applications to build robust
tion to VT-d, the memory manager benefits from Intel’s fault detection, threat detection, and system recovery
recent extended page table (EPT) hardware feature. Using sub-systems.
EPT, guest OSs are able to directly manage their local
memory page tables, no longer requiring assistance from LynxSecure’s Type Zero hypervisor core design satisfies
the hypervisor which removes a significant bottleneck in the size, efficiency, determinism, security, and reliability
guest OS memory access performance. requirements of embedded mission-critical systems, while
leaving the need for flexibility up to the higher level vir-
• Hypercall API - The Hypercall API is a privileged hyper- tualization framework. By selecting a minimum set of
visor interface utilized by the virtualization framework functionality and utilizing Intel’s hardware assistance,
to provide guest OSs and bare-metal applications a the size and complexity of the core components are drasti-
facility for inter-guest communication, guest OS man- cally reduced to assure vital security and reliability logic
agement, audit, and maintenance management. is correct, while the software computational overhead is
minimized to improve latency for a stronger deterministic
• Interrupt Handler - The interrupt handler manages behavior.

28 Engineers’ Guide to Embedded Linux and Android 2013

 EECatalog SPECIAL FEATURE

Summary Arun Subbarao is Vice President of Engineering

Virtualization is a powerful technology that is changing at LynuxWorks, responsible for the develop-
the way organizations of all shapes and sizes do business ment of security, virtualization and operating-
through the greatly offered cost saving and security benefits. system products, as well as consulting services.
Up until now, however, virtualization has been confined to He has 20 years of experience in the software
IT server and PC environments leaving a world of untapped industry working on security, virtualization,
opportunity for technology producers to explore. With the operating systems and networking technologies. In this role,
help from advancements in hardware assisted virtualization he spearheaded the development of the award-winning
features from chip vendors like Intel, combined with the LynxSecure separation kernel and hypervisor product as well
vision from embedded RTOS company, LynuxWorks, the Type as software innovation in the areas of security, safety and
Zero hypervisor emerges to give the embedded community virtualization. He has also been a panelist and presenter at
the tools they need to deliver the benefits of virtualization several industry conferences. He holds a BS in Computer Sci-
beyond the realm of enterprise IT, into new industries with ence from India, MS in Computer Science from SUNY Albany
the most demanding security and reliability requirements. and an MBA from Santa Clara University.

Will Keegan is a technical specialist at LynuxWorks,

Inc., where he upholds a strategic role in supporting
sales, marketing, and engineering. He has over 7 years
of experience working in enterprise IT, safety-critical,
and security-critical industries. He previously served
as a product engineer for OIS where he worked on the
development and marketing of various high assurance cryptographic
network and embedded middleware products. Will also served as a
network engineer for USAA, building and maintaining world class
data centers. He graduated from the University of Texas at Austin in
2005, earning a B.S. in Computer Science.

www.eecatalog.com/embeddedlinux 29