HardFails: Insights into Software-Exploitable

Hardware Bugs
Ghada Dessouky and David Gens, Technische Universität Darmstadt; Patrick Haney and
Garrett Persyn, Texas A&M University; Arun Kanuparthi, Hareesh Khattri, and Jason
M. Fung, Intel Corporation; Ahmad-Reza Sadeghi, Technische Universität Darmstadt;
Jeyavijayan Rajendran, Texas A&M University
https://www.usenix.org/conference/usenixsecurity19/presentation/dessouky

This paper is included in the Proceedings of the

28th USENIX Security Symposium.
August 14–16, 2019 • Santa Clara, CA, USA
978-1-939133-06-9

Open access to the Proceedings of the

28th USENIX Security Symposium
is sponsored by USENIX.
 HardFails: Insights into Software-Exploitable Hardware Bugs

Ghada Dessouky† , David Gens† , Patrick Haney∗ , Garrett Persyn∗ , Arun Kanuparthi◦ ,
Hareesh Khattri◦ , Jason M. Fung◦ , Ahmad-Reza Sadeghi† , Jeyavijayan Rajendran∗
† Technische Universität Darmstadt, Germany. ∗ Texas A&M University, College Station, USA.
◦ Intel Corporation, Hillsboro, OR, USA.

ghada.dessouky@trust.tu-darmstadt.de,david.gens@trust.tu-darmstadt.de,
prh537@tamu.edu,gpersyn@tamu.edu,arun.kanuparthi@intel.com,
hareesh.khattri@intel.com,jason.m.fung@intel.com,
ahmad.sadeghi@trust.tu-darmstadt.de,jv.rajendran@tamu.edu
Abstract ticated attacks that combine software and hardware bugs to
exploit computing platforms at runtime [20, 23, 36, 43, 45, 64,
Modern computer systems are becoming faster, more efficient, 69, 72, 74]. These cross-layer attacks disrupt traditional threat
and increasingly interconnected with each generation. Thus, models, which assume either hardware-only or software-only
these platforms grow more complex, with new features con- adversaries. Such attacks may provoke physical effects to in-
tinually introducing the possibility of new bugs. Although the duce hardware faults or trigger unintended microarchitectural
semiconductor industry employs a combination of different states. They can make these effects visible to software adver-
verification techniques to ensure the security of System-on- saries, enabling them to exploit these hardware vulnerabilities
Chip (SoC) designs, a growing number of increasingly so- remotely. The affected targets range from low-end embedded
phisticated attacks are starting to leverage cross-layer bugs. devices to complex servers, that are hardened with advanced
These attacks leverage subtle interactions between hardware defenses, such as data-execution prevention, supervisor-mode
and software, as recently demonstrated through a series of execution prevention, and control-flow integrity.
real-world exploits that affected all major hardware vendors.
In this paper, we take a deep dive into microarchitectural Hardware vulnerabilities. Cross-layer attacks circumvent
security from a hardware designer’s perspective by reviewing many existing security mechanisms [20, 23, 43, 45, 64, 69, 72,
state-of-the-art approaches used to detect hardware vulnera- 74], that focus on mitigating attacks exploiting software vul-
bilities at design time. We show that a protection gap currently nerabilities. Moreover, hardware-security extensions are not
exists, leaving chip designs vulnerable to software-based at- designed to tackle hardware vulnerabilities. Their implemen-
tacks that can exploit these hardware vulnerabilities. Inspired tation remains vulnerable to potentially undetected hardware
by real-world vulnerabilities and insights from our industry bugs committed at design-time. In fact, deployed extensions
collaborator (a leading chip manufacturer), we construct the such as SGX [31] and TrustZone [3] have been targets of suc-
first representative testbed of real-world software-exploitable cessful cross-layer attacks [69, 72]. Research projects such
RTL bugs based on RISC-V SoCs. Patching these bugs may as Sanctum [18], Sanctuary [8], or Keystone [39] are also not
not always be possible and can potentially result in a product designed to ensure security at the hardware implementation
recall. Based on our testbed, we conduct two extensive case level. Hardware vulnerabilities can occur due to: (a) incor-
studies to analyze the effectiveness of state-of-the-art security rect or ambiguous security specifications, (b) incorrect design,
verification approaches and identify specific classes of vulner- (c) flawed implementation of the design, or (d) a combination
abilities, which we call HardFails, which these approaches thereof. Hardware implementation bugs are introduced either
fail to detect. Through our work, we focus the spotlight on through human error or faulty translation of the design in
specific limitations of these approaches to propel future re- gate-level synthesis.
search in these directions. We envision our RISC-V testbed SoC designs are typically implemented at register-transfer
of RTL bugs providing a rich exploratory ground for future level (RTL) by engineers using hardware description lan-
research in hardware security verification and contributing to guages (HDLs), such as Verilog and VHDL, which are synthe-
the open-source hardware landscape. sized into a lower-level representation using automated tools.
Just like software programmers introduce bugs to the high-
level code, hardware engineers may accidentally introduce
1 Introduction bugs to the RTL code. While software errors typically cause
a crash which triggers various fallback routines to ensure the
The divide between hardware and software security research safety and security of other programs running on the platform,
is starting to take its toll, as we witness increasingly sophis- no such safety net exists for hardware bugs. Thus, even mi-

USENIX Association 28th USENIX Security Symposium 213

nor glitches in the implementation of a module within the first deploys formal verification to perform exhaustive and
processor can compromise the SoC security objectives and complete verification of a hardware design, while the second
result in persistent/permanent denial of service, IP leakage, or leverages formal verification and path sensitization to check
exposure of assets to untrusted entities. for illegal data flows and fault tolerance.
Our second case study revealed that certain properties of
Detecting hardware security bugs. The semiconductor in-
RTL bugs pose challenges for state-of-the-art verification
dustry makes extensive use of a variety of techniques, such
techniques with respect to black-box abstraction, timing flow,
as simulation, emulation, and formal verification to detect
and non-register states. This causes security bugs in the RTL
such bugs. Examples of industry-standard tools include In-
of real-world SoCs to slip through the verification process.
cisive [10], Solidify [5], Questa Simulation and Questa For-
Our results from the two case studies indicate that particu-
mal [44], OneSpin 360 [66], and JasperGold [11]. These were
lar classes of hardware bugs entirely evade detection—even
originally designed for functional verification with security-
when complementing systematic tool-based verification ap-
specific verification incorporated into them later.
proaches with manual inspection. RTL bugs arising from
While a rich body of knowledge exists within the software complex and cross-modular interactions in SoCs render these
community (e.g., regarding software exploitation and tech- bugs extremely difficult to detect in practice. Furthermore,
niques to automatically detect software vulnerabilities [38, such bugs are exploitable from software, and thus can com-
46]), security-focused HDL analysis is currently lagging be- promise the entire platform. We call such bugs HardFails.
hind [35, 57]. Hence, the industry has recently adopted a To the best of our knowledge, this is the first work to pro-
security development lifecycle (SDL) for hardware [68] — vide a systematic and in-depth analysis of state-of-the-art
inspired by software practices [26]. This process combines hardware verification approaches for security-relevant RTL
different techniques and tools, such as RTL manual code au- bugs. Our findings shed light on the capacity of these tools and
dits, assertion-based testing, dynamic simulation, and auto- demonstrate reproducibly how bugs can slip through current
mated security verification. However, the recent outbreak of hardware security verification processes. Being also software-
cross-layer attacks [20, 23, 37, 43, 45, 47, 48, 49, 51, 52, 53, exploitable, these bugs pose an immense security threat to
64, 69, 74] poses a spectrum of difficult challenges for these SoCs. Through our work, we highlight why further research
security verification techniques, because they exploit complex is required to improve state-of-the-art security verification of
and subtle inter-dependencies between hardware and software. hardware. To summarize, our main contributions are:
Existing verification techniques are fundamentally limited in • Systematic evaluation and case studies: We compile
modeling and verifying these interactions. Moreover, they a comprehensive test harness of real-world RTL bugs, on
also do not scale with the size and complexity of real-world which we base our two case studies: (1) Hack@DAC’18,
SoC designs. in which 54 independent teams of researchers competed
Goals and Contributions. In this paper, we show that cur- worldwide over three months to find these bugs using
rent hardware security verification techniques are fundamen- manual RTL inspection and simulation techniques, and
tally limited. We provide a wide range of results using a (2) an investigation of the bugs using industry-leading
comprehensive test harness, encompassing different types formal verification tools that are representative of the
of hardware vulnerabilities commonly found in real-world current state of the art. Our results show that particular
platforms. To that end, we conducted two case studies to classes of bugs entirely evade detection, despite combin-
systematically and qualitatively assess existing verification ing both tool-based security verification approaches and
techniques with respect to detecting RTL bugs. Together manual analysis.
with our industry partners, we compiled a list of 31 RTL • Stealthy hardware bugs: We identify HardFails as
bugs based on public Common Vulnerabilities and Exposures RTL bugs that are distinctly challenging to detect using
(CVEs) [37, 43, 50, 54, 55] and real-world errata [25]. We in- industry-standard security verification techniques. We
jected bugs into two open-source RISC-V-based SoC designs, explain the fundamental limitations of these techniques
which we will open-source after publication. in detail using concrete examples.
We organized an international public hardware security • Open-sourcing: We will open-source our bugs testbed
competition, Hack@DAC, where 54 teams of researchers at publication to the community.
competed for three months to find these bugs. While a number
of bugs could not be detected by any of the teams, several 2 SoC Verification Processes and Pitfalls
participants also reported new vulnerabilities of which we
had no prior knowledge. The teams used manual RTL inspec- Similar to the Security Development Lifecycle (SDL) de-
tion and simulation techniques to detect the bugs. In industry, ployed by software companies [26], semiconductor compa-
these are usually complemented by automated tool-based and nies [15, 35, 40] have recently adapted SDL for hardware
formal verification approaches. Thus, our second case study design [57]. We describe next the conventional SDL process
focused on two state-of-the-art formal verification tools: the for hardware and the challenges thereof.

214 28th USENIX Security Symposium USENIX Association

 tedious, and complex process even for industry experts. Exist-
ing techniques largely rely on human expertise to define the
security test cases and run the tests. The correct security spec-
ifications must be exhaustively anticipated, identified, and
accurately and adequately expressed using security properties
that can be captured and verified by the tools. We discuss
these challenges further in Section 7.
Besides the specifications, the techniques and tools them-
selves are not scalable and are less effective in capturing
F IGURE 1: Typical Security Development Lifecycle (SDL) subtle semantics that are relevant to many vulnerabilities,
process followed by semiconductor companies. which is the focus of this work. We elaborate next on the lim-
itations of state-of-the-art hardware security verification tools
2.1 The Security Development Lifecycle commonly used by industry. To investigate the capabilities of
these tools, we then construct a comprehensive test-harness
(SDL) for Hardware
of real-world RTL vulnerabilities.
SDL is conducted concurrently with the conventional hard-
ware development lifecycle [68], as shown in Figure 1. The
3 Assessing Hardware Security Verification
top half of Figure 1 shows the hardware development lifecy-
cle. It begins with design exploration followed by defining In this section, we focus on why the verification of the secu-
the specifications of the product architecture. After the archi- rity properties of modern hardware is challenging and provide
tecture specification, the microarchitecture is designed and requirements for assessing existing verification techniques
implemented in RTL. Concurrently, pre-silicon verification under realistic conditions. First, we describe how these ver-
efforts are conducted until tape-out to detect and fix all func- ification techniques fall short. Second, we provide a list of
tional bugs that do not meet the functional specification. After common and realistic classes of hardware bugs, which we
tape-out and fabrication, iterations of post-silicon validation, use to construct a test harness for assessing the effectiveness
functional testing, and tape-out "spins" begin. This cycle is re- of these verification techniques. Third, we discuss how these
peated until no defects are found and all quality requirements bugs relate to the common security goals of a chip.
are met. Only then does the chip enter mass production and
is shipped out. Any issues found later in-field are debugged,
and the chip is then either patched if possible or recalled. 3.1 Limitations of Automated Verification
After architectural features are finalized, a security assess- Modern hardware designs are highly complex and incorpo-
ment is performed, shown in the bottom half of Figure 1. The rate hundreds of in-house and third-party Intellectual Property
adversary model and the security objectives are compiled in (IP) components. This creates room for vulnerabilities to be
the security specification. This typically entails a list of assets, introduced in the inter-modular interactions of the design hi-
entry points to access these assets, and the adversary capa- erarchy. Multi-core architectures typically have an intricate
bilities and architectural security objectives to mitigate these interconnect fabric between individual cores (utilizing com-
threats. These are translated into microarchitectural security plex communication protocols), multi-level cache controllers
specifications, including security test cases (both positive and with shared un-core and private on-core caches, memory and
negative). After implementation, pre-silicon security verifica- interrupt controllers, and debug and I/O interfaces.
tion is conducted using dynamic verification (i.e., simulation For each core, these components contain logical modules
and emulation), formal verification, and manual RTL reviews. such as fetch and decode stages, an instruction scheduler, indi-
The chip is not signed off for tape-out until all security specifi- vidual execution units, branch prediction, instruction and data
cations are met. After tape-out and fabrication, post-silicon se- caches, the memory subsystem, re-order buffers, and queues.
curity verification commences. The identified security bugs in These are implemented and connected using individual RTL
both pre-silicon and post-silicon phases are rated for severity modules. The average size of each module is several hundred
using the industry-standard scoring systems such as the Com- lines of code (LOC). Thus, real-world SoCs can easily ap-
mon Vulnerability Scoring System (CVSS) [30] and promptly proach 100,000 lines of RTL code, and some designs may
fixed. Incident response teams handle issues in shipped prod- even have millions of LOC. Automatically verifying, at the
ucts and provide patches, if possible. RTL level, the respective interconnections and checking them
against security specifications raises a number of fundamen-
2.2 Challenges with SDL tal challenges for the state-of-the-art approaches. These are
described below.
Despite multiple tools and security validation techniques used L-1: Cross-modular effects. Hardware modules are inter-
by industry to conduct SDL, it remains a highly challenging, connected in a highly hierarchical design with multiple inter-

USENIX Association 28th USENIX Security Symposium 215

dependencies. Thus, an RTL bug located in an individual security vulnerabilities in their interactions, such as illegal
module may trigger a vulnerability in intra- and inter-modular information leakages across different privilege levels. Identi-
information flows spanning multiple complex modules. Pin- fying RTL bugs that trigger such vulnerabilities is beyond the
pointing the bug requires analyzing these flows across the capabilities of existing techniques.
relevant modules, which is highly cumbersome and unreliable L-4: Hardware-software interactions. Some RTL bugs re-
to achieve by manual inspection. It also pushes formal veri- main indiscernible to hardware security verification tech-
fication techniques to their limits, which work by modeling niques because they are not explicitly vulnerable unless trig-
and analyzing all the RTL modules of the design to verify gered by the software. For instance, although many SoC ac-
whether design specifications (expressed using security prop- cess control policies are directly implemented in hardware,
erty assertions, invariants and disallowed information flows) some are programmable by the overlying firmware to allow
and implementation match. for post-silicon flexibility. Hence, reasoning on whether an
Detecting such vulnerabilities requires loading the RTL RTL bug exists is inconclusive when considering the hardware
code of all the relevant modules into the tools to model and RTL in isolation. These vulnerabilities would only materialize
analyze the entire state space, thus driving them quickly when the hardware-software interactions are considered, and
into state explosion due to the underlying modeling algo- existing techniques do not handle such interactions.
rithms [16, 21]. Alleviating this by providing additional com-
putational resources and time is not scalable as the complexity
of SoCs continues to increase. Selective "black-box" abstrac- 3.2 Constructing Real-World RTL Bugs
tion of some of the modules, state space constraining, and To systematically assess the state of the art in hardware se-
bounded-model checking are often used. However, they do curity verification with respect to the limitations described
not eliminate the fundamental problem and rely on interactive above, we construct a test harness by implementing a large
human expertise. Erroneously applying them may introduce number of RTL bugs in RISC-V SoC designs (cf. Table 1).
false negatives, leading to missed vulnerabilities. To the best of our knowledge, we are the first to compile and
L-2: Timing-flow gap. Current industry-standard techniques showcase such a collection of hardware bugs. Together with
are limited in capturing and verifying security properties re- our co-authors at Intel, we base our selection and construc-
lated to timing flow (in terms of clock cycle latency). This tion of bugs on a solid representative spectrum of real-world
leads to vast sources of information leakage due to software- CVEs [47, 48, 49, 51, 52, 53] as shown in Table 1. For in-
exploitable timing channels (Section 8). A timing flow exists stance, bug #22 was inspired by a recent security vulnerability
between the circuit’s input and output when the number of in the Boot ROM of video gaming mobile processors [56],
clock cycles required for the generation of the output depends which allowed an attacker to bring the device into BootROM
on input values or the current memory/register state. This can Recovery Mode (RCM) via USB access. This buffer over-
be exploited to leak sensitive information when the timing flow vulnerability affected many millions of devices and is
variation is discernible by an adversary and can be used to popularly used to hack a popular video gaming console1 .
infer inputs or memory states. This is especially problematic We extensively researched CVEs that are based on
for information flows and resource sharing across different software-exploitable hardware and firmware bugs and clas-
privilege levels. This timing variation should remain indis- sified them into different categories depending on the weak-
tinguishable in the RTL, or should not be measurable from nesses they represent and the modules they impact. We repro-
the software. However, current industry-standard security ver- duced them by constructing representative bugs in the RTL
ification techniques focus exclusively on the functional in- and demonstrated their software exploitability and severity
formation flow of the logic and fail to model the associated by crafting a real-world software exploit based on one of
timing flow. The complexity of timing-related security issues these bugs in Appendix D. Other bugs were constructed with
is aggravated when the timing flow along a logic path spans our collaborating hardware security professionals, inspired
multiple modules and involves various inter-dependencies. by bugs that they have previously encountered and patched
L-3: Cache-state gap. State-of-the-art verification tech- during the pre-silicon phase, which thus never escalated into
niques only model and analyze the architectural state of a CVEs. The chosen bugs were implemented to achieve cover-
processor by exclusively focusing on the state of registers. age of different security-relevant modules of the SoC.
However, they do not support analysis of non-register states, Since industry-standard processors are based on proprietary
such as caches, thus completely discarding modern proces- RTL implementations, we mimic the CVEs by reproducing
sors’ highly complex microarchitecture and diverse hierarchy and injecting them into the RTL of widely-used RISC-V SoCs.
of caches. This can lead to severe security vulnerabilities aris- We also investigate more complex microarchitecture features
ing due to state changes that are unaccounted for, e.g., the of another RISC-V SoC and discover vulnerabilities already
changing state of shared cache resources across multiple privi- existing in its RTL (Section 4). These RTL bugs manifest as:
lege levels. Caches represent a state that is influenced directly
or indirectly by many control-path signals and can generate 1 https://github.com/Cease-and-DeSwitch/fusee-launcher

216 28th USENIX Security Symposium USENIX Association

 • Incorrect assignment bugs due to variables, registers, • Hardware Vulnerability: The attacker has knowledge of
and parameters being assigned incorrect literal values, a vulnerability in the hardware design of the SoC (i.e., at
incorrectly connected or left floating unintended. the RTL level) and can trigger the bug from software.
• Timing bugs resulting from timing flow issues and in- • User Access: The attacker has complete control over a user-
correct behavior relevant to clock signaling such as in- space process, and thus can issue unprivileged instructions
formation leakage. and system calls in the basic RISC-V architecture.
• Incorrect case statement bugs in the finite state ma- • Secure Software: Software vulnerabilities and resulting
chine (FSM) models such as incorrect or incomplete attacks, such as code-reuse [65] and data-only attacks [27]
selection criteria, or incorrect behavior within a case. against the software stack, are orthogonal to the problem
• Incorrect if-else conditional bugs due to incorrect of cross-layer bugs. Thus, we assume all platform software
boolean conditions or incorrect behavior described is protected by defenses such as control-flow integrity [1]
within either branch. and data-flow integrity [13], or is formally verified.
• Specification bugs due to a mismatch between a spec- The goal of an adversary is to leverage the vulnerability
ified property and its actual implementation or poorly on the chip to provoke unintended functionality, e.g., access
specified / under-specified behavior. to protected memory locations, code execution with elevated
These seemingly minor RTL coding errors may constitute privileges, breaking the isolation of other processes running
security vulnerabilities, some of which are very difficult to on the platform, or permanently denying services. RTL bugs
detect during verification. This is because of their intercon- in certain hardware modules might only be exploitable with
nection and interaction with the surrounding logic that affects physical access to the victim device, for instance, bugs in de-
the complexity of the subtle side effects they generate in their bug interfaces. However, other bugs are software-exploitable,
manifestation. Some of these RTL bugs may be patched by and thus have a higher impact in practice. Hence, we focus on
modifying parts of the software stack that use the hardware software-exploitable RTL vulnerabilities, such as the exploit
(e.g., using firmware/microcode updates) to circumvent them showcased in Appendix D. Persistent denial of service (PDoS)
and mitigate specific exploits. However, since RTL is usually attacks that require exclusive physical access are out of scope.
compiled into hardwired integrated circuitry logic, the under- JTAG attacks, though they require physical access, are still in
lying bugs cannot, in principle, be patched after production. scope as the end user may be the attacker and might attempt to
The limited capabilities of current detection approaches in unlock the device to steal manufacturer secrets. Furthermore,
modeling hardware designs and formulating and capturing rel- exploiting the JTAG interface often requires a combination of
evant security assertions raise challenges for detecting some both physical access and privilege escalation by means of a
of these vulnerabilities, which we investigate in depth in this software exploit to enable the JTAG interface. We also note
work. We describe next the adversary model we assume for that an adversary with unprivileged access is a realistic model
our vulnerabilities and our investigation. for real-world SoCs: Many platforms provide services to other
devices over the local network or even over the internet. Thus,
the attacker can obtain some limited software access to the
3.3 Adversary Model
platform already, e.g., through a webserver or an RPC inter-
In our work, we investigate microarchitectural details at the face. Furthermore, we emphasize that this work focuses only
RTL level. However, all hardware vendors keep their propri- on tools and techniques used to detect bugs before tape-out.
etary industry designs and implementations closed. Hence,
we use an open-source SoC based on the popular open-source
4 HardFails: Hardware Security Bugs
RISC-V [73] architecture as our platform. RISC-V supports
a wide range of possible configurations with many standard In light of the limitations of state-of-the-art verification tools
features that are also available in modern processor designs, (Section 3.1), we constructed a testbed of real-world RTL
such as privilege level separation, virtual memory, and multi- bugs (Section 3.2) and conducted two extensive case stud-
threading, as well as optimization features such as config- ies on their detection (described next in Sections 5 and 6).
urable branch prediction and out-of-order execution. Based on our findings, we have identified particular classes of
RISC-V RTL is freely available and open to inspection hardware bugs that exhibit properties that render them more
and modification. While this is not necessarily the case for challenging to detect with state-of-the-art techniques. We call
industry-leading chip designs, an adversary might be able these HardFails. We now describe different types of these
to reverse engineer or disclose/steal parts of the chip using HardFails encountered during our analysis of two RISC-V
existing tools23 . Hence, we consider a strong adversary that SoCs, Ariane [59] and PULPissimo [61]. In Section 5.3, we
can also inspect the RTL code. describe the actual bugs we instantiated for our case studies.
In particular, we make the following assumptions: Ariane is a 6-stage in-order RISC-V CPU that implements
2 https://www.chipworks.com/ the RISC-V draft privilege specification and can run Linux
3 http://www.degate.org/ OS. It has a memory management unit (MMU) consisting of

USENIX Association 28th USENIX Security Symposium 217

TABLE 1: Detection results for bugs in PULPissimo SoC based on formal verification (SPV and FPV, i.e., JasperGold Security
Path Verification and Formal Property Verification) and our hardware security competition (M&S, i.e., manual inspection and
simulation). Check and cross marks indicate detected and undetected bugs, respectively. Bugs marked inserted were injected
by our team and based on the listed CVEs, while bugs marked native were already present in the SoC and discovered by the
participants during Hack@DAC. LOC denotes the number of lines of code, and states denotes the total number of logic states
for the modules needed to attempt to detect this bug.

# Bug Type SPV FPV M&S Modules LOC # States

1 Address range overlap between peripherals SPI Master and SoC Inserted (CVE-2018-12206 / 3 3 3 91 6685 1.5×1020
CVE-2019-6260 / CVE-2018-8933)

2 Addresses for L2 memory is out of the specified range. Native 3 3 3 43 6746 3.5×1013

3 Processor assigns privilege level of execution incorrectly from CSR. Native 7 3 3 2 1186 2.1×1096

4 Register that controls GPIO lock can be written to with software. Inserted (CVE-2017-18293) 3 3 7 2 1186 2.1×1096

5 Reset clears the GPIO lock control register. Inserted (CVE-2017-18293) 3 3 7 2 408 1

6 Incorrect address range for APB allows memory aliasing. Inserted (CVE-2018-12206 / 3 3 7 1 110 2
CVE-2019-6260)

7 AXI address decoder ignores errors. Inserted (CVE-2018-4850) 7 3 7 1 227 2

8 Address range overlap between GPIO, SPI, and SoC control peripherals. Inserted (CVE-2018-12206 / 3 3 3 68 14635 9.4×1021
(CVE-2017-5704)

9 Incorrect password checking logic in debug unit. Inserted (CVE-2018-8870) 7 3 7 4 436 1

10 Advanced debug unit only checks 31 of the 32 bits of the password. Inserted (CVE-2017-18347 / 7 3 7 4 436 16
CVE-2017-7564)

11 Able to access debug register when in halt mode. Native (CVE-2017-18347 / 7 3 3 2 887 1

12 Password check for the debug unit does not reset after successful check. Inserted (CVE-2017-7564) 7 3 3 4 436 16

13 Faulty decoder state machine logic in RISC-V core results in a hang. Native 7 3 3 2 1119 32

14 Incomplete case statement in ALU can cause unpredictable behavior. Native 7 3 3 2 1152 4

15 Faulty logic in the RTC causing inaccurate time calculation for security-critical flows, e.g., DRM. Native 7 3 7 1 191 1

16 Reset for the advanced debug unit not operational. Inserted (CVE-2017-18347) 7 7 3 4 436 16

17 Memory-mapped register file allows code injection. Native 7 7 3 1 134 1

18 Non-functioning cryptography module causes DOS. Inserted 7 7 7 24 2651 1

19 Insecure hash function in the cryptography module. Inserted (CVE-2018-1751) 7 7 7 24 2651 N/A

20 Cryptographic key for AES stored in unprotected memory. Inserted (CVE-2018-8933 / 7 7 7 57 8955 1
CVE-2014-0881 / CVE-2017-5704)

21 Temperature sensor is muxed with the cryptography modules. Inserted 7 7 3 1 65 1

22 ROM size is too small preventing execution of security code. Inserted (CVE-2018-6242 / ) 7 7 3 1 751 N/A
CVE-2018-15383)

23 Disabled the ability to activate the security-enhanced core. Inserted (CVE-2018-12206) 7 7 7 1 282 N/A

24 GPIO enable always high. Inserted (CVE-2018-1959) 7 7 7 1 392 1

25 Unprivileged user-space code can write to the privileged CSR. Inserted (CVE-2018-7522 / 7 7 3 1 745 1
CVE-2017-0352)

26 Advanced debug unit password is hard-coded and set on reset. Inserted (CVE-2018-8870) 7 7 3 1 406 16

27 Secure mode is not required to write to interrupt registers. Inserted (CVE-2017-0352) 7 7 3 1 303 1

28 JTAG interface is not password protected. Native 7 7 3 1 441 1

29 Output of MAC is not erased on reset. Inserted 7 7 3 1 65 1

30 Supervisor mode signal of a core is floating preventing the use of SMAP. Native 7 7 3 1 282 1

31 GPIO is able to read/write to instruction and data cache. Native 7 7 3 1 151 4

218 28th USENIX Security Symposium USENIX Association

data and instruction translation lookaside buffers (TLBs), a = Security Vulnerability

hardware page table walker, and a branch prediction unit to L2 L2

JTAG
Advanced ROM AXI AXI
enable speculative execution. Figure 4 in Appendix A shows Debug Unit Bank Bank
its high-level microarchitecture. Tightly Coupled Data Memory Interconnect
PULPissimo is an SoC based on a simpler RISC-V core
with both instruction and data RAM as shown in Figure 2. It
GPIO GPIO APB Event RISC-V
provides an Advanced Extensible Interface (AXI) for access- HWPE
Core
ing memory from the core. Peripherals are directly connected
to an Advanced Peripheral Bus (APB) which connects them
I2S I2S uDMA Peripheral Interface
to the AXI through a bridge module. It provides support for
autonomous I/O, external interrupt controllers and features a SPI Camera
I2C UART CLK Timer Debug
debug unit and an SPI slave. Master Interface

TLB Page Fault Timing Side Channel (L-1 & L-2). SPI CPI I2C UART
While analyzing the Ariane RTL, we noted a timing
side-channel leakage with TLB accesses. TLB page faults F IGURE 2: Hardware overview of the PULPissimo SoC. Each
due to illegal accesses occur in a different number of clock bug icon indicates the presence of at least one security vulner-
cycles than page faults that occur due to unmapped memory ability in the module.
(we contacted the developers and they acknowledged the
vulnerability). This timing disparity in the RTL manifests were pre-fetched into the cache (at the system privilege level)
in the microarchitectural behavior of the processor. Thus, do not get flushed. These shared cache entries are visible to
it constitutes a software-visible side channel due to the user-space software, thus enabling timing channels between
measurable clock-cycle difference in the two cases. Previous privileged and unprivileged software.
work already demonstrated how this can be exploited by Verifying the implementation of all the flush control signals
user-space adversaries to probe mapped and unmapped and their behavior in all different states of the processor
pages and to break randomization-based defenses [24, 29]. requires examining at least eight modules: ariane.sv -
Timing flow properties cannot be directly expressed by controller.sv - frontend.sv - id_stage.sv - icache.sv - fetch_fifo
simple properties or modeled by state-of-the-art verification - ariane_pkg.sv - csr_regfile.sv (see Figure 5). This is complex
techniques. Moreover, for this vulnerability, we identify at because it requires identifying and defining all the relevant
least seven RTL modules that would need to be modeled, security properties to be checked across these RTL modules.
analyzed and verified in combination, namely: mmu.sv - Since current industry-standard approaches do not support
nbdcache.sv - tlb.sv instantiations - ptw.sv - load_unit.sv expressive capturing and the verification of cache states, this
- store_unit.sv. Besides modeling their complex inter- and issue in the RTL can only be found by manual inspection.
intra-modular logic flows (L-1), the timing flows need to be
modeled to formally prove the absence of this timing channel Firmware-Configured Memory Ranges (L-4).
leakage, which is not supported by current industry-standard In PULPissimo, we added peripherals with injected bugs to
tools (L-2). Hence, the only alternative is to verify this reproduce bugs from CVEs. We added an AES encryption/de-
property by manually inspecting and following the clock cryption engine whose input key is stored and fetched from
cycle transitions across the RTL modules, which is highly memory tightly coupled to the processor. The memory ad-
cumbersome and error-prone. However, the design must still dress the key is stored in is unknown, and whether it is within
be analyzed to verify that timing side-channel resilience is the protected memory range or not is inconclusive by observ-
implemented correctly and bug-free in the RTL. This only ing the RTL alone. In real-world SoCs, the AES key is stored
becomes far more complex for real-world industry-standard in programmable fuses. During secure boot, the bootload-
SoCs. We show the RTL hierarchy of the Ariane core in er/firmware senses the fuses and stores the key to memory-
Figure 5 in Appendix A to illustrate its complexity. mapped registers. The access control filter is then configured
to allow only the AES engine access to these registers, thus
Pre-Fetched Cache State Not Rolled Back (L-1 & L-3). protecting this memory range. Because the open-source SoC
Another issue in Ariane is with the cache state when a system we used did not contain a fuse infrastructure, the key storage
return instruction is executed, where the privilege level of the was mimicked to be in a register in the Memory-Mapped I/O
core is not changed until this instruction is retired. Before (MMIO) space.
retirement, linear fetching (guided by branch prediction) of Although the information flow of the AES key is defined
data and instructions following the unretired system return in hardware, its location is controlled by the firmware.
instruction continues at the current higher system privilege Reasoning on whether the information flow is allowed or
level. Once the instruction is retired, the execution mode of the not using conventional hardware verification approaches is
core is changed to the unprivileged level, but the entries that inconclusive when considering the RTL code in isolation.

USENIX Association 28th USENIX Security Symposium 219

The vulnerable hardware/firmware interactions cannot be respectively. Phase 1 was conducted remotely over a two-
identified unless they are co-verified. Unfortunately, current month period. Phase 2 was conducted in an 8-hour time frame
industry-standard tools do not support this. co-located with DAC (Design Automation Conference).
For Phase 1, we chose the Pulpino [60] SoC since it was
Memory Address Range Overlap (L-1 & L-4). a real-world, yet not an overly complex SoC design for the
PULPissimo provides I/O support to its peripherals by map- teams to work with. It features a RISC-V core with instruction
ping them to different memory address ranges. If an address and data RAM, an AXI interconnect for accessing memory,
range overlap bug is committed at design-time or by firmware, with peripherals on an APB accessing the AXI through a
this can break access control policies and have critical secu- bridge module. It also features a boot ROM, a debug unit and
rity consequences, e.g., privilege escalation. We injected an a serial peripheral interface (SPI) slave. We inserted security
RTL bug where there is address range overlap between the bugs in multiples modules of the SoC, including the AXI,
SPI Master Peripheral and the SoC Control Peripheral. This APB, debug unit, GPIO, and bridge.
allowed the untrusted SPI Master to access the SoC Control For Phase 2, we chose the more complex PULPissimo [61]
memory address range over the APB bus. SoC, shown in Figure 2. It additionally supports hardware pro-
Verifying issues at the SoC interconnect in such complex cessing engines, DMA, and more peripherals. This allowed us
bus protocols is challenging since too many modules needed to extend the SoC with additional security features, making
to support the interconnect have to be modeled to properly room for additional bugs. Some native security bugs were dis-
verify their security. This increases the scope and the com- covered by the teams and were reported to the SoC designers.
plexity of potential bugs far beyond just a few modules, as
shown in Table 1. Such an effect causes an explosion of the
state space since all the possible states have to be modeled
5.2 Competition Objectives
accurately to remain sound. Proof kits for accelerated verifica- For Hack@DAC, we first implemented additional security
tion of advanced SoC interconnect protocols were introduced features in the SoC, then defined the security objectives and
to mitigate this for a small number of bus protocols (AMBA3 adversary model and accordingly inserted the bugs. Specify-
and AMBA4). However, this requires an add-on to the default ing the security goals and the adversary model allows teams to
software and many protocols are not supported4 . define what constitutes a security bug. Teams had to provide
a bug description, location of RTL file, code reference, the se-
5 Crowdsourcing Detection curity impact, adversary profile, and the proposed mitigation.
Security Features: We added password-based locks on the
We organized and conducted a capture-the-flag competition, JTAG modules of both SoCs and access control on certain
Hack@DAC, in which 54 teams (7 from leading industry peripherals. For the Phase-2 SoC, we also added a crypto-
vendors and 47 from academia) participated. The objective graphic unit implementing multiple cryptographic algorithms.
for the teams was to detect as many RTL bugs as possi- We injected bugs into these features and native features to
ble from those we injected deliberately in real-world open- generate security threats as a result.
source SoC designs (see Table 1). This is designed to mimic Security Goals: We provided the three main security goals
real-world bug bounty programs from semiconductor com- for the target SoCs to the teams. Firstly, unprivileged code
panies [17, 32, 62, 63]. The teams were free to use any tech- should not escalate beyond its privilege level. Secondly, the
niques: simulation, manual inspection, or formal verification. JTAG module should be protected against an adversary with
physical access. Finally, the SoCs should thwart software
5.1 Competition Preparation adversaries from launching denial-of-service attacks.

RTL of open-source RISC-V SoCs was used as the testbed

for Hack@DAC and our investigation. Although these SoCs 5.3 Overview of Competition Bugs
are less complex than high-end industry proprietary designs, As described earlier in Section 3.2, the bugs were selected
this allows us to feasibly inject (and detect) bugs into less and injected together with our Intel collaborators. They are
complex RTL. Thus, this represents the best-case results for inspired by their hardware security expertise and real-world
the verification techniques used during Hack@DAC and our CVEs (cf. Table 1) and aim to achieve coverage of different
investigation. Moreover, it allows us to open-source and show- security-relevant components of the SoC. Several participants
case our testbed and bugs to the community.Hack@DAC con- also reported a number of native bugs already present in the
sisted of two phases: a preliminary Phase 1 and final Phase 2, SoC that we did not deliberately inject. We describe below
which featured the RISC-V Pulpino and PULPissimo SoCs, some of the most interesting bugs.
4 http://www.marketwired.com/press-release/jasper- UDMA address range overlap: We modified the memory
introduces-intelligent-proof-kits-faster-more-accurate- address range of the UDMA so that it overlaps with the master
verification-soc-interface-1368721.htm port to the SPI. This bug allows an adversary with access to

220 28th USENIX Security Symposium USENIX Association

the UMDA memory to escalate its privileges and modify the cated intuitively that exploiting this parameter would lead
SPI memory. This bug is an example of the "Memory Address to privilege escalation attacks. The teams reported that they
Range Overlap" HardFail type in Section 4. Other address prioritized inspecting security-relevant modules of the SoC,
range configuration bugs (#1, 2, 6 and 8) were also injected such as the debug interfaces.
in the APB bus for different peripherals. Undetected bugs: Many inserted bugs were not detected.
GPIO errors: The address range of the GPIO memory was One was in the advanced debug unit, where the password bit
erroneously declared. An adversary with GPIO access can index register has an overflow (bug #9). This is an example of
escalate its privilege and access the SPI Master and SoC Con- a security flaw that would be hard to detect by methods other
trol. The GPIO enable was rigged to display a fixed erroneous than verification. Moreover, the presence of many bugs within
status of ’1’, which did not give the user a correct display of the advanced debug unit password checker further masked
the actual GPIO status. The GPIO lock control register was this bug. Another bug was the cryptographic unit key storage
made write-accessible by user-space code, and it was flawed in unprotected memory (bug #20). The teams could not detect
to clear at reset. Bugs #4, 5, 24 and 31 are such examples. it as they focused on the RTL code in isolation and did not
Debug/JTAG errors: The password-checking logic in the consider HW/FW interactions.
debug unit was flawed and its state was not being correctly Techniques used by the teams: The teams were free to use
reset after a successful check. We hard-coded the debug unit any techniques to detect the bugs but most teams eventually
password, and the JTAG interface was not password protected. relied on manual inspection and simulation.
Bugs #9, 10, 11, 16, 26, and 28 are such examples. • Formal verification: One team used an open-source
Untrusted boot ROM: A native bug (bug #22) would allow formal verification tool (VeriCoq), but they reported little
unprivileged compromise of the boot ROM and potentially success because these tools (i) did not scale well with
the execution of untrusted boot code at a privileged level, thus the complete SoC and (ii) required expertise to use and
disclosing sensitive information. define the security properties. Some teams deployed
Erroneous AXI finite-state machine: We injected a bug their in-house verification techniques, albeit with little
(bug #7) in the AXI address decoder such that, if an error success. They eventually resorted to manual analysis.
signal is generated on the memory bus while the underlining • Assertion-based simulation: Some teams prepared
logic is still handling an outstanding transaction, the next sig- RTL testbenches and conducted property-based simu-
nal to be handled will instead be considered operational by the lations using SystemVerilog assertion statements.
module unconditionally. This bug can be exploited to cause • Manual inspection: All teams relied on manual inspec-
computational faults in the execution of security-critical code tion methods since they are the easiest and most accessi-
(we showcase how to exploit this vulnerability—which was ble and require less expertise than formal verification, es-
not detected by all teams—in Appendix D). pecially when working under time constraints. A couple
Cryptographic unit bugs: We injected bugs in a crypto- of teams reported prioritizing the inspection of security-
graphic unit that we inserted to trigger denial-of-service, a critical modules such as debug interfaces.
broken cryptographic implementation, insecure key storage, • Software-based testing: One team detected software-
and disallowed information leakage. Bugs #18, 19, 20, 21, exposure and privilege escalation bugs by running C
and 29 are such examples. code on the processor and attempting to make arbitrary
reads/writes to privileged memory locations. In doing
this, they could detect bugs #4, #8, #15, and #17.
5.4 Competition Results Limitations of manual analysis: While manual inspection
Various insights were drawn from the submitted bug reports can detect the widest array of bugs, our analysis of the
and results, which are summarized in Table 1. Hack@DAC results reveals its limitations. Manual analysis
is qualitative and difficult to scale to cross-layer and more
Analyzing the bug reports: Bug reports submitted by teams complex bugs. In Table 1, out of 16 cross-module bugs (span-
revealed which bug types were harder to detect and analyze ning more than one module) only 9 were identified using
using existing approaches. We evaluated the submissions and manual inspection. Three of them (#18, #19, and #20) were
rated them for accuracy and detail, e.g., bug validity, method- also undetected by formal verification methods, which is 10%
ology used, and security impact. of the bugs in our case studies.
Detected bugs: Most teams easily detected two bugs in
PULPissimo. The first one is where debug IPs were used 6 Detection Using State-of-The-Art Tools
when not intended. The second bug was where we declared
a local parameter PULP_SEC, which was always set to ’1’, Our study reveals two results: (1) a number of bugs could not
instead of the intended PULP_SECURE. The former was de- be detected by means of manual auditing and other ad-hoc
tected because debugging interfaces represent security-critical methods, and (2) the teams were able to find bugs already
regions of the chip. The latter was detected because it indi- existing in the SoC which we did not inject and were not

USENIX Association 28th USENIX Security Symposium 221

aware of. This prompted us to conduct a second in-house
7
case study to further investigate whether formal verification SPV
6 FPV
techniques can be used to detect these bugs. In practice, M&S
hardware-security verification engineers use a combination of 5 Undetected
techniques such as formal verification, simulation, emulation,

# of bugs
4
and manual inspection. Our first case study covered manual
inspection, simulation and emulation techniques. Thus, we 3

focused our second case study on assessing the effectiveness 2

of industry-standard formal verification techniques usually 1
used for verifying pre-silicon hardware security.
In real-world security testing (see Section 2), engineers will
Privilege DoS Secret Code
not have prior knowledge of the specific vulnerabilities they escalation leakage injection
Bug class
are trying to find. Our goal, however, is to investigate how an
industry-standard tool can detect RTL bugs that we deliber- F IGURE 3: Verification results grouped by bug class and
ately inject in an open-source SoC and have prior knowledge the number of bugs in each class detected by Security Path
of (see Table 1). Since there is no regulation or explicitly de- Verification (SPV), Formal Property Verification (FPV) and
fined standard for hardware-security verification, we focus our manual inspection and simulation techniques (M&S).
investigation on the most popular and de-facto standard for- niques to exhaustively and formally check if unauthorized
mal verification platform used in industry [11]. This platform data propagates (through a functional path) from a source
encompasses a representative suite of different state-of-the-art to a destination signal. To specify the SPV properties, we
formal verification techniques for hardware security assur- identified source signals where the sensitive information was
ance. As opposed to simulation and emulation techniques, located and destination signals where it should not propagate.
formal verification guarantees to model the state space of the We then identified the bounding preconditions to constrain the
design and formally prove the desired properties. We empha- paths the tool searches to alleviate state and time explosion.
size that we deliberately fix all other variables involved in the Similar to FPV, we identified the modules that are required
security testing process, in order to focus in a controlled set- to capture the information flow of interest. This must include
ting on testing the capacity and limitations of the techniques source, destination and intermediate modules, as well as mod-
and tools themselves. Thus, our results reflect the effective- ules that generate control signals which interfere with the
ness of tools in a best case where the bug is known a priori. information flow.
This eliminates the possibility of writing an incorrect security
property assertion which fails to detect the bug.
6.2 Detection Results
6.1 Detection Methodology Of the 31 bugs we investigated, shown in Table 1, using the
We examined each of the injected bugs and its nature in order formal verification techniques described above, only 15 (48%)
to determine which formal technique would be best suited to were detected. While we attempted to detect all 31 bugs for-
detect it. We used two formal techniques: Formal Property mally, we were able to formulate security properties for only
Verification (FPV) and JasperGold’s Security Path Verifica- 17 bugs. This indicates that the main challenge with using
tion (SPV) [12]. They represent the state of the art in hardware formal verification tools is identifying and expressing security
security verification and are used widely by the semiconductor properties that the tools are capable of capturing and checking.
industry [4], including Intel. Bugs due to ambiguous specifications of interconnect logic,
FPV checks whether a set of security properties, usually for instance, are examples of bugs that are difficult to create
specified as SystemVerilog Assertions (SVA), hold true for security properties for.
the given RTL. To describe the assertions correctly, we exam- Our results, shown in Figure 3, indicate that privilege es-
ined the location of each bug in the RTL and how its behavior calation and denial-of-service (DoS) bugs were the most de-
is manifested with the surrounding logic and input/output re- tected at 60% and 67% respectively. Secret leakage only had
lationships. Once we specified the security properties using a 17% detection rate due to incorrect design specification for
assert, assume and cover statements, we determined which one bug, state explosion and the inability to express proper-
RTL modules we need to model to prove these assertions. ties that the tool can assert for the remaining bugs. The code
If a security property is violated, the tool generates a coun- injection bug was undetected by formal techniques. Bugs at
terexample; this is examined to ensure whether the intended the interconnect level of the SoC such as bugs #1 and #2 were
security property is indeed violated or is a false alarm. especially challenging since they involved a large number of
SPV detects bugs which specifically involve unauthorized highly complex and inter-connected modules that needed to be
information flow. Such properties cannot be directly captured loaded and modeled by the tool (see L-1 in Section 3.1). Bug
using SVA/PSL assertions. SPV uses path sensitization tech- #20, which involves hardware/firmware interactions, was also

222 28th USENIX Security Symposium USENIX Association

detected by neither the state-of-the-art FPV nor SPV since resilient software5 . For vulnerabilities that can be patched,
they analyze the RTL in isolation (see L-4 in Section 3.1). We patches at this higher abstraction level in the firmware only
describe these bugs in more detail in Appendix C. act as a "symptomatic" fix that circumvent the RTL bug. How-
ever, they do not fundamentally patch the bug in the RTL,
which is already realized as hardwired logic. Thus, microcode
6.3 State-Explosion Problem patching is a fallback for RTL bugs discovered after produc-
Formal verification techniques are quickly driven into state tion, when you can not patch the RTL. They may also incur
space explosion when analyzing large designs with many performance impact 6 that could be avoided if the underlying
states. Many large interconnected RTL modules, like those problem is discovered and fixed during design.
relevant to bugs #1 and #2, can have states in the order of
magnitude of 1020 . Even smaller ones, like these used for bugs
#3 and #4, can have a very large number of states, as shown
in Table 1. When combined, the entire SoC will have a total 7.2 Additional Challenges in Practice
number of states significantly higher than any of the results
in Table 1. Attempting to model the entire SoC drove the tool Functional vs. Security Specifications. As described in Sec-
into state explosion, and it ran out of memory and crashed. tion 2, pre- and post-silicon validation efforts are conducted
Formal verification tools, including those specific to security to verify that the implementation fully matches both its func-
verification are currently incapable of handling so many states, tional and security specifications. The process becomes in-
even when computational resources are increased. This is creasingly difficult (almost impossible) as the system com-
further aggravated for industry-standard complex SoCs. plexity increases and specification ambiguity arises. Devi-
Because the entire SoC cannot be modeled and analyzed at ations from specification occur due to either functional or
once, detecting cross-modular bugs becomes very challeng- security bugs, and it is important to distinguish between them.
ing. Engineers work around this (not fundamentally solve While functional bugs generate functionally incorrect results,
it) by adopting a divide-and-conquer approach and selecting security bugs are not reflected in functionality. They arise due
which modules are relevant for the properties being tested to unconsidered and corner threat cases that are unlikely to
and which can be black-boxed or abstracted. However, this get triggered, thus making them more challenging to detect
is time-consuming, non-automated, error-prone, and requires and cover. It is, therefore, important to distinguish between
expertise and knowledge of both the tools and design. By functional and security specifications, since these are often
relying on the human factor, the tool can no longer guarantee the references for different verification teams working con-
the absence of bugs for the entire design, which is the original currently on the same RTL implementation.
advantage of formal verification. Specification Ambiguity. Another challenge entails antic-
ipating and identifying all the security properties that are
required in a real-world scenario. We analyzed the efficacy
7 Discussion and Future Work of industry-standard tools in a controlled setting—where we
have prior knowledge of the bugs. However, in practice hard-
We now describe why microcode patching is insufficient for ware validation teams do not have prior knowledge of the
RTL bugs while emphasizing the need for advancing the hard- bugs. Security specifications are often incomplete and am-
ware security verification process. We discuss the additional biguous, only outlining the required security properties under
challenges of the overall process, besides the limitations of an assumed adversary model. These specifications are inval-
the industry-standard tools, which is the focus of this work. idated once the adversary model is changed. This is often
the case with IP reuse, where the RTL code for one product
is re-purposed for another with a different set of security re-
7.1 Microcode Patching quirements and usage scenarios. Parameters may be declared
multiple times and get misinterpreted by the tools, thus caus-
While existing industry-grade SoCs support hotfixes by mi- ing bugs to slip undetected. Furthermore, specs usually do
crocode patching for instance, this approach is limited to a not specify bugs and information flows that should not exist,
handful of changes to the instruction set architecture, e.g., and there is no automated approach to determine whether one
modifying the interface of individual complex instructions is proving the intended properties. Thus, a combination of
and adding or removing instructions [25]. Some vulnerabili- incomplete or incorrect design decisions and implementation
ties cannot even be patched by microcode, such as the recent errors can easily introduce bugs to the design.
Spoiler attack [33]. Fundamentally mitigating this requires
fixing the hardware of the memory subsystem at the hardware 5 https://www.intel.com/content/www/us/en/security-
design phase. For legacy systems, the application developer is center/advisory/intel-sa-00238.html
advised to follow best practices for developing side channel- 6 https://access.redhat.com/articles/3307751

USENIX Association 28th USENIX Security Symposium 223

7.3 Future Research Directions estingly, this includes verifying that the test mechanisms are
correctly implemented in the RTL, otherwise they may consti-
Through our work, we shed light on the limitations of state- tute security vulnerabilities when used after fabrication (see
of-the-art verification techniques. In doing so, we hope to bugs#9,#10,#11,#12,#16, #26 of the JTAG/debug interface).
motivate further research in advancing these techniques to
Proof assistant and theorem-proving methods rely on
adequately capture and detect these vulnerabilities.
mathematically modeling the system and the required secu-
Although manual RTL inspection is generally useful and
rity properties into logical theorems and formally proving if
can potentially cover a wide array of bugs, its efficacy de-
the model complies with the properties. VeriCoq [7] based on
pends exclusively on the expertise of the engineer. This can
the Coq proof assistant transforms the Verilog code that de-
be inefficient, unreliable and ad hoc in light of rapidly evolv-
scribes the hardware design into proof-carrying code.VeriCoq
ing chip designs. Exhaustive testing of specifications through
supports the automated conversion of only a subset of Verilog
simulation requires amounts of resources exponential in the
code into Coq. However, this assumes accurate labeling of the
size of the input (i.e., design state space) while coverage
initial sensitivity labels of each and every signal in order to
must be intelligently maximized. Hence, current approaches
effectively track the flow of information. This is cumbersome,
face severe scalability challenges, as diagnosing software-
error-prone, generates many faluse positives, and does not
exploitable bugs that reside deep in the design pipeline can
scale well in practice beyond toy examples. Moreover, timing
require simulation of trillions of cycles [14]. Our results indi-
(and other) side-channel information flows are not modeled.
cate that it is important to first identify high-risk components
Finally, computational scalability to verifying real-world com-
due to software exposure, such as password checkers, crypto
plex SoCs remains an issue given that the proof verification
cores, and control registers, and prioritize analyzing them.
for a single AES core requires ≈ 30 minutes to complete [6].
Scalability due to complex inter-dependencies among mod-
Model checking-based approaches check a given prop-
ules is one challenge for detection. Vulnerabilities associated
erty against the modeled state space and possible state tran-
with non-register states (such as caches) or clock-cycle depen-
sitions using provided invariants and predefined conditions.
dencies (i.e., timing flows) are another open problem. Initial
They face scalability issues as computation time scales ex-
research is underway [71] to analyze a limited amount of
ponentially with the model and state space size. This can
low-level firmware running on top of a simulated RTL de-
be alleviated by using abstraction to simplify the model or
sign for information and timing flow violations. However,
constraining the state space to a bounded number of states
these approaches are still in their infancy and yet to scale for
using assumptions and conditions. However, this introduces
real-world SoC designs.
false positives, may miss vulnerabilities, and requires expert
knowledge. Most industry-leading tools, such as the one we
8 Related Work use in this work, rely on model checking algorithms such as
boolean satisfiability problem solvers and property specifica-
We now present related work in hardware security verifica- tion schemes, e.g., assertion-based verification to verify the
tion while identifying limitations with respect to detecting required properties of a given hardware design.
HardFails. We also provide an overview of recent software Side-channel leakage modeling and detection remain
attacks exploiting underlying hardware vulnerabilities. an open problem. Recent work [76] uses the Murϕ model
checker to verify different hardware cache architectures for
side-channel leakage against different adversary models. A
8.1 Current Detection Approaches formal verification methodology for SGX and Sanctum en-
Security-aware design of hardware has gained significance claves under a limited adversary was introduced in [67]. How-
only recently as the critical security threat posed by hardware ever, such approaches are not directly applicable to hardware
vulnerabilities became acutely established. Confidentiality implementation. They also rely exclusively on formal veri-
and integrity are the commonly investigated properties [19] fication and remain inherently limited by the underlying al-
in hardware security. They are usually expressed using infor- gorithms in terms of scalability and state space explosion,
mation flow properties between entities at different security besides demanding particular expertise to use.
levels. Besides manual inspection and simulation-based tech- Information flow analysis (such as SPV) works by assign-
niques, systematic approaches proposed for verifying hard- ing a security label (or a taint) to a data input and monitoring
ware security properties include formal verification methods the taint propagation. In this way, the designer can verify
such as proof assistance, model-checking, symbolic execu- whether the system adheres to the required security policies.
tion, and information flow tracking. We exclude the related Recently, information flow tracking (IFT) has been shown ef-
work in testing mechanisms, e.g., JTAG/scan-chain/built-in fective in identifying security vulnerabilities, including timing
self-test, because they are leveraged for hardware testing af- side channels and information-leaking hardware Trojans.
ter fabrication. However, the focus of this work is on veri- IFT techniques are proposed at different levels of abstrac-
fying the security of the hardware before fabrication. Inter- tion: gate-, RT, and language-levels. Gate-level information

224 28th USENIX Security Symposium USENIX Association

flow tracking (GLIFT) [2, 58, 70] performs the IFT analysis hard to detect in RTL due to the many cross-module connec-
directly at gate-level by generating GLIFT analysis logic that tions involved and the timing-flow leakage. The timing flow
is derived from the original logic and operates in parallel to it. leakage is caused by the software triggering clock cycle differ-
Although gate-level IFT logic is easy to automatically gener- ences in accesses that map to the same bank below cache line
ate, it does not scale well. Furthermore, when IFT uses strict granularity, thus breaking constant-time implementations.
non-interference, it taints any information flow conservatively The TLBleed [23] attack shows how current TLB imple-
as a vulnerability [34] which scales well for more complex mentations can be exploited to break state-of-the-art cache
hardware, but generates too many false positives. side-channel protections. As described in Section 4, TLBs
At the language level, Caisson [42] and Sapper [41] are are typically highly interconnected with complex processor
security-aware HDLs that use a typing system where the de- modules, such as the cache controller and memory manage-
signer assigns security "labels" to each variable (wire or reg- ment unit, making vulnerabilities therein very hard to detect
ister) based on the security policies required. However, they through automated verification or manual inspection.
both require redesigning the RTL using a new hardware de- BranchScope [20] extracts information through the direc-
scription language which is not practical. SecVerilog [22, 75] tional branch predictor, thus bypassing software mitigations
overcomes this by extending the Verilog language with a dy- that prevent leakage via the BTB. We classify it as a cache-
namic security type system. Designers assign a security label state gap in branch prediction units, which is significantly
to each variable (wire or register) in the RTL to enable a challenging to detect using existing RTL security verification
compile-time check of hardware information flow. However, tools, which cannot capture and verify cache states. Melt-
this involves complex analysis during simulation to reason down [43] exploits speculative execution on modern proces-
about the run-time behavior of the hardware state and depen- sors to completely bypass all memory access restrictions. Van
dencies across data types for precise flow tracking. Bulck et al. [72] also demonstrated how to apply this to Intel
Hardware/firmware co-verification to capture and verify SGX. Similarly, Spectre [37] exploits out-of-order execution
hardware/firmware interactions remains an open challenge across different user-space processes as arbitrary instruction
and is not available in widely used industry-standard tools. A executions would continue during speculation. We recognize
co-verification methodology [28] addresses the semantic gap these vulnerabilities are hard to detect due to scalability chal-
between hardware and firmware by modeling hardware and lenges in existing tools, since the out-of-order scheduling
firmware using instruction-level abstraction to leverage soft- module is connected to many subsystems in the CPU. Addi-
ware verification techniques. However, this requires modeling tionally, manually inspecting these interconnected complex
the hardware that interacts with firmware into an abstraction RTL modules is very challenging and cumbersome.
which is semi-automatic, cumbersome, and lossy. CLKScrew [69] abuses low-level power-management func-
While research is underway [71] to analyze a limited tionality that is exposed to software to induce faults and
amount of low-level firmware running on top of a simulated glitches dynamically at runtime in the processor. We cat-
RTL design these approaches are still under development and egorize CLKScrew to have vulnerable hardware-firmware
not scalable. Current verification approaches focus on register- interactions and timing-flow leakage, since it directly exposes
state information-flow analysis, e.g., to monitor whether sensi- clock-tuning functionality to attacker-controlled software.
tive locations are accessible from unprivileged signal sources.
Further research is required to explicitly model non-register
9 Conclusion
states and timing explicitly alongside the existing capabilities
of these tools. Software security bugs and their impact have been known for
many decades, with a spectrum of established techniques to
8.2 Recent Attacks detect and mitigate them. However, the threat of hardware
We present and cautiously classify the underlying hardware security bugs has only recently become significant as cross-
vulnerabilities of recent cross-layer exploits (see Table 2 in layer exploits have shown that they can completely undermine
Appendix B), using the categories introduced in 3.1. We do software security protections. While some hardware bugs can
not have access to proprietary processor implementations, be patched with microcode updates, many cannot, often leav-
so our classification is only based on our deductions from ing millions of affected chips in the wild. In this paper, we
the published technical descriptions. Yarom et al. demon- presented the first testbed of RTL bugs and systematically
strate that software-visible side channels can exist even below analyzed the effectiveness of state-of-the-art formal verifica-
cache-line granularity in CacheBleed [74]–undermining a tion techniques, manual inspection and simulation methods
core assumption of prior defenses, such as scatter-gather [9]. in detecting these bugs. We organized an international hard-
MemJam [45] exploits false read-after-write dependencies in ware security competition and an in-house study. Our results
the CPU to maliciously slow down victim accesses to mem- have shown that 54 teams were only able to detect 61% of
ory blocks within a cache line. We categorize the underlying the total number of bugs, while with industry-leading formal
vulnerabilities of CacheBleed and MemJam as potentially verification techniques, we were only able to detect 48% of

USENIX Association 28th USENIX Security Symposium 225

the bugs. We showcase that the grave security impact of many [5] Averant. Solidify. http://www.averant.com/storage/
of these undetected bugs is only further exacerbated by being documents/Solidify.pdf, 2018.
software-exploitable. [6] M.-M. Bidmeshki, X. Guo, R. G. Dutta, Y. Jin, and Y. Makris. Data Se-
Our investigation revealed the limitations of state-of-the- crecy Protection Through Information Flow Tracking in Proof-Carrying
art verification/detection techniques with respect to detecting Hardware IP—Part II: Framework Automation. IEEE Transactions on
Information Forensics and Security, 12(10):2430–2443, 2017.
certain classes of hardware security bugs that exhibit partic-
ular properties. These approaches remain limited in the face [7] M.-M. Bidmeshki and Y. Makris. VeriCoq: A Verilog-to-Coq Con-
of detecting vulnerabilities that require capturing and verify- verter for Proof-Carrying Hardware Automation. IEEE International
ing complex cross-module inter-dependencies, timing flows, Symposium on Circuits and Systems, pages 29–32, 2015.
cache states, and hardware-firmware interactions. While these [8] F. Brasser, D. Gens, P. Jauernig, A.-R. Sadeghi, and E. Stapf. SANC-
effects are common in SoC designs, they are difficult to model, TUARY: ARMing TrustZone with User-space Enclaves. Network and
capture, and verify using current approaches. Our investiga- Distributed System Security Symposium (NDSS), 2019.
tive work highlights the necessity of treating the detection [9] E. Brickell, G. Graunke, M. Neve, and J.-P. Seifert. Software mitiga-
of hardware bugs as significantly as that of software bugs. tions to hedge AES against cache-based software side channel vulnera-
Through our work, we highlight the pressing call for further bilities. IACR Cryptology ePrint Archive, 2006:52, 2006.
research to advance the state of the art in hardware security [10] Cadence. Incisive Enterprise Simulator. https://www.cadence.com/
verification. Particularly, our results indicate the need for in- content/cadence-www/global/en_US/home/tools/system-
creased scalability, efficacy and automation of these tools, design-and-verification/simulation-and-testbench-
making them easily applicable to large-scale commercial SoC verification/incisive-enterprise-simulator.html, 2014.
designs—without which software protections are futile. [11] Cadence. JasperGold Formal Verification Platform. https:
//www.cadence.com/content/cadence-www/global/en_US/
home/tools/system-design-and-verification/formal-
Acknowledgments and-static-verification/jasper-gold-verification-
platform.html, 2014.

We thank our anonymous reviewers and shepherd, Stephen [12] Cadence. JasperGold Security Path Verification App.
Checkoway, for their valuable feedback. The work was sup- https://www.cadence.com/content/cadence-www/global/en_
ported by the Intel Collaborative Research Institute for Col- US/home/tools/system-design-and-verification/formal-
and-static-verification/jasper-gold-verification-
laborative Autonomous & Resilient Systems (ICRI-CARS), platform/security-path-verification-app.html, 2018. Last
the German Research Foundation (DFG) by CRC 1119 accessed on 09/09/18.
CROSSING P3, and the Office of Naval Research (ONR
[13] M. Castro, M. Costa, and T. Harris. Securing software by enforcing
Award #N00014-18-1-2058). We would also like to ac- data-flow integrity. USENIX Symposium on Operating Systems Design
knowledge the co-organizers of Hack@DAC: Dan Holcomb and Implementation, pages 147–160, 2006.
(UMass-Amherst), Siddharth Garg (NYU), and Sourav Sudhir
(TAMU), and the sponsors of Hack@DAC: the National Sci- [14] D. P. Christopher Celio, Krste Asanovic. The Berkeley Out-of-Order
Machine. https://riscv.org/wp-content/uploads/2016/01/
ence Foundation (NSF CNS-1749175), NYU CCS, Mentor - a Wed1345-RISCV-Workshop-3-BOOM.pdf, 2016.
Siemens Business and CROSSING, as well as the participants
of Hack@DAC. [15] Cisco. Cisco: Strengthening Cisco Products. https://www.
cisco.com/c/en/us/about/security-center/security-
programs/secure-development-lifecycle.html, 2017.

References [16] E. M. Clarke, W. Klieber, M. Nováček, and P. Zuliani. Model check-

ing and the state explosion problem. Tools for Practical Software
[1] M. Abadi, M. Budiu, U. Erlingsson, and J. Ligatti. Control-flow in- Verification, 2012.
tegrity. ACM conference on Computer and communications security,
pages 340–353, 2005. [17] K. Conger. Apple announces long-awaited bug bounty program.
https://techcrunch.com/2016/08/04/apple-announces-
long-awaited-bug-bounty-program/, 2016.
[2] A. Ardeshiricham, W. Hu, J. Marxen, and R. Kastner. Register Trans-
fer Level Information Flow Tracking for Provably Secure Hardware
[18] V. Costan, I. A. Lebedev, and S. Devadas. Sanctum: Minimal Hardware
Design. Design, Automation & Test in Europe, pages 1695–1700, 2017.
Extensions for Strong Software Isolation. USENIX Security Symposium,
pages 857–874, 2016.
[3] ARM. Security technology building a secure system using trust-
zone technology (white paper). http://infocenter.arm.com/ [19] O. Demir, W. Xiong, F. Zaghloul, and J. Szefer. Survey of ap-
help/topic/com.arm.doc.prd29-genc-009492c/PRD29-GENC- proaches for security verification of hardware/software systems. https:
009492C_trustzone_security_whitepaper.pdf, 2009. //eprint.iacr.org/2016/846.pdf, 2016.

[4] R. Armstrong, R. Punnoose, M. Wong, and J. Mayo. Sur- [20] D. Evtyushkin, R. Riley, N. C. Abu-Ghazaleh, D. Ponomarev, et al.
vey of Existing Tools for Formal Verification. Sandia Na- BranchScope: A New Side-Channel Attack on Directional Branch
tional Laboratories https://prod.sandia.gov/techlib-noauth/ Predictor. ACM Conference on Architectural Support for Programming
access-control.cgi/2014/1420533.pdf, 2014. Languages and Operating Systems, pages 693–707, 2018.

226 28th USENIX Security Symposium USENIX Association

[21] F. Farahmandi, Y. Huang, and P. Mishra. Formal Approaches to Hard- [38] C. Lattner and V. S. Adve. LLVM: A compilation framework for
ware Trust Verification. The Hardware Trojan War, 2018. lifelong program analysis & transformation. International Symposium
on Code Generation and Optimization, 2004.
[22] A. Ferraiuolo, R. Xu, D. Zhang, A. C. Myers, and G. E. Suh. Verifi-
cation of a Practical Hardware Security Architecture Through Static [39] D. Lee. Keystone enclave: An open-source secure enclave for risc-v.
Information Flow Analysis. ACM Conference on Architectural Support https://keystone-enclave.org/, 2018.
for Programming Languages and Operating Systems, pages 555–568,
2017. [40] Lenovo. Lenovo: Taking Action on Product Security.
https://www.lenovo.com/us/en/product-security/about-
[23] B. Gras, K. Razavi, H. Bos, and C. Giuffrida. Translation Leak-aside lenovo-product-security, 2017.
Buffer: Defeating Cache Side-channel Protections with TLB Attacks.
USENIX Security Symposium, 2018. [41] X. Li, V. Kashyap, J. K. Oberg, M. Tiwari, V. R. Rajarathinam, R. Kast-
ner, T. Sherwood, B. Hardekopf, and F. T. Chong. Sapper: A Language
[24] D. Gruss, C. Maurice, A. Fogh, M. Lipp, and S. Mangard. Prefetch Side- for Hardware-level Security Policy Enforcement. International Con-
Channel Attacks: Bypassing SMAP and Kernel ASLR. Proceedings of ference on Architectural Support for Programming Languages and
the 2016 ACM SIGSAC Conference on Computer and Communications Operating Systems, pages 97–112, 2014.
Security, pages 368–379, 2016.
[42] X. Li, M. Tiwari, J. K. Oberg, V. Kashyap, F. T. Chong, T. Sherwood,
[25] M. Hicks, C. Sturton, S. T. King, and J. M. Smith. SPECS: and B. Hardekopf. Caisson: A Hardware Description Language for
A Lightweight Runtime Mechanism for Protecting Software from Secure Information Flow. ACM SIGPLAN Conference on Programming
Security-Critical Processor Bugs. In Proceedings of the International Language Design and Implementation, 46(6):109–120, 2011.
Conference on Architectural Support for Programming Languages and
Operating Systems, ASPLOS. ACM, 2015. [43] M. Lipp, M. Schwarz, D. Gruss, T. Prescher, W. Haas, S. Mangard,
P. Kocher, D. Genkin, Y. Yarom, and M. Hamburg. Meltdown. https:
[26] M. Howard and S. Lipner. The Security Development Lifecycle. Mi- //arxiv.org/abs/1801.01207, 2018.
crosoft Press Redmond, 2006.
[44] Mentor. Questa Verification Solution. https://www.mentor.com/
[27] H. Hu, S. Shinde, A. Sendroiu, Z. L. Chua, P. Saxena, and Z. Liang. products/fv/questa-verification-platform, 2018.
Data-oriented programming: On the expressiveness of non-control data
attacks. IEEE Symposium on Security and Privacy, 2016. [45] A. Moghimi, T. Eisenbarth, and B. Sunar. MemJam: A false depen-
dency attack against constant-time crypto implementations in SGX.
[28] B.-Y. Huang, S. Ray, A. Gupta, J. M. Fung, and S. Malik. Formal Se- Cryptographers’ Track at the RSA Conference, pages 21–44, 2018.
curity Verification of Concurrent Firmware in SoCs Using Instruction- 10.1007/978-3-319-76953-0_2.
level Abstraction for Hardware. ACM Annual Design Automation
Conference, pages 91:1–91:6, 2018. [46] F. Nielson, H. R. Nielson, and C. Hankin. Principles of program
analysis. Springer, 1999.
[29] R. Hund, C. Willems, and T. Holz. Practical timing side channel attacks
against kernel space ASLR. Symposium on Security and Privacy, 2013. [47] NIST. HP: Remote update feature in HP LaserJet printers does not re-
quire password. https://nvd.nist.gov/vuln/detail/CVE-2004-
[30] F. Inc. Common Vulnerability Scoring System v3.0. https://www. 2439, 2004.
first.org/cvss/cvss-v30-specification-v1.8.pdf, 2018.
[48] NIST. Microsoft: Hypervisor in Xbox 360 kernel allows attackers
[31] Intel. Intel Software Guard Extensions (Intel SGX). https:// with physical access to force execution of the hypervisor syscall with a
software.intel.com/en-us/sgx, 2016. Last accessed on 09/05/18. certain register set, which bypasses intended code protection. https:
//nvd.nist.gov/vuln/detail/CVE-2007-1221, 2007.
[32] Intel. Intel Bug Bounty Program. https://www.intel.
com/content/www/us/en/security-center/bug-bounty- [49] NIST. Apple: Multiple heap-based buffer overflows in the AudioCodecs
program.html, 2018. library in the iPhone allows remote attackers to execute arbitrary code
or cause DoS via a crafted AAC/MP3 file. https://nvd.nist.gov/
[33] S. Islam, A. Moghimi, I. Bruhns, M. Krebbel, B. Gulmezoglu, T. Eisen- vuln/detail/CVE-2009-2206, 2009.
barth, and B. Sunar. SPOILER: Speculative Load Hazards Boost
Rowhammer and Cache Attacks. https://arxiv.org/abs/1903. [50] NIST. Broadcom Wi-Fi chips denial of service. https://nvd.nist.
00446, 2019. gov/vuln/detail/CVE-2012-2619, 2012.
[34] R. Kastner, W. Hu, and A. Althoff. Quantifying Hardware Security [51] NIST. Vulnerabilities in Dell BIOS allows local users to bypass in-
Using Joint Information Flow Analysis. IEEE Design, Automation & tended BIOS signing requirements and install arbitrary BIOS images.
Test in Europe, pages 1523–1528, 2016. https://nvd.nist.gov/vuln/detail/CVE-2013-3582, 2013.
[35] H. Khattri, N. K. V. Mangipudi, and S. Mandujano. Hsdl: A security [52] NIST. Google: Escalation of Privilege Vulnerability in MediaTek
development lifecycle for hardware technologies. IEEE International WiFi driver. https://nvd.nist.gov/vuln/detail/CVE-2016-
Symposium on Hardware-Oriented Security and Trust, pages 116–121, 2453, 2016.
2012.
[53] NIST. Samsung: Page table walks conducted by MMU during Virtual
[36] Y. Kim, R. Daly, J. Kim, C. Fallin, J. H. Lee, D. Lee, C. Wilkerson, to Physical address translation leaves in trace in LLC. https://nvd.
K. Lai, and O. Mutlu. Flipping bits in memory without accessing them: nist.gov/vuln/detail/CVE-2017-5927, 2017.
An experimental study of DRAM disturbance errors. ACM SIGARCH
Computer Architecture News, 42(3):361–372, 2014. [54] NIST. AMD: Backdoors in security co-processor ASIC. https://
nvd.nist.gov/vuln/detail/CVE-2018-8935, 2018.
[37] P. Kocher, D. Genkin, D. Gruss, W. Haas, M. Hamburg, M. Lipp,
S. Mangard, T. Prescher, M. Schwarz, and Y. Yarom. Spectre At- [55] NIST. AMD: EPYC server processors have insufficient access con-
tacks: Exploiting Speculative Execution. http://arxiv.org/abs/ trol for protected memory regions. https://nvd.nist.gov/vuln/
1801.01203, 2018. detail/CVE-2018-8934, 2018.

USENIX Association 28th USENIX Security Symposium 227

[56] NIST. Buffer overflow in bootrom recovery mode of nvidia tegra mo- [74] Y. Yarom, D. Genkin, and N. Heninger. CacheBleed: a timing attack on
bile processors. https://nvd.nist.gov/vuln/detail/CVE-2018- OpenSSL constant-time RSA. Journal of Cryptographic Engineering,
6242, 2018. 7(2):99–112, 2017. 10.1007/s13389-017-0152-y.

[57] J. Oberg. Secure Development Lifecycle for Hardware Becomes an Im- [75] D. Zhang, Y. Wang, G. E. Suh, and A. C. Myers. A Hardware De-
perative. https://www.eetimes.com/author.asp?section_id= sign Language for Timing-Sensitive Information-Flow Security. In-
36&doc_id=1332962, 2018. ternational Conference on Architectural Support for Programming
Languages and Operating Systems, pages 503–516, 2015.
[58] J. Oberg, W. Hu, A. Irturk, M. Tiwari, T. Sherwood, and R. Kastner. The-
oretical Analysis of Gate Level Information Flow Tracking. IEEE/ACM [76] T. Zhang and R. B. Lee. New Models of Cache Architectures Char-
Design Automation Conference, pages 244–247, 2010. acterizing Information Leakage from Cache Side Channels. ACSAC,
pages 96–105, 2014.
[59] PULP Platform. Ariane. https://github.com/pulp-platform/
ariane, 2018.

[60] PULP Platform. Pulpino. https://github.com/pulp-platform/ Appendix

pulpino, 2018.

[61] PULP Platform. Pulpissimo. https://github.com/pulp-

A Ariane Core and RTL Hierarchy
platform/pulpissimo, 2018. Figure 4 shows the high-level microarchitecture of the Ariane
[62] Qualcomm. Qualcomm Announces Launch of Bounty Program. core to visualize its complexity. This RISC-V core is far
https://www.qualcomm.com/news/releases/2016/11/17/ less complex than an x86 or ARM processor and their more
qualcomm-announces-launch-bounty-program-offering- sophisticated microarchitectural and optimization features.
15000-usd-discovery, 2018.
Figure 5 illustrates the hierarchy of the RTL components of
[63] Samsung. Rewards Program. https://security.samsungmobile. the Ariane core. This focuses only on the core and excludes all
com/rewardsProgram.smsb, 2018. uncore components, such as the AXI interconnect, peripherals,
the debug module, boot ROM, and RAM.
[64] M. Seaborn and T. Dullien. Exploiting the DRAM rowhammer bug to
gain kernel privileges. Black Hat, 15, 2015.

[65] H. Shacham. The geometry of innocent flesh on the bone: return-into-

B Recent Microarchitectural Attacks
libc without function calls (on the x86). ACM Symposium on Computer
and Communication Security, pages 552–561, 2007.
We reviewed recent microarchitectural attacks with respect
to existing hardware verification approaches and their limita-
[66] O. Solutions. OneSpin 360. https://www.onespin.com/ tions. We observe that the underlying vulnerabilities would
fileadmin/user_upload/pdf/datasheet_dv_web.pdf, 2013. be difficult to detect due to the properties that they exhibit,
[67] P. Subramanyan, R. Sinha, I. Lebedev, S. Devadas, and S. A. Seshia. A rendering them as potential HardFails. We do not have access
Formal Foundation for Secure Remote Execution of Enclaves. ACM to their proprietary RTL implementation and cannot inspect
SIGSAC Conference on Computer and Communications Security, pages the underlying vulnerabilities. Thus, we only infer from the
2435–2450, 2017.
published technical descriptions and errata of these attacks
[68] Sunny .L He and Natalie H. Roe and Evan C. L. Wood and Noel the nature of the underlying RTL issues. We classify in Ta-
Nachtigal and Jovana Helms. Model of the Product Development ble 2 the properties of these vulnerabilities that represent
Lifecycle. https://prod.sandia.gov/techlib-noauth/access- challenges for state-of-the-art hardware security verification.
control.cgi/2015/159022.pdf, 2015.

[69] A. Tang, S. Sethumadhavan, and S. Stolfo. CLKSCREW: exposing C Details on the Pulpissimo Bugs
the perils of security-oblivious energy managemen. USENIX Security
Symposium, pages 1057–1074, 2017. We present next more detail on some of the RTL bugs used in
[70] M. Tiwari, H. M. Wassel, B. Mazloom, S. Mysore, F. T. Chong, and our investigation.
T. Sherwood. Complete Information Flow Tracking from the Gates Up. Bugs in crypto units and incorrect usage: We extended
ACM International Conference on Architectural Support for Program- the SoC with a faulty cryptographic unit with a multiplexer
ming Languages and Operating Systems, pages 109–120, 2009.
to select between AES, SHA1, MD5, and a temperature sen-
[71] Tortuga Logic. Verifying Security at the Hardware/Software Boundary. sor. The multiplexer was modified such that a race condition
http://www.tortugalogic.com/unison-whitepaper/, 2017. occurs if more than one bit in the status register is enabled,
causing unreliable behavior in these security critical modules.
[72] J. Van Bulck, F. Piessens, and R. Strackx. Foreshadow: Extracting the
Keys to the Intel SGX Kingdom with Transient Out-of-Order Execution. Furthermore, both SHA-1 and MD5 are outdated and bro-
USENIX Security Symposium, 2018. ken cryptographic hash functions. Such bugs are not de-
tectable by formal verification, since they occur due to a
[73] A. Waterman, Y. Lee, D. A. Patterson, and K. Asanovic. The
RISC-V Instruction Set Manual. Volume 1: User-Level ISA, Version
specification/design issue and not an implementation flaw,
2.0. https://content.riscv.org/wp-content/uploads/2017/ therefore they are out of the scope of automated approaches
05/riscv-spec-v2.2.pdf, 2014. and formal verification methods. The cryptographic key is

228 28th USENIX Security Symposium USENIX Association

 F IGURE 4:Ariane
High-level
RISC-V architecture of the
Core RTL Module Ariane core [59].
Hierarchy
ariane

std_cache_
frontend id_stage issue_stage ex_stage commit_stage csr_regfile perf_counters controller
subsystem

alu branch_unit lsu csr_buffer icache

nbdcache mmu load_unit store_unit lsu_arbiter lsu_bypass data_s tag_

lfsr
ram sram

itlb dtlb ptw Acronym Legend

ras return address stack
btb branch trace buffer
bht branch history table
scoreboard re_name issue_read_operands dltb data translation lookaside buffer
iltb instruction translation lookaside buffer
mmu memory management unit
ariane_regfile
fetch instr_ nbdcache non-blocking data cache
ras btb bht
_fifo scan lsu load/store unit
csr configuration status register
id_stage instruction decode stage

F IGURE 5: Illustration of the RTL module hierarchy of the Ariane core.

Memory Information Cross- HW/FW- Cache-State Timing-Flow

Attack Privilege Level HardFail
Corruption Leakage modular Interaction Gap Gap

Cachebleed [74] unprivileged 7 3 7 7 7 3 3

TLBleed [23] unprivileged 7 3 3 7 3 3 3
BranchScope [20] unprivileged 7 3 7 7 3 7 3
Spectre [37] unprivileged 7 3 3 7 3 7 3
Meltdown [43] unprivileged 7 3 3 7 3 7 3
MemJam [45] supervisor 7 3 3 7 7 3 3
CLKScrew [69] supervisor 3 3 7 3 7 3 3
Foreshadow [72] supervisor 3 3 3 3 3 7 3

TABLE 2: Classification of the underlying vulnerabilities of recent microarchitectural attacks by their HardFail properties.

USENIX Association 28th USENIX Security Symposium 229

stored and read from unprotected memory, allowing an at-
Userspace
tacker access to the key. The temperature sensor register value
is incorrectly muxed as output instead of the crypto engine
Kernel
output and vice versa, which are illegal information flows that Task A Task B 4
could compromise the cryptographic operations.
3
L ISTING 1: Incorrect use of crypto RTL: The key input for OS Kernel
the AES (g_input) is connected to signal b. This signal is then Task B
IVT PCBA PCBB MM
passed through various modules until it connects directly to a 6
tightly coupled memory in the processor.
input logic [127:0] b, Task A
Core Core

DRAM
...
aes_1cc aes( Memory
.clk(0),
... NULL
2 Interconnect 5 PCBB 1
.rst(1),
.g_input(b),
.e_input(a), F IGURE 6: Our attack exploits a bug in the implementation
.o(aes_out) of the memory bus of the PULPissimo SoC: by 1 spamming
); the bus with invalid transactions an adversary can make 4
malicious write requests be set to operational.
Bugs in security modes: We replaced the standard under certain conditions (see bug number #7 in Table 1). In
PULP_SECURE parameter in the riscv_cs_registers and the first step 1 , the attacker generates a user program (Task
riscv_int_controller modules with another constant param- A) that registers a dummy signal handler for the segmenta-
eter to permanently disable the security/privilege checks for tion fault (SIGSEGV) access violation. Task A then executes a
these two modules. Another bug we inserted is switching the loop with 2 a faulting memory access to an invalid memory
write and read protections for the AXI bus interface, causing address (e.g., LW x5, 0x0). This will generate an error in
erroneous checks for read and write accesses. the memory subsystem of the processor and issue an invalid
Bugs in the JTAG module: We implemented a JTAG memory access interrupt (i.e., 0x0000008C) to the processor.
password-checker and injected multiple bugs in it, includ- The processor raises this interrupt to the running software (in
ing the password being hardcoded in the password checking this case the OS), using the pre-configured interrupt handler
file. The password checker also only checks the first 31 bits, routines in software. The interrupt handler in the OS will then
which reduces the computational complexity of brute-forcing forward this as a signal to the faulting task 3 , which keeps
the password. The password checker does not reset the state looping and continuously generating invalid accesses. Mean-
of the correctness of the password when an incorrect bit is while, the attacker launches a separate Task B, which will
detected, allowing for repeated partial checks of passwords then issue a single memory access 4 to a privileged memory
to end up unlocking the password checker. This is also facil- location (e.g., LW x6, 0xf77c3000). In this situation, multi-
itated by the fact that the index overflows after the user hits ple outstanding memory transactions will be generated on the
bit 31, allowing for an infinite cycling of bit checks. memory bus, all of which but one will be flagged as faulty by
the address decoder. An invalid memory access will always
D Exploiting Hardware Bugs From Software proceed the single access of Task B. Due to the bug in the
memory bus address decoder, 5 the malicious memory ac-
We now explain how one of our hardware bugs can be ex-
cess will become operational instead of triggering an error.
ploited in real-world by software. This RTL vulnerability
Thus, the attacker can issue read and write instructions to
manifests in the following way. When an error signal is gen-
arbitrary privileged (and unprivileged) memory by forcing the
erated on the memory bus while the underlining logic is still
malicious illegal access to be preceded with a faulty access.
handling an outstanding transaction, the next signal to be han-
Using this technique the attacker can eventually leverage this
dled will instead be considered operational by the module
read-write primitive, e.g., 6 to escalate privileges by writing
unconditionally. This lets erroneous memory accesses slip
the process control block (PCBB ) for his task to elevate the
through hardware checks at runtime. Armed with the knowl-
corresponding process to root. This bug leaves the attacker
edge about this vulnerability, an adversary can force memory
with access to a root process, gaining control over the en-
access errors to evade the checks. As shown in Figure 6, the
tire platform and potentially compromising all the processes
memory bus decoder unit (unit of the memory interconnect)
running on the system.
is assumed to have the bug. This causes errors to be ignored

230 28th USENIX Security Symposium USENIX Association