Chapter 17

Pipeline Hazards

1
 What Is Pipelining

• Laundry Example
• Ann, Brian, Cathy, Dave A B C D
each have one load of clothes
to wash, dry, and fold
• Washer takes 30 minutes

• Dryer takes 40 minutes

• “Folder” takes 20 minutes

Appendix A - Pipelining 2
 What Is Pipelining
6 7 8 9 10 11 Midnigh
PM Time t

30 40 20 30 40 20 30 40 20 30 40 20
T
a A
s
k
B
O
r
d C
e
r
D
Sequential laundry takes 6 hours for 4 loads
If they learned pipelining, how long would laundry take?
Appendix A - Pipelining 3
 What Is Pipelining
Start work ASAP
6 7 8 9 10 11 Midnigh
PM Time t

30 40 40 40 40 20
T
a A
s • Pipelined laundry takes
k 3.5 hours for 4 loads
B
O
r
d C
e
r
D

Appendix A - Pipelining 4
 What Is Pipelining
Lessons
Pipelining
• Pipelining doesn’t help
6 7 8 9 latency of single task, it helps
PM Time
throughput of entire workload
• Pipeline rate limited by
T slowest pipeline stage
a 30 40 40 40 40 20 • Multiple tasks operating
s simultaneously
k A
• Potential speedup = Number
pipe stages
O
r B • Unbalanced lengths of pipe
d stages reduces speedup
e • Time to “fill” pipeline and time
r C to “drain” it reduces speedup

Appendix A - Pipelining 5
 The Basic Pipeline For MIPS
Cycle Cycle Cycle Cycle Cycle Cycle Cycle
1 2 3 4 5 6 7
Re

ALU
Ifetc Re DMe
h g m g
I
n
s Re

ALU
Ifetc Re DMe
t h g m g

r.
Re

ALU
Ifetc Re DMe
O h g m g

r
d Re

ALU
Ifetc Re DMe
e h g m g

Figure
3.3 Appendix A - Pipelining 6
 Pipeline Hurdles
A.1 What is Pipelining?
A.2 The Major Hurdle of Pipelining-
Structural Hazards
-- Structural Hazards Limits to pipelining: Hazards prevent next
– Data Hazards instruction from executing during its designated
– Control Hazards clock cycle
A.3 How is Pipelining Implemented – Structural hazards: HW cannot support this
combination of instructions (single person to fold
A.4 What Makes Pipelining Hard to
Implement?
and put clothes away)
– Data hazards: Instruction depends on result of
A.5 Extending the MIPS Pipeline to
Handle Multi-cycle Operations prior instruction still in the pipeline (missing
sock)
– Control hazards: Pipelining of branches &
other instructions that change the PC
– Common solution is to stall the pipeline until the
hazard is resolved, inserting one or more
“bubbles” in the pipeline

Appendix A - Pipelining 7
 Pipeline Hurdles
Definition
• conditions that lead to incorrect behavior if not fixed
• Structural hazard
– two different instructions use same h/w in same cycle
• Data hazard
– two different instructions use same storage
– must appear as if the instructions execute in correct order
• Control hazard
– one instruction affects which instruction is next

Resolution
• Pipeline interlock logic detects hazards and fixes them
• simple solution: stall ­
• increases CPI, decreases performance
• better solution: partial stall ­
• some instruction stall, others proceed better to stall early than late

Appendix A - Pipelining 8
 Structural Hazards
When two or
Time (clock more different
cycles)
Cycle Cycle Cycle Cycle Cycle Cycle Cycle instructions want
1 2 3 4 5 6 7 to use same
hardware
Re

ALU
Ifetc Re DMe
I Loa h g m g resource in same
cycle
n d
s Re

ALU
Ifetc Re DMe
Instr h g m g e.g., MEM uses
t the same memory
r. 1 Re
port as IF as

ALU
Ifetc Re DMe
Instr h g m g shown in this
slide.
O 2 Re

ALU
Ifetc Re DMe
r Instr h g m g

d 3
e Re

ALU
Ifetc Re DMe
Instr h g m g
r
4
Figure
3.6 Appendix A - Pipelining 9
 Structural Hazards
Time (clock
cycles)
Cycle Cycle Cycle Cycle Cycle Cycle Cycle This is another
1 2 3 4 5 6 7 way of looking
Re at the effect of

ALU
Ifetc Re DMe
I Loa h g m g
a stall.
n d
s Re

ALU
Ifetc Re DMe

t Instr h g m g

r. 1
Re

ALU
Ifetc Re DMe
Instr h g m g

O 2
r Bu Bu Bu Bu Bu
d Stal bb bbl bbl bbl bbl
le e e e e
e l
Re

ALU
Ifetc Re DMe
r Instr h g m g

3
Figure
3.7 Appendix A - Pipelining 10
 Structural Hazards

This is another way to represent the stall we saw on

the last few pages.

Appendix A - Pipelining 11
 Structural Hazards
Dealing with Structural Hazards
Stall
• low cost, simple
• Increases CPI
• use for rare case since stalling has performance effect
Pipeline hardware resource
• useful for multi-cycle resources
• good performance
• sometimes complex e.g., RAM
Replicate resource
• good performance
• increases cost (+ maybe interconnect delay)
• useful for cheap or divisible resources

Appendix A - Pipelining 12
 Structural Hazards
Structural hazards are reduced with these rules:
• Each instruction uses a resource at most once
• Always use the resource in the same pipeline stage
• Use the resource for one cycle only
Many RISC ISA’a designed with this in mind
Sometimes very complex to do this. For example, memory of
necessity is used in the IF and MEM stages.

Some common Structural Hazards:

• Memory - we’ve already mentioned this one.
• Floating point - Since many floating point instructions require
many cycles, it’s easy for them to interfere with each other.
• Starting up more of one type of instruction than there are
resources. For instance, the PA-8600 can support two ALU + two
load/store instructions per cycle - that’s how much hardware it
has available.

Appendix A - Pipelining 13
 Structural Hazards
We want to compare the performance of two machines. Which machine is faster?
• Machine A: Dual ported memory - so there are no memory stalls
• Machine B: Single ported memory, but its pipelined implementation has a 1.05
times faster clock rate
Assume:
• Ideal CPI = 1 for both
• Loads are 40% of instructions executed

SpeedUpA = Pipeline Depth/(1 + 0) x (clockunpipe/clockpipe)

= Pipeline Depth
SpeedUpB = Pipeline Depth/(1 + 0.4 x 1)
x (clockunpipe/(clockunpipe / 1.05)
= (Pipeline Depth/1.4) x 1.05
= 0.75 x Pipeline Depth
SpeedUpA / SpeedUpB = Pipeline Depth / (0.75 x Pipeline Depth) = 1.33

• Machine A is 1.33 times faster

Appendix A - Pipelining 14
 Data Hazards
A.1 What is Pipelining?
A.2 The Major Hurdle of Pipelining-
Structural Hazards
-- Structural Hazards
These occur when at any time, there are
– Data Hazards instructions active that need to access the
– Control Hazards same data (memory or register) locations.
A.3 How is Pipelining Implemented
A.4 What Makes Pipelining Hard to Where there’s real trouble is when we have:
Implement?
A.5 Extending the MIPS Pipeline to instruction A
Handle Multi-cycle Operations instruction B

and B manipulates (reads or writes) data

before A does. This violates the order of the
instructions, since the architecture implies
that A completes entirely before B is executed.

Appendix A - Pipelining 15
 Data Hazards
Execution Order is:
Read After Write (RAW)
InstrI
InstrJ tries to read operand before InstrI writes it
InstrJ

I: add r1,r2,r3
J: sub r4,r1,r3
• Caused by a “Dependence” (in compiler nomenclature).
This hazard results from an actual need for
communication.

Appendix A - Pipelining 16
 Data Hazards
Execution Order is:
Write After Read (WAR)
InstrI
InstrJ tries to write operand before InstrI reads i
– Gets wrong operand
InstrJ

I: sub r4,r1,r3
J: add r1,r2,r3
K: mul r6,r1,r7
– Called an “anti-dependence” by compiler writers.
This results from reuse of the name “r1”.

• Can’t happen in MIPS 5 stage pipeline because:

– All instructions take 5 stages, and
– Reads are always in stage 2, and
– Writes are always in stage 5

Appendix A - Pipelining 17
 Data Hazards
Execution Order is:
Write After Write (WAW)
InstrI
InstrJ tries to write operand before InstrI writes it
InstrJ
– Leaves wrong result ( InstrI not InstrJ )

I: sub r1,r4,r3
J: add r1,r2,r3
K: mul r6,r1,r7

• Called an “output dependence” by compiler writers

This also results from the reuse of name “r1”.

• Can’t happen in MIPS 5 stage pipeline because:

– All instructions take 5 stages, and
– Writes are always in stage 5

• Will see WAR and WAW in later more complicated pipes

Appendix A - Pipelining 18
Data Hazards
Simple Solution to RAW

• Hardware detects RAW and stalls

• Assumes register written then read each cycle
+ low cost to implement, simple
-- reduces IPC
• Try to minimize stalls

Minimizing RAW stalls

• Bypass/forward/short­circuit (We will use the word “forward”)

• Use data before it is in the register
+ reduces/avoids stalls
-- complex
• Crucial for common RAW hazards

Appendix A - Pipelining 19
 Data Hazards
Time (clock
cycles)
IF ID/RF E ME W
X MDMe BRe
I

ALU
Ifetc Re
add h g m g
n
s
r1,r2,r3 Re

ALU
Ifetc Re DMe

t sub h g m g

r. r4,r1,r3 Re

ALU
Ifetc Re DMe
and h g m g

O
r6,r1,r7
r Re

ALU
Ifetc Re DMe

d or h g m g

e r8,r1,r9 Re

ALU
Ifetc Re DMe

r xor h g m g

r10,r1,r11
The use of the result of the ADD instruction in the next three instructions causes a
hazard, since the register is not written until after those instructions read it.

Figure
3.9 Appendix A - Pipelining 20
 Forwarding is the concept of making data

Data Hazards
available to the input of the ALU for
subsequent instructions, even though the
generating instruction hasn’t gotten to WB
Forwarding To Avoid in order to write the memory or registers.
Data Hazard
Time (clock
cycles)
I

ALU
Ifetc Re DMe Re
n add h g m g
s
t
r1,r2,r3

ALU
Ifetc Re DMe Re
r. sub h g m g

r4,r1,r3
O

ALU
Ifetc Re DMe Re

r and h g m g

d r6,r1,r7
e Re

ALU
Ifetc Re DMe

r or h g m g

r8,r1,r9

ALU
Ifetc Re DMe Re

xor h g m g

r10,r1,r11
Figure
3.10 Appendix A - Pipelining 21
 The data isn’t loaded until after
Data Hazards the MEM stage.
Time (clock
cycles)
Re

ALU
Ifetc Re DMe
I lw r1, h g m g

n 0(r2)
s
Re

ALU
Ifetc Re DMe
t sub h g m g

r.
r4,r1,r6

ALU
Ifetc Re DMe Re
O and h g m g

r
d
r6,r1,r7
e Re

ALU
Ifetc Re DMe

r or h g m g

r8,r1,r9
There are some instances where hazards occur, even with forwarding.
Figure
3.12 Appendix A - Pipelining 22
Data Hazards The stall is necessary as shown
here.
Time (clock
cycles)
I
n Re

ALU
s lw r1, Ifetc
h
Re
g
DMe
m g

t 0(r2) B
r. u
Ifetc Re b DMe Re

AL
sub

U
h g m g
b
O B
l
r r4,r1,r6 u
e
d Ifetc b Re DMe Re

AL
U
e and h b
B
g m g

l
r r6,r1,r7 u
e
b Ifetc Re DMe

AL
U
h g
b m
or l
r8,r1,r9 e

There are some instances where hazards occur, even with forwarding.
Figure
3.13 Appendix A - Pipelining 23
 This is another

Data Hazards representation

of the stall.

LW R1, 0(R2) IF ID EX MEM WB

SUB R4, R1, R5 IF ID EX MEM WB

AND R6, R1, R7 IF ID EX MEM WB

OR R8, R1, R9 IF ID EX MEM WB

LW R1, 0(R2) IF ID EX MEM WB

SUB R4, R1, R5 IF ID stall EX MEM WB

AND R6, R1, R7 IF stall ID EX MEM WB

OR R8, R1, R9 stall IF ID EX MEM WB

Appendix A - Pipelining 24
 Control Hazards
A.1 What is Pipelining?
A.2 The Major Hurdle of Pipelining-
Structural Hazards
-- Structural Hazards
– Data Hazards A control hazard is when we
– Control Hazards need to find the destination
A.3 How is Pipelining Implemented of a branch, and can’t fetch
A.4 What Makes Pipelining Hard to any new instructions until
Implement?
we know that destination.
A.5 Extending the MIPS Pipeline to
Handle Multi-cycle Operations

Appendix A - Pipelining 25
 Control Hazard on
Control Hazards Branches
Three Stage Stall

ALU
10: beq Ifetch Reg DMem Reg

r1,r3,36

ALU
Ifetc
14: and r2,r3,r5 Reg DMem Reg
h

ALU
18: or Ifetch Reg DMem Reg

r6,r1,r7

ALU
Reg
22: add Ifetch Reg DMem

r8,r1,r9

ALU
Ifetc
36: xor r10,r1,r11 h
Reg DMem Reg

Appendix A - Pipelining 26
Control Hazards Branch Stall Impact
• If CPI = 1, 30% branch, Stall 3 cycles => new CPI = 1.9!
(Whoa! How did we get that 1.9???)
• Two part solution to this dramatic increase:
– Determine branch taken or not sooner, AND
– Compute taken branch address earlier

• MIPS branch tests if register = 0 or ^ 0

• MIPS Solution:
– Move Zero test to ID/RF stage
– Adder to calculate new PC in ID/RF stage
• must be fast
• can't afford to subtract
• compares with 0 are simple
• Greater-than, Less-than test sign­bit, but not-equal must OR all bits
• more general compares need ALU
– 1 clock cycle penalty for branch versus 3

Appendix A - Pipelining 27
Control Hazards Five Branch Hazard
Alternatives
#1: Stall until branch direction is clear

#2: Predict Branch Not Taken

– Execute successor instructions in sequence
– “Squash” instructions in pipeline if branch actually taken
– Advantage of late pipeline state update
– 47% MIPS branches not taken on average
– PC+4 already calculated, so use it to get next instruction

#3: Predict Branch Taken

– 53% MIPS branches taken on average
– But haven’t calculated branch target address in MIPS
• MIPS still incurs 1 cycle branch penalty
• Other machines: branch target known before outcome

Appendix A - Pipelining 28
Control Hazards Five Branch Hazard
Alternatives
#4: Execute Both Paths

#5: Delayed Branch

– Define branch to take place AFTER a following instruction

branch instruction
sequential successor1
sequential successor2
........
sequential successorn Branch delay of length
branch target if taken n

– 1 slot delay allows proper decision and branch target address in 5

stage pipeline
– MIPS uses this

Appendix A - Pipelining 29
Control Hazards Delayed Branch
• Where to get instructions to fill branch delay slot?
– Before branch instruction
– From the target address: only valuable when branch taken
– From fall through: only valuable when branch not taken
– Cancelling branches allow more slots to be filled

• Compiler effectiveness for single branch delay slot:

– Fills about 60% of branch delay slots
– About 80% of instructions executed in branch delay slots useful in
computation
– About 50% (60% x 80%) of slots usefully filled

• Delayed Branch downside: 7-8 stage pipelines, multiple instructions

issued per clock (superscalar)

Appendix A - Pipelining 30
Control Hazards Compiler “Static”
Prediction of
Taken/Untaken Branches

• Improves strategy for placing instructions in delay slot

• Two strategies
– Backward branch predict taken, forward branch not taken
– Profile-based prediction: record branch behavior, predict branch
based on prior run

Appendix A - Pipelining 31
Summary of Pipelining Basics
• Hazards limit performance
– Structural: need more HW resources
– Data: need forwarding, compiler scheduling
– Control: early evaluation & PC, delayed branch, prediction
• Increasing length of pipe increases impact of hazards; pipelining
helps instruction bandwidth, not latency
• Interrupts, Instruction Set, FP makes pipelining harder
• Compilers reduce cost of data and control hazards
– Load delay slots
– Branch delay slots
– Branch prediction

Appendix A - Pipelining 32
 Summary
A.1 What is Pipelining?
A.2 The Major Hurdle of Pipelining-Structural Hazards
– Data Hazards
– Control Hazards
A.3 How is Pipelining Implemented
A.4 What Makes Pipelining Hard to Implement?
A.5 Extending the MIPS Pipeline to Handle Multi-cycle Operations

Appendix A - Pipelining 33