Recap: Sequential Laundry

6 PM 7 8 9 10 11 12 1 2 AM

T 3030 30 30 3030 30 30 3030 30 30 3030 30 30

a Time
A
s
k B
C
O
r D
d
e
r ° Sequential laundry takes 8 hours for 4 loads
° If they learned pipelining, how long would laundry
take?
cs 152 L1 3 .1 DAP Fa97,  U.CB
 Recap: Pipelining Lessons (it’s
intuitive!) ° Pipelining doesn’t help
latency of single task, it
6 PM 7 8 9 helps throughput of entire
workload
Time
T ° Multiple tasks operating
a 3030 30 30 30 3030 simultaneously using
different resources
s A ° Potential speedup =
k Number pipe stages
B ° Pipeline rate limited by
O C slowest pipeline stage
r ° Stages should take about
d D same amount of time
e ° Unbalanced lengths of
pipe stages reduces
r speedup
° Time to “fill” pipeline and
time to “drain” it reduces
speedup
cs 152 L1 3 .2
° Stall for Dependences
DAP Fa97,  U.CB
 Recap: Ideal
Pipelining Assume instructions
are completely independent!

IF DCD EX MEM WB

IF DCD EX MEM WB

IF DCD EX MEM WB

IF DCD EX MEM WB
IF DCD EX MEM WB

Maximum Speedup  Number of stages

Speedup Time for unpipelined operation
Time for longest stage

Example: 40ns data path, 5 stages, Longest stage is 10 ns, Speedup 4

cs 152 L1 3 .3 DAP Fa97,  U.CB

 Recap: Graphically Representing Pipelines

° Can help with answering questions like:

• how many cycles does it take to execute this code?
• what is the ALU doing during cycle 4?
• use this representation to help understand datapaths

cs 152 L1 3 .4 DAP Fa97,  U.CB

 Recap: Can pipelining get us into
trouble?
° Yes: Pipeline Hazards
• structural hazards: attempt to use the same resource two
different ways at the same time
- e.g., multiple memory accesses, multiple register writes
- solutions: multiple memories, stretch pipeline
• control hazards: attempt to make a decision before condition is
evaulated
- e.g., any conditional branch
- solutions: prediction, delayed branch
• data hazards: attempt to use item before it is ready
- e.g., add r1,r2,r3; sub r4, r1 ,r5; lw r6, 0(r7); or r8, r6 ,r9
- solutions: forwarding/bypassing, stall/bubble

cs 152 L1 3 .5 DAP Fa97,  U.CB

 Recap: Pipelined Datapath with Data Stationary
Control
IAU

npc Just like Time-State!

I mem

Regs lw $2,20($5) PC
Operand Register Selects
B A im n op rw
ALU Op <= PC + 4 + immed
alu
S

MEM Op
D mem

Regs Result Reg Select and Enable

cs 152 L1 3 .6 DAP Fa97,  U.CB
 Recap
° Pipelining is a fundamental concept
• multiple steps using distinct resources

° Utilize capabilities of the Datapath by pipelined

instruction processing
• start next instruction while working on the current one
• limited by length of longest stage (plus fill/flush)
• detect and resolve hazards

° What makes it easy

• all instructions are the same length
• just a few instruction formats
• memory operands appear only in loads and stores

° Hazards make it hard

° We’ll build a simple pipeline and look at these
issues
cs 152 L1 3 .7 DAP Fa97,  U.CB
 The Big Picture: Where are We Now?

° The Five Classic Components of a Computer

Processor
Input
Control
Memory

Datapath Output

° This Section´s Topics:

• Recap last lecture
• Pipelined Control/ Do it yourself Pipelined Control
• Hazards/Forwarding
• Exceptions
• Review MIPS R3000 pipeline
• Advanced Pipelining?

cs 152 L1 3 .8 DAP Fa97,  U.CB

 Recap: Control Diagram
IR <- Mem[PC]; PC < PC+4;

A <- R[rs]; B<– R[rt]

S <– A + B; S <– A or ZX; S <– A + SX; S <– A + SX; If Cond PC

< PC+SX;

M <– S M <– S M <– Mem[S] Mem[S] <- B

Equal
R[rd] <– S; R[rt] <– S; R[rd] <– M;

Reg.
Inst. Mem

File
Exec
Reg

A M
Next PC

File

S
PC

IR

Access
Mem

Mem
Data
D
cs 152 L1 3 .9 DAP Fa97,  U.CB
 6.3 (Parts of 6.6) Pipelined Control

To specify control for the pipeline, we need only set

the control values during each pipeline stage.
Because each control line is associated with a
component active in only a single pipeline stage,
we divide the control lines into five groups.
The PC and Pipeline registers are written every clock
cycle, so there is no seperate write signal for them.
Since the control lines start with the EX stage, the
control information is created during instruction
decode. The signals are then used in the
appropriate pipeline stage as the instr moves down
the pipe. The signals for the stage are stored in the
pipeline regs.
If control bit into 2 way multiplexor = 1 then
multiplexor selects the input corresponding to 1, if
control bit = 0 then multiplexor selects the input
corresponding to 0.
cs
cs 152
152 L1
L1 33 .10
.10 DAP Fa97,  U.CB
 Specification of 9 Control Signals (Figs. 6.26 and 6.27)

• Instruction Fetch: Control signals to read Inst Mem and write

PC are always asserted (nothing special to control).

• ID / Reg file read: same as previous stage (no optional control

lines to set).

• EX / Address Calculation: signals to be set are RegDest

(selects result reg), ALUOp (2 control bits assert ALU
operation, a 6-bit function code field specified in the immediate
field of the instr saved in the ID/EX pipe reg, see fig 6.26) and
ALUSrc (Read data reg 2 or a sign-extended immediate for the
ALU).

• Mem Access: control lines to be set are Branch (set by branch

instr), MemRead (load instr)and MemWrite (store instr).
PCSrc selects the next sequential address unless control
asserts Branch and the ALU result was zero.

• Write back: control lines are MemtoReg (decides between

sending the ALU result or the memory value to the reg file) and
RegWrite (writes the chosen value).

cs 152 L1 3 .11 DAP Fa97,  U.CB

 But recall use of “Data Stationary
Control”
° The Main Control generates the control signals during
Reg/Dec
• Control signals for Exec (ExtOp, ALUSrc, ...) are used 1 cycle later
• Control signals for Mem (MemWr Branch) are used 2 cycles later
• Control signals for Wr (MemtoReg MemWr) are used 3 cycles later
Reg/Dec Exec Mem Wr

ExtOp ExtOp
ALUSrc ALUSrc
IF/ID Register

Ex/Mem Register
ALUOp ALUOp

Mem/Wr Register
ID/Ex Register

Main RegDst RegDst

Control
MemWr MemWr MemWr
Branch Branch Branch
MemtoReg MemtoReg MemtoReg MemtoReg
RegWr RegWr RegWr RegWr

cs 152 L1 3 .12 DAP Fa97,  U.CB

 Datapath + Data Stationary
Control
IR v v v
fun rw rw rw
Inst. Mem

Decode
wb wb wb
rt me me WB
rs ex Mem
op Ctrl
Ctrl
rs rt im

Reg.
File
Reg
File A M

Exec
S

Access
Mem

Mem
Data
D
Next PC

PC

cs 152 L1 3 .13 DAP Fa97,  U.CB

 Optimized Data Path

Suppose extra HW (branch address adder moved

from EX to IF stage and comparisson logic) is
added in the ID / RegFetch stage so that branches
are resolved in the second cycle of branch
execution. Next PC is known in second cycle of
branch ---> saves 1 clock cycle of penalty.

cs
cs 152
152 L1
L1 33 .14
.14 DAP Fa97,  U.CB
 Let’s Try it
Out

10 lw r1, r2(35)
14 addI r2, r2, 3
20 sub r3, r4, r5
24 beq r6, r7, 100
30 ori r8, r9, 17 these addresses are octal
34 add r10, r11, r12

100 and r13, r14, 15

cs 152 L1 3 .15 DAP Fa97,  U.CB

 Start: Fetch
10 n n
Inst. Mem n n

Decode
WB
Mem
Ctrl
Ctrl
IR im
rs rt

Reg.
File
Reg

A M
File

Exec
S
B

Access
=

Mem

Mem
Data
D IF 10 lw r1, r2(35)
14 addI r2, r2, 3
Next PC

20 sub r3, r4, r5

10

24 beq r6, r7, 100

30 ori r8, r9, 17
34 add r10, r11, r12
PC

cs 152 L1 3 .16
100 and r13, r14, 15
DAP Fa97,  U.CB
 Fetch 14, Decode 10
n n n
lw r1, r2(35)
Inst. Mem

Decode
WB
Mem
Ctrl
Ctrl
IR im
2 rt

Reg.
File
Reg

A M
File

Exec
S
B

Access
=

Mem

Mem
Data
D ID 10 lw r1, r2(35)
IF 14 addI r2, r2, 3
Next PC

20 sub r3, r4, r5

14

24 beq r6, r7, 100

30 ori r8, r9, 17
34 add r10, r11, r12
PC

cs 152 L1 3 .17
100 and r13, r14, 15
DAP Fa97,  U.CB
 Fetch 20, Decode 14, Exec 10
n n
addI r2, r2, 3
Inst. Mem

Decode

lw r1
WB
Mem
Ctrl
Ctrl
IR

35
2 rt

Reg.
File
Reg

r2 M
File

Exec
S
B

Access
=

Mem

Mem
Data
D EX 10 lw r1, r2(35)
ID 14 addI r2, r2, 3
Next PC

IF 20 sub r3, r4, r5

20

24 beq r6, r7, 100

30 ori r8, r9, 17
34 add r10, r11, r12
PC

cs 152 L1 3 .18
100 and r13, r14, 15
DAP Fa97,  U.CB
 Fetch 24, Decode 20, Exec 14, Mem 10
n
sub r3, r4, r5

addI r2, r2, 3

Inst. Mem

Decode

lw r1
WB
Mem
Ctrl
Ctrl
IR
4 5
3

Reg.
r2+35

File
Reg

r2 M
File

Exec
B

Access
=

Mem

Mem
Data
D M 10 lw r1, r2(35)
EX 14 addI r2, r2, 3
Next PC

ID 20 sub r3, r4, r5

24

IF 24 beq r6, r7, 100

30 ori r8, r9, 17
34 add r10, r11, r12
PC

cs 152 L1 3 .19
100 and r13, r14, 15
DAP Fa97,  U.CB
 Fetch 30, Dcd 24, Ex 20, Mem 14, WB
10
beq r6, r7 100
Inst. Mem

Decode

addI r2
sub r3

lw r1
WB
Mem
Ctrl
Ctrl
IR
6 7

M[r2+35]

Reg.
File
Reg

r4

r2+3
File

Exec
r5

Access
=

Mem

Mem
Data
D WB 10 lw r1, r2(35)
M 14 addI r2, r2, 3
Next PC

EX 20 sub r3, r4, r5

ID 24
30

beq r6, r7, 100

IF 30 ori r8, r9, 17
34 add r10, r11, r12
PC

Note Delayed Branch: always execute ori after beq

cs 152 L1 3 .20
100 and r13, r14, 15
DAP Fa97,  U.CB
 Fetch 100, Dcd 30, Ex 24, Mem 20, WB
14
ori r8, r9 17
Inst. Mem

Decode

addI r2
sub r3
WB

beq
Mem
Ctrl
Ctrl
IR

r1=M[r2+35]
100
9 xx

Reg.
r2+3

File
Reg

r4-r5
r6
File

Exec
r7

Access
=

Mem

Mem
Data
D 10 lw r1, r2(35)
WB 14 addI r2, r2, 3
M 20
Next PC

sub r3, r4, r5

EX 24
100

beq r6, r7, 100

ID 30 ori r8, r9, 17
34 add r10, r11, r12
PC

IF 100 and r13, r14, 15

cs 152 L1 3 .21 DAP Fa97,  U.CB
 Fetch 104, Dcd 100, Ex 30, Mem 24, WB
20
Inst. Mem

Decode
?
WB
Mem
Ctrl
Ctrl
IR

Reg.
File
Reg
File

Exec

Access
=

Mem

Mem
Data
D 10 lw r1, r2(35)
14 addI r2, r2, 3
Next PC

WB 20 sub r3, r4, r5

___

M 24 beq r6, r7, 100

EX 30 ori r8, r9, 17
34 add r10, r11, r12
PC

Fill it in yourself!
ID 100 and r13, r14, 15
cs 152 L1 3 .22 DAP Fa97,  U.CB
 Fetch 110, Dcd 104, Ex 100, Mem 30, WB
24
Inst. Mem

Decode
? ?
WB
Mem
Ctrl
Ctrl
IR ?

Reg.
File
Reg

?
File

Exec

Access
=

Mem

Mem
Data
D 10 lw r1, r2(35)
14 addI r2, r2, 3
Next PC

20 sub r3, r4, r5

___

WB 24 beq r6, r7, 100

M 30 ori r8, r9, 17
34 add r10, r11, r12
PC

Fill it in yourself!
cs 152 L1 3 .23
EX 100 and r13, r14, 15
DAP Fa97,  U.CB
 Fetch 114, Dcd 110, Ex 104, Mem 100, WB
30
Inst. Mem

Decode
? ? ?
WB
Mem
Ctrl
Ctrl
IR ?

Reg.
File
Reg

? ?
File

Exec

Access
=

Mem

Mem
Data
D 10 lw r1, r2(35)
14 addI r2, r2, 3
Next PC

20 sub r3, r4, r5

___

24 beq r6, r7, 100

WB 30 ori r8, r9, 17
34 add r10, r11, r12
PC

Fill it in yourself!
M 100 and r13, r14, 15
cs 152 L1 3 .24 DAP Fa97,  U.CB
 Pipeline Hazards Again
I-Fet ch DCD MemOpFetch OpFetch Exec Store

IFetch DCD °°°

Structural
Hazard

I-Fet ch DCD OpFetch Jump Control Hazard

IFetch DCD °°°

IF DCD EX Mem WB RAW (read after write) Data Hazard

IF DCD EX Mem WB
WAW Data Hazard
IF DCD EX Mem WB (write after write)

IF DCD OF Ex Mem

IF DCD OF Ex WB WAR Data Hazard

(write after read)
cs 152 L1 3 .25 DAP Fa97,  U.CB
 6.4 Data Hazards

° Avoid some “by design”

• eliminate WAR by always fetching operands early (DCD) in pipe
• eliminate WAW by doing all WBs in order (last stage, static)

° Detect and resolve remaining ones

• stall or forward (if possible)
• Introduce a Forwarding Unit in the datapath

IF DCD EX Mem WB RAW Data Hazard

IF DCD EX Mem WB
WAW Data Hazard
IF DCD EX Mem WB

IF DCD OF Ex Mem

IF DCD OF Ex WB WAR Data Hazard

cs 152 L1 3 .26 DAP Fa97,  U.CB

 Compiler Scheduling Solution

sub $2, $1, $3

nop
nop
and $12, $2, $5
or $13, $6, $2
add $14, $2, $2
sw $15, 100($2)
This code works properly but the 2 nops occupy 2
clock cycles that do no useful work. These
dependencies occur too often to rely on compilers.
*

cs 152 L1 3 .27 DAP Fa97,  U.CB

 Hazard Detection
° Suppose instruction i is about to be issued and a predecessor
instruction j is in the instruction pipeline.
° A RAW hazard exists on register if Rregs( i ) Wregs( j )
• Keep a record of pending writes (for inst's in the pipe) and compare
with operand regs of current instruction.
• When instruction issues, reserve its result register.
• When on operation completes, remove its write reservation.

° A WAW hazard exists on register if Wregs( i ) Wregs( j )

° A WAR hazard exists on register if Wregs( i ) Rregs( j )

cs 152 L1 3 .28 DAP Fa97,  U.CB

 Forwarding

If we can take the inputs to the ALU from any pipeline

register rather than just the ID/EX, then we can
forward the proper data.

By adding multiplexors to the input of the ALU and

with the proper controls, we can run the pipeline at
full speed in the presence of some data
dependencies.
Notation:
rs: 1st regs source operand
rt: 2nd regs source operand
rw (rd in text book): regs destination operand
*
cs
cs 152
152 L1
L1 33 .29
.29 DAP
DAP Fa97,  U.CB
Fa97,  U.CB
 Record of Pending
Writes
IAU

npc
° Current operand
I mem registers
Regs op rw rs rt PC
° Pending writes
° hazard <=
B A im n op rw ((rs == rwex) & regWex) OR
((rs == rwmem) & regWme) OR
alu
((rs == rwwb) & regWwb) OR
S n op rw
((rt == rwex) & regWex) OR
D mem ((rt == rwmem) & regWme) OR
((rt == rwwb) & regWwb)
m
n op rw
Regs
cs 152 L1 3 .30 DAP Fa97,  U.CB
 Resolve RAW by forwarding
IAU ° Detect nearest
valid write op
npc operand register
and forward into
I mem op latches,
Regs bypassing
op rw rs rt PC remainder of the
Forward
mux
pipe
• Increase muxes to
B A im n op rw add paths from
pipeline registers
alu
• Data Forwarding =
S n op rw Data Bypassing
D mem

m
n op rw
Regs
cs 152 L1 3 .31 DAP Fa97,  U.CB
 6.5 What about memory operations?
° If instructions are initiated in order and
operations always occur in the same
stage, there can be no hazards between op Rd Ra Rb
memory operations!

° What does delaying WB on arithmetic

operations cost?
– cycles ?
– hardware ? op Rd Ra Rb A B
° What about data dependence on loads?
R1 <- R4 + R5
R2 <- Mem[ R2 + I ]
R3 <- R2 + R1 Rd R
=>
"Delayed Loads"

Rd T
Instr is in the ID stage means:
the instruction’s control signals are in to reg
file
the IF/ID register.

cs 152 L1 3 .32 DAP Fa97,  U.CB

 Load Data Hazard

Time (in clock cycles)

Program CC 1 CC 2 CC 3 CC 4 CC 5 CC 6 CC 7 CC 8 CC 9
execution
order
(in instructions)
lw $2, 20($1) IM Reg DM Reg

and $4, $2, $5 IM Reg DM Reg

or $8, $2, $6 IM Reg DM Reg

add $9, $4, $2 IM Reg DM Reg

slt $1, $6, $7 IM Reg DM Reg

cs
cs 152
152 L1
L1 33 .33
.33 DAP Fa97,  U.CB
 No Solution by Forwarding

Since the dependence between the load and the

following instr goes backwards in time, this hazard
cannot be solved by forwarding.
It is necessary to stall the pipeline. Include a Hazard
Detection Unit which operates during the ID stage
Condition for this load hazard detection:
if (ID/EX.MemRead and
(( ID/EX RegsRt = IF/ID.RegsRs ) or
ID/EX RegsRt = IF/ID.RegsRt )))
stall the pipe one clock cycle
If instr is load and destination regs field in EX stage
matches either source regs of the instr in the ID
stage, then stall.
After the stall the forwarding unit handles the
dependency to avoid loosing one more cycle.
cs 152 L1 3 .34 DAP Fa97,  U.CB
 Bubbles Caused by Stalls

Program Time (in clock cycles)

execution CC 1 CC 2 CC 3 CC 4 CC 5 CC 6 CC 7 CC 8 CC 9 CC 10
order
(in instructions)

lw $2, 20($1) IM Reg DM Reg

and $4, $2, $5 IM Reg Reg DM Reg

or $8, $2, $6 IM IM Reg DM Reg

bubble

add $9, $4, $2 IM Reg DM Reg

slt $1, $6, $7 IM Reg DM Reg

cs 152 L1 3 .35 DAP Fa97,  U.CB

 Stall/Bubble in the Pipeline

Or, more
accurately…
cs 152 L1 3 .36 Chapter 4 — The Processor
DAP Fa97, — 36
U.CB
 Stalling

If the instruction in the ID stage is stalled, then the

instr in the IF stage must also be stalled (so we
don’t loose the fetched instr).
Stalling mechanism: prevent the PC and the IF/ID
pipe regs from changing.
The instr in the IF stage will continue to be read using the same
PC, and the regs in the ID stage will continue to be read using the
same instr fields in the IF/ID pipe regs. Like restarting the
washing machine with the same clothes while letting the dryer
continue working empty.
It is also necessary to set the EX, MEM and WB control signals of
the ID/ EX pipe regs to 0 (the hazard has already been detected in
the ID stage). These values are propagated forward in the pipe
and no regs or memory locations are changed. Actually only the
signals RegWrite and MemWrite need to be 0.
In the example in previous figure, the HW repeats in cycle 4 what it
did for instrs and and or in clock 3. The execution time for these
instrs is stretched and the fetch of the add instr is delayed.
The bubble delays everything behind it and proceeds down the
pipe.
cs 152 L1 3 .37 DAP Fa97,  U.CB
 Hazard Detection and Forwarding Units

The Forwarding Unit controls the ALU multiplexors

to replace the value from a general purpose regs
with the value from the proper pipe regs.

The Hazard Detection Unit controls the writing of the

PC and IF/ID regs plus the multiplexor that chooses
between the real control values and all 0s. It stalls
and deasserts the control fields if the load-use
hazard test is true.

cs 152 L1 3 .38 DAP Fa97,  U.CB

 Cycles 2 and 3 (Single-Cycle Diagrams)
and $4, $2, $5 lw $2, 20($1) before<1> before<2> before<3>
Hazard
ID/EX.MemRead
detection
1 unit ID/EX
X
11
WB

IF/IDWrite
EX/MEM
M
Control u M WB
x MEM/WB
0
IF/ID EX M WB

1 $1
PCWrite

M
Instruction
X u
x
Registers
Instruction Data
PC ALU
memory memory M
$X
u
M x
u
x

1
X
2
M
u
x
ID/EX.RegisterRt Forwarding
unit

Clock 2

or $4, $4, $2 and $4, $2, $5 lw $2, 20($1) before<1> before<2>

Hazard
ID/EX.MemRead
detection
2 unit ID/EX
5
00 11
WB
IF/IDWrite

EX/MEM
M
Control u M WB
x MEM/WB
0
IF/ID EX M WB

2 $2 $1
PCWrite

M
Instruction

5 u
x
Registers
Instruction Data
PC ALU
memory memory M
$5 $X
u
M x
u
x

2 1
5 X
2 M
4 u
x
ID/EX.RegisterRt Forwarding
unit

Clock 3
cs 152 L1 3 .36
.39 DAP Fa97,  U.CB
 Cycles 4 and 5
or $4, $4, $2 and $4, $2, $5 bubble lw $2, . . . before<1>
Hazard
ID/EX.MemRead
detection
2 unit ID/EX
5
10 00
WB

IF/IDWrite
EX/MEM
M 11
Control u M WB
x MEM/WB
0
IF/ID EX M WB

2 $2 $2
PCWrite

M
Instruction

5 u
x
Registers
Instruction Data
PC ALU
memory memory M
$5 $5
u
M x
u
x

2 2
5 5

M 2
4 4 u
x
ID/EX.RegisterRt Forwarding
unit

Clock 4

add $9, $4, $2 or $4, $4, $2 and $4, $2, $5 bubble lw $2, . . .
Hazard
ID/EX.MemRead
detection
4 unit ID/EX
2
10 10
WB
IF/IDWrite

EX/MEM
M 0
Control u M WB
x MEM/WB
0
11
IF/ID EX M WB

4 $4 $2
PCWrite

M
Instruction

2 u
x
Registers
Instruction 2 Data
PC ALU
memory memory M
$2 $5
u
M x
u
x

4 2
2 5
M 2
4 4 u
x
ID/EX.RegisterRt Forwarding
unit

Clock 5

cs 152 L1 3 .39
.40 DAP Fa97,  U.CB
 Cycles 6 and 7
after<1> add $9, $4, $2 or $4, $4, $2 and $4, . . . bubble
Hazard ID/EX.MemRead
detection
4
unit ID/EX
2
10 10
WB

IF/IDWrite
EX/MEM
M 10
Control u M WB
x MEM/WB
0
0
IF/ID EX M WB

4 $4 $4
PCWrite

Instruction
2 u
x
Registers
Instruction Data
PC ALU
memory memory M
$2 $2
u
M x
u
x

4 4
2 2

M 4
9 4 u
x
ID/EX.RegisterRt Forwarding
unit

Clock 6

after<2> after<1> add $9, $4, $2 or $4, . . . and $4, . . .

Hazard
detection ID/EX.MemRead
unit ID/EX
10 10
WB
IF/IDWrite

EX/MEM
M 10
Control u M WB
x MEM/WB
0
1
IF/ID EX M WB

$4
PCWrite

M
Instruction

u
x
Registers
Instruction 4 Data
PC ALU
memory memory M
$2
u
M x
u
x

4
2

M 4 4
9 u
x
ID/EX.RegisterRt Forwarding
unit

Clock 7
cs 152 L1 3 ..41 DAP Fa97,  U.CB