COE 485 Sem 1 2023

Advanced Computer Architecture

Pipelined Datapath Execution

 Pipelining
• Overlapped execution of instructions
• Instruction level parallelism (concurrency)
• Physical pipeline: Automobile assembly line
• Response time for any instruction is the same
• Instruction throughput increases
• Speedup  number of steps (stages)
– Reality: Pipelining introduces overhead
 Pipelined Datapath
M
IF/ID ID/EX EX/MEM MEM/WB
u
x

Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 MIPS Pipeline
• MIPS subset
– Memory access: lw and sw
– Arithmetic and logic: and, sub, and, or, slt
– Branch: beq
• Steps (pipeline segments)
– IF: fetch instruction from memory
– ID: decode instruction and read registers
– EX: execute the operation or calculate address
– MEM: access an operand in data memory
– WB: write the result into a register
 Execution of lw (1/5)
Instruction Fetch

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Execution of lw (2/5)
Instruction Decode

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Execution of lw (3/5)
Execution

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Execution of lw (4/5)
Memory

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Execution of lw (5/5)
Write Back

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Datapath Resources Used for lw

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
Multiple-clock-cycle Diagram
 Single-cycle Diagram (Cycle 1)
IF: lw $10, 20($1)

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Single-cycle Diagram (Cycle 2)
IF: sub $11, $2, $3 ID: lw $10, 20($1)

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Single-cycle Diagram (Cycle 3)
ID: sub $11, $2, $3 EX: lw $10, 20($1)

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Single-cycle Diagram (Cycle 4)
EX: sub MEM: lw

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Single-cycle Diagram (Cycle 5)
MEM: sub WB: lw

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Single-cycle Diagram (Cycle 6)
WB: sub

IF/ID ID/EX EX/MEM MEM/WB

M
u
x
Add
Shift
left 2
4
Add
M
u
Regs M ALU x
u
Instr. x Data
PC Mem Mem

Sign
extend
 Limits to Pipelining
• Hazards prevent next instruction from
executing during its designated clock cycle
– Structural hazards
• HW cannot support this combination of instructions
– Control hazards
• Branches stall the pipeline until condition is
evaluated
– Data hazards
• Instruction depends on result of prior instruction
 Structural Hazards
• Structural hazard: inadequate hardware to simultaneously
support all instructions in the pipeline in the same clock cycle
• E.g., suppose single – not separate – instruction and data
memory in pipeline below with one read port
– then a structural hazard between first and fourth lw instructions

Program
execution 2 4 6 8 10 12 14
Time
order
(in instructions)
Instruction Data
lw $1, 100($0) Reg ALU Reg
fetch access
Pipelined
Instruction Data
lw $2, 200($0) 2 ns Reg ALU Reg
fetch access Hazard if single memory
Instruction Data
lw $3, 300($0) 2 ns Reg ALU Reg
fetch access
Instruction Data
lw $4, 400($0) Reg ALU Reg
• MIPS was designed to be pipelined: structural hazards are easy
2 ns fetch access

to avoid! 2 ns 2 ns 2 ns 2 ns 2 ns
 Control Hazards
• Control hazard: need to make a decision based on the
result of a previous instruction still executing in pipeline
• Solution 1 Stall the pipeline
Program
execution 2 4 6 8 10 12 14 16
order Time
(in instructions)

add $4, $5, $6

Instruction
Reg ALU
Data
Reg Note that branch outcome is
fetch access
computed in ID stage with
beq $1, $2, 40
Instruction
Reg ALU
Data
Reg added hardware (later…)
2ns fetch access

Instruction Data
lw $3, 300($0) bubble Reg ALU Reg
fetch access

4 ns 2ns

Pipeline stall
 Control Hazards
• Solution 2 Predict branch outcome
– e.g., predict branch-not-taken :
Program
execution 2 4 6 8 10 12 14
order Time
(in instructions)
Instruction Data
add $4, $5, $6 fetch
Reg ALU
access
Reg

Instruction Data
beq $1, $2, 40 Reg ALU Reg
2 ns fetch access

Instruction Data
lw $3, 300($0) Reg ALU Reg
2 ns fetch access

Prediction success
Program
execution 2 4 6 8 10 12 14
order Time
(in instructions)
Instruction Data
add $4, $5 ,$6 Reg ALU Reg
fetch access

Instruction Data
beq $1, $2, 40 Reg ALU Reg
fetch access
2 ns
bubble bubble bubble bubble bubble

Instruction Data
or $7, $8, $9 Reg ALU Reg
fetch access
4 ns
Prediction failure: undo (=flush) lw
 Control Hazards
• Solution 3 Delayed branch: always execute the
sequentially next statement with the branch executing after
one instruction delay – compiler’s job to find a statement
that can be put in the slot that is independent of branch
outcome
– MIPS does this – but it is an option in SPIM (Simulator -> Settings)
Program
execution 2 4 6 8 10 12 14
order Time
(in instructions)

beq $1, $2, 40 Instruction Data

Reg ALU Reg
fetch access

add $4, $5, $6 Instruction Data

Reg ALU Reg
(d elayed branch slot) 2 ns fetch access

Instruction Data
lw $3, 300($0) Reg ALU Reg
2 ns fetch access

2 ns

Delayed branch beq is followed by add that is

independent of branch outcome
 Data Hazards
• Data hazard: instruction needs data from the result of a
previous instruction still executing in pipeline
• Solution Forward data if possible…
2 4 6 8 10
Time
Instruction pipeline diagram:
IF ID
add $s0, $t0, $t1 EX MEM WB shade indicates use –
left=write, right=read

Program
execution 2 4 6 8 10
order Time
(in instructions)
add $s0, $t0, $t1 IF ID EX MEM WB Without forwarding – blue
line – data has to go back
in time;
sub $t2, $s0, $t3 IF ID EX MEM WB with forwarding – red line
–
data is available in time
 Data Hazards
• Forwarding may not be enough
– e.g., if an R-type instruction following a load uses the result of the
load – called load-use data hazard
2 4 6 8 10 12 14
Program Time
execution
order
(in instructions)

lw $s0, 20($t1) IF ID EX MEM WB

Without a stall it is impossible
to provide input to the sub
instruction in time
sub $t2, $s0, $t3 IF ID EX MEM WB

2 4 6 8 10 12 14
Program Time
execution
order
(in instructions)

lw $s0, 20($t1) IF ID
With a one-stage stall, forwarding
EX MEM WB
can get the data to the sub
instruction in time
bubble bubble bubble bubble bubble

sub $t2, $s0, $t3 IF ID EX MEM WB

Reordering Code to Avoid Pipeline
Stall (Software Solution)
• Example:
lw $t0, 0($t1)
lw $t2, 4($t1)
Data hazard
sw $t2, 0($t1)
sw $t0, 4($t1)

• Reordered code:
lw $t0, 0($t1)
lw $t2, 4($t1)
sw $t0, 4($t1)
Interchanged
sw $t2, 0($t1)
 Conclusions
• Caution: Pipelining is not as simple as it might appear
– Different resources used per cycle per instruction
– More complex than multicycle DP
– Multiple representations
• Pipeline contents representation
– Single-cycle datapath: convenient
– Single- and Multi-cycle diagrams