CS 6290

Branch Prediction
Outline
• Control Dependence • Micro-architecture
and Branch – Branch Target Buffer
• Static Branch – Return Address
Prediction Stack
• Dynamic Branch • Branch Prediction in
Prediction Real World
– One Bit
– Two Bits
• Global Branch History
• Hybrid Branch
Predictor
Control Dependencies

• Branches are very frequent

– Approx. 20% of all instructions
• Can not wait until we know where it
goes
– Long pipelines
• Branch outcome known after B cycles
• No scheduling past the branch until outcome
known
– Superscalars (e.g., 4-way)
• Branch every cycle or so!
• One cycle of work, then bubbles for ~B cycles?
Surviving Branches: Prediction

• Predict Branches
– And predict them well!
• Fetch, decode, etc. on the predicted
path
– Option 1: No execute until branch
resovled
– Option 2: Execute anyway (speculation)
• Recover from mispredictions
– Restart fetch from correct path
Branch Prediction
• Need to know two things
– Whether the branch is taken or not (direction)
– The target address if it is taken (target)

• Direct jumps, Function calls

– Direction known (always taken), target easy to
compute
• Conditional Branches (typically PC-relative)
– Direction difficult to predict, target easy to compute
• Indirect jumps, function returns
– Direction known (always taken), target difficult
Branch Prediction: Direction

• Needed for conditional branches

– Most branches are of this type
• Many, many kinds of predictors for
this
– Static: fixed rule, or compiler annotation
(e.g. “BEQL” is “branch if equal likely”)
– Dynamic: hardware prediction
• Dynamic prediction usually history-
based
– Example: predict direction is the same
as the last time this branch was executed
Static Prediction
• Always predict NT
– easy to implement
– 30-40% accuracy … not so good
• Always predict T
– 60-70% accuracy
• BTFNT
– loops usually have a few iterations, so
this is like always predicting that the loop
is taken
– don’t know target until decode
One-Bit Branch Predictor
Branch history
K bits of branch
table of 2^K entries,
instruction address
1 bit per entry

Index Use this entry to

predict this branch:

0: predict not taken

1: predict taken

When branch direction resolved,

go back into the table and
update entry: 0 if not taken, 1 if taken
One-Bit Branch Predictor
(cont’d)

0xDC08: for(i=0; i < 100000; i++)

{
0xDC44: if( ( i % 100) == 0 )
T
tick( );

0xDC50: if( (i & 1) == 1)

odd( ); N

}
The Bit Is Not Enough!

• Example: short loop (8 iterations)

– Taken 7 times, then not taken once
– Not-taken mispredicted (was taken previously)
• Execute the same loop again
– First always mispredicted
(previous outcome was not taken)
– Then 6 predicted correctly
– Then last one mispredicted again
• Each fluke/anomaly in a stable pattern
results in two mispredicts per loop
Examples

DC08: TTTTTTTTTTT ... TTTTTTTTTTNTTTTTTTTT …

100,000 iterations
NT
How often is branch outcome != previous outcome? TN
2 / 100,000
99.998%
DC44: TTTTT ... TNTTTTT … TNTTTTT …
Prediction
2 / 100 Rate
98.0%
DC50: TNTNTNTNTNTNTNTNTNTNTNTNTNTNT …

2/2 0.0%
Two Bits are Better Than One

Predict NT
Predict T
Transistion on T outcome
2 3
Transistion on NT outcome

0 1
0 1

FSM for Last-Outcome FSM for 2bC

Prediction (2-bit Counter)
Example

1bC: Initial Training/Warm-up

0 1 1 1 1 1 1 0 1 1
… …
T T T T T T N T T T

         
2bC:
0 1 2 3 3 3 3 2 3 3
… …
T T T T T T N T T T

         

Only 1 Mispredict per N branches now!

DC08: 99.999% DC04: 99.0%
Still Not Good Enough

These are We can

good live with
these

This is bad!
Importance of Branches
• 98%  99%
– Who cares?
– Actually, it’s 2% misprediction rate  1%
– That’s a halving of the number of mispredictions
• So what?
– If misp rate equals 50%, and 1 in 5 insts is a branch, then
number of useful instructions that we can fetch is:
5*(1 + ½ + (½)2 + (½)3 + … ) = 10
– If we halve the miss rate down to 25%:
5*(1 + ¾ + (¾)2 + (¾)3 + … ) = 20
– Halving the miss rate doubles the number of useful
instructions that we can try to extract ILP from
How about the Branch at
0xdc50?
• 1bc and 2bc don’t do too well (50% at
best)
• But it’s still obviously predictable
• Why?
– It has a repeating pattern: (NT)*
– How about other patterns? (TTNTN)*

• Use branch correlation

– The outcome of a branch is often related to
previous outcome(s)
Idea: Track the History of a
Branch
An (m, n) predictor uses the behavior of the last m branches to choose from 2^m
branch predictors, each of which is an n-bit predictor for a single branch.
(1, 2) predictor
Previous Outcome
PC
Counter if prev=0
1 3 0
Counter if prev=1

1 3 3 prev = 1 3 0 prediction = N
prev = 0 3 0 prediction = T
prev = 1 3 3 prediction = T prev = 1 3 0 prediction = N

prev = 0 3 2 prediction = T prev = 0 3 0 prediction = T

prev = 1 3 2 prediction = T
prev = 1 3 3 prediction = T
Deeper History Covers More
Patterns
(3, 2) predictor

Last 3 Outcomes Counter if prev=000

Counter if prev=001

PC Counter if prev=010

0 0 1 1 3 1 0 3 2 0 2

Counter if prev=111

• What pattern has this branch predictor entry

learned?
001  1; 011  0; 110  0; 100  1
00110011001… (0011)*
Global vs. Local Branch History
• Local Behavior
– What is the predicted direction of Branch
A given the outcomes of previous
instances of Branch A?
• Global Behavior
– What is the predicted direction of Branch
Z given the outcomes of all* previous
branches A, B, …, X and Y?
* number of previous branches tracked limited by the history
length
Why Global Correlations Exist
• Example: related branch conditions

A: p = findNode(foo);
if ( p is parent )
do something;

do other stuff; /* may contain more branches */

Outcome of second
branch is always
B: if ( p is a child ) opposite of the first
branch
do something else;
Other Global Correlations
• Testing same/similar conditions
– code might test for NULL before a function call,
and the function might test for NULL again
– in some cases it may be faster to recompute a
condition rather than save a previous
computation in memory and re-load it
– partial correlations: one branch could test for
cond1, and another branch could test for cond1 &&
cond2 (if cond1 is false, then the second branch
can be predicted as false)
– multiple correlations: one branch tests cond1, a
second tests cond2, and a third tests cond1 
cond2 (which can always be predicted if the first
two branches are known).
Tournament Predictors

• No predictor is clearly the best

– Different branches exhibit different
behaviors
• Some “constant”, some global, some local
• Idea:
Let’s have a predictor to predict
which predictor will predict better 
Tournament Hybrid Predictors
Meta- table of 2-/3-bit counters
Pred0 Pred1
Predictor

Meta
Pred0 Pred1
Final Prediction Update
  ---
If meta-counter MSB = 0,
use pred0 else use pred1   Inc
  Dec
  ---
Common Combinations
• Global history + Local history
• “easy” branches + global history
– 2bC and gshare
• short history + long history

• Many types of behaviors, many

combinations
Direction Predictor Accuracy
Target Address Prediction
• Branch Target Buffer
– IF stage: need to know fetch addr every cycle
– Need target address one cycle after fetching a branch
– For some branches (e.g., indirect) target known
only after EX stage, which is way too late
– Even easily-computed branch targets need to wait
until instruction decoded and direction predicted in ID
stage
(still at least one cycle too late)
– So, we have a quick-and-dirty predictor for the target
that only needs the address of the branch instruction
Branch Target Buffer

• BTB indexed by instruction address

• We don’t even know if it is a branch!
Direction prediction
can be factored out
• If address matches a BTB entry, it intois
separate table
predicted to be a branch
• BTB entry tells whether it is taken
(direction) and where it goes if taken
• BTB takes only the instruction address, so
while we fetch one instruction in the IF
stage
we are predicting where to fetch the next
one from
Branch-Target Buffer
• Need high instruction bandwidth!
– Branch-Target buffers
• Next PC prediction buffer, indexed by current PC

Copyright © 2012, Elsevier Inc. All rights

Branch Folding
• Optimization:
– Larger branch-target buffer
– Add target instruction into buffer to deal
with longer decoding time required by
larger buffer
– “Branch folding”

Copyright © 2012, Elsevier Inc. All rights

Return Address Stack (RAS)
• Function returns are frequent, yet
– Address is difficult to compute
(have to wait until EX stage done to know it)
– Address difficult to predict with BTB
(function can be called from multiple places)
• But return address is actually easy to
predict
– It is the address after the last call instruction
that we haven’t returned from yet
– Hence the Return Address Stack
Return Address Stack (RAS)
• Call pushes return address into the RAS
• When a return instruction decoded,
pop the predicted return address from RAS
• Accurate prediction even w/ small RAS
Example 1: Alpha 21264

• Hybrid predictor
– combines local history and global history
components with a meta-predictor
Example 2: Pentium-M

• Also hybrid, but uses tag-based

selection mechanism
Pentium-M (cont’d)

• Local component also has support for

loops
– accurately predict branches of the form
(TkN)*
Pentium-M (cont’d)

• Special target prediction for indirect

branches
– common in object-oriented code (vtables)
– assumes correlation with global history