Chapter 7

Memory System Design

1
In the past few decades, CPU processing speed
as measured by the number of instructions
executed per second has doubled every 18
months, for the same price.
Computer memory has experienced a similar
increase along a different dimension,
quadrupling in size every 36 months, for the
same price.
Memory speed, however, has only increased at
a rate of less than 10% per year.
2
Thus, while processing speed increases at the
same rate that memory size increases, the gap
between the speed of the processor and the speed
of memory also increases.
As the gap between processor and memory
speeds grows, architectural solutions help bridge
the gap.
A typical computer contains several types of
memory, ranging from fast, expensive internal
registers to slow, inexpensive removable disks.
3
 Location
CPU (registers and L1 cache)
 Internal Memory (main)
 External (secondary)
Capacity
Word Size - typically equal to the number of
bits used to represent a number and to the
instruction length.
Number of Words

4
Unit of Transfer
Word
 Block
Physical Characteristics
 volatile - information decays or is lost when power
is lost.
non-volatile - information remains without
deterioration until changed.
 non-erasable- information cannot be altered with a
normal memory access cycle.
Erasable 5
Access Method
Sequential Access-access must be made in a
specific linear sequence.
-the time to access an arbitrary record is
highly variable.
Direct Access- access is by direct access to
general vicinity of desired information, then
some search.
-access time is still variable, but not as much
as sequential access
6
Random Access-
each addressable location has a unique,
physical location.
access is by direct access to desired
location.
 access time is constant and independent
of prior accesses.
Performance

7
Memory in a conventional digital computer is
organized in a hierarchy.
A memory system is a hierarchy of storage
devices with different capacities, costs, and
access times.
 CPU registers hold the most frequently used
data. Small, fast cache memories nearby the
CPU act as staging areas for a subset of the data
and instructions stored in the relatively slow
main memory.
8
At the top of the hierarchy are registers that
are matched in speed to the CPU, but tend to be
large and consume a significant amount of
power.
There are normally only a small number of
registers in a processor, on the order of a few
hundred or less.
At the bottom of the hierarchy are secondary
and off-line storage memories such as hard
magnetic disks and magnetic tapes,
9
in which the cost per stored bit is small in
terms of money and electrical power, but the
access time is very long when compared with
registers.
Between the registers and secondary storage
are a number of other forms of memory that
bridge the gap between the two.

10
11
The memory hierarchy can be characterized
by a number of parameters.
 Among these parameters are:
the access type,
capacity,
cycle time,
latency,
bandwidth, and
 cost.

12
The term access refers to the action that
physically takes place during a read or
write operation.(sequential, direct,
random…etc)
The capacity of a memory level is usually
measured in bytes.
 The cycle time is defined as the time
elapsed from the start of a read operation
to the start of a subsequent read.
13
The latency is defined as the time interval
between the request for information and the
access of the first bit of that information.
The bandwidth provides a measure of the
number of bits per second that can be
accessed.
The cost of a memory level is usually
provided as Dollars per megabytes.

14
Smaller access time -> greater cost per bit
 Greater capacity -> smaller cost per bit
 Greater capacity -> greater access time

15
The term random access refers to the fact
that any access to any memory location takes
the same fixed amount of time regardless of
the actual memory location and/or the
sequence of accesses that take place.

16
The efficiency of a memory hierarchy
depends on the principle of moving
information into the fast memory infrequently
and accessing it many times before replacing it
with new information.
 This principle is possible due to a well-
known phenomenon called locality of
reference, i.e. within a given period of time,
programs tend to reference relatively confined
area of memory repeatedly.
17
There exists two forms of locality.
Spatial locality refers to the phenomenon that when
a given address has been referenced, it is most likely
that addresses near it will be referenced within a
short period of time, e.g. consecutive instruction in a
straightline program.
 Temporal locality, on the other hand, refers to the
phenomenon that once a particular memory item has
been referenced, it is most likely that it will be
referenced again within a short period of time, e.g.
an instruction in a program loop.
18
As the name implies, the main memory
provides the main storage for a computer.
Fig.below shows interface between the main
memory the CPU.
Two CPU registers are used to interface the
CPU to the main memory.
These are the Memory Address Register (MAR)
and the Memory Data Register (MDR).
The MDR is used to hold the data to be stored
and/or retrieved in/from the memory location
whose address in held in the MAR. 19
20
features
RAM is packaged as a chip.
Basic storage unit is a cell (one bit per cell)
Multiple RAM chips form a memory.
In this context the term “random” means
that any memory location can be accessed in
the same amount of time, regardless of its
position in the memory.

21
A simplified pinout of a RAM chip is shown
in Figure below. An m-bit address

22
23
Random-access memory (RAM) comes in two
varieties—static and dynamic.
Static RAM (SRAM) is faster and significantly
more expensive than Dynamic RAM (DRAM).
SRAM is used for cache memories, both
on and off the CPU chip.
DRAM is used for the main memory plus the
frame buffer of a graphics system.
Typically, a desktop system will have no more
than a few megabytes of SRAM, but hundreds or
thousands of megabytes of DRAM.
24
Each cell stores bit with a six-transistor
circuit.
Retains value indefinitely, as long as it is kept
powered.
stores data in traditional flip-flop logic gates
Relatively insensitive to disturbances such as
electrical noise.
Faster and more expensive than DRAM.
This circuit has the property that it can stay
indefinitely in either of two different voltage
configurations, or states. 25
26
27
DRAM stores each bit as charge on a capacitor.
 This capacitor is very small—typically around
30 femto farads, that is, 30 × 10−15 farads.
 Each cell stores bit with a capacitor and
transistor.
Value must be refreshed every 10-100 ms.
Sensitive to disturbances.
 Slower and cheaper than SRAM.

28
29
The cells (bits) in a DRAM chip are partitioned
into “d” supercells, each consisting of “w” DRAM
cells.
 A d × w DRAM stores a total of dw bits of
information.
The supercells are organized as a rectangular
array with “r” rows and “c” columns, where rc = d.
Each supercell has an address of the form (i, j),
where i denotes the row, and j denotes the column.

30
For example, Figure below shows the
organization of a 16 × 8 DRAM chip with d = 16
supercells, w = 8 bits per supercell, r = 4 rows,
and c = 4 columns.
The shaded box denotes the supercell at
address (2, 1).
Information flows in and out of the chip via
external connectors called pins.
Each pin carries a 1-bit signal.

31
Figure below shows two of these sets
of pins: eight data pins that can
transfer 1 byte in or out of the chip,
and two addr pins that carry two-bit.
row and column supercell addresses.
Other pins that carry control
information are not shown.

32
Conventional DRAM
Organization
d x w DRAM:
 dw total bits organized as d supercells of
size w bits 16 x 8 DRAM chip
cols
0 1 2 3
2 bits 0
/
addr
1
rows
memory supercell
2
controller (2,1)
(to CPU)
3
8 bits
/
data

internal row buffer

Each DRAM chip is connected to some
circuitry, known as the memory controller, that
can transfer w bits at a time to and from each
DRAMchip.
To read the contents of supercell (i, j), the
memory controller sends the row address i to the
DRAM, followed by the column address j.
The DRAM responds by sending the contents of
supercell (i, j) back to the controller.

34
The row address “i” is called a RAS (Row Access
Strobe) request.
The column address “j” is called a CAS (Column
Access Strobe) request.
Notice that the RAS and CAS requests share the
same DRAM address pins.
For example, to read supercell (2, 1) from the 16 ×
8 DRAM in Figure above, the memory controller
sends row address 2, as shown in Figure below.
The DRAM responds by copying the entire
contents of row 2 into an internal row buffer.
35
 Reading DRAM Supercell
(2,1)
Step 1(a): Row access strobe (RAS) selects row 2.

Step 1(b): Row 2 copied from DRAM array to row

buffer.
16 x 8 DRAM chip
cols
0 1 2 3
RAS = 2
2
/ 0
addr
1
rows
memory
controller 2

8 3
/
data

internal row buffer

Next, the memory controller sends
column address 1, as shown in Figure
below.
The DRAM responds by copying the
8 bits in supercell (2, 1) from the row
buffer and sending them to the memory
controller.

37
 Reading DRAM Supercell
(2,1)
Step 2(a): Column access strobe (CAS) selects column1

Step 2(b): Supercell (2,1) copied from buffer to

data lines, and eventually back to the CPU.
16 x 8 DRAM chip
cols
0 1 2 3
CAS = 1
2
/ 0
addr
To CPU 1
rows
memory
controller 2

supercell 3
8
(2,1) /
data

supercell
internal row buffer
(2,1)
One reason circuit designers organize DRAMs as
two-dimensional arrays instead of linear arrays is
to reduce the number of address pins on the chip.
For example, if our example 128-bit DRAM were
organized as a linear array of 16 supercells with
addresses 0 to 15, then the chip would need four
address pins instead of two.
The disadvantage of the two-dimensional array
organization is that addresses must be sent in two
distinct steps, which increases the access time.

39
DRAM chips are packaged in memory modules
that plug into expansion slots on the main system
board (motherboard).
Common packages include the 168-pin dual
inline memory module (DIMM), which transfers
data to and from the memory controller in 64-bit
chunks, and the 72-pin single inline memory
module (SIMM), which transfers data in 32-bit
chunks.

40
Figure below shows the basic idea of a memory
module. The example module stores a total of 64
MB (megabytes) using eight 64-Mbit 8M × 8
DRAM chips, numbered 0 to 7.
 Each supercell stores 1 byte of main memory,
and each 64-bit doubleword1 at byte address A in
main memory is represented by the eight supercells
whose corresponding supercell address is (i, j).
In the example in Figure below, DRAM 0
stores the first (lower-order) byte, DRAM 1 stores
the next byte, and so on.
41
To retrieve a 64-bit doubleword at memory
address A, the memory controller converts A to a
supercell address (i, j) and sends it to the memory
module, which then broadcasts i and j to each
DRAM.
In response,each DRAM outputs the 8-bit
contents of its (i, j) supercell.
Circuitry in the module collects these outputs
and forms them into a 64-bit doubleword, which
it returns to the memory controller.
42
Memory Modules
addr (row = i, col = j)
: supercell (i,j)
DRAM 0
64 MB
memory module
consisting of
DRAM 7
eight 8Mx8 DRAMs

bits bits bits bits bits bits bits bits

56-6348-5540-47 32-3924-3116-23 8-15 0-7

63 56 55 4847 40 39 3231 2423 16 15 8 7 0

Memory
controller
64-bit doubleword at main memory address A

64-bit doubleword
DRAMs and SRAMs are volatile in the sense
that they lose their information if the supply
voltage is turned off.
Nonvolatile memories, on the other hand,
retain their information even when they are
powered off.
There are a variety of nonvolatile memories.
 For historical reasons, they are referred to
collectively as read-only memories (ROMs), even
though some types of ROMs can be written to as
well as read. 44
ROMs are distinguished by the number of
times they can be reprogrammed (written to)
and by the mechanism for reprogramming
them.
Types of ROMs
Programmable ROM (PROM)
Eraseable programmable ROM (EPROM)
Electrically eraseable PROM (EEPROM)
Flash memory

45
A programmable ROM (PROM):
can be programmed exactly once.
PROMs include a sort of fuse with each memory
cell that can be blown once by zapping it with a high
current.
An erasable programmable ROM (EPROM):
has a transparent quartz window that permits light
to reach the storage cells.
The EPROM cells are cleared to zeros by shining
ultraviolet light through the window.

46
Programming an EPROM is done by using a
special device to write ones into the EPROM.
An EPROM can be erased and reprogrammed on
the order of 1000 times.
An electrically erasable PROM (EEPROM)
is akin to an EPROM, but does not require a
physically separate programming device, and thus
can be reprogrammed in-place on printed circuit
cards.
An EEPROM can be reprogrammed on the order of
105 times before it wears out.
47
Flash memory is a type of nonvolatile memory,
based on EEPROMs, that has become an
important storage technology.
Flash memories are everywhere, providing fast
and durable nonvolatile storage for a slew of
electronic devices, including digital cameras, cell
phones, music players, PDAs, and laptop,
desktop, and server computer systems.

48
Data flows back and forth between the
processor and the DRAM main memory over
shared electrical circuits called buses.
Each transfer of data between the CPU and
memory is accomplished with a series of
steps called a bus transaction.
 A read transaction transfers data from the
main memory to the CPU.
A write transaction transfers data from the
CPU to the main memory.
49
A bus is a collection of parallel wires that carry
address, data, and control signals.
 Depending on the particular bus design, data
and address signals can share the same set of
wires, or they can use different sets.
Also, more than two devices can share the same
bus.
The control wires carry signals that
synchronize the transaction and identify what
kind of transaction is currently being performed.
50
Typical Bus Structure Connecting
CPU and Memory
A bus is a collection of parallel wires that
carry address, data, and control signals.
Buses are typically shared by multiple
devices.
CPU chip

register file

ALU

system bus memory bus

I/O main
bus interface
bridge memory
Memory Read Transaction
(1)
CPU places address A on the memory
bus.register file
ALU
%eax

main memory
I/O bridge 0
A
bus interface x A
 Memory Read Transaction
(2) memory reads A from the memory
-Main
bus, retrieves word x, and places it on
the bus.
register file

ALU
%eax

main memory
I/O bridge x 0

bus interface x A
 Memory Read Transaction
(3) read word x from the bus and
-CPU
copies it into register %eax.
register file

ALU
%eax x

main memory
I/O bridge 0

bus interface x A
Memory Write Transaction
(1)
- CPU places address A on bus. Main memory reads
it and waits for the corresponding data word to
arrive.
register file

ALU
%eax y

main memory
I/O bridge 0
A
bus interface A
Memory Write Transaction
(2)
CPU places data word y on the bus.
register file

ALU
%eax y
main memory
I/O bridge 0
y
bus interface A
Memory Write Transaction
(3)
Main memory read data word y from
the bus and stores it at address A.
register file

ALU
%eax y

main memory
I/O bridge 0

bus interface y A
To store a single bit, we can use
∗ Flip flops or latches
Larger memories can be built by
∗ Using a 2D array of these 1-bit devices
- “Horizontal” expansion to increase word
size
- “Vertical” expansion to increase number
of words
Design concepts are mostly independent of the
actual technique used to store a bit of data
58
59
60
61
62
63
64
Disks are constructed from platters.
Each platter consists of two sides, or surfaces,
that are coated with magnetic recording material.
 A rotating spindle in the center of the platter
spins the platter at a fixed rotational rate,
typically between 5400 and 15,000 revolutions
per minute (RPM).
A disk will typically contain one or more of
these platters encased in a sealed container.

65
66
67
The maximum number of bits that can be
recorded by a disk is known as its maximum
capacity, or simply capacity.
Disk capacity is determined by the
following technology factors:
Recording density (bits/in)
Track density (tracks/in)
Areal density (bits/in2)

68
Disk Capacity
 Capacity: maximum number of bits that can be stored.

Vendors express capacity in units of gigabytes (GB),
where 1 GB = 10^9.
 Capacity is determined by these technology factors:

Recording density (bits/in): number of bits that can be
squeezed into a 1 inch segment of a track.

Track density (tracks/in): number of tracks that can
be squeezed into a 1 inch radial segment.

Areal density (bits/in2): product of recording and
track density.
 Modern disks partition tracks into disjoint subsets called
recording zones

Each track in a zone has the same number of sectors,
determined by the circumference of innermost track.

Each zone has a different number of sectors/track
Disk manufacturers work tirelessly to
increase areal density (and thus capacity),
and this is doubling every few years.
The original disks, designed in an age of
low areal density, partitioned every track
into the same number of sectors, which was
determined by the number of sectors that
could be recorded on the innermost track.

70
The capacity of a disk is given by the
following formula:

71
For example, suppose we have a disk with
 five platters,
 512 bytes per sector,
20,000 tracks per surface, and
an average of 300 sectors per track.
Then the capacity of the disk is

72
73
What is the capacity of a disk with
 two platters
10,000 cylinders,
 an average of 400 sectors per track, and
512 bytes per sector

74
Disks read and write bits stored on the
magnetic surface using a read/write head
connected to the end of an actuator arm,
as shown in Figure below.
By moving the arm back and forth along
its radial axis, the drive can position the
head over any track on the surface.
This mechanical motion is known as a
seek.
75
 Once the head is positioned over the
desired track, then as each bit on the track
passes underneath, the head can either
sense the value of the bit (read the bit) or
alter the value of the bit (write the bit).
Disks with multiple platters have a
separate read/write head for each surface,
as shown in Figure below.
76
Disk Operation (Single-Platter
View)
The disk
The read/write head
surface
is attached to the end
spins at a
of the arm and flies over
fixed
the disk surface on
rotational
a thin cushion of air.
rate

spindle
spindle
spindle
spindle

By moving radially, the

arm can position the
read/write head over
any track.
Disk Operation (Multi-Platter
View) read/write heads
move in unison
from cylinder to cylinder

arm

spindle
79
80
81
82
83
84
85
86
87
88
Disks read and write data in sector-sized
blocks.
The access time for a sector has three
main components:
seek time,
rotational latency, and
 transfer time

89
To read the contents of some target sector, the arm
first positions the head over the track that contains
the target sector.
The time required to move the arm is called the
seek time.
 The seek time, Tseek, depends on the previous
position of the head and the speed that the arm
moves across the surface.
The average seek time in modern drives, Tavg
seek, measured by taking the mean of several
thousand seeks to random sectors, is typically on the
order of 3 to 9 ms. 90
Once the head is in position over the track,
the drive waits for the first bit of the target
sector to pass under the head.
The performance of this step depends on
both the position of the surface when the head
arrives at the target sector and the rotational
speed of the disk.
In the worst case, the head just misses the
target sector and waits for the disk to make a
full rotation.
91
When the first bit of the target sector
is under the head, the drive can begin
to read or write the contents of the
sector.
The transfer time for one sector
depends on the rotational speed and the
number of sectors per track.

92
Disk Access Time
 Average time to access some target sector approximated by :
 Taccess = Tavg seek + Tavg rotation + Tavg transfer

 Seek time (Tavg seek)

 Time to position heads over cylinder containing target

sector.
 Typical Tavg seek = 9 ms

 Rotational latency (Tavg rotation)

 Time waiting for first bit of target sector to pass under r/w

head.

Tavg rotation = 1/2 x 1/RPMs x 60 sec/1 min
 Transfer time (Tavg transfer)
 Time to read the bits in the target sector.

 Tavg transfer = 1/RPM x 1/(avg # sectors/track) x 60

secs/1 min.
Disk Access Time Example
Given:
 Rotational rate = 7,200 RPM
 Average seek time = 9 ms.
 Avg # sectors/track = 400.
Derived:
 Tavg rotation = 1/2 x (60 secs/7200 RPM) x 1000 ms/sec = 4
ms.
 Tavg transfer = 60/7200 RPM x 1/400 secs/track x 1000 ms/sec
= 0.02 ms
 Taccess = 9 ms + 4 ms + 0.02 ms
Important points:
 Access time dominated by seek time and rotational latency.
 First bit in a sector is the most expensive, the rest are free.
 SRAM access time is about 4 ns/doubleword, DRAM about 60
ns

Disk is about 40,000 times slower than SRAM,

2,500 times slower then DRAM.
I/O Bus
CPU chip
register file

ALU
system bus memory bus

I/O main
bus interface
bridge memory

I/O bus Expansion slots for

other devices such
USB graphics disk as network adapters.
controller adapter controller

mouse
keyboard monitor
disk
 Reading a Disk Sector (1)
CPU chip
CPU initiates a disk read by writing
register file
a command, logical block number,
and destination memory address to
ALU
a port (address) associated with
disk controller.

main
bus interface
memory

I/O bus

USB graphics disk

controller adapter controller

mouse
keyboard monitor
disk
 Reading a Disk Sector (2)
CPU chip
Disk controller reads the sector and
register file
performs a direct memory access
(DMA) transfer into main memory.
ALU

main
bus interface
memory

I/O bus

USB graphics disk

controller adapter controller

mouse
keyboard monitor
disk
 Reading a Disk Sector (3)
CPU chip
When the DMA transfer completes,
register file
the disk controller notifies the CPU
with an interrupt (i.e., asserts a
ALU
special “interrupt” pin on the CPU)

main
bus interface
memory

I/O bus

USB graphics disk

controller adapter controller

mouse
keyboard monitor
disk