CSCI3150 Introduction to

Operating Systems

Lecture 12: Memory Management

Part III: Swapping
Hong Xu
https://github.com/henryhxu/CSCI3150
https://github.com/henryhxu/CSCI3150
Overview

 Mechanisms
 Policies

CSCI 3150 Intro to OS 2

Beyond Physical Memory: Mechanisms

 Use part of disk as memory

 OS needs a place to stash away portions of address space that currently
aren’t in great demand.

 In modern systems, this role is usually served by a hard disk drive.

Registers

Cache

Main Memory

Mass Storage (hard disk, tape, etc...)

Memory hierarchy in modern system

CSCI 3150 Intro to OS 3

Swap Space

 Reserve some space on the disk for moving pages back and forth.
 OS needs to remember the swap space, in page-sized unit.

PFN 0 PFN 1 PFN 2 PFN 3

Physical Proc 0 Proc 1 Proc 1 Proc 2

Memory [VPN 0] [VPN 2] [VPN 3] [VPN 0]

Block 0 Block 1 Block 2 Block 3 Block 4 Block 5 Block 6 Block 7

Swap Proc 0 Proc 0 Proc 1 Proc 1 Proc 3 Proc 2 Proc 3

[Free]
Space [VPN 1] [VPN 2] [VPN 0] [VPN 1] [VPN 0] [VPN 1] [VPN 1]

Physical Memory and Swap Space

CSCI 3150 Intro to OS 4

Present Bit

 Add some machinery higher up in the system to support swapping

the pages to and from the disk.
 When the hardware looks in the PTE, it may find that the page is not
present in physical memory.

Value Meaning
1 page is present in physical memory
0 The page is not in memory but rather on disk.

CSCI 3150 Intro to OS 5

Concepts

 Page fault
 Accessing page that is not in physical memory.

 If a page is not present and has been swapped to disk, the OS needs to
swap the page back into memory in order to service the page fault.

 Page replacement
 The OS likes to page out some pages to make room for the new ones it is
about to bring in

 The process of picking a page to evict (or replace) is known as page-

replacement policy.

CSCI 3150 Intro to OS 6

When to Perform Page Replacement

 Lazy approach…
 OS waits until memory is full to start replacing pages.

 This is clearly unrealistic… Do not procrastinate!

 Swap Daemon, Page Daemon

 When there are fewer than LW (low watermark) pages available, a
background thread that is responsible for freeing memory runs.

 The thread evicts pages until there are HW (high watermark) pages
available.

CSCI 3150 Intro to OS 7

Page Fault Control Flow

 Bits used for data such as the page’s PFN are used for a disk address
in the PTE
3. Check storage whether the page exists.
kernel Secondary Storage

2.Trap
1. Reference

Load M

i
6. reinstruction
Page Frame

Page Frame

Page Table ... 4. Get the page

Page Frame
5. Reset Page Table. Page Frame

Address Space
When the OS receives a page fault, it looks in the PTE and issues the request to disk.

CSCI 3150 Intro to OS 8

Page Fault Control Flow – Hardware

1: VPN = (VirtualAddress & VPN_MASK) >> SHIFT

2: (Success, TlbEntry) = TLB_Lookup(VPN)
3: if (Success == True) // TLB Hit
4: if (CanAccess(TlbEntry.ProtectBits) == True)
5: Offset = VirtualAddress & OFFSET_MASK
6: PhysAddr = (TlbEntry.PFN << SHIFT) | Offset
7: Register = AccessMemory(PhysAddr)
8: else RaiseException(PROTECTION_FAULT)
9: else // TLB Miss
10: PTEAddr = PTBR + (VPN * sizeof(PTE))
11: PTE = AccessMemory(PTEAddr)
12: if (PTE.Valid == False)
13: RaiseException(SEGMENTATION_FAULT)
14: else
15: if (CanAccess(PTE.ProtectBits) == False)
16: RaiseException(PROTECTION_FAULT)
17: else if (PTE.Present == True)
18: // assuming hardware-managed TLB
19: TLB_Insert(VPN, PTE.PFN, PTE.ProtectBits)
20: RetryInstruction()
21: else if (PTE.Present == False)
22: RaiseException(PAGE_FAULT)

CSCI 3150 Intro to OS 9

Page Fault Control Flow – Software

1: PFN = FindFreePhysicalPage()
2: if (PFN == -1) // no free page found
3: PFN = EvictPage() // run replacement algorithm
4: DiskRead(PTE.DiskAddr, PFN) // sleep (waiting for I/O)
5: PTE.present = True // update page table with present
6: PTE.PFN = PFN // bit and translation (PFN)
7: RetryInstruction() // retry instruction

 The OS must find a physical frame for the soon-be-faulted-in page to

reside within.

 If there is no such page, waiting for the replacement algorithm to run

CSCI 3150 Intro to OS 10

Summary

 Swapping: making the part of disk as memory

 Present bit required

CSCI 3150 Intro to OS 11

 Swapping Policies

CSCI 3150 Intro to OS 12

Goal of Cache Management

 To minimize cache misses

 To improve average memory access time (AMAT)

𝐴𝑀𝐴𝑇 = 𝑃𝐻𝑖𝑡 ∗ 𝑇𝑀 + (𝑃𝑀𝑖𝑠𝑠 ∗ 𝑇𝐷 )

Notation Meaning
𝑇𝑀 The cost of accessing memory
𝑇𝐷 The cost of accessing disk
𝑃𝐻𝑖𝑡 The probability of hitting the data item in the cache
𝑃𝑀𝑖𝑠𝑠 The probability of not finding the data in the cache

CSCI 3150 Intro to OS 13

The Optimal Replacement Policy

 Lead to the fewest number of misses overall

 Replace the page that will be accessed furthest in the future.

 Result in the fewest possible cache misses.

 Serve only as a comparison point, to know how close we are to

perfection.

CSCI 3150 Intro to OS 14

Tracing the Optimal Policy
Reference Row

0 1 2 0 1 3 0 3 1 2 1

Access Hit/Miss? Evict Resulting Cache State

0 Miss 0
1 Miss 0,1
2 Miss 0,1,2
0 Hit 0,1,2
1 Hit 0,1,2
3 Miss 2 0,1,3
0 Hit 0,1,3
3 Hit 0,1,3
1 Hit 0,1,3
2 Miss 3 0,1,2
1 Hit 0,1,2

𝐻𝑖𝑡𝑠
Hit rate is = 𝟓𝟒. 𝟔% Future is unknown!
𝐻𝑖𝑡𝑠+𝑀𝑖𝑠𝑠𝑒𝑠

CSCI 3150 Intro to OS 15

A Simple Policy: FIFO

 Pages were placed in a queue when they enter the system.

 When a replacement occurs, the page on the head of the queue (the
“first-in” page) is evicted.
 Simple to implement

 Agnostic to the importance of pages

CSCI 3150 Intro to OS 16

Tracing the FIFO Policy
Reference Row

0 1 2 0 1 3 0 3 1 2 1

Access Hit/Miss? Evict Resulting Cache State

0 Miss 0
1 Miss 0,1
2 Miss 0,1,2
0 Hit 0,1,2
1 Hit 0,1,2
3 Miss 0 1,2,3
0 Miss 1 2,3,0
3 Hit 2,3,0
1 Miss 3,0,1
2 Miss 3 0,1,2
1 Hit 0,1,2

Hit rate is
𝐻𝑖𝑡𝑠
= 𝟑𝟔. 𝟒% Even though page 0 had been accessed a number of
𝐻𝑖𝑡𝑠+𝑀𝑖𝑠𝑠𝑒𝑠 times, FIFO still kicks it out.

CSCI 3150 Intro to OS 17

Belady’s Anomaly

 We would expect the cache hit rate to increase when the cache gets
larger. But in this case, with FIFO, it gets worse.

Reference Row

1 2 3 4 1 2 5 1 2 3 4 5

14

12
Page Fault Count

10

0
1 2 3 4 5 6 7
Page Frame Count

CSCI 3150 Intro to OS 18

Another Simple Policy: Random

 Picks a random page to replace under memory pressure.

 It doesn’t really try to be too intelligent in picking which page to evict

 Random does depends entirely upon how lucky Random gets in its choice.

Access Hit/Miss? Evict Resulting Cache State

0 Miss 0
1 Miss 0,1
2 Miss 0,1,2
0 Hit 0,1,2
1 Hit 0,1,2
3 Miss 0 1,2,3
0 Miss 1 2,3,0
3 Hit 2,3,0
1 Miss 3 2,0,1
2 Hit 2,0,1
1 Hit 2,0,1

CSCI 3150 Intro to OS 19

Using History

 Learn on the past and use history.

 Two types of historical information.

Historical
Meaning Algorithms
Information
The more recently a page has been accessed, the more likely it
Recency LRU
will be accessed again
If a page has been accessed many times, It should not be
Frequency LFU
replaced as it clearly has some value

CSCI 3150 Intro to OS 20

Using History: LRU

 Replace the least-recently-used page.

Reference Row

0 1 2 0 1 3 0 3 1 2 1

Access Hit/Miss? Evict Resulting Cache State

0 Miss 0
1 Miss 0,1
2 Miss 0,1,2
0 Hit 1,2,0
1 Hit 2,0,1
3 Miss 2 0,1,3
0 Hit 1,3,0
3 Hit 1,0,3
1 Hit 0,3,1
2 Miss 0 3,1,2
1 Hit 3,2,1

CSCI 3150 Intro to OS 21

Workload Example : The No-Locality Workload

 Each reference is to a random page in a set of 100 pages

 Workload has 100 accesses over time.

 Choosing the next page to refer to at random

The No-Locality Workload

100%

80%

When the cache is large enough to fit

Hit Rate

60% the entire workload,

OPT
it also doesn’t matter which policy
LRU
FIFO you use.
40%
RAND

20%

20 40 60 80 100
Cache Size (Blocks)

CSCI 3150 Intro to OS 22

Workload Example : The 80-20 Workload

 Exhibits locality: 80% of the reference are made to 20% of the page

 The remaining 20% of the reference are made to the remaining 80% of the
pages.

The 80-20 Workload

100%

80%
LRU is more likely to
hold onto the hot pages.
Hit Rate

60%
OPT
LRU
FIFO
40%
RAND

20%

20 40 60 80 100
Cache Size (Blocks)

CSCI 3150 Intro to OS 23

Workload Example : The Looping Sequential

 Refer to 50 pages in sequence.

 Starting at 0, then 1, … up to page 49, and then we loop, repeating those accesses,
for total of 10,000 accesses to 50 unique pages.

The Looping-Sequential Workload

100%

80%
Hit Rate

60%
OPT
LRU
FIFO
40%
RAND

20%

20 40 60 80 100
Cache Size (Blocks)

CSCI 3150 Intro to OS 24

Approximating LRU: Clock Algorithm
 Require hardware support: use bit
 Whenever a page is referenced, the use bit is set by hardware to 1.

 Hardware never clears the bit, though; that is the responsibility of the OS

 Clock Algorithm
 All pages of the system arranges in a circular list.

 A clock hand points to a page to begin with.

 The algorithm continues until it finds a use bit that is set to 0.

A
H B
Use bit Meaning
G C 0 Evict the page
1 Clear use bit and advance hand
F D
E
The Clock page replacement algorithm

CSCI 3150 Intro to OS 25

Workload with Clock Algorithm

 Clock algorithm doesn’t do as well as LRU; but better than approaches

that don’t consider history at all

The 80-20 Workload

100%

80%
Hit Rate

60%
OPT
LRU
Clock
40%
FIFO
RAND
20%

20 40 60 80 100
Cache Size (Blocks)

CSCI 3150 Intro to OS 26

Considering Dirty Pages

 The hardware includes a modified bit (a.k.a dirty bit)

 Page has been modified and is thus dirty, it must be written back to disk
to evict it.

 Page has not been modified; the eviction is free.

CSCI 3150 Intro to OS 27

Prefetching

 The OS guesses that a page is about to be used, and thus bring it in

ahead of time.

Page 1 is brought into memory

Page n
Page 3

Page 4

Page 5
…

Physical Memory
Page 1
Page 2
Page 3
Page 4
…

Secondary
Storage
Page 2 likely soon be accessed and
should be brought into memory too

CSCI 3150 Intro to OS 28

Clustering, Grouping

 Collect a number of pending writes together in memory and write

them to disk in one write.
 Perform a single large write more efficiently than many small ones.

Pending writes

Page n
Page 1
Page 2

Page 3

Page 4

Page 5
…

Physical Memory
write in one write
Page 1
Page 2
Page 3
Page 4

Secondary
Storage

CSCI 3150 Intro to OS 29

Thrashing

 Memory is oversubscribed and the memory demands of the set of running

processes exceeds the available physical memory.
 Decide not to run a subset of processes.

 Reduced set of processes working sets fit in memory.

CPU
Utilization

Trashing

Degree of multiprogramming

CSCI 3150 Intro to OS 30

Current Trends in Memory Management

 Less critical now

 Memory is cheap → larger physical memory (well except for Apple’s
devices…)

 Larger page sizes

 Better TLB coverage; smaller page tables

 Larger virtual address space

 64-bit address

 File I/O using the virtual memory system

 Memory mapped I/O: mmap()
Summary

 Swapping: use part of disk as memory

 Page replacement (page/swap out, page/swap in)
 LRU, LFU, Random, FIFO

 Approximation to LRU: Clock

 Batching the disk IO

 Clustering

 Grouping

 Prefetching

CSCI 3150 Intro to OS 32