Carnegie Mellon

Dynamic Memory Alloca/on:

Basic Concepts
15-213: Introduc;on to Computer Systems
19th Lecture, Nov. 3, 2015

Instructors:
Randal E. Bryant and David R. O’Hallaron

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 1

 Carnegie Mellon

Today
¢ Basic concepts
¢ Implicit free lists

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 2

 Carnegie Mellon

Dynamic Memory Alloca/on

¢ Programmers use Applica/on
dynamic memory Dynamic Memory Allocator
allocators (such as
Heap
malloc) to acquire VM
at run /me.
§ For data structures whose User stack
size is only known at
run;me.
Top of heap
¢ Dynamic memory (brk ptr)
Heap (via malloc)
allocators manage an
area of process virtual Unini/alized data (.bss)
memory known as the Ini/alized data (.data)
heap. Program text (.text)

0
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 3
 Carnegie Mellon

Dynamic Memory Alloca/on

¢ Allocator maintains heap as collec/on of variable sized

blocks, which are either allocated or free
¢ Types of allocators
§ Explicit allocator: applica;on allocates and frees space
E.g., malloc and free in C
§
§ Implicit allocator: applica;on allocates, but does not free space
§ E.g. garbage collec;on in Java, ML, and Lisp

¢ Will discuss simple explicit memory alloca/on today

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 4

 Carnegie Mellon

The malloc Package

#include <stdlib.h>
void *malloc(size_t size)
§ Successful:
§ Returns a pointer to a memory block of at least size bytes
aligned to an 8-byte (x86) or 16-byte (x86-64) boundary
§ If size == 0, returns NULL
§ Unsuccessful: returns NULL (0) and sets errno
void free(void *p)
§ Returns the block pointed at by p to pool of available memory
§ p must come from a previous call to malloc or realloc
Other func/ons
§ calloc: Version of malloc that ini;alizes allocated block to zero.
§ realloc: Changes the size of a previously allocated block.
§ sbrk: Used internally by allocators to grow or shrink the heap
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 5
 Carnegie Mellon

malloc Example
#include <stdio.h>!
#include <stdlib.h>!
!
void foo(int n) {!
int i, *p;!
!
/* Allocate a block of n ints */!
p = (int *) malloc(n * sizeof(int));!
if (p == NULL) {!
perror("malloc");!
exit(0);!
}!
!
/* Initialize allocated block */!
for (i=0; i<n; i++)!
p[i] = i;!
!
!
/* Return allocated block to the heap */!
free(p);!
}!

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 6

 Carnegie Mellon

Assump/ons Made in This Lecture

¢ Memory is word addressed.
¢ Words are int-sized.

Allocated block Free block

(4 words) (3 words) Free word
Allocated word

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 7

 Carnegie Mellon

Alloca/on Example
p1 = malloc(4)

p2 = malloc(5)

p3 = malloc(6)

free(p2)

p4 = malloc(2)

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 8

 Carnegie Mellon

Constraints
¢ Applica/ons
§ Can issue arbitrary sequence of malloc and free requests
§ free request must be to a malloc’d block

¢ Allocators
§ Can’t control number or size of allocated blocks
§ Must respond immediately to malloc requests
i.e., can’t reorder or buffer requests
§
§ Must allocate blocks from free memory
§ i.e., can only place allocated blocks in free memory
§ Must align blocks so they sa;sfy all alignment requirements
§ 8-byte (x86) or 16-byte (x86-64) alignment on Linux boxes
§ Can manipulate and modify only free memory
§ Can’t move the allocated blocks once they are malloc’d
§ i.e., compac;on is not allowed
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 9
 Carnegie Mellon

Performance Goal: Throughput

¢ Given some sequence of malloc and free requests:
§ R0, R1, ..., Rk, ... , Rn-1

¢ Goals: maximize throughput and peak memory u/liza/on

§ These goals are o`en conflic;ng

¢ Throughput:
§ Number of completed requests per unit ;me
§ Example:
§ 5,000 malloc calls and 5,000 free calls in 10 seconds
§ Throughput is 1,000 opera;ons/second

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 10

 Carnegie Mellon

Performance Goal: Peak Memory U/liza/on

¢ Given some sequence of malloc and free requests:
§ R0, R1, ..., Rk, ... , Rn-1
¢ Def: Aggregate payload Pk
§ malloc(p) results in a block with a payload of p bytes
§ A`er request Rk has completed, the aggregate payload Pk is the sum of
currently allocated payloads

¢ Def: Current heap size Hk

§ Assume Hk is monotonically nondecreasing
§ i.e., heap only grows when allocator uses sbrk

¢ Def: Peak memory u@liza@on aAer k+1 requests

§ Uk = ( maxi<=k Pi ) / Hk

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 11

 Carnegie Mellon

Fragmenta/on
¢ Poor memory u/liza/on caused by fragmenta@on
§ internal fragmenta;on
§ external fragmenta;on

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 12

 Carnegie Mellon

Internal Fragmenta/on
¢ For a given block, internal fragmenta@on occurs if payload is
smaller than block size

Block

Internal Internal
Payload
fragmenta/on fragmenta/on

¢ Caused by
§ Overhead of maintaining heap data structures
§ Padding for alignment purposes
§ Explicit policy decisions
(e.g., to return a big block to sa;sfy a small request)

¢ Depends only on the paVern of previous requests

§ Thus, easy to measure
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 13
 Carnegie Mellon

External Fragmenta/on
¢ Occurs when there is enough aggregate heap memory,
but no single free block is large enough

p1 = malloc(4)

p2 = malloc(5)

p3 = malloc(6)

free(p2)

p4 = malloc(6) Oops! (what would happen now?)

¢ Depends on the paVern of future requests

§ Thus, difficult to measure
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 14
 Carnegie Mellon

Implementa/on Issues
¢ How do we know how much memory to free given just a
pointer?

¢ How do we keep track of the free blocks?

¢ What do we do with the extra space when alloca/ng a

structure that is smaller than the free block it is placed in?

¢ How do we pick a block to use for alloca/on -- many

might fit?

¢ How do we reinsert freed block?

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 15
 Carnegie Mellon

Knowing How Much to Free

¢ Standard method
§ Keep the length of a block in the word preceding the block.
This word is o`en called the header field or header
§
§ Requires an extra word for every allocated block

p0

p0 = malloc(4) 5

block size payload

free(p0)

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 16

 Carnegie Mellon

Keeping Track of Free Blocks

¢ Method 1: Implicit list using length—links all blocks

5 4 6 2

¢ Method 2: Explicit list among the free blocks using pointers

5 4 6 2

¢ Method 3: Segregated free list

§ Different free lists for different size classes

¢ Method 4: Blocks sorted by size

§ Can use a balanced tree (e.g. Red-Black tree) with pointers within each
free block, and the length used as a key
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 17
 Carnegie Mellon

Today
¢ Basic concepts
¢ Implicit free lists

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 18

 Carnegie Mellon

Method 1: Implicit List

¢ For each block we need both size and alloca/on status
§ Could store this informa;on in two words: wasteful!
¢ Standard trick
§ If blocks are aligned, some low-order address bits are always 0
§ Instead of storing an always-0 bit, use it as a allocated/free flag
§ When reading size word, must mask out this bit
1 word

Size a a = 1: Allocated block

a = 0: Free block
Format of
allocated and Payload
Size: block size
free blocks
Payload: applica/on data
(allocated blocks only)
Op/onal
padding
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 19
 Carnegie Mellon

Detailed Implicit Free List Example

Unused
Start
of 8/0 16/1 32/0 16/1 0/1
heap

Double-word Allocated blocks: shaded

aligned Free blocks: unshaded
Headers: labeled with size in bytes/allocated bit

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 20

 Carnegie Mellon

Implicit List: Finding a Free Block

¢ First fit:
§ Search list from beginning, choose first free block that fits:
p = start;
while ((p < end) && \\ not passed end
((*p & 1) || \\ already allocated
(*p <= len))) \\ too small
p = p + (*p & -2); \\ goto next block (word addressed)

§ Can take linear ;me in total number of blocks (allocated and free)
§ In prac;ce it can cause “splinters” at beginning of list
¢ Next fit:
§ Like first fit, but search list star;ng where previous search finished
§ Should o`en be faster than first fit: avoids re-scanning unhelpful blocks
§ Some research suggests that fragmenta;on is worse
¢ Best fit:
§ Search the list, choose the best free block: fits, with fewest bytes le` over
§ Keeps fragments small—usually improves memory u;liza;on
§ Will typically run slower than first fit
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 21
 Carnegie Mellon

Implicit List: Alloca/ng in Free Block

¢ Alloca/ng in a free block: spliQng
§ Since allocated space might be smaller than free space, we might want
to split the block

4 4 6 2

addblock(p, 4)

4 4 4 2 2

void addblock(ptr p, int len) {

int newsize = ((len + 1) >> 1) << 1; // round up to even
int oldsize = *p & -2; // mask out low bit
*p = newsize | 1; // set new length
if (newsize < oldsize)
*(p+newsize) = oldsize - newsize; // set length in remaining
} // part of block
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 22
 Carnegie Mellon

Implicit List: Freeing a Block

¢ Simplest implementa/on:
§ Need only clear the “allocated” flag
void free_block(ptr p) { *p = *p & -2 }

§ But can lead to “false fragmenta;on”

4 4 4 2 2

free(p) p

4 4 4 2 2

malloc(5) Oops!

There is enough free space, but the allocator won’t be able to find it

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 23

 Carnegie Mellon

Implicit List: Coalescing

¢ Join (coalesce) with next/previous blocks, if they are free
§ Coalescing with next block

4 4 4 2 2
logically
p
free(p) gone
4 4 6 2 2

void free_block(ptr p) {
*p = *p & -2; // clear allocated flag
next = p + *p; // find next block
if ((*next & 1) == 0)
*p = *p + *next; // add to this block if
} // not allocated

§ But how do we coalesce with previous block?

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 24

 Carnegie Mellon

Implicit List: Bidirec/onal Coalescing

¢ Boundary tags [Knuth73]
§ Replicate size/allocated word at “booom” (end) of free blocks
§ Allows us to traverse the “list” backwards, but requires extra space
§ Important and general technique!

4 4 4 4 6 6 4 4

Header Size a a = 1: Allocated block

a = 0: Free block
Format of
Size: Total block size
allocated and Payload and
padding
free blocks Payload: Applica/on data
(allocated blocks only)
Boundary tag Size a
(footer)

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 25

 Carnegie Mellon

Constant Time Coalescing

Case 1 Case 2 Case 3 Case 4

Allocated Allocated Free Free

Block being
freed
Allocated Free Allocated Free

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 26

 Carnegie Mellon

Constant Time Coalescing (Case 1)

m1 1 m1 1

m1 1 m1 1
n 1 n 0

n 1 n 0
m2 1 m2 1

m2 1 m2 1

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 27

 Carnegie Mellon

Constant Time Coalescing (Case 2)

m1 1 m1 1

m1 1 m1 1
n 1 n+m2 0

n 1
m2 0

m2 0 n+m2 0

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 28

 Carnegie Mellon

Constant Time Coalescing (Case 3)

m1 0 n+m1 0

m1 0
n 1

n 1 n+m1 0
m2 1 m2 1

m2 1 m2 1

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 29

 Carnegie Mellon

Constant Time Coalescing (Case 4)

m1 0 n+m1+m2 0

m1 0
n 1

n 1
m2 0

m2 0 n+m1+m2 0

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 30

 Carnegie Mellon

Disadvantages of Boundary Tags

¢ Internal fragmenta/on

¢ Can it be op/mized?
§ Which blocks need the footer tag?
§ What does that mean?

Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 31

 Carnegie Mellon

Summary of Key Allocator Policies

¢ Placement policy:
§ First-fit, next-fit, best-fit, etc.
§ Trades off lower throughput for less fragmenta;on
§ Interes@ng observa@on: segregated free lists (next lecture)
approximate a best fit placement policy without having to search en;re
free list

¢ Splifng policy:
§ When do we go ahead and split free blocks?
§ How much internal fragmenta;on are we willing to tolerate?
¢ Coalescing policy:
§ Immediate coalescing: coalesce each ;me free is called
§ Deferred coalescing: try to improve performance of free by deferring
coalescing un;l needed. Examples:
§ Coalesce as you scan the free list for malloc
§ Coalesce when the amount of external fragmenta;on reaches
some threshold 32
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on
 Carnegie Mellon

Implicit Lists: Summary

¢ Implementa/on: very simple
¢ Allocate cost:
§ linear ;me worst case
¢ Free cost:
§ constant ;me worst case
§ even with coalescing
¢ Memory usage:
§ will depend on placement policy
§ First-fit, next-fit or best-fit

¢ Not used in prac/ce for malloc/free because of linear-

/me alloca/on
§ used in many special purpose applica;ons

¢ However, the concepts of splifng and boundary tag

coalescing are general to all allocators
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec;ve, Third Edi;on 33