CSE 380 – Computer Game ProgrammingMemory Management

Pocket WORM, a rip-off of Snake

Your first game industry job interview• Employer: “Hello”

• You: “Hi”

• Employer: “Well your resume looks good. If you have a student project to show I’d like to see it.”

• You: “Sure, I have two games I made that you can download from my Web page.”

• Employer: “Ok, great. Just a few questions then. What are the memory budgets for those games? How does your memory management system work? Oh, and please on this piece of paper, override the new and delete keywords and write your own definitions. In fact, do the same for malloc and free.”

C & C++

• Why are these languages used?– performance– they give you precise control over system resources

• Game programmers need to know:– how to implement game systems in these languages– how to employ the performance advantages of these

languages

• Note: this is a career-long learning process

Constrained systems

• Fixed resources for developer to work with– memory– disk space

• Consoles, cell phones, handheld platforms (i.e. Nintendo DS), etc.

• PCs have more resources (they cost more too)– but of course they can do many things

An Example of Capabilities• Xbox 360

– 512 MB RAM

• PlayStation 2:

– 32 MB RAM

• PlayStation 3:

– 256 MB RAM

– 256 MB Video RAM

• PSP

– 32 MB RAM

• Nintendo Wii

– 512 MB RAM Source: CNET Reviews & Specs

But we have tons of memory now

Computing Power

• As memory grows so do the demands on it– designers & artists want more data

What is a memory budget?

• A memory usage limit for the game

• Typically divided among game subsystems:– graphics, physics, sound, AI, networking, etc.

• Budgets can be hard or soft– soft means systems can request more memory larger

than their budgets allow• memory management system may give permission

Memory Management Systems

• To enforce a limit, you must know your game’s usage

• Most games have their own memory management system– allocates memory

– deallocates memory

– compiles statistics

– enforces rules/budgets

• 2 Options:– for development only (debug mode)– for live system (release mode as well)

Developing on a state of the art PC• Pros:

– faster development time

– less interference from system errors

• Cons:

– false negatives• error doesn’t happen to you, but will happen to others

– superior system may be good at overcoming flaws

• Console rule of thumb:

– the game will play completely different on a console than on the PC developing for it

The Console Developer’s Obsession

• Stay under budget

• What is:– the memory footprint of the application?– acceptable before problems will occur?

• PC game programmers need to be concerned with memory too– it’s just console programmers take it a few steps further

Staying under budget

• Approaches we’ll look at:• no memory leaks (or dangling references of course)

• minimize data when possible

• recycle data when possible

• fix data structure sizes when possible

• minimize dynamic memory allocation

• pre-compute when possible

• compress data when appropriate

• yell at artists, audio people, & game designers

Staying under budget

• Building a memory management system that:– optimizes memory usage– minimizes fragmentation– maximizes cache hit ratios– maintains statistics for analysis

No Memory Leaks• What’s a memory leak?

– a program allocates memory for an object or struct, but when done with it, doesn’t de-allocate it

• Why is that bad?– a program’s memory footprint will grow & grow & grow

– results: • slower, slower, & slower performance

• eventual crash

• Good rule of thumb:– if you put something on the heap, you have to take it off

new & delete• new creates an object on the heap and returns a pointer to it

– allocates a block of memory the size of:

• the object’s data

• plus object header info

• delete frees the memory on the heap referenced by a pointer

– invokes object’s deconstructor

• C++ rule of thumb:

– when you add a statement with new, also add your delete statement at the appropriate place

• when you won’t need that object anymore

Constructors & Destructors

• Constructors– called when an object is created with new– initializes instance variables

• Destructors– called when an object is destroyed with delete– destroys what’s at the end of instance variable pointers

if necessary. If necessary?• if no one else still needs them

malloc & free

• C methods for managing blocks of memory– void *malloc(size_t size);– void free(void *pointer);

• malloc allocates blocks on the heap– new calls malloc

• free releases blocks on the heap– delete calls free

* & new goes on the heap• We can create an object/struct/array on the stack

• Ex:

void myMethod()

{

RECT rect1;

RECT *rect2 = new RECT();

…

}• rect1 will be popped off the stack when the method returns• rect2 needs to be deleted unless given to another object to be

used as an instance variable

Common Sources of Memory Leaks

• Construct an object in a method, forget to delete it before the method ends

• An object has a pointer instance variable:– when you assign a new object, you forget to delete the

old one

OR– when you delete the object, you forget to delete the

object instance variable

• You have an STL container of objects:– when you want to remove or replace the contents, you

forget to delete the old contents

Constructing * objects in a method

• Good practice:– whenever you write new, decide where to put delete

Setting a * instance variable

• When you set a *, you must:– first delete the old one– then assign the new one

• Typically done inside a single set method

Deleting *objects from an STL container

• Iterate through the container

• For each item:– get the object– move the iterator onto the next item– remove the object from the container– delete the object

Detecting Memory Leaks

• Easiest way, open the Windows Task Manager– while the program is running, is memory increasing?

• Better ways exist:1.Use VS _Crt libraries and add hook for notification of

any memory allocation changes

2.Override new & delete and count memory allocation/freeing

3.Manage memory yourself

(We’ll look at these approaches too)

Beware the Dangling Reference

• When memory is freed up but someone still wants to use it– causes a fatal error

• Common causes:– you delete a * in an STL before removing it– you construct a * and give it to set method, then delete

it, thinking it was a temporary variable

Remember this?

• Approaches we’ll look at today:• no memory leaks (or dangling references of course)

• minimize data when possible

• pre-compute when possible

• recycle data when possible

• fix data structure sizes when possible

• minimize dynamic memory allocation

• compress data when appropriate

• yell at artists, audio people, & game designers

Minimize data when possible• Carefully choose data types

– sometimes means trading computation for memory

• How could we improve our RenderItems?– an int is 4 bytes, max of 2,147,483,647

– a 40X40 world with 64X64 tiles has a max coordinate of 2650

– Do we really need something that big?

– How about making x & y shorts?

• Console developer might take it a step further– a custom primitive

Minimizing data via a custom primitive• A single number to store data for multiple instance variables

– this is for memory extremists

• 2650 can be fit into 12 bits, so:– x & y can be fit into 24 bits

– we could store both in a single int

– we then have 8 bits left to store something else:• alpha fits perfectly, 8 bits

• We would now need methods for adding and retrieving data

A custom primitive for RenderItem’s x,y,aconst unsigned int CLEAR_X = 1048575;

const unsigned int CLEAR_Y = 4293918975;

const unsigned int CLEAR_A = 4294967040;

class RenderItem

{

private:

unsigned int x_y_a;

…

public:

RenderItem() { x_y_a = 0; }

…

// GET & SET METHODS

void setX(short initX)

{

unsigned int xAsInt = initX;

xAsInt <<= 20;

x_y_a &= CLEAR_X;

x_y_a |= xAsInt;

}

void setY(short initY)

{

unsigned int yAsInt = initY;

yAsInt <<= 8;

x_y_a &= CLEAR_Y;

x_y_a |= yAsInt;

}

void setA(byte initA)

{

x_y_a &= CLEAR_A;

x_y_a |= initA;

}

short getX()

{

unsigned int x = x_y_a;

x >>= 20;

return (short)x;

}

short getY()

{

unsigned int y = x_y_a;

y &= CLEAR_X;

y >>= 8;

return (short)y;

}

byte getA()

{

unsigned int a = x_y_a;

a &= CLEAR_X;

a &= CLEAR_Y;

return (byte)a;

}

};

Pre-compute when possible

• Games are approximations– they just need to be fun

• Game programmers love constants computed offline:

const unsigned int CLEAR_X = 1048575;

const unsigned int CLEAR_Y = 4293918975;

const unsigned int CLEAR_A = 4294967040;

• These are bit strings representing:– 11111111111100000000000011111111

– 00000000000011111111111111111111

– 11111111111111111111111100000000

Recycle data when possible

• How could we improve our particle system management?

• When a particle’s life has expired, rather than delete it from the data structure, reuse it for another particle

• How?– give each Particle a bool alive instance variable

– only render those in the list that are alive

– when time to add a new particle, pick one that’s not alive, fill it with data, and bring it back to life

Fix data structure sizes when possible

• How could we improve our rendering?– determine the maximum number of render items we will allow

– construct a data structure with that many RenderItem objects

– don’t let game conditions need more than the max

• Each frame:– fill in RenderItems in list at index 0 – (N-1), where N is our

counter

– Render items in list at indices 0 – (N-1)

Unused data structures vs. dynamic memory allocation

• Generally speaking, dynamic memory allocation is to be avoided when possible

• Why?– computationally expensive for memory management

system– increases potential for memory leaks– memory fragmentation

Compress data when possible

• Use algorithms to reduce footprint– we saw this with the last example

• Ex:– Data Compression

• RLE – Run Length Encoding

• LZW – Lempel Ziv Welch

– Fixed Point Math• for systems with no floating-point unit

RLE• Lossless compression algorithm

• Replace long sequences of identical data with a single piece of data and a count

• Ex: Background Tile Repetition

– suppose we wanted to store a tiled layer character sequence:

• 0,0,0,0,0,0,0.0,0.0,1c,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

• Could be replaced by 10(0),1(1c),16(0)

• 52 characters replaced by 17

• Many variations on this theme

– commonly used in constrained systems

Lempel Ziv Welch

• Lossless algorithm

• Replaces strings of characters with a code (number)– similar to RLE

• Adds new strings to a table of strings

• Longer strings are built from smaller string codes

Input String = /WED/WE/WEE/WEB/WET

Character Input Code Output New code value New String

/W / 256 /W

E W 257 WE

D E 258 ED

/ D 259 D/

WE 256 260 /WE

/ E 261 E/

WEE 260 262 /WEE

/W 261 263 E/W

EB 257 264 WEB

/ B 265 B/

WET 260 266 /WET

EOF T

http://marknelson.us/1989/10/01/lzw-data-compression/

Yell at artists, audio people, & game designers

• I’m kidding, I’m kidding

• Game programmers should know their system’s true limitations

• Game designers & artists:

– may not be fully aware of these limits

– will try to push these limits

• An alternative to employing the size of data is to reduce the amount of data

• Common strategy for game engines:

– maximum triangle counts per mesh

– maximum sizes for art, sound, music

– etc.

Detecting Memory Leaks

• Easiest way, open the Windows Task Manager– while the program is running, is memory increasing?

• More accurate ways exist:1.Use VS _Crt libraries and add hook for notification of

any memory allocation changes

2.Override new & delete and count memory allocation/freeing

3.Manage memory yourself

Windows Memory Methods

• Methods starting with _ are only built (and thus executed) when in debug mode

• Windows has a series of _CrtXXX methods for analyzing memory allocation

• What for?– detecting memory leaks

• Uses callback methods• You tell windows the callback method to call whenever

memory is allocated or de-allocated:

_CrtSetAllocHook(MyCustomAllocHook);• You then define a response in this method

_CrtSetAllocHook

• What kind of response might we program?

• We could count the number of objects constructed & deconstructed

• If the gap between them rises, we likely have a memory leak

• Drawbacks to this technique:– size of allocation is available– size of de-allocation is not– Windows-specific methods (not portable)

int MyCustomAllocHook( int nAllocType, void *userData, size_t size, int nBlockType, long requestNumber, const unsigned char *filename, int lineNumber)

{

if( nBlockType == _CRT_BLOCK)

return TRUE ; // better to not handle

switch(nAllocType)

{

case _HOOK_ALLOC :

objectConstructed++;

// add the code for handling

// the allocation requests

break ;

case _HOOK_FREE :

objectDestructed++;

// add the code for handling

// the free requests

break ;

}…

NOTE: These are

our counting variables

Ref: http://www.codeproject.com/KB/cpp/MLFDef.aspx

Overriding new & delete

• new & delete are C++ operators

• That means they can be overridden

• How?– redefine them globally so all classes use your

definition

• Headers:– void* operator new (size_t size) {…– void* operator new[] (size_t size) {…– void operator delete (void *p) {…– void operator delete[] (void *p) {…

• What is void*?– address of the object we are constructing or freeing

What do new & delete do?• Very simple stuff

• They call malloc & free– C memory allocation & de-allocation methods

• Result is system dependant, but for all systems:– malloc:

• sets aside memory on the heap equal to the size of the object

• returns a pointer to that memory address– free:

• releases a memory address

• What’s the sizeof an object?

– the sum of the sizes of its instance variables

Visual Studio’s new & deletevoid* operator new (size_t size)

{

void *p=malloc(size);

if (p==0) // did malloc succeed?

throw std::bad_alloc();

// ANSI/ISO compliant behavior

return p;

}

void operator delete (void *p)

{

free(p);

}

Ref:

Our new & deletevoid* operator new (size_t size)

{

void *p=our_malloc(size);

if (p==0) // did malloc succeed?

throw std::bad_alloc();

// ANSI/ISO compliant behavior

return p;

}

void operator delete (void *p)

{

our_free(p);

}

• NOTE: We would then use the same code for the array versions of new & delete

What are our malloc & free going to do?

• Same as before. How?– we’ll call C’s malloc and free from them

• Problem:– delete doesn’t get size of memory being freed

• Solution:– when mallocing, add a little extra memory for some

header info– when freeing, extract header info– what kind of info?

• size of allocation

• checksum for error checking

Time to practice pointer arithmetic

• What’s that?

• Feature of C language

• If you have a pointer char *text, it points to memory address storing text

• text + 1 points to 1 byte after start of text– might be second character– might not– you need to know what you’re moving your pointer to

• Why do we care?– we need to stick our info at the front of our object

malloc_info

• Our header– we’ll stick in in front of each piece of data we allocate

#define MALLOC_CHECKSUM 123456789

struct malloc_info

{

int checksum;

size_t bytes;

};

Ref: Grad Student Rick Spillane: http://www.fsl.cs.sunysb.edu/~rick/

void* our_malloc(size_t bytes)

{

void *data;

struct malloc_info *info;

data = malloc(bytes + sizeof(struct malloc_info));

if (!data)

return NULL;

else {

size_t headerSize = sizeof(struct malloc_info);

totalAlloc += bytes + headerSize;

dataAlloc += bytes;

info = (struct malloc_info *)data;

info->checksum = MALLOC_CHECKSUM;

info->bytes = bytes;

char *data_char = (char*)data;

data_char += headerSize;

return (void*)data_char;

}

}

void our_free(void *data)

{

struct malloc_info *info;

void *data_n_header;

size_t header_size = sizeof(struct malloc_info);

char *data_char = (char*)data;

data_char = data_char - header_size;

data_n_header = (void*)data_char;

info = (struct malloc_info *)data_n_header;

if (info->checksum != MALLOC_CHECKSUM)

throw std::bad_alloc();

totalFreed += info->bytes + sizeof(struct malloc_info);

dataFreed += info->bytes;

free(data_n_header);

}

So what?

• So we can monitor the following differences:totalAlloc–totalFreed

dataAlloc–dataFreed

• We can also reset them if we wish

• Why?– reset before a method starts– check differences after method completes

• only relevant for methods that are supposed to have a 0 net object creation

– if differences > 0, memory leak may exist

Another option: optimize memory management

• How?

– manage it yourself

– not the heap directly, just blocks of memory

• Why?

– to minimize dynamic allocation• minimize calls to malloc & free

– to minimize fragmentation – what’s that?

– to tightly cap budgets

• Companies typically have their own memory management systems

Memory Management Systems

• Draw up a budget of all the data you need

• Divide your data into separate blocks– blocks tightly pack like data

– each block has a header with size & used/not used info

• Allocate your data in these blocks– allocations need not call malloc, just do pointer arithmetic on a

block & then return a void*

• We’ll see an implementation of this in Box2D when we do realistic physics later this semester

Memory Pools

• What do you think is the most expensive part of a system’s memory allocation (malloc)?– finding an appropriate block of memory to return

– worse when memory is fragmented

– a search may have to look through many blocks before finding one it can use

• Solution: Memory Pools– pre-allocated memory used to allocate objects of a particular

size at runtime

– when released, object memory is returned to the pool, not freed on the heap

Ref: [1]

Advantages of Memory Pools

• No malloc & heap search overhead

• No heap fragmentation

• Spatial coherence– improve data cache hits

• One approach:– allocate on giant block of memory for your system– divide up this block yourself into subsystems– this further improves spatial coherence and minimizes

fragmentation

Disadvantages of Memory Pools

• Slack space– memory allocated to your system but never used

• An additional system to manage– and implement, test, maintain, etc. of course

A Memory Pools Strategy1. Allocate one large memory block

– think of it as made up of memory slices that we will give out

– these slices (blocks, chunks, etc.) start out marked as free

• we may use a linked list of free blocks

• add headers to each block to point to next & prev

2. When a new request comes in:

– grab a block in list:

• remove it from the list

• return pointer to caller

3. When a delete request comes in:

– add that memory to the list of free blocks (at head)

– order doesn’t matter

Memory Pool Algorithms• What sizes should the blocks be?

– typically various for a single system allocator

• Work on a free list of blocks

• Sequential fit methods:

– First fit

– Best fit

– Optimal fit

– Worst fit

– may also involve subdividing blocks

• Buddy-system method

What’s the size of a struct?

• Are Dummy1 & Dummy2 the same size?

int sizeOfDummy1 = sizeof(Dummy1);

int sizeOfDummy2 = sizeof(Dummy2);

• http://www.codeguru.com/forum/showthread.php?t=276622

struct Dummy1{

int num1;char letter1;int num2;char letter2;int num3;char letter3;

};

struct Dummy2{

int num1;int num2;int num3;char letter1;char letter2;char letter3;

};

24

16

References• C++ Memory Management: From Fear to Triumph

– http://www.linuxdevcenter.com/pub/a/linux/2003/05/08/cpp_mm-1.html

• 3D Game Engine Design by David Eberly

– Chapter 19: Memory Management

• Run Length Encoding (RLE)

– http://www.data-compression.info/Algorithms/RLE/index.htm

• LZW Data Compression by Mark Nelson

– http://marknelson.us/1989/10/01/lzw-data-compression/

References• Introduction to Game Development by Steve Rabin

– http://www.charlesriver.com/Books/BookDetail.aspx?productID=99109

• C++ Memory Management: From Fear to Triumph

– http://www.linuxdevcenter.com/pub/a/linux/2003/05/08/cpp_mm-1.html

• 3D Game Engine Design by David Eberly

– Chapter 19: Memory Management

• Box2D by Eric Catto

– http://www.box2d.org/