Dealing With Leaks
-
Upload
gobara-dhan -
Category
Documents
-
view
235 -
download
0
Transcript of Dealing With Leaks
© 2003 Mark Bartosik, www.leakbrowser.com
Dealing with Leaks
© 2003 Mark Bartosik, www.leakbrowser.com
What is a resource leak ?
• Repeated allocation of a resource without timely deallocation
Probable resources:• memory
virtual memory, non-paged pool memory
• address space• OS objects / handles
files, threads, events, semaphores, sockets, windows, timers, etc.
• disk space• application level / non OS recourses e.g. database records
© 2003 Mark Bartosik, www.leakbrowser.com
Unreferenced resources
int global = 20;
void foo(){
int b = 22;HANDLE h = CreateEvent(....);
int * int_ptr = int(1000); int * array_ptr = new int[1024];}
© 2003 Mark Bartosik, www.leakbrowser.com
Unreferenced heap memoryProcess / User memory
static storage (fixed size)
0x38C
22
21
stackgrows ondem and
stack (one per thread)
1000
? ar? n mod
1024 e lements(4096 bytes)
heap
0xDE ADBEAF
0x17E D54230xD EA DBEAF
0x1 7E D5423
heapgrows ondem and
20
Kerne l / O S memory
0x80101234
0x80101DC0 event ob ject0x801013580x38C
0x388
0x390
int global = 20;
void foo(){ int b = 22; HANDLE h = CreateEvent(...); int * int_ptr = int(1000); int * array_ptr = new int[1024];
execution is here}
int main(){ int a = 21; foo();
return EXIT_SUCCESS;}
© 2003 Mark Bartosik, www.leakbrowser.com
Unreferenced heap memoryProcess / User memory
static storage (fixed size)
21
stackgrows ondem and
stack (one per thread)
1000
? ar? n mod
1024 e lements(4096 bytes)
heap
0xD EA DBEAF
0x1 7E D5423
heapgrows ondem and
20
Kerne l / O S memory
0x80101234
0x80101DC0 event ob ject0x801013580x38C
0x388
0x390
int global = 20;
void foo(){ int b = 22; HANDLE h = CreateEvent(...); int * int_ptr = int(1000); int * array_ptr = new int[1024];
}
int main(){ int a = 21; foo(); execution is here return EXIT_SUCCESS;}
© 2003 Mark Bartosik, www.leakbrowser.com
Unbounded resource consumptionstd::map<unsigned, transaction_t> transactions;
void do_transaction(P1_t p1, P2_t p2)
{
transactions[id] = transaction_t(p1,p2);
}
© 2003 Mark Bartosik, www.leakbrowser.com
C++, avoiding the problem
• encapsulate ownership within objects or smart pointersbstr_t CComPtr shared_ptr auto_ptr scoped_ptr auto_handle std::string, avoid BSTR, char*
• only use raw pointers to implement tightly encapsulated objects,e.g. when implementing shared_ptr
• never use raw pointers to transfer ownership• never use raw pointers with business logic
exception: indication optional objects - use of NULL• Use a garbage collector• Problem:
No language enforcement -- trusts the programmer
© 2003 Mark Bartosik, www.leakbrowser.com
VB, avoiding the problem
• Stay within Visual Basic 5/6 environment (avoid C / OS APIs).
• Release objects (with events) in “unload” methods
set some_object = nothing
• Forms reference objects (with events) and the objects can reference the form. This is a cyclic reference.
• VB7 is implemented on .NET only and garbage collected.
© 2003 Mark Bartosik, www.leakbrowser.com
Java, .NET, avoiding the problem
• Stay within JVM or .NET VM (avoid C / OS APIs).You have no destructors, and finalizers are restrictive.
• Cyclic references -- no problem they are garbage collected.
• Don’t get lax, watch out for unbounded resource consumption.
© 2003 Mark Bartosik, www.leakbrowser.com
Garbage collection - the solution
Process / U se r memory
static sto rage (fixed size)
stackgrows ondem and
stack (one per thread)
stuff
managedheap
0x17E D 5423
0x17E D5423
heapgrows ondem and
17ED78AC
stuff
0x17E D78A C
stuff
garbage
garbage
• pause the entire program
• find every object that can be reached from the static storage or thread stacks
• mark those objects with an “age”
• delete all older objects, in any order you like
© 2003 Mark Bartosik, www.leakbrowser.com
Garbage collection - the solution
Process / U se r memory
static sto rage (fixed size)
stackgrows ondem and
stack (one per thread)
stuff
managedheap
0x17E D 5423
0x17E D5423
17ED78AC
stuff
0x17E D78A C
stuff
• pause the entire program
• find every object that can be reached from the static storage or thread stacks
• mark those objects with an “age”
• delete all older objects, in any order you like
© 2003 Mark Bartosik, www.leakbrowser.com
Garbage collection - the penalties• Pause the entire program!
When?For how long?
• Find EVERY reachable object?Page in every object from disk!Must run in physical memory to be efficient.Working set will be same as used memory!
• Delete all the garbage, in what order?Any order!So how do I test deterministically?So I cannot use finalizers?
• These may or may not be acceptable compromises• Remember it only solves half the problem!
© 2003 Mark Bartosik, www.leakbrowser.com
Workarounds• Watch dogs
Who watches the watch dog?What can the watch dog do? Bark!
• COM+Memory and lifetime limits can be configured for COM servers
• Heap limitsMaximum heap size can be configured
• WRITE GOOD CODE!• FIND THE LEAKS!
© 2003 Mark Bartosik, www.leakbrowser.com
Available tools
• Bounds checker• Purify• Heap Agent• Great Circle• Lint• Parasoft• C++ debug heap
© 2003 Mark Bartosik, www.leakbrowser.com
Typical restrictions
• Crashes itself on large scale programs• Crashes large scale programs• Only reports unreferenced memory not unbounded allocations• Only reports malloc, free etc• Does not report OS API leaks like SysAllocString• Only supports C++• Requires recompilation• Reports many false positives• Requires many hours to run• Grinds target program to a halt
© 2003 Mark Bartosik, www.leakbrowser.com
perfmon
© 2003 Mark Bartosik, www.leakbrowser.com
Identifying the problem
• Monitor resourcesperfmon “process private bytes”
“process handle count”“process thread count”
taskmgrVM Size, Handle count, Thread count, GDI Objects
• Exercise the program• Coverage test -- does it leak?• Use case tests -- when does it leak (most)?
© 2003 Mark Bartosik, www.leakbrowser.com
Quantifying the problem
reso
urce
usa
ge
time and/or number of use cases
custom eracceptab lerestart tim e
sim ilar to MT BF
m ax acceptab lereso urce usage
• Must stay out of the red
• Must consistently reach the green, without crashing
• The restart period might be... 1 hour 1 working day 1 working week 1 year 1000 transactions never!
• Max resources might be... 100 KB 2 GB
© 2003 Mark Bartosik, www.leakbrowser.com
Needle in a haystack• 3 processes• 250 component DLLs• 3500 source files• 1,500,000 lines of source code• 100 KB of leaks per transaction• exercising 15% of the code
great only 225,000 lines of source to check!• 10,000 allocations per transaction• caused by probably no more than half a dozen lines of code
I have hidden a few leaks in the 1,500,000 lines of D3000 code.We are going to find them NOW!
© 2003 Mark Bartosik, www.leakbrowser.com
Ignore one time initialization
reso
urce
usa
ge
time and/or number of use cases
custom eracceptab le
restart tim esim ilar to M T BF
m ax acceptab leresource usage
without one-timein itia lization leaks
problem isthe slope
with one-timein itia lization leaks
• Ignore the first few transactions
• They might cause one-time initialization leaks, but who cares?
© 2003 Mark Bartosik, www.leakbrowser.com
Cycle and observe everything
• Choose a small prime number
• Perform this number of transactions or cycles
• Track all allocations and deallocations
reso
urce
s
cycles / transactions
cycle
leaks percycle
cycles toignore
cycles tomonitor (7) cycles to
continue
tota
lle
aks
© 2003 Mark Bartosik, www.leakbrowser.com
Cycle again observing deallocationsstrtype a = b;
strtype & strtype::operator=(const strtype & rhs){ if (m_len < rhs.m_len) { char * temp = new char[rhs.m_len]; delete m_p; m_p = temp; } m_len = rhs.m_len; memmove(m_p, rhs.m_p, m_len); return *this;}
• Allocations made in the last cycle, may not be deleted until the next
• Perform additional cycles, only tracking deallocations
© 2003 Mark Bartosik, www.leakbrowser.com
Leak Browser learns
• Now generate a report of the leak• Leak Browser has just “learnt”, what you are interesting in• Next time Leak Browser will examine these few areas of code in
greater detail (requiring more CPU)
• Clear the leaks• Repeat 7 monitoring cycles• Repeat some additional cycles• Regenerate the report
© 2003 Mark Bartosik, www.leakbrowser.com
Not everything is a leak
• Background tasks may provide “background noise”for example, incoming news or price updates
• Caches may delay some deallocations• Queues and logs may include records of the last 1000
transactions
© 2003 Mark Bartosik, www.leakbrowser.com
Capabilities
• Resource profiling - finding the bloat• Fast over 10,000 allocations per second can be tracked• Tracks about 300 key APIs, not just malloc & free• Finds both unreferenced leaks and unbounded leaks• Does not require code recompilation• Is not intrusive with large applications• Remote installation and debugging• It works!
© 2003 Mark Bartosik, www.leakbrowser.com
How does it work?• Traps key APIs• API is redirected
into Leak Browser stub
• Stack trace is recorded with allocation details
• In-memory database is maintained
• If no PDB further instrumentation can be done
Core OS
C/C++ runtime
C/C++ runtime
W in32
Your code
© 2003 Mark Bartosik, www.leakbrowser.com
Challenges• Must be thread safe but...
– Cannot hold a thread lock and call the OS– Cannot allocate memory– Restrictions on using thread local storage
(it may not be allocated yet, or it may have been deallocated)
• Has only micro-seconds to trace the stack– Standard Microsoft algorithm takes seconds– Cannot afford to call the OS
• Must cope with optimized code– for stack tracing, and instrumentation– must modify hundreds or thousands of functions without side effects
© 2003 Mark Bartosik, www.leakbrowser.com
Limitations
• Need to be able to cycle the application• No true garbage detection (yet)• .NET interop only (so far)• No JVM support - none planned• Slow display -- too much HTML• Occasional crashes, but more reliable than the rest
© 2003 Mark Bartosik, www.leakbrowser.com
Summary
• Use garbage collection where appropriate• Use best practice for the programming language of choice• Monitor resource usage during development• Are you within your customer acceptable restart period?• If not
– Leak Browserwww.leakbrowser.com