Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu,...
-
Upload
morris-park -
Category
Documents
-
view
217 -
download
0
Transcript of Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu,...
![Page 1: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/1.jpg)
Conditional Correlation Analysis forSafe Region-based Memory Management
Xi Wang, Zhilei Xu, Xuezheng Liu,Zhenyu Guo, Xiaoge Wang, Zheng Zhang
Microsoft Research Asia, Tsinghua University
PLDI, June 9th 2008, Tucson
![Page 2: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/2.jpg)
Region-based Memory Management
• Memory Region (a.k.a. Pool) is widely used– Apache web server– SVN (subversion) version control system– RC compiler– …
![Page 3: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/3.jpg)
Ownership relation
Region Usage
r = region_create();
a = region_alloc(r);b = region_alloc(r);c = region_alloc(r);
region_destroy(r);
• Allocate objects in regions• A region Owns objects• Destroy a region, delete all objects it owns
r
a b c
![Page 4: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/4.jpg)
Subregion
y = region_create(x);z = region_create(x);w = region_create(y);
region_destroy(x);
• One region can be Subregion of its parent• Detroy a parent, destroy all its subregions
– Parent lives longer than subx
y z
w
Subregion relation
Subregion relation
![Page 5: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/5.jpg)
Dangling pointer between regions
foo(parent){sub = region_create(parent);table = region_alloc(sub);iterater = make(parent, table);region_destroy(sub);}
• Object can Access object in another region• When pointee’s region get destroyed earlier,
pointer become danglingmake(r, table){iterator = region_alloc(r);iterator.f = table;}
Accessrelation
parent iterator
tablesub
Danglingpointer
![Page 6: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/6.jpg)
Harmful & Really exists (in svn…)
foo(parent){sub = region_create(parent);table = region_alloc(sub);iterater = make(parent, table);region_destroy(sub);}
• Further Deref. -- crash• No Deref. -- Pointer lives longer than necessary
& cause memory wastemake(r, table){iterator = region_alloc(r);iterator.f = table;}
parent iterator
tablesub
Danglingpointer
![Page 7: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/7.jpg)
The problem is not easy
• Correlated Relations– Ownership : Object - Region– Subregion : Region - Region– Access : Object - Object
• Existing solution to safe region usage– reference counting– …
![Page 8: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/8.jpg)
Solution: RegionWiz
• To verifyObject P may access object Q, P’s owner region must be descendant of Q’s owner region.(consistent conditional correlation)
• Static analysis to infer the ownership, subregion & access relations
• Verify correlation, find dangling pointers
• Context-sensitive, heap cloning
![Page 9: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/9.jpg)
Highlights
• Conditional correlation analysis framework for safe region usage
• Context-sensitive analysis with heap cloning
• Found bugs in real-world applications written in C (100+KLOC)– 12 dangling pointers in 6 software packages– 13 false alarms
• Case study & experience
![Page 10: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/10.jpg)
FrameworkSource Code
Context-sensitive
program info
Correlation Analysis(datalog)
Compiler plug-in (Phoenix in VC)
program infointo DB tables
ContextClone
Relation Computation
(datalog)
OwnershipSubregion
Access relations
Dangling pointer Report
Post processing
![Page 11: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/11.jpg)
Extract Program InformationSource Code
Context-sensitive
program info
Correlation Analysis(datalog)
Compiler plug-in (Phoenix in VC)
program infointo DB tables
ContextClone
Relation Computation
(datalog)
OwnershipSubregion
Access relations
Dangling pointer Report
Post processing
![Page 12: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/12.jpg)
Extract Program Information
• Extract program information into DB tables• Following analysis can be datalog inference
• Example
– Call(foo , make)
– Call(make , region_alloc)make( r , table ){ region_alloc(…)}
foo( parent ){ make(…)}
![Page 13: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/13.jpg)
Context CloningSource Code
Context-sensitive
program info
Correlation Analysis(datalog)
Compiler plug-in (Phoenix in VC)
program infointo DB tables
ContextClone
Relation Computation
(datalog)
OwnershipSubregion
Access relations
Dangling pointer Report
Post processing
![Page 14: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/14.jpg)
Context Cloning
make
foo
ralloc
foo(parent){sub = region_create(parent);table = region_alloc(sub);iterater = make(parent, table);region_destroy(sub);}
make(r, table){iterator = region_alloc(r);iterator.f = table;}
bar(parent){table = region_alloc(parent);iterater = make(parent, table);}
bar
![Page 15: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/15.jpg)
Context Cloning
make
foo(1)
ralloc
foo(parent){sub = region_create(parent);table = region_alloc(sub);iterater = make(parent, table);region_destroy(sub);}
make(r, table){iterator = region_alloc(r);iterator.f = table;}
bar(parent){table = region_alloc(parent);iterater = make(parent, table);}
bar(1)
makemake(1)
make(2)
rallocrallocrallocralloc(1)
ralloc(2)
ralloc(3)
ralloc(4)
Heap Cloning(Specialization)
![Page 16: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/16.jpg)
Context Cloningfoo(1)
foo(parent){sub = region_create(parent);table = region_alloc(sub);iterater = make(parent, table);region_destroy(sub);}
make(r, table){iterator = region_alloc(r);iterator.f = table;}
bar(parent){table = region_alloc(parent);iterater = make(parent, table);}
bar(1)
make(1)
make(2)
ralloc(1)
ralloc(2)
ralloc(3)
ralloc(4)
iterator(1)
table(1)
iterator(2)
table(2)
![Page 17: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/17.jpg)
Program Information -- Cloned
• From now on program information tables are decorated with context information
• Example
– Call(bar - 1, make - 2)
– Call(bar - 1 , region_alloc - 4)
– Call(make - 2 , region_alloc - 3)
bar(1)
make(2)
ralloc(3) ralloc
(4)
![Page 18: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/18.jpg)
Relation ComputationSource Code
Context-sensitive
program info
Correlation Analysis(datalog)
Compiler plug-in (Phoenix in VC)
program infointo DB tables
ContextClone
Relation Computation
(datalog)
OwnershipSubregion
Access relations
Dangling pointer Report
Post processing
![Page 19: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/19.jpg)
Relation computation using datalog• Datalog rules
– How new relations can be computed from existing relations
• Basic Rules for regionsNewRegion( context ) :- Call( _ , “region_alloc” – context)NewObject( context ) :- Call( _ , “region_alloc” – context)
• Context- & Field- sensitive Points-to analysis in ~20 rules
![Page 20: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/20.jpg)
Compute the needed relations• Access(P , Q) :-
Object P points-to Q through whatever field.
• Own(Rgn , Obj) :-region_alloc(r) called, return value assigned to v,r points-to Rgn, v points-to Obj.
• Subregion(Sub , Parent) :-region_create(u) called, return value assigned to v,v points-to Sub, u points-to Parent.
Descendant(Des , Ans) :- Des=Ans , NewRegion(Des), NewRegion(Ans).Descendant(Des , Ans) :- Descendant(Des , r), Subregion(r, Ans).
![Page 21: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/21.jpg)
FrameworkSource Code
Context-sensitive
program info
Correlation Analysis(datalog)
Compiler plug-in (Phoenix in VC)
program infointo DB tables
ContextClone
Relation Computation
(datalog)
OwnershipSubregion
Access relations
Dangling pointer Report
Post processing
![Page 22: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/22.jpg)
Detect unsafe usage
• Rule for detecting dangling pointer:P access Q; region1 own P; region2 own Q;not ( region1 is a descendant of region2 ).
Warning(P , Q)
• Report Warning(P , Q) when there is some, or report that the correlation is consistent
• Heuristics to filter out very unlikely warnings, reduce the total warnings
![Page 23: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/23.jpg)
Framework Source Code
Context-sensitive
program info
Correlation Analysis(datalog)
Compiler plug-in (Phoenix in VC)
program infointo DB tables
ContextClone
Relation Computation
(datalog)
OwnershipSubregion
Access relations
Dangling pointer Report
Post processing
![Page 24: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/24.jpg)
Experiment
• Conducted on Intel Xeon 2.0GHz / 32G RAM• Latest stable version applications tested
– RCC RC Compiler– Apache 2.2.6 HTTP web server &
utilities– freeswitch 1.0b1 Telephony platform shell– jxta-c 2.5.2 P2P framework shell– lklftpd FTP server– SVN (subversion) 1.4.5Version control system
![Page 25: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/25.jpg)
Bugs foundKLOC #exe #Warnings #Dangling Pointers
RCC 37 1 1 1
Apache 42 9 1 0
freeswitch 109 1 0 0
jxta-c 114 1 0 0
lklftpd 5 1 2 2
svn 240 9 21 9
Total N/A N/A 25 12use APR
~ 200KLOCnot count in
![Page 26: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/26.jpg)
Time consumption & relation sizestime #subregion #ownership #access
RCC 19m21s 9 1,577 746,940
Apache httpd 34m04s 18 2,341 2,273
freeswitch 14m55s 46 3,065 2,499
jxta-c 58m24s 16 27 10
lklftpd 2m34s 6 622 565
svn svn 25h59m53s 542,402 897,921,834 280,671,987<1hacceptable
Unacceptable!
They blow up because ofContext cloning!
• Current context: full call-path• Future work: better context definition
![Page 27: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/27.jpg)
Case study – 1 (svn)• Region structure should be
consistent with program logic– Iterator vs. Hash table– Request vs. Connection
• RegionWiz can effectively find this kind of bugs
parent iterator
tablesub
Danglingpointer
![Page 28: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/28.jpg)
Case study – 2 (rcc)
• r1 and r2 are totally independent
• Destroying r2 earlier will cause dangling pointer
r1
r2
config
name
![Page 29: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/29.jpg)
Case study – 2 (rcc)
• r1 and r2 are totally independent
• Destroying r2 earlier will cause dangling pointer
• To use immutable string, it’s better to make a private copy in pointer’s own region
r1
r2
config
name
copy of name
![Page 30: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/30.jpg)
Case study – 3 (svn)
• Temporary unsafe usage• Often involves branches• Path-sensitivity
• Dangerous as code evolves
• Re-organize code to avoid even temporary unsafe usage
svn_do_open(……){lock = region_alloc(parent);if (Conditon) { hash = region_alloc(sub); lock.f = hash;}……if (Condition) { lock.f = NULL ;}region_destroy(sub);}
![Page 31: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/31.jpg)
Related work
• Language support for regions– Reap [OOPSLA '2002], Cyclone [PLDI '2002], RC
[PLDI '2001], Ownership types[PLDI '2003]• Correlation Analysis
– Locksmith [PLDI '2006], Chord [PLDI '2006, POPL '2007]
• Context-sensitive Analysis– bddbddb [PLDI '2004, PODS '2005]
![Page 32: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/32.jpg)
Conclusion
• Use memory regions safely is not trivial
• RegionWiz can detect dangling pointers between regions through static conditional correlation analysis
• RegionWiz is efficient to find real bugs in real applications & improve safety of region-based memory management
• We believe the correlation analysis framework can solve other problems
![Page 33: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/33.jpg)
Thank you!
Q/A
![Page 34: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/34.jpg)
Heuristics
• For Warning(Pointer, Pointee)• Pointer’s type mismatch with pointee’s type –
less possible• Pointer & pointee never allocated from the
same region under some context – more possible
• Examined 205 lower-ranked warnings, all but one are really false alarms
![Page 35: Conditional Correlation Analysis for Safe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f215503460f94c39937/html5/thumbnails/35.jpg)
Limitations
• Function pointer – standard inter-procedural propagation of function pointer values, but only propagate through variables & parameters, not through heap objects
• Pointer arithmetic (variable as array index) not supported, just ignored
• Limited thread support, no support for Asynchronous event, Callbacks, etc.
• Heuristics bring unsoundness