Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou,...
-
Upload
clifton-porter -
Category
Documents
-
view
216 -
download
0
Transcript of Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou,...
![Page 1: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/1.jpg)
Language-Based Replay via
Data Flow CutMicrosoft Research Asia
Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang
MITFan Long, Xi Wang, Zhilei Xu
![Page 2: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/2.jpg)
OutlineMotivationObservationChallengesModeling Replay InterfaceGenerating Replay InterfaceRecord and ReplayEvaluationConclusion
![Page 3: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/3.jpg)
MotivationReplay is important due to non-determinism
Caused by time, user input, network I/O, thread interleaving
Makes postmortem debug hardExisting replay tools
Incurs significant overhead: interposition & logging
Hard to be adopted, especially for deployed system
How to mitigate recording overhead?Using efficient way to find the necessary
information to log
![Page 4: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/4.jpg)
ObservationReplay interface between program and
environmentOnly part of the program needs to be
replayedneon’sroutine
status->code
req->respbuf
recv
*respbuf(1MB)
atoi
return value
struct ne_request { ne_status status; char respbuf[];};int read_status_line( ne_request *req, ne_status *status, ...) { ne_sock_readline(…, req->respbuf, …); if (...) status->code = atoi(buffer + 4); else if (ne_parse_statusline(buffer, status)) {...}}
4B1MB
![Page 5: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/5.jpg)
Challenges
Finding a complete replay interfaceFinding a replay interface incurring low
recording overhead
![Page 6: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/6.jpg)
OutlineMotivationObservationChallengesModeling Replay InterfaceGenerating Replay InterfaceRecord and ReplayEvaluationConclusion
![Page 7: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/7.jpg)
Execution Flow Graph(EFG)f() { cnt = 0; g(&cnt); printf("%d\n", cnt); g(&cnt); printf("%d\n", cnt);}g(int *p) { a = random(); *p += a;}
// execution1 cnt1 <- 02 a1 <- random()3 cnt2 <- cnt1 + a1
4 print cnt2
5 a2 <- random()6 cnt3 <- cnt2 + a2
7 print cnt3
Inst1
Inst2 Inst
3
Inst5
cnt1
a1
cnt2
a2
Inst6cnt3
Inst7
Inst4
Cut 1
Cut 2
replay target
non-deterministic
deterministic
![Page 8: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/8.jpg)
OutlineMotivationObservationChallengesModeling Replay InterfaceGenerating Replay InterfaceRecord and ReplayEvaluationConclusion
![Page 9: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/9.jpg)
Static Flow GraphReplay interface on EFG only optimal for specific runSound approximation of execution flow graph
Scan whole programOperation node for function, value node for variable
Interpret instruction as read/writey = x + 1 read x and write y
Alias analysis
fcnt
a
g
![Page 10: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/10.jpg)
Functions without Source CodeConservatively consider them as non-
deterministic by defaultrecv(fd, buf, len, flags)
Annotate functions to provide write edgerecv([in]fd, [out, bsize(return)] buf, [in]len,
[in]flags)Annotate functions as deterministic
Math functions: abs(), sqrt()Memory and string: memcpy(), strcat()
![Page 11: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/11.jpg)
OutlineMotivationObservationChallengesModeling Replay InterfaceGenerating Replay InterfaceRecord and ReplayEvaluationConclusion
![Page 12: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/12.jpg)
Replay RuntimeCalls
Record call from function in non-replay space to function which may be in replay space
Replay callee if it does belong to replay spaceWrites
Where to issue writesWhen to issue writes
g
x
Replayed
Non-replayed
f
h
iDowncall Upcall
![Page 13: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/13.jpg)
Other Subtle Non-determinismsMemory Management
Address of variables in replay space should not change
Separated deterministic memory pool for replay space
Separate stacks for replay and non-replay functions
Thread InterleavingSynchronization logDeterministic multi-threading
![Page 14: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/14.jpg)
EvaluationImplemented iTarget for C program on
WindowsUsing Phoenix compiler framework for
instrumentationBenchmarks:
Apache HTTP Server, Berkeley DB, neon HTTP client, wget, SPEC CINT2000
Modular and monolithic programsCompared to R2 (OSDI 2008)
![Page 15: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/15.jpg)
Apache HTTP serverless than 1% slowdown
![Page 16: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/16.jpg)
ConclusionA model
Reduce the problem of finding an optimal replay interface to that of finding the minimum cut in a data flow graph
A system: iTargetemploy programming language techniques to
achieve both correctness and low recording overhead
![Page 17: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/17.jpg)
Thanks! Q&A
![Page 18: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/18.jpg)
ProfilingResults tend not to be sensitive to the
profiling workload scale
![Page 19: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/19.jpg)
Related WorkLibrary-based replay tools:
RecPlay (TOCS 1999)Flashback (USENIX ATC 2004)Liblog (USENIX ATC 2006)R2 (OSDI 2008)
Instruction level replayiDNA (VEE 2006)
Other language runtimeJava, ML, MPI
![Page 20: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/20.jpg)
Memory ManagementAddress of variables in replay space should
not changeVariables allocated in heap
Non-replay space function may allocate memory
Separate deterministic memory pool for replay space
Variables allocated on stackRecord ESP at a call from non-replay to replay spaceReset the ESP during replay
f
g
hi
f
Run-time
i
Replaystack
Recording stack
ESP
Recorded ESP
lower address
higher address
![Page 21: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/21.jpg)
Memory ManagementAddress of variables in replay space should
not changeVariables allocated in heap
Non-replay space function may allocate memory
Separate deterministic memory pool for replay space
Variables allocated on stackRecord ESP at a call from non-replay to replay spaceReset the ESP during replay
f
g
hi
f
Run-time
i
Replaystack
Recording stack
ESP
Recorded ESP
Current ESP
lower address
higher address
![Page 22: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/22.jpg)
Monolithic ProgramNeon and Wget
![Page 23: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/23.jpg)
Functions without Source CodeConservatively consider them as non-
deterministic by defaultrecv(fd, buf, len, flags)System global variable: errno
Annotate functions to provide write edgerecv([in]fd, [out, bsize(return)] buf, [in]len,
[in]flags)Annotate functions as deterministic
Math functions: abs(), sqrt()Memory and string: memcpy(), strcat()
![Page 24: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/24.jpg)
Memory ManagementAddress of variables in replay space should
not changeVariables allocated in heap
Non-replay space function may allocate memory
Separate deterministic memory pool for replay space
Variables allocated on stackRecord ESP at a call from non-replay to replay spaceReset the ESP during replay
![Page 25: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/25.jpg)
Thread ManagementThread interleaving introduce another source
of non-determinismGuarantee same write order during replay as
recording runSynchronization log
Track causal dependencyUtilize deterministic multi-threading model
![Page 26: Microsoft Research Asia Ming Wu, Haoxiang Lin, Xuezheng Liu, Zhenyu Guo, Huayang Guo, Lidong Zhou, Zheng Zhang MIT Fan Long, Xi Wang, Zhilei Xu.](https://reader030.fdocuments.net/reader030/viewer/2022013004/56649e4d5503460f94b439df/html5/thumbnails/26.jpg)
Execution Flow Graph(EFG)f() { cnt = 0; g(&cnt); printf("%d\n", cnt); g(&cnt); printf("%d\n", cnt);}g(int *p) { a = random(); *p += a;}
// execution1 cnt1 <- 02 a1 <- random()3 cnt2 <- cnt1 + a1
4 print cnt2
5 a2 <- random()6 cnt3 <- cnt2 + a2
7 print cnt3
f
cnt1
a1
cnt2
a2
cnt3
g1
g2
Cut 2