Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher...
-
Upload
melinda-megan-andrews -
Category
Documents
-
view
221 -
download
5
Transcript of Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher...
![Page 1: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/1.jpg)
Brett D. Higgins^, Kyungmin Lee*, Jason Flinn*, T.J. Giuli+, Brian Noble*, and Christopher Peplin+
Arbor Networks^ University of Michigan* Ford Motor Company+
The future is cloudy: Reflecting prediction error in mobile
applications
![Page 2: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/2.jpg)
Mobile applications are adaptive
2Kyungmin Lee
![Page 3: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/3.jpg)
How do applications adapt?
3Kyungmin Lee
Make predictions
Choose optimal strategy
Execute it!
![Page 4: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/4.jpg)
How do applications adapt?
4Kyungmin Lee
Make predictions
Choose optimal strategy
Execute it!
![Page 5: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/5.jpg)
How do applications adapt?
5Kyungmin Lee
Make predictions
Choose optimal strategy
Execute it!
![Page 6: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/6.jpg)
How do applications adapt?
6Kyungmin Lee
Make predictions
Choose optimal strategy
Execute it!
CloneCloud ’11MAUI ’10Chroma ’07Spectra ‘02
![Page 7: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/7.jpg)
How do applications adapt?
7Kyungmin Lee
Make predictions
Choose optimal strategy
Execute it!
What can possibly go wrong?
![Page 8: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/8.jpg)
Predictions are not perfect
8Kyungmin Lee
Make predictions
Choose optimal strategy
Execute it!
Need to consider predictor errors!
![Page 9: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/9.jpg)
Need to consider redundancy
9Kyungmin Lee
Make predictions
Choose optimal strategy
Execute it!
![Page 10: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/10.jpg)
Re-evaluate the environment
10Kyungmin Lee
Make predictions
Choose optimal strategy
Execute it!
Needs to constantly re-evaluate the
environment
![Page 11: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/11.jpg)
Embracing uncertainty
• Our library chooses the best strategy– Incorporates prediction errors– Single strategy or redundant– Balances cost & benefit of redundancy
• Benefit (time saved)• Cost (energy + cellular data)
– Re-evaluates the environment
11Kyungmin Lee
![Page 12: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/12.jpg)
Outline
• Motivation• Uncertainty-aware decision-making methods
– Library overview– Our three methods– Re-evaluation from new information
• Evaluation• Conclusion
12Kyungmin Lee
![Page 13: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/13.jpg)
Library overview
13Kyungmin Lee
Application provides
Our libraryprovides
Strategies
Predictors Predictors
Errordistribution
Environment reevaluation
Decision mechanism
![Page 14: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/14.jpg)
14
90%
10%
Remote response time
1 sec100 sec
100%
Local response time
20 sec
Remote vs. Local
Kyungmin Lee
Localexpected time: 20 sec
Remoteexpected time: 10.9 sec
Uncertain server load
![Page 15: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/15.jpg)
15
90%
10%
Remote response time
1 sec100 sec
Remote vs. Local
Kyungmin Lee
Remoteexpected time: 10.9 sec
Uncertain server load
100%
Local response time
20 sec
Localexpected time: 20 sec
![Page 16: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/16.jpg)
16
90%
10%
Remote response time
1 sec100 sec
Let’s consider redundancy
Kyungmin Lee
Redundancyexpected time: 2.9 sec
Remoteexpected time: 10.9 sec
Uncertain server load
100%
Local response time
20 sec
![Page 17: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/17.jpg)
Incorporating prediction errors
17Kyungmin Lee
• Use redundancy?– When predictions are too uncertain– Benefit (time) > Cost (energy + cellular data)
• Our library provides three methods– Brute force, error bounds, Bayesian estimation– Hides complexity from the application
![Page 18: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/18.jpg)
Brute force
18Kyungmin Lee
• Compute error upon new measurement• Weighted sum over joint error distribution
– For redundant strategies:• Time: min across all strategies• Cost: sum across all strategies
• Simple, but computationally expensive
![Page 19: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/19.jpg)
Error bounds
• Obtain bound for new measurement• Calculate bound on net gain of redundancy
max(benefit) – min(cost) = max(net gain)
9876543210
BP1 BP2
Band
wid
th (M
bps)
Network bandwidth
9876543210
T1 T2
Tim
e (s
econ
ds)
Time to send 10Mb
Max time savingsfrom redundancy
19Kyungmin Lee
![Page 20: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/20.jpg)
Bayesian estimation
• Basic idea:– Given a prior belief about the world,– and some new evidence,– update our beliefs to account for the evidence,
• AKA obtaining posterior distribution
– using the likelihood of the evidence
• Via Bayes’ Theorem: posterior = likelihood * prior p(evidence) Normalization factor;
ensures posterior sums to 1
20Kyungmin Lee
![Page 21: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/21.jpg)
Bayesian estimation
• Applied to decision making:– Prior: completion time measurements– Evidence: complet. time prediction + implied decision– Posterior: new belief about completion time– Likelihood:
• When local wins, how often has the prediction agreed?• When remote wins, how often has the prediction agreed?
• Via Bayes’ Theorem: posterior = likelihood * prior p(evidence)
21Kyungmin Lee
Normalization factor;ensures posterior sums to 1
![Page 22: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/22.jpg)
22
90%
10%
Remote response time
1 sec100 sec
Reevaluation: conditional distributions
Kyungmin Lee
Expected time: 20sec Expected time: 10.9sec
Decision
Elapsed Time
Remote
0 11s 31s …. 100s
Uncertain server load
100%
Local response time
20 sec
Remote Remote & local
![Page 23: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/23.jpg)
Outline
• Motivation• Uncertainty-aware decision-making methods
– Library overview– Our three methods– Re-evaluation from new information
• Evaluation• Conclusion
23Kyungmin Lee
![Page 24: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/24.jpg)
24
Evaluation: methodology
• Network trace replay (walking & driving)– Speech recognition, network selection app
• Metric: weighted cost function– time + cenergy * energy + cdata * data
No-cost
Low-cost
Mid-cost
High-cost
cenergy 0 0.00001 0.0001 0.001
Battery life reduction under average use (normally 20 hours)
N/A 6 min 36 sec 3.6 sec
Kyungmin Lee
![Page 25: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/25.jpg)
25
No cost High cost0
0.2
0.4
0.6
0.8
1
1.2
Local-onlyRemote-preferredAdaptiveOur library
Speech recognition, server loadW
eigh
ted
cost
(nor
m.)
Kyungmin Lee
Our library matches the best strategy
23%
Redundancy is less beneficial as cost increases
![Page 26: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/26.jpg)
26
No cost High cost0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
Cellular-onlyRemote-preferredAdaptiveOur library
Network selection, walking traceW
eigh
ted
cost
(nor
m.)
Kyungmin Lee
Our library matches the best strategy
24%
2x
![Page 27: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/27.jpg)
27
Discussion
• Our library provides the best strategy• Which method is the best?
– Brute force: Accurate, but expensive– Error bounds: Leans toward redundancy– Bayesian: Mixed bag
• No clear winner
Kyungmin Lee
![Page 28: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/28.jpg)
28
Conclusion
• Need to consider uncertainty in predictions• Redundancy is powerful!• Our library helps apps to choose best strategy• Source code at
– https://github.com/brettdh/instruments– https://github.com/brettdh/libcmm
Kyungmin Lee
![Page 29: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/29.jpg)
29
Questions?
Kyungmin Lee
![Page 30: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/30.jpg)
30Kyungmin Lee
![Page 31: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/31.jpg)
31
Speech recognition, server loadW
eigh
ted
cost
(nor
m.)
Kyungmin Lee
No cost High cost0
0.2
0.4
0.6
0.8
1
1.2
Brute forceError boundsBayesian
Error boundsleans towardsredundancy
![Page 32: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/32.jpg)
32
Network selection, walking traceW
eigh
ted
cost
(nor
m.)
No cost Low cost Mid cost High cost0
0.20.40.60.8
11.21.41.61.8
2
2x
24%
Low-resource strategies improve
Meatballs matches the best strategy
Error boundsleans towardsredundancy
Simple Our library
Kyungmin Lee
![Page 33: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/33.jpg)
33
Speech recognition, server load
No cost Low cost Mid cost High cost0
0.2
0.4
0.6
0.8
1
1.2
1.4
23%
Meatballs matches the best strategy
Simple
Error boundsleans towardsredundancy
Wei
ghte
d co
st (n
orm
.)
Kyungmin Lee
Our library
![Page 34: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/34.jpg)
34
Network selection, driving trace
No cost Low cost Mid cost High cost0
0.5
1
1.5
2
Not much benefitfrom using WiFi
Simple
Wei
ghte
d co
st (n
orm
.)
Kyungmin Lee
Our library
![Page 35: Brett D. Higgins ^, Kyungmin Lee *, Jason Flinn *, T.J. Giuli +, Brian Noble *, and Christopher Peplin + Arbor Networks ^ University of Michigan * Ford.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2aa0c/html5/thumbnails/35.jpg)
35
Speech recognition, walking trace
No cost Low cost Mid cost High cost0
0.2
0.4
0.6
0.8
1
1.2
1.4
Benefit of redundancy persists more
23-35%
>2x
Meatballs matches the best strategy
Simple
Wei
ghte
d co
st (n
orm
.)
Kyungmin Lee
Our library