© Asifuddin Mohammad Empirical Model for HTTP Network Traffic Asifuddin Mohammad Club presentation...

© Asifuddin Mohammad

Empirical Model for HTTP Network Traffic

Asifuddin MohammadClub presentation

27 February 2009 Abbreviated Paper Title 1



• Paper I– An Empirical Model of HTTP Network Traffic– Bruce A. Bah, INFOCOM ‘97

• Paper II– Empirical Models of TCP and UDP End–User Network

Traffic from NETI@home Data Analysis– IEEE 2006



An Empirical Model of HTTP Network Traffic

[Mah-INFOCOM-1997]

© 2009–Asifuddin Mohammad27 February 2009

Asifuddin Mohammad

Club presentation

[email protected]

http://www.people.ku.edu/~asifm


1 January 2000 Abbreviated Paper Title 4


Abstract• The workload of the global Internet is dominated by the

Hypertext Transfer Protocol (HTTP), an application protocol used by World Wide Web clients and servers. Simulation studies of this environment will require a model of the traffic patterns of the World Wide Web, in order to investigate the performance aspects of this increasingly popular application. We have developed an empirical model of network traffic produced by HTTP. Instead of relying on server or client logs, our approach is based on gathering packet traces of HTTP network conversations. Through traffic analysis, we have determined statistics and distributions for higher-level quantities such as the size of HTTP items retrieved, the number of items per “Web page”, think time, and user browsing behavior. These quantities form a model can then be used by simulations to mimic World Wide Web network applications in wide-area IP internetworks.



An Empirical Model of HTTP Network Traffic Outline

• Introduction• Background• Prior Work• Methodology• Model• Experimental Results• Conclusions• References

note: outline foils do not count toward your total foil budget




Introduction• Introduction• Background• Prior Work• Methodology• Model• Experimental Results• Conclusions• References




Introduction• Develop an empirical model for HTTP

– Accurate models of the system under study to yield useful data

– Provides a synthetic workload to simulation of wide area IP internetwork

– Based on network packet traces

• At Lowest Level– Describes the sizes of individual web files

• At highest level– Describes the browsing behavior of the user




Background• Introduction• Background• Prior Work• Methodology• Model• Experimental Results• Conclusions• References




Background• WWW

– Collection of documents– Each document consist of # of files– E.g. multipart document = text (in HTML) + images

• HTTP request-response protocol• HTTP uses TCP for reliable transfer

– Non-Persistent TCP – Persistent TCP was proposed but not implemented




Prior Work• Introduction• Background• Prior Work• Methodology• Model• Experimental Results• Conclusions• References




Prior Work

• Approaches taken to characterize internet application– Server logs– Client logs– Packet or Traffic traces

• Server Logs– Logs ranging from operational monitoring to collecting the

demographic info about the user– Easiest way– 2 dis-adv

• No easy capture of users access pattern across multiple servers

• No HTTP overhead capture




Prior Work

• Client Logs– No problem in capturing user accesses b/w multiple servers– Characterization of client side caching of documents– Need the browser code to log request size– Supporting variety of browsers may be difficult if modified

• Packet Trace– Analyzing the HTTP packet trace taken from a subnet– Used in number of traffic studies ; eliminates dis-adv– 2 dis-adv

• More effort to reconstruct the TCP connection contents• Effects of client caching of documents are difficult to

determine




Methodology• Introduction• Background• Prior Work• Methodology• Model• Experimental Results• Conclusions• References



An Empirical Model of HTTP Network Traffic Methodology

• Packet Trace approach– No higher level info like actual file accessed– TCPdump packet capture utility– More than dozen networks with 100’s of hosts– TCP port 80– Recorded packet loss of 0.014%



An Empirical Model of HTTP Network Traffic Model

• Introduction• Background• Prior Work• Methodology• Model• Experimental Results• Conclusions• References



An Empirical Model of HTTP Network Traffic Model

• Empirical distribution for different quantities




Experimental Results• Introduction• Background• Prior Work• Methodology• Model• Experimental Results• Conclusions• References




Experimental Results

• Few Results



Experimental Results• Page Length:




An Empirical Model of HTTP Network Traffic Experimental Results

• Server Selection– No proper results– Approximate the distribution to Zipf’s law

• Zipf’s Law– Probability of selecting the ith most popular item in a

set is proportional to 1/i




Conclusions• Introduction• Background• Prior Work• Methodology• Model• Experimental Results• Conclusions• References



An Empirical Model of HTTP Network Traffic Conclusions

• An Empirical Model is developed




References• Introduction• Background• Prior Work• Methodology• Model• Experimental Results• Conclusions• References



• Paper I– An Empirical Model of HTTP Network Traffic– Bruce A. Bah, INFOCOM ‘97

• Paper II– Empirical Models of TCP and UDP End–User Network

Traffic from NETI@home Data Analysis– Charles R. Simpson, Jr. Dheeraj Reddy, George F.

Riley– IEEE 2006




Empirical Models of TCP and UDP End–User Network Traffic from NETI@home

Data Analysis Abstract

• The simulation of computer networks requires accurate models of user behavior. To this end, we present empirical models of end–user network traffic derived from the analysis of NETI@home data. There are two forms of models presented. The first models traffic for a specific TCP or UDP port. The second models all TCP or UDP traffic for an end–user. These models are meant to be network–independent and contain aspects such as bytes sent, bytes received, and user think time. The empirical models derived in this study can then be used to enable more realistic simulations of computer networks.




Data Analysis Outline

• Introduction• Background & Related Work• Methodology• Experimental Results• Simulation Results• Future Work• Conclusions• References




Data Analysis Introduction





Data Analysis Introduction

• Simulation– Popular method to evaluate characteristics of

networks

• Need – For accurate Models for Simulator component

• E.g. End-User Traffic

– To update the Models frequent




Data Analysis Background & Related Work





Data Analysis Background & Related Work

• Expansion of Bah’s empirical distribution• Network Intelligence at Home (NETI@home)

– Software is used instead of packet trace– Reports end to end flow summary stats

• Bah’s study was conducted on specific port– TCP port 80

• Experiment conducted on any given TCP or UDP port




Data Analysis Methodology





Data Analysis Methodology

• 2 categories of models– TCP or UDP port– Aggregate of all port-specific models

• Dataset collected over one year period– Oct 1,2004 to Sep 30,2005– 36 million TCP flows and 93 million UDP flows

• Aspects– Bytes send and bytes received– User think time– Consecutive contacts– Contact selection




Data Analysis Experimental Results






• Bytes sent





• Bytes Received





• User Think Time to the same IPs





• User Think Time to differing IPs





• Consecutive Contacts





• Contact selection




Data Analysis Simulation Results





Data Analysis Simulation Results

• Implemented in GTNets – Data collected for NETI as well as Bah’s empirical

Models– Network topology used is shown

© Asifuddin MohammadEmpirical Models of TCP and UDP End–User

Network Traffic from NETI@home Data Analysis

Simulation Results

INET@home Bah’s Model





Data Analysis Future Work





Data Analysis Future Work

• Useful to Model Idle time• Determine Correlation between different aspects of

this Model– E.g. correction between bytes sent and byte received

• Enhancements to consecutive contacts and contact selection using a memory based model– Markov model

• Extend to other protocols beyond TCP or UDP • Model network-dependent characteristics of

internet• Develop a analytical models from empirical




Data Analysis Conclusions





Data Analysis Conclusion

• This paper developed an empirical model for the number of bytes sent, number of bytes received, the user think time to the same destination, the user think time to a different destination, the number of times a destination will be contacted consecutively, and the popularity of specific destinations.




References• [1] D. P. Anderson and et al. SETI@home: Search for extraterrestrial intelligence at home. Software on-line:

http://setiathome.ssl.berkeley.edu, 2003.• [2] C. Barakat, P. Thiran, G. Iannaccone, C. Diot, and P. Owezarski. Modeling internet backbone traffic at the flow level.

IEEE Transactions on Signal Processing – Special Issue on Networking, 51(8), August 2003.• [3] P. Barford and M. Crovella. Generating representative web workloads for network and server performance

evaluation. In ACM SIGMETRICS, 1998.• [4] J. Cao, W. S. Cleveland, Y. Gao, K. Jeffay, F. D. Smith, and M. C. Weigle. Stochastic models for generating synthetic

HTTP source traffic. In IEEE INFOCOMM, March 2004.• [5] Y.-C. Cheng, U. Holzle, N. Cardwell, S. Savage, and G. M. Voelker. Monkey see, monkey do: A tool for TCP tracing

and replaying. In Proceedings of USENIX Technical Conference, June 2004.• [6] H.-K. Choi and J. O. Limb. A behavioral model of web traffic. In ICNP, 1999.• [7] M. Christiansen, K. Jeffay, D. Ott, and F. D. Smith. Tuning RED for web traffic. IEEE/ACM Transactions on Networking,

9(3):249–264, June 2001.• [8] S. Floyd and V. Paxson. Difficulties in simulating the internet. IEEE/ACMTransactions on Networking, 9(4):392–403,

August 2001.• [9] J. B. Grizzard, C. R. Simpson, Jr., S. Krasser, H. L. Owen, and G. F. Riley. Flow based observations from NETI@home

and honeynet data. In Proceedings from the sixth IEEE Systems, Man and Cybernetics Information Assurance Workshop, pages 244–251, June 2005.

• [10] F. Hernandez-Campos, A. B. Nobel, F. D. Smith, and K. Jeffay. Understanding patterns of TCP connection usage with statistical clustering. In IEEE MASCOTS, 2005.

• [11] F. Hernandez-Campos, F. D. Smith, and K. Jeffay. Generating realistic TCP workloads. In Computer Measurement Group International Conference, December 2004.

• [12] L. Le, J. Aikat, K. Jeffay, and F. D. Smith. The effects of active queue management on web performance. In ACM• SIGCOMM, pages 265–276, August 2003.• [13] B. A. Mah. An empirical model of HTTP network traffic. In IEEE INFOCOMM, April 1997.




References• [14] G. F. Riley. The Georgia Tech Network Simulator. In Proceedings of the ACM SIGCOMM workshop on Models,

methods and tools for reproducible network research, pages 5–12, 2003.• [15] C. R. Simpson, Jr. NETI@home. Software on-line: http://neti.gatech.edu, 2003. Georgia Institute of

Technology.• [16] C. R. Simpson, Jr. and G. F. Riley. NETI@home: A distributed approach to collecting end-to-end network

performance measurements. In PAM2004 - A workshop on Passive and Active Measurements, April 2004.• [17] F. D. Smith, F. Hernandez-Campos, K. Jeffay, and D. Ott. What TCP/IP protocol headers can tell us about the

web. In ACM SIGMETRICS, pages 245–256, 2001.• [18] J. Sommers, H. Kim, and P. Barford. Harpoon: A flow–level traffic generator for router and network tests. In

ACM SIGMETRICS, June 2004.• [19] M. Weigle, K. Jeffay, and F. D. Smith. Delay–based early congestion detection and adaptation in TCP: Impact

on web performance. ACM Computer Communications Review, 28(8):837–850, May 2005.• [20] J. Xu andW. Lee. Sustaining availability of web services under distributed denial of service attacks. IEEE

Transactions on Computers, 52(2):195–208, February 2003.

© Asifuddin Mohammad Empirical Model for HTTP Network Traffic Asifuddin Mohammad Club presentation...

Documents

Transcript of © Asifuddin Mohammad Empirical Model for HTTP Network Traffic Asifuddin Mohammad Club presentation...