Multiscale Traffic Processing Techniques for Network Inference and Control

Multiscale Traffic Processing Techniques for Network Inference and Control

Richard Baraniuk Edward Knightly Robert Nowak Rolf Riedi

Rice University INCITE ProjectSeptember 2000

Rice University | INCITE Project | September 2000

INCITEInterNet Control and Inference Tools at the Edge

• Overall Objective:

Scalable, edge-based tools for on-line network analysis, modeling, and measurement

• Theme for DARPA NMS Research:

Multiscale traffic analysis, modeling, and processing via multifractals

• Expertise:

Statistical signal processing, mathematics, network QoS


Technical Challenges

Poor understanding of origins of complex network dynamics

Lack of adequate modeling techniques for network dynamics

Internal network inaccessible

Need: Manageable, reduced-complexity models with characterizable accuracy


Innovative Tools - 1

• Multifractals: a natural “language” and toolset for traffic with

– bursts and high variability on multiple time scales

– short-range dependencies (SRD)

– long-range dependencies (LRD)

packetscheduling

sessionlifetime

networkmanagement

round-triptime

< 1 msec 10s msec minutes hours

_________ _________


Innovative Tools - 2

• Reduced-complexity statistical traffic models based on multifractal trees and cascades

– realistic capture multiscale variability, SRD+LRD, non-Gaussianity

– analytically tractable eg: queuing analysis

– linear complexity algorithms O(N)

• Statistical inference tools for model fitting, end-to-end path modeling


Multiscale Traffic ModelingTime

Scale

Multiplicative innovations


Multifractal Wavelet Model (MWM)

• Random multiplicativeinnovations Aj,k on [0,1]

eg: beta

• Parsimonious modeling(one parameter per scale)

• Strong ties with rich theory of multifractals


Multiscale Traffic Trace Matching

4ms

16ms

64ms

Auckland 2000 MWM matchscale


Marginal Matching

4ms

16ms

64ms

scale Auckland 2000 MWM Gaussian


Multiscale Queuing


End-to-End Path Modeling

• Abstract network dynamics into a single bottleneck queue driven by “effective cross-traffic”

• Goal: Estimate volume of cross-traffic


Probing

• Ideally:

delay spread of packet pair spaced by T sec

correlates with

cross-traffic volume at time-scale T


Probing Uncertainty Principle

• Should not allow queue to empty between probe packets

• Small T for accurate measurements– but probe traffic would disturb

cross-traffic (and overflow bottleneck buffer!)

• Larger T leads to measurement uncertainties– queue could empty between probes

• To the rescue: model-based inference


Multifractal Cross-Traffic Inference

• Model bursty cross-traffic using MWM


Efficient Probing: Packet Chirps

• MWM tree inspires geometric chirp probe• MLE estimates of cross-traffic at multiple scales


Chirp Probe Cross-Traffic Inference


ns-2 Simulation

• Inference improves with increased utilization

Low utilization (39%) High utilization (65%)


ns-2 Simulation (Adaptivity)

• Inference improves as MWM parameters adapt

MWM parameters Inferred x-traffic


Adaptivity (MWM Cross-Traffic)


Challenges: Path Modeling

Packet chirps balance measurement accuracy vs. disturbance to network and cross-traffic

Enhancements needed: rigorous statistical accuracy analysis multiple bottleneck queues passive monitoring deal with losses as well as delays closed loop paths (feedback) practical implementation issues

(clock jitter, estimating bottleneck service rate, ...)

Verification with real Internet experiments(need “ground truth” info on cross-traffic)


INCITE: Near-term Goals

• Multifractal analysis, modeling, synthesis toolbox

• Path modeling theory and toolbox

• Preliminary verification– simulations (ns-2)– Rice testbed– Enron, Nokia, Texas Instruments– IPEX / XIWT


INCITE: Longer-Term Goals

• New traffic models, inference algorithms– theory, simulation, real implementation

• Applications to control, QoS, network meltdown early warning

• TI Avalanche measurement system

• Leverage from our other projects– ATR program (DARPA, ONR, ARO)

– RENE– NSF ITR


Rice ATR Project

• Modeling, compression, automatic target recognition of multi-modal images, maps, …D. Healy (DARPA), W. Masters, W. Martinez (ONR), W. Sander (ARO)


Leverage from Other Rice Projects

• RENE (NSF, Nokia, TI)– large wireless networking project (6 PIs)– substantial traffic modeling component

• ITR/INDRA (NSF SPN, ITR)– $5M collaboration between

Rice/CMU/Virginia/Berkeley – scalable services: QoS communication,

multicast and mirroring/caching– three core services:

transfer, replication, and storage


Natural Synergies• Modeling team: New insights into

– key traffic features models should capture– origins of complex network dynamics

• Simulation team– fast synthesis of realistic aggregate traffic

• Measurement team– novel model-based inference schemes– what and where to measure

• Emulation team– level of detail for desired realism

• Design – “what if?”– new approaches to control


Natural Synergies

• What we need:

– critique of our models

– insight into the physical network mechanisms to inspire new modeling simplifications

eg: how many bottlenecks on a typical path?

– discussions on practical implementation issues

– verification experiments (“ground truth”) (scale up from ns and Rice testbed)

Multiscale Traffic Processing Techniques for Network Inference and Control

Documents

Transcript of Multiscale Traffic Processing Techniques for Network Inference and Control