Post on 24-Jan-2016
Link Failure Monitoring Using Network Coding
Hamed FiroozSumit Roy, Linda Bai
firooz,sroy,lyb3@u.washington.edu
Fundamentals of Networking Lab(FunLab)
Outline Network Tomography
Introduction (Network Monitoring) Approaches:
Deterministic vs. Stochastic Active vs Passive
Challenges: Overhead, Identifiability Network Coding
Applications to network monitoring: new method Optimization : speed/complexity tradeoffs OPNET Implementation
Fundamentals of Networking Lab(FunLab)
Network Tomography
Networks: set of nodes, links modeled as graph G(V,E)
Network monitoring Involves collection of network performance
statistics (link delay, link loss or failure status)
Important for QoS guarantees (media streaming, interactive video applications)
Challenges Choice of appropriate measurement
technique and algorithmics
1
2
34
56
7
8
Node or network
A Logical Network
G(V,E)
Fundamentals of Networking Lab(FunLab)
Measurement Methods Node-oriented: These
methods are based on cooperation among network nodes, e.g. ping or traceroute Using Ping, round trip delay to
every node can be measured. Uses Internet control message
protocol (ICMP) packets Many routers do NOT
respond to these packets Many service providers do not
own the entire network
R
R
R1lD
2lD
1
2
Fundamentals of Networking Lab(FunLab)
Measurement Methods
Edge-oriented: Access is available to nodes at the edge only (and not to any in the interior) Does not require exchanging
special control messages between interior nodes
Inverse problem: estimate link level status from end-2-end (path level) measurements
S
SS
S
Network(?)Network(?)
Fundamentals of Networking Lab(FunLab)
Measurement Methods
Active (sending probe packets)
- Adds overhead to normal data traffic by
introducing new control packets Passive (insitu traffic analysis)
- No overhead; temporal and spatial dependence might bias measurement
Our method: edge-oriented, active network tomography Given a network, and a limited number of
end hosts, when can we infer failure status of the links?
Network?
Fundamentals of Networking Lab(FunLab)
End1
End2 End3
router1
link1
link2
link3
11032
10131
01121
321
EndEnd
EndEnd
EndEnd
linklinklink
Routing matrix A
End-to-End Probing
• Probes are inserted into a data stream, and end-to-end properties on that route measured.• Probes are exchanged between end nodes using routing matrix of the graph
Fundamentals of Networking Lab(FunLab)
End-to-End Probes
Routing matrix relates link attribute to route attribute
For some parameters like delay or path loss, this relation is linear under some assumptions
End1
End2 End3
R
1
23
3
2
1
110
101
011
32
31
21
l
l
l
EndEnd
EndEnd
EndEnd
D
D
D
D
DD
Fundamentals of Networking Lab(FunLab)
Deterministic
Link attributes (e.g. delay) are considered unknown, constant
Goal: estimate constants Link attributes are typically time varying
method is suitable for periods of local ‘stationarity’
Fundamentals of Networking Lab(FunLab)
Stochastic
Link attribute specified by a suitable probability distribution e.g. link delay follows a Gaussian distribution
Estimation problem: unknown model parameters
based on path observation in the presence of additive noise
Fundamentals of Networking Lab(FunLab)
Deterministic vs. Stochastic Methods
Stochastic Bayesian - requires a prior distribution
incorrect choice leads to biases in the estimates More computationally intensive
Deterministic Lower complexity but suffers from generic non-
identifiability
Fundamentals of Networking Lab(FunLab)
Link Failure Model
congestedis1
okis0
i
il l
lx
i
End1 End2R1 R2
1 2 3
..1
okis,,ofall0 32121 wo
llly endend
Define an indicator function for status of each link
Fundamentals of Networking Lab(FunLab)
Binary Deterministic Model
32121 lllendend xorxorxy
y = Ax
A: N-by-M binary routing matrix
x: M-by-1 binary vector, the status of each link
y: N-by-1 binary vector, the status of each path (measurements)
End1 End2R1 R2
1 2 3
Fundamentals of Networking Lab(FunLab)
Failure Monitoring
Network G(V,E) with set of paths P x, y are binary vectors A path is congested if at least one of its links
is congested
}1,0{,
110
101
011
32
31
21
1
3
2
1
3
2
1
321
l
l
l
l
x
x
x
x
EndEnd
EndEnd
EndEnd
y
yy
lll
32
31
21
)(
)()(
3
2
1
ll
ll
ll
xORx
xORxxORx
y
yy
|||| }1,0{,}1,0{ PE yx
End1
End2 End3
Router
1
2 3
Fundamentals of Networking Lab(FunLab)
Identifiability y = Ax Problem: Estimate x from y with
A (N-by-M) : binary routing matrix x (M-by-1) : binary link failure status y (N-by-1) : end-to-end measurements
Identifiability: a network is identifiable if y = Ax has a unique solution Usually, M ( # of links in network) >> N (# of
measurements), so network is generically NOT identifiable.
6 links, 3 End-to-End routes N=6, M=3
Fundamentals of Networking Lab(FunLab)
Identifiability: Binary Model Solution: limit (maximum) number of failed links
inside the network Suppose at most k links can fail simultaneously
Defn: k-Identifiability Network is k-identifiable if
from end-to-end observation it is possible to uniquely identify up to k congested links
k01|E| x
2121020121 ,k,ks.t., AxAxxxxxxx Only one link can be congested
1016 x
Fundamentals of Networking Lab(FunLab)
Example of 1-identifiability
1016 x
-- 1 2 3 4 5 6
0 1 1 0 0 0 0
0 1 0 1 1 0 1
0 0 0 0 0 1 1
21y
31y
32y
1
2 3
45
6
111000
100101
001011
A
Fundamentals of Networking Lab(FunLab)
Example: k=2 identifiability
2016 x
0
1
1
32
31
21
y
y
y
y
1
2 3
45
6
0
1
1
32
31
21
y
y
y
y
Ambiguity
111000
100101
001011
A
Fundamentals of Networking Lab(FunLab)
1-Identifiability
A network with an intermediate degree two node is not 1-identifiable If path End1End2 is congested, it is
impossible to determine which link among 1 and 2 is congested .
Necessary but not sufficient!
`
`
l1
l2
End1
End2
11 211 EndEndl yx
11 212 EndEndl yx
Fundamentals of Networking Lab(FunLab)
k=1 Identifiability
1-identifiability Theorem: End-to-End probe based measurements can detect a unique congested link in a network if and only if there are no two identical columns in the network routing matrix
10000000
01000000
00011000
00100100
00000010
00000001
P1 P3
P1
P3
Fundamentals of Networking Lab(FunLab)
k- identifiability
k-identifiability Theorem: End-to-End probe based measurements can detect a unique congested link in a network only if there are no k+1 dependent columns in the network routing matrix
10000000
01000000
00011000
00100100
00000010
00000001
Fundamentals of Networking Lab(FunLab)
Example: k=2 identifiability
2016 x
0
1
1
32
31
21
y
y
y
y
1
2 3
45
6
0
1
1
32
31
21
y
y
y
y
Ambiguity
111000
100101
001011
A
Fundamentals of Networking Lab(FunLab)
Shortest Path Routing Revisited
Packets are sent on shortest path between two end nodes
- sub-graphs = tree starting from a boundary (source) node Node 4 has degree two in all graphs
But node 4 has degree four in the original network
Fundamentals of Networking Lab(FunLab)
Revisiting Shortest Path Routing What if we could change routing
matrix ?
Example: in place of shortest path routing, route packets through longer paths, e.g. n124n2
Now network is 1-identifiable ! Intrinsic limitation for end-to-end
measurement methods based on shortest path routes probes transmitted along such paths
contain only minimum information
Fundamentals of Networking Lab(FunLab)
Solution
Look to exchange probes between boundary nodes via other (non-shortest) paths?
Changing the routing tables violates tomography assumption
Use Network Coding; exploit broadcast nature of network coding, a transmitted probe will traverse almost every path between two boundary nodes
Fundamentals of Networking Lab(FunLab)
Network Coding: Short Review Present: routers just forward incoming packets, i.e. copy
the packets on an input link onto the output links Proposed: What if each node in a network performs
some computation on received data prior to forwarding?
y1
y2
y1
y2
y1
y2
y3
f1(y1,y2,y3)
f2(y1,y2,y3)
Fundamentals of Networking Lab(FunLab)
How does NC work? (1)
sender sreceiver t2
receiver t1
A
B
C D
“Butterfly” network: All links have the same capacity 1 b/s s wants to send data bits a, b to both t1 and t2
Bottleneck is CD
Fundamentals of Networking Lab(FunLab)
How does NC work?(2)
Node C XORs received messages on each of its links
sender sreceiver t2
receiver t1
A
B
XOR
Da
b a+b
a
b
Fundamentals of Networking Lab(FunLab)
How does NC work?(3)
t1 and t2 know both a and b Now s can send data at rate 2 b/s/receiver
sender sreceiver t2
receiver t1
A
B
XOR
Da
b a+ba+ba+b
a
b
Fundamentals of Networking Lab(FunLab)
Linear Network Coding
Network Coding is a coding at layer three The coding is conducted over the finite field Fu, u=2q
each coded symbol can be represented by q-bits within an IP layer frame
Signal Y(j) on an outgoing link j of node v, is a linear combination of signals Y(i) on incoming link i of v: We assume there is no process generated at node v
})(:{
)()(vldll lYjY
Fundamentals of Networking Lab(FunLab)
Received Symbols
Pi : i-th route from source to destination Source sends α over Pi
βi depends on topology G hence βi(G)
Coef. NCPath )(
),(2
i
q
i
Plli
iPl
l
G
FGy
S D
γ1 γ2
γ3
γ4 γ5
α)(121 Gy
Fundamentals of Networking Lab(FunLab)
Received Symbols: Linear Model
ek one of source outgoing links Pek : collection of all paths between source and
destination starts at ek
Source sends αk over ek. By superposition destination receives
||
1, )(
ke
k
kei i
P
ieik
PP Pllk Gy
S D
γ1 γ2
γ3
γ4 γ5
e1
1eP
)()(11 ,2,11531211 eey
αα1
Fundamentals of Networking Lab(FunLab)
Received Symbols: Linear Model
Source sends out symbols αk over ek using superposition once more
In vector format: y=αtβ(G) β(G) is total network coding vector
K
k
P
ieik
ke
kGy
1
||
1, )(
S D
γ1 γ2
γ3
γ4 γ5
e1
1eP
αα1
α2
Fundamentals of Networking Lab(FunLab)
Received Symbols: Linear Model
Source sends symbols in M succ. time slots:
11 )( NNMM GAy
t
P
eN
P
eNe
P
eNeeN
Ke
KK
ee
G
,,,1,,2,11
2
221
1
1111)(
Fundamentals of Networking Lab(FunLab)
Link Failure Model
If a link is severely congested, packets are significantly delayed and assumed lost at the destination
We model the network with link in congestion state by its edge deleted subgraph denoted by G(V,E)
S D
γ1
γ3
γ4 γ5
Fundamentals of Networking Lab(FunLab)
Link Failure Model Total network coding vecor of G(V;E), β(G) is
different from β(G)
if the congested link doesn’t belong to i-th path from source to destination, Pi, it will not affect packets going through those paths It is zero otherwise
..0
)()()( ,
,wo
dPlifGG
ieei
leikk
k
S D
γ1 γ2
γ3
γ4 γ5
e1
e2
211 )( G
542 )( G
0)(11 lG
)()( 22 1GGl
1
Fundamentals of Networking Lab(FunLab)
Link Failure Model
Training sequence is A y : vector of symbols observed at the
destination in M time slots with link congested
Potential for identifying: received symbols change uniquely in response to link congestion
11 )( NlNMlM GAy
1 2
1 1
1 1
lM M
l lM M
y y
y y
Fundamentals of Networking Lab(FunLab)
Example
11, 1 1 1e
S D
11
2
3 2
e1
e2
1eP
2eP
12, 1 2 2 3e
22, 3 2 1e
1
( ) 3
1
G
1 1 2
3 3 3A
-- e1 e2 1
2 3
1st time slot 0 2 2 3 1 1
2nd time slot 2 3 1 0 1 3
Fundamentals of Networking Lab(FunLab)
Theorem 1: Sufficient Conditions
If Rank(A)= deg(S), and for all Pek set of paths between source and
destination starting at ek
then
jj
P
jejj
ke
i
00||
1,
EllGAGA
ElGAGA
ll
l
21,)()(
)()(
21
(more next slide)
Fundamentals of Networking Lab(FunLab)
Theorem 1
Condition means For a set of paths having ek in common, Pek , NC
coefficient of the paths are independent !
jj
P
jejj
ke
i
00||
1,
S D
γ1 γ2
γ3
γ4 γ5
e1
e2
1eP
2eP
t independen531,2
21,1
1
1
e
e
54,1 2 e
t
P
eN
P
eNe
P
eNee
Ke
KK
ee
,,,1,,2,1
2
221
1
1111
Independent Independent
Fundamentals of Networking Lab(FunLab)
Example
11, 1 1 1e
S D
11
2
3 2
e1
e2
1eP
2eP
12, 1 2 2 3e
22, 3 2 1e
1
( ) 3
1
G
1 1 2
3 3 3A
( ) 2 deg( )Rank A S
-- e1 e2 1
2 3
1st time slot 0 2 2 3 1 1
2nd time slot 2 3 1 0 1 3
Independent
Fundamentals of Networking Lab(FunLab)
Complexity/Speed First condition of Theorem 1:
In previous example M=2=deg(S) Number of time slots: at least the number of
outgoing links of source Is it possible to decrease number of time
slots? faster monitoring Possible by increasing number of bits in LNC
coeff. more complexity
)deg( implies)deg()Rank( SMSA NM
Fundamentals of Networking Lab(FunLab)
Example
q=3 A=[1 1 4]
-- e1 e2 1
2 3
1st time slot 6 4 2 5 7 1
S D
11
2
3 2
e1
e2
Fundamentals of Networking Lab(FunLab)
Theorem 2: Complexity/Speed tradeoff
Ni=|Pi| q bits per symbol are used in network coding M number of (desired) time slots Let Z={1,2,…,K} K degree of source ZM: collection of all partitions of Z with size M
K=3, 2 Z={1,2,3} ZM={ {{1,2},{3}} , {{1,3},{2}} , {{2,3},{3}} }
M
ijiiMM HHZHHHHZ
121 },|},...,,{{
S
K links
Fundamentals of Networking Lab(FunLab)
Theorem 2: Complexity/speed tradeoff
Network is 1-identifiable if
Rank(A)=M
iMi Hj
jiZMiH
Nq maxmin},...,1,{
Fundamentals of Networking Lab(FunLab)
Theorem 3: Random LNC
Random linear network coding is a distributed approach achieving capacity asymptotically
Intermediate node choose their NC coefficients uniformly from the elements of Fu (u=2q)
Exponential increase with q (number of bits) and M (number of time slots)
Quadratic decrease with size of network
1( is1-identifiable) 1 | | (| | 1)( )
2M
qP G E E
Fundamentals of Networking Lab(FunLab)
Multi-source Multi-destination
So far, considered only Single source Single destination
Easily extendable to Multi-source Multi-destination
Fundamentals of Networking Lab(FunLab)
Simulation Simulation environment
OPNET 14.5 MATLAB 7.1 (finite field operations)
Evaluation University of Washington’s Electrical Engineering
network Thirteen subnets 3 backbone routers Full Duplex Ethernet links
Fundamentals of Networking Lab(FunLab)
Simulation Set-Up Implementation of Network Coding (NC) within OPNET We employ network coding at transport layer (instead
of IP layer) Easier to implement
Routers model is modified to distinguish between non-NC/NC packets through the use of a flag bit within the UDP header NC packets are sent for separate processing non-NC packets are processed normally
We assign a q-bit field called LNC field within the TCP/UDP header, for linear network coding.
UDP packetLNC field1
Fundamentals of Networking Lab(FunLab)
RECEIVE/SEND interface
Inherently network coding operates on unidirectional links
Each interface within a router mode is designated as a SEND or RECEIVE interface only for the network coded packets operating regularly with non-network coded
packets Finite field operation is done in MATLAB
Using MATLAB API within OPNET
Fundamentals of Networking Lab(FunLab)
RECEIVE/SEND
Fundamentals of Networking Lab(FunLab)
Evaluation
Fundamentals of Networking Lab(FunLab)
UW EE Network
Fundamentals of Networking Lab(FunLab)
UW EE Network-lookup table
Fundamentals of Networking Lab(FunLab)
Fundamentals of Networking Lab(FunLab)
Network Tomography: A Stochastic Model [1]
Passage of probes can be modeled as two stochastic process: {X(i)} and {Z(i)} for each node k
Z(i) time delay process of link k
X(i) called bookkeeping process: cumulative probe from root to k
[1] V. Arya, N. Duffield, D. Veitch “ Temporal Delay Tomography”, IEEE Infocom 2008
Fundamentals of Networking Lab(FunLab)
Network Tomography: Stochastic Method
Discretize delay D={0,b,2b,…,mb,∞} mb is delay threshold X(i)=X’(i)+Z(i)
k-1
k
X(i)
Z(i)
’
X’(i)
..
if
if
])(Pr[
1
0
])(|)(Pr[
wo
vd
vd
vdiZ
viXdiX
k
ll