Trouble Shooting CERNET NOC Pengfei Li. 2 Contents Trouble Shooting Tools Trouble Shooting BGP...

download Trouble Shooting CERNET NOC Pengfei Li. 2 Contents Trouble Shooting Tools Trouble Shooting BGP Trouble Shooting OSPF Trouble Shooting IS-IS Trouble Shooting

If you can't read please download the document

  • date post

    17-Dec-2015
  • Category

    Documents

  • view

    1.233
  • download

    11

Embed Size (px)

Transcript of Trouble Shooting CERNET NOC Pengfei Li. 2 Contents Trouble Shooting Tools Trouble Shooting BGP...

  • Slide 1
  • Trouble Shooting CERNET NOC Pengfei Li
  • Slide 2
  • 2 Contents Trouble Shooting Tools Trouble Shooting BGP Trouble Shooting OSPF Trouble Shooting IS-IS Trouble Shooting CEF Trouble Shooting IP Multicast Case Studies
  • Slide 3
  • Trouble Shooting Tools Key of trouble shooting Excellent NMS
  • Slide 4
  • 4 Route Monitoring
  • Slide 5
  • 5 Routing (BGP summary)
  • Slide 6
  • 6 Routing Mornitoring
  • Slide 7
  • 7 BGP Statistics (current status)
  • Slide 8
  • 8 Looking-glass
  • Slide 9
  • 9 BGP Monitoring (TEIN2-NORTH)
  • Slide 10
  • 10 BGP Monitoring (TEIN2-SOUTH)
  • Slide 11
  • 11 BGP Monitoring (TEIN2-JP)
  • Slide 12
  • 12 AS Path Entries
  • Slide 13
  • 13 Community Entries
  • Slide 14
  • 14 IPv4 Prefix
  • Slide 15
  • 15 IPv6 Prefix
  • Slide 16
  • 16 Basic tools Ping Traceroute telnet Show commands etc
  • Slide 17
  • Trouble Shooting BGP
  • Slide 18
  • 18 BGP in Large Scale Networks
  • Slide 19
  • 19 Avoid the Problem in the First Place Use simple configurations maintain a consistent policy throughout the AS Promote stable networks nail-down your routes use loopback interfaces Grow into your network use peer-groups and RRs for scalability
  • Slide 20
  • 20 Agenda Basic Tools Peer Establishment UPDATE Exchange Selection Algorithm Route Reflectors Route Flap Damping
  • Slide 21
  • 21 BGP Troubleshooting Tools show commands debug output Log messages
  • Slide 22
  • 22 show Commands router#show ip bgp ? A.B.C.D IP prefix /, e.g., 35.0.0.0/8 A.B.C.D Network in the BGP routing table to display cidr-only Display only routes with non-natural netmasks community Display routes matching the communities community-list Display routes matching the community-list dampened-paths Display paths suppressed due to dampening filter-list Display routes conforming to the filter-list flap-statistics Display flap statistics of routes inconsistent-as Display only routes with inconsistent origin ASs neighbors Detailed information on TCP and BGP neighbor connections paths Path information peer-group Display information on peer-groups quote-regexp Display routes matching the AS path "regular expression" regexp Display routes matching the AS path regular expression summary Summary of BGP neighbor status | Output modifiers
  • Slide 23
  • 23 show Commands (Cont.) router#show ip bgp neighbors x.x.x.x ? advertised-routes Display the routes advertised to a BGP neighbor dampened-routes Display the dampened routes received from neighbor flap-statistics Display flap statistics of the routes learned from neighbor paths Display AS paths learned from neighbor received Display information received from a BGP neighbor received-routes Display the received routes from neighbor routes Display routes learned from neighbor | Output modifiers
  • Slide 24
  • 24 router#show ip bgp BGP table version is 9, local router ID is 7.72.6.1 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 3.0.0.0 0.0.0.0 0 32768 i *> 5.0.0.0 0.0.0.0 0 32768 i *> 6.0.0.0 6.72.6.2 4294967294 0 2 i * i 6.72.6.2 4294967294 100 0 2 i *> 7.0.0.0 0.0.0.0 0 32768 i *> 8.0.0.0/5 0.0.0.0 0 32768 i *> 17.0.0.0 6.72.6.2 4294967294 0 2 i * i 6.72.6.2 4294967294 100 0 2 i *> 23.0.0.0 6.72.6.2 4294967294 0 2 i * i 6.72.6.2 4294967294 100 0 2 i *> 35.0.0.0 6.72.6.2 4294967294 0 2 i * i 6.72.6.2 4294967294 100 0 2 i The BGP Table
  • Slide 25
  • 25 The BGP Table (Cont.) router#show ip bgp 6.0.0.0 BGP routing table entry for 6.0.0.0/8, version 2 Paths: (2 available, best #1) Advertised to non peer-group peers: 7.25.14.4 7.72.6.3 7.75.7.1 2 6.72.6.2 from 6.72.6.2 (7.72.6.2) Origin IGP, metric 4294967294, localpref 100, valid, external, best 2 6.72.6.2 from 7.75.7.1 (7.75.7.1) Origin IGP, metric 4294967294, localpref 100, valid, internal
  • Slide 26
  • 26 show ip bgp Summary router#show ip bgp summary BGP router identifier 7.72.6.1, local AS number 1 BGP table version is 9, main routing table version 9 8 network entries and 12 paths using 1176 bytes of memory 3 BGP path attribute entries using 144 bytes of memory 1 BGP AS-PATH entries using 24 bytes of memory BGP activity 8/0 prefixes, 12/0 paths Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 6.72.6.2 4 2 6885 6882 9 0 0 4d18h 4 7.25.14.4 4 3 6882 6883 9 0 0 4d18h 0 7.72.6.3 4 1 6880 6886 9 0 0 4d18h 0 7.75.7.1 4 1 6884 6885 9 0 0 4d18h 4
  • Slide 27
  • 27 show ip bgp neighbors router#show ip bgp neighbors 6.72.6.2 BGP neighbor is 6.72.6.2, remote AS 2, external link Index 1, Offset 0, Mask 0x2 BGP version 4, remote router ID 7.72.6.2 BGP state = Established, table version = 9, up for 4d21h Last read 00:00:56, last send 00:00:48 Hold time 180, keepalive interval 60 seconds Neighbor NLRI negotiation: Configured for unicast routes only Peer negotiated unicast and multicast routes Exchanging unicast routes only Received route refresh capability from peer Minimum time between advertisement runs is 30 seconds Received 7044 messages, 0 notifications, 0 in queue Sent 7041 messages, 0 notifications, 0 in queue Prefix advertised 4, suppressed 0, withdrawn 0 Route refresh request: received 0, sent 0 Inbound path policy configured Route map for incoming advertisements is k Connections established 1; dropped 0 Last reset never Number of unicast/multicast prefixes received 4/0 External BGP neighbor may be up to 255 hops away. Connection state is ESTAB, I/O status: 1, unread input bytes: 0 Local host: 3.72.6.1, Local port: 179 Foreign host: 6.72.6.2, Foreign port: 11014
  • Slide 28
  • 28 router#debug ip bgp ? A.B.C.D BGP neighbor address dampeningBGP dampening events BGP events keepalives BGP keepalives updates BGP updates debug ip bgp Remember can be dangerous! Use only in the lab or If advised by the TAC To make a little safer: logging buffered no logging console
  • Slide 29
  • 29 Session Establishment (debug ip bgp ) 16:06:30: BGP: 7.72.6.1 sending OPEN, version 4 16:06:31: BGP: 7.72.6.1 OPEN rcvd, version 4 16:06:31: BGP: 7.72.6.1 rcv OPEN w/ OPTION parameter len: 12 16:06:31: BGP: 7.72.6.1 rcv OPEN w/ option parameter type 2 (Capability) len 6 16:06:31: BGP: 7.72.6.1 OPEN has CAPABILITY code: 1, length 4 16:06:31: BGP: 7.72.6.1 OPEN has MP_EXT CAP for afi/safi: 1/1 16:06:31: BGP: 7.72.6.1 rcv OPEN w/ option parameter type 2 (Capability) len 2 16:06:31: BGP: 7.72.6.1 OPEN has CAPABILITY code: 128, length 0 16:06:31: BGP: 7.75.7.1 passive open 16:06:31: BGP: 7.75.7.1 OPEN rcvd, version 4 16:06:31: BGP: 7.75.7.1 sending OPEN, version 4 16:06:31: BGP: 7.75.7.1 rcv OPEN w/ OPTION parameter len: 12 16:06:31: BGP: 7.75.7.1 rcv OPEN w/ option parameter type 2 (Capability) len 6 16:06:31: BGP: 7.75.7.1 OPEN has CAPABILITY code: 1, length 4 16:06:31: BGP: 7.75.7.1 OPEN has MP_EXT CAP for afi/safi: 1/1 16:06:31: BGP: 7.75.7.1 rcv OPEN w/ option parameter type 2 (Capability) len 2 16:06:31: BGP: 7.75.7.1 OPEN has CAPABILITY code: 128, length 0
  • Slide 30
  • 30 Session Establishment (debug ip bgp events) 17:31:39: BGP: 7.72.6.1 went from Idle to Active 17:32:00: BGP: 7.72.6.1 went from Active to OpenSent 17:32:00: BGP: 7.72.6.1 went from OpenSent to OpenConfirm 17:32:00: BGP: 7.72.6.1 went from OpenConfirm to Established 17:31:59: BGP: 7.75.7.1 went from Idle to Active 17:32:00: BGP: 7.75.7.1 went from Active to Idle 17:32:00: BGP: 7.75.7.1 went from Idle to Connect 17:32:00: BGP: 7.75.7.1 went from Connect to OpenSent 17:32:00: BGP: 7.75.7.1 went from OpenSent to OpenConfirm 17:32:00: BGP: 7.75.7.1 went from OpenConfirm to Established
  • Slide 31
  • 31 Looking at the Updates router#debug ip bgp updates? Access list Access list (expanded range) router#debug ip bgp x.x.x.x updates? Access list Access list (expanded range) Use an access-list to limit the output!
  • Slide 32
  • 32 debug ip bgp Updates BGP: 6.72.6.2 computing updates, neighbor version 0, table version 13, starting at 0.0.0.0 BGP: 6.72.6.2 send UPDATE 3.0.0.0/8, next 3.72.6.1 BGP:, metric 0, path 1 BGP: 6.72.6.2 send UPDATE 5.0.0.0/8 (chgflags: 0x0), next 3.72.6.1 BGP: 6.72.6.2 send UPDATE 7.0.0.0/8 (chgflags: 0x0), next 3.72.6.1 BGP: 6.72.6.2 1 updates enqueued (average=56, maximum=56) BGP: 6.72.6.2 update run completed, ran for 0ms, neighbor version 0, start version 13, throttled to 13, check point net 0.0.0.0 Peer Address Prefix Being Advertised NEXT_HOP
  • Slide 33
  • 33 debug ip bgp Updates (Cont.) BGP: 6.72.6.2 rcv UPDATE w/ attr: nexthop 6.72.6.2, origin i, metric 294, path 2 1 BGP: 6.72.6.2 rcv UPDATE about 3.0.0.0/8 -- DENIED due to: as-path contains our own AS; BGP: 6.72.6.2 rcv UPDATE about 7.0.0.0/8 -- DENIED due to: as-path contains our own AS; BGP: 6.72.6.2 rcv UPDATE w/ attr: nexthop 6.72.6.2, origin i, metric 494, path 2 BGP: 6.72.6.2 rcv UPDATE about 6.0.0.0/8 BGP: 6.72.6.2 rcv UPDATE about 17.0.0.0/8 BGP: 6.72.6.2 rcv UPDATE about 23.0.0.0/8 BGP: 6.72.6.2 rcv UPDATE about 35.0.0.0/8 Prefixes in the Same UPDATE Attributes Apply to All Prefixes Peer Address
  • Slide 34