Troubleshooting Routing Protocols - alcatron.net Live 2015 Melbourne/Cisco...• OSPF not enabled on...

#clmel

Troubleshooting Routing Protocols-BGP/OSPF/EIGRP

BRKRST-2619

Faraz Shamim – Technical Leader

© 2015 Cisco and/or its affi liates. All rights reserved.BRKRST-2619 Cisco Public

Agenda

• OSPF Overview

• OSPF Troubleshooting

• EIGRP Overview

• EIGRP Troubleshooting

• BGP Overview

• BGP Troubleshooting

OSPF Overview


OSPF Areas

• OSPF uses a 2 level hierarchical model

• Areas defined with 32 bit number

–Defined in IP address format

–Can also be defined using single decimal value (i.e., Area 0.0.0.0, or Area 0)

• 0.0.0.0 reserved for the backbone area

Area 0

Area 1

Area 2

Area 3


OSPF LSAs• Router and network LSA’s within

an area

• Summary LSA type 3 outside the area

• Summary LSA Type 4 and Type 5 for redistributed routes

• Partial SPF for summary and External LSA’s

Area 0

Area 1

Area 2

Area 3

OSPF/OSPFv3 Troubleshooting


• OSPF not enabled on the interface

• Tip: Check for the wrong network statement; re-enter the network statement / For OSPFv3, check interface configs for missing ipv6 ospf area command

R3#show ip/ipv6 ospf neighborR3#

R3#show ip ospf interface serial 2Serial2 is up, line protocol is up

OSPF not enabled on this interface

In 12.0:

R3#show ip/ipv6 ospf interface serial 2

R3#

Adjacency Is Not Coming Up


Adjacency Is Not Coming Up (Not Applicable to OSPFv3)

• Mismatched subnet mask

R3#show ip ospf neighborR3#

R3#debug ip ospf adj

OSPF adjacency events debugging is onR3#

OSPF: Mismatched hello parameters from 192.1.1.4Dead R 40 C 40, Hello R 10 C 10 Mask R 255.255.255.192 C 255.255.255.0


OSPF: Rcv pkt from 192.1.1.4, Ethernet0, area 0.0.0.1 mismatch area 0.0.0.2 in the header

neighbour is in area 2 but we are not

%OSPF-4-ERRRCV: Received invalid packet: mismatch area ID, from backbone area must be virtual-link but not found from 192.1.1.4, Ethernet0

R4#show ip/ipv6 ospf neighbor

R4#

R4#debug ip/ipv6 ospf adj

OSPF adjacency events debugging is on


• Mismatched area ID



• Mismatched transit/stub/NSSA option

OSPF: Hello from 18.10.0.2 with mismatched Stub/Transit area option bit

R7#show ip/ipv6 ospf neighbor

R7#

R7#debug ip/ipv6 ospf adj

OSPF adjacency events debugging is on

7.7.7.7 8.8.8.8

R8R7Area 2

Area 2 NSSA

18.10.0.218.10.0.1


Options

Normal area: OSPF: Send DBD to 141.108.97.1 on Serial0 seq 0xBC4 opt 0x2 flag 0x3 len 492

E bit is 1, Allow externals, option: 0x2(HEX) = 00000010(Bin)

Stub area: OSPF: Send DBD to 141.108.97.1 on Serial0 seq 0x1866 opt 0x0 flag 0x3 len 372

E bit is 0, no external allowed, options: 0x0 = 00000000

NSSA:OSPF: Send DBD to 141.108.97.1 on Serial0 seq 0x118 opt 0x8 flag 0x3 len 372

N/P bit is on, options: 0x8 = 00001000

DC:OSPF: Send DBD to 141.108.97.1 on Serial0 seq 0x1A1E opt 0x20 flag 0x3 len 392

DC bit is negotiated, options: 0x20 = 00100000

O DC EA N/P MC E

* *


Adjacency Is Not Coming Up – OSPFv3 Only• Mismatched instance-ID

TIP: Either change R2 instance ID to 2 or change R1 instance ID back to 0

R1#show ipv6 ospf neighbor

R1#

R1#sh ipv6 ospf int e0/0 | inc Instance

Area 1, Process ID 1, Instance ID 2, Router ID 192.168.1.130

R2#sh ipv6 ospf int e0/0 | inc Instance

Area 1, Process ID 1, Instance ID 0, Router ID 172.16.1.2

192.168.1.130 172.16.1.2

R2R1Area 1

Area 1


Adjacency Is Not Coming Up – OSPFv3 Only

• No IPv6 address on the interface

TIP: OSPFv3 runs over link local address. Either a link local or unicast global address needs to be enabled on the interface for OSPFv3 to form

adj.


R2#

R1#sh ipv6 int e0/0

192.168.1.130 172.16.1.2

R2R1Area 1

Area 1


Adjacency Is Not Coming Up – OSPFv3 Only

• No IPv6 unicast-routing

TIP: ipv6 unicast-routing is not enabled by default in Cisco IOS. The router will be unable to route without this command.


R2#

R1#sh ipv6 ospf int e0/0

%OSPFv3: OSPF not enabled on Ethernet0/0

R1#

192.168.1.130 172.16.1.2

R2R1Area 1

Area 1


Stuck in INIT

• One side is blocking the hello packet with access-list

• One side is translating (NAT) OSPF hello

• One side multicast capabilities is broken (Layer 2)

Reasons


Stuck in EXSTART/EXCHANGE

• MTU mismatch—EXCHANGE

– Note: If Cisco IOS is < 12.0.3 neighbour will show stuck in EXCHANGE

• Neighbour RID is same as ours—EXSTART

– Note: If Cisco IOS is > 12.0.7, it displays msg: %OSPF-3-DUP_RTRID and OSPF neighbour list will be empty

• Unicast is broken—EXCHANGE

1. MTU problem, can’t ping across with more than certain length packet

2. Access-list blocking unicast; after two-way OSPF send unicast packet except p2p links

3. NAT is translating unicast packet

Reasons


Mismatched Network Types

Area 0

R3

3.3.3.3

R6

6.6.6.6

18.10.0.4/30

R3#show ip/IPv6 ospf interface serial 0Serial0 is up, line protocol is up

Internet Address 18.0.0.5/30, Area 0Process ID 1, Router ID 3.3.3.3, Network Type POINT_TO_POINT, Cost: 64

R6#show ip/IPv6 ospf interface serial 0Serial0 is up, line protocol is up

Internet Address 18.0.0.6/30, Area 0Process ID 1, Router ID 6.6.6.6, Network Type BROADCAST, Cost: 64

R3#show ip/IPv6 ospf neighbor

Neighbor ID Pri State Dead Time Address Interface 6.6.6.6 1 FULL/ - 00:00:30 18.0.0.6 Serial0


R618.10.0.4/30

Area 0

R3

3.3.3.3 6.6.6.6

Mismatched Network Types OSPFv2R3#show ip ospf database router 3.3.3.3

. . .

Link ID = 6.6.6.6 Router id of the neighbor

Link Data = 18.10.0.5 IP interface address

Type = 1 This is a point-to-point link

# TOS metrics = 0

metric = 8

. . .

R3#show ip ospf database router 6.6.6.6

. . .

Link ID = 18.10.0.6 IP address of the DR

Link Data = 18.10.0.6 Interface address

Type = 2 This is a transit link

# TOS metrics = 0

metric = 8


R618.10.0.4/30

Area 0

R3

3.3.3.3 6.6.6.6

Mismatched Network Types – OSPFv3

R3#show ipv6 ospf database router self

. . .

Link connected to: another Router (point-to-point)

Link Metric: 10

Local Interface ID: 2

Neighbor Interface ID: 2

Neighbor Router ID: 6.6.6.6

. . .

R3#show ipv6 ospf database router adv 6.6.6.6

. . .

Link connected to: a Transit Network

Link Metric: 10

Local Interface ID: 2

Neighbor (DR) Interface ID: 2

Neighbor (DR) Router ID: 3.3.3.3


Different Mask or IP Subnet on P2P Links – OSPFv2 Only

Area 0

R3

3.3.3.3

R6

6.6.6.6

19.10.0.5/30 18.10.0.6/30

R3#show interface serial 0Serial0 is up, line protocol is up

Hardware is HD64570 Internet address is 19.10.0.5/24

R6#show interface serial 0

Serial0 is up, line protocol is up Hardware is HD64570 Internet address is 18.10.0.6/30

R3#show ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface 6.6.6.6 1 FULL/ - 00:00:30 18.0.0.6 Serial0


R6

Area 0

R3

3.3.3.3 6.6.6.6

Different Mask or IP Subnet on P2P Links (Cont.)


. . .




# TOS metrics = 0

metric = 8

. . .


. . .




# TOS metrics = 0

metric = 8


SPF Running Constantly

• LSA Flaps Due To:–Duplicate RID/IP address

–Constant link flapping in an area

Reasons


SPF Running Constantly – OSPFv2/v3R3#sh ip/ipv6 ospf stat

Area 0: SPF algorithm executed 42 times


SPF calculation time

Delta T Intra D-Intra Summ D-Summ Ext D-Ext Total Reason

00:22:00 0 0 0 0 0 0 0 R, N, SN,

00:21:44 0 0 4 0 0 0 4 R, SN, X

00:21:34 0 0 4 0 0 0 4 R, SN, X

00:21:24 0 0 0 4 0 0 4 R, SN, X

00:21:14 0 0 0 0 0 0 0 R,

00:21:04 0 0 0 0 0 0 0 R, N, SN,

00:20:54 0 0 0 0 0 0 0 X

00:20:44 0 0 4 0 0 0 4 R, SN, X

00:20:34 0 0 0 0 0 0 0 X

00:00:17 4 0 0 0 0 0 4 R, N, SN, SA, X

. . .

R=Router LSA; N=NetworkLSA; SN=Summary Network LSA; SA=Summary ASBR

LSA; X=External LSA

Requires Enable Mode


Show IP OSPF Stat – OSPFv2/v3R1#show ip/ipv6 ospf stat detailOSPF process ID 100

------------------------------------------


SPF 1 executed 1w0d ago, SPF type Full

SPF calculation time (in msec):

SPT Intra D-Intr Summ D-Summ Ext7 D-Ext7 Total0 0 0 0 0 0 0 0

LSIDs processed R:1 N:0 Stub:1 SN:0 SA:0 X7:0

Change record R,

LSIDs changed 1

Last 10 LSIDs:

4.4.4.4(R)

Summary OSPF SPF statistic

SPF calculation timeDelta T Intra D-Intra Summ D-Summ Ext D-Ext Total Reason

1w0d 0 0 0 0 0 0 0 R,

Requires Enable Mode


R3#deb ip/ipv6 ospf mon

OSPF: Schedule SPF in area 1Change in LS ID 1.1.1.1, LSA type R,

OSPF: schedule SPF: spf_time 0ms wait_interval 861421816s

OSPF: Begin SPF at 0x33585480ms, process time 752ms

spf_time 0ms, wait_interval 861421816s

OSPF: End SPF at 0x33585488ms, Total elapsed time 8ms

Intra: 4ms, Inter: 0ms, External: 0ms




R3#show ip/ipv6 ospf databaseOSPF Router with ID (3.3.3.3) (Process ID 1)

Router Link States (Area 0)

Link ID ADV Router Age Seq# Checksum Link count3.3.3.3 3.3.3.3 106 0x80000009 0xC3F1 3

. . .

Summary Net Link States (Area 0)

Link ID ADV Router Age Seq# Checksum

18.10.0.0 7.7.7.7 3 (DNA) 0x80000008 0x3DC2

18.10.0.0 8.8.8.8 1396 0x80000004 0x27D8 . . .

Router Link States (Area 1)

Link ID ADV Router Age Seq# Checksum Link count

1.1.1.1 1.1.1.1 2 0x80000016 0xE6CD 2

. . .

EIGRP Overview


EIGRP Basics• Advanced distance vector,

TLV based protocol, uses DUAL algorithm

• Runs directly on top of IP/IPv6 protocol 88

• Uses multicast capabilities(Neighbour relationship &Updates)

• Support unequal cost path load balancing


EIGRP Basics• Metric is based on the bandwidth

and delay (minimum bandwidth along the path and composite delay)

• K1, K2, K3, K4, K5 =Constants

• Metric = [K1 x BW + (K2 x BW) / (256 -Load) + K3 x Delay] x [K5 / (Reliability + K4)]

– By Default: K1 = 1, K2 = 0, K3 = 1, K4 = K5 = 0

– Metric = [107/BWmin + Σdelays] x 256

EIGRP Troubleshooting


Adjacency Is Not Coming Up – EIGRP v4 only

• Mismatch subnet

R3#show ip eigrp neighborR3#

R3#

%DUAL-6-NBRINFO: EIGRP-IPv4 1: Neighbor 10.122.1.1 (Ethernet0/0) is blocked: not on common subnet (10.121.1.2/30)



• Mismatch AS number

R3#show ip/ipv6 eigrp neighborR3#

R3#show run | i eigrp

router eigrp 1R2#show run | i eigrp

router eigrp 2

Compare the AS number on both sides


Adjacency Is Not Coming Up• Unidirectional link issue

R3#show ip/ipv6 eigrp neighborIP-EIGRP neighbors for process 1

H Address Interface Hold Uptime SRTT RTO Q Seq(sec) (ms) Cnt Num

1 10.88.18.2 S0 14 01:26:30 149 894 0 291

R2#show ip/ipv6 eigrp neighborIP-EIGRP neighbors for process 1

H Address Interface Hold Uptime SRTT RTO Q Seq(sec) (ms) Cnt Num

1 10.88.18.2 S0 14 00:00:30 149 894 4 0


Troubleshooting the Stuck in ACTIVE

• Show ip eigrp topology active

• Useful only while the problem is occurring

• If the problem isn’t occurring at the time, it is very difficult to find the reason the routes are getting stuck


Chasing Active Routes

Why Is RtrA Reporting SIA Routes?Let’s Look at a Problem in Progress

RtrA Is Waiting on RtrB

20.1

.1.0

/24

10.1.1.0/24 10.1.2.0/24 10.1.3.0/24

.1 .2 .1 .2 .1 .2

A B C D

RtrA#show ip eigrp topology active

IP-EIGRP Topology Table for AS(1)/ID(20.1.1.1)

A 20.1.1.0/24, 1 successors, FD is Inaccessible

1 replies, active 00:01:17, query-origin: Local origin

via Connected (Infinity/Infinity), Ethernet1/0

Remaining replies:

via 10.1.1.2, r, Ethernet0/0



So Why Hasn’t RtrB Replied?

RtrB#show ip eigrp topology active


A 20.1.1.0/24, 1 successors, FD is Inaccessible

1 replies, active 00:01:26, query-origin: Successor Origin

via 10.1.1.1 (Infinity/Infinity), Ethernet0/0

Remaining replies:

via 10.1.2.2, r, Ethernet1/0

RtrB is Waiting on RtrC

20.1

.1.0

/24

10.1.1.0/24 10.1.2.0/24 10.1.3.0/24

.1 .2 .1 .2 .1 .2

A B C D



What’s RtrC’s Problem?

RtrC#show ip eigrp topology active


A 20.1.1.0/24, 1 successors, FD is Inaccessible, Qqr

1 replies, active 00:01:33, query-origin: Successor Origin, retries(1)

via 10.1.2.1 (Infinity/Infinity), Ethernet0/0, serno 20

via 10.1.3.2 (Infinity/Infinity), rs, q, Ethernet1/0, serno 19, anchored

RtrC Is Waiting on RtrD

20.1

.1.0

/24

10.1.1.0/24 10.1.2.0/24 10.1.3.0/24

.1 .2 .1 .2 .1 .2

A B C D



Why Isn’t RtrD Answering?

RtrD#show ip eigrp topology active


RtrD#

20.1

.1.0

/24

10.1.1.0/24 10.1.2.0/24 10.1.3.0/24

.1 .2 .1 .2 .1 .2

A B C D



No; RtrC Is Still Waiting on RtrD; What’s the Deal?

RtrC#show ip eigrp topology active


A 20.1.1.0/24, 1 successors, FD is Inaccessible, Qqr

1 replies, active 00:01:52, query-origin: Successor Origin, retries(1)

via 10.1.2.1 (Infinity/Infinity), Ethernet0/0, serno 20

via 10.1.3.2 (Infinity/Infinity), rs, q, Ethernet1/0, serno 19, anchored

RtrC is waiting on RtrD

20.1

.1.0

/24

10.1.1.0/24 10.1.2.0/24 10.1.3.0/24

.1 .2 .1 .2 .1 .2

A B C D



Let’s see why they don’t seem to agree about the active route

RtrC#show ip eigrp neighbors

IP-EIGRP neighbors for process 1

H Address Interface Hold Uptime SRTT RTO Q Seq

(sec) (ms) Cnt Num

0 10.1.3.2 Et1/0 13 00:00:14 0 5000 1 0

1 10.1.2.1 Et0/0 13 01:22:54 227 1362 0 385

Looks like something’s broken between RtrC and RtrD

20.1

.1.0

/24

10.1.1.0/24 10.1.2.0/24 10.1.3.0/24

.1 .2 .1 .2 .1 .2

A B C D



RtrC#ping 10.1.3.2

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.1.3.2, timeout is 2 seconds:

.....

Success rate is 0 percent (0/5)

Okay—we can’t ping; we need to fix this before EIGRP stands a chance of working

20.1

.1.0

/24

10.1.1.0/24 10.1.2.0/24 10.1.3.0/24

.1 .2 .1 .2 .1 .2

A B C D


Likely Causes for Stuck-in-Active

• Bad or congested links

• Query range is “too long”

• Excessive redundancy

• Overloaded router (high CPU)

• Router memory shortage

• Software defects (seldom)

43


EIG

RP

OS

PF

Multiple Points of Redistribution

• A route is injected into EIGRP as an external; this route is redistributed into OSPF by RtrB

• The route is transmitted through OSPF to RtrA , who redistributes it back into EIGRP

• Depending on the manually set metrics, RtrB may prefer this redistributed route, building a routing loop

• Depending on the timing, the loop can be persistent or transient. Either way, a bad thing!

A

Metric 10 Metric 2816000

10.1.1.0/24

Metric

2688000


B


Redistribution Design

• There are three primary methods used to prevent this routing loop:– Redistributing live routing information in only one direction

– Filtering routes based on the prefixes advertised to prevent feedback loop

– Filtering routes using routing tags to prevent feedback


EIG

RP

OS

PF


• Redistribute a static in one direction, and between protocols in the other direction

• A route is injected into EIGRP as an external; this route is then redistributed into OSPF by RtrB

• The route is transmitted to RtrAthrough OSPF; the route is not redistributed back into EIGRP, since redistribution between OSPF and EIGRP is not configured

Live Routing Information in Only One Direction

router ospf 100

redistribute eigrp 100 metric 10

router eigrp 100

redistribute static metric 10000 1000 255 1 1500

10

.1.0

.0/1

6

10

.2.0

.0/1

6

A

BMetric 10 Metric 2816000

Metric 25

ip route 10.2.0.0 255.255.0.0 serial 0/0

10.1.1.0/24

Metric 2560256



• Configure access/prefix lists which match the address ranges used in each section of the network and filter based on these ACLs

• The route is injected into EIGRP as an external; this route is then redistributed into OSPF by RtrB

• The route is transmitted through OSPF and reaches RtrA

• The route is now blocked by distribute list 20, which breaks the routing loop

Filtering Based on Prefixes

10

.2.0

.0/1

6

router ospf 100

redistribute eigrp 100 metric 10

distribute-list 10 out

router eigrp 100

redistribute ospf 100 metric 1000 1 255 1 1500

distribute-list 20 out

EIG

RP

OS

PF

A

B

access-list 10 permit 10.1.0.0 0.0.255.255

access-list 20 permit 10.2.0.0 0.0.255.255


Metric 25

10.1.1.0/24

Metric 2560256

10

.1.0

.0/1

6


10

.2.0

.0/1

6


• Set route tags when redistributing between the protocols; deny tagged routes at the redistribution point

• The route is injected into EIGRP as an external; it is redistributed into OSPF by RtrB and a tag is set

• The route is transmitted to RtrA through OSPF

• The route is blocked from being redistributed into EIGRPbecause of the route tag

Filtering Based on Tags

router ospf 100

redistribute eigrp 100 metric 10 route-map usetags

router eigrp 100

redistribute ospf 100 metric 1000 1 255 1 1500 route-map usetags

EIG

RP

OS

PF

A

B

route-map usetags deny 10

match tag 1000

route-map usetags permit 20

set tag 1000


10.1.1.0/24

Metric 2560256

Metric 25

10

.1.0

.0/1

6


IPv6 Unique Issues

• IPv6 Router-ID

• IPv6 Interfaces

• IPv6 Peer Addresses

• IPv6 Shutdown


IPv6 Router-id

• EIGRP IPv6 will not work without a router-id

• EIGRP uses the same router-id selection process used by IPV4– Highest IPv4 address on a Loopback interface

– If no Loopbacks, highest IPv4 address on non-loopback interface

• If no IPv4 address is available to use, manually set the router-id under the “ipv6 router eigrp x” configuration– “eigrp router-id 1.1.1.1”

– Note that in some older versions, the leading “eigrp” in the command above wasn’t required.


IPv6 Interfaces

• EIGRP does not use a network statement to specify its IPv6 interfaces– Interfaces may not have a globally routable address (may have only link-local)

• Two different methods exist, depending on the configuration method– Classic mode uses the “ipv6 eigrp <AS>” command on each interface

– Named mode defaults to having all interfaces enabled• Will need to “shutdown” under af-interface <interface> in order to not include some interfaces


IPv6 Interfaces

Classic ModeR2#conf t

Enter configuration commands, one per line. End

with CNTL/Z.

R2(config)#ipv6 router eigrp 1

R2(config-rtr)#interface s4/0

R2(config-if)#ipv6 eigrp 1

R2#sh run interface s4/0

Interface Serial4/0

ipv6 address 1:2::2/64

ipv6 eigrp 1

end

R2#sh run | section ipv6 router

ipv6 router eigrp 1

R2#sh ipv6 eigrp topology

P 1:2::/64, 1 successors, FD is 2169856

via Connected, Serial4/0

Named modeR1# conf t

Enter configuration commands, one per line. End

with CNTL/Z.

R1(config)#router eigrp foo

R1(config-router)#address-family ipv6 unicast auto 1

R1#sh run | section router eigrp

router eigrp foo

!

address-family ipv6 unicast autonomous-system 1

!

topology base

exit-af-topology

exit-address-family

R1#sh eigrp address-family ipv6 topology

P 1:1::/64, 1 successors, FD is 1735175958

via Connected, Serial4/0


IPv6 Shutdown

• When EIGRP was originally coded, the router process was defined to be Shutdown by default

– This was done because of the lack of network statement and the fact that interface commands could start EIGRP before filtering was defined

• This default behaviour has confused users and testers so we’ve changed it in the latest code

– Just be warned that the default behaviour for shutdown in EIGRP IPv6 is different in different versions!

BGP Overview


BGP Basics

• Runs over TCP (port 179)

• Path vector protocol

• Incremental updates

• Keepalive messages exchanged

AS 100 AS 101

AS 102

E

B D

A C

Peering


External Neighbours (eBGP)

AS 109

AS 110

131.108.0.0/16

A

B

150.10.0.0/16

131.108.10.0/24

.

1

.2

• BGP speakers in different AS

• Should be directly connected

• Configuration:

–Router B

router bgp 110network 150.10.0.0 neighbour 131.108.10.1 remote-as 109

–Router A

router bgp 109network 131.108.0.0 neighbour 131.108.10.2 remote-as 110


Internal Neighbours (iBGP)• BGP peer within the same AS

• Not required to be directly connected

• iBGP neighbours should be fully meshed!

AS 100

A

E

B

D

Configuration:

Router B

router bgp 109neighbour 131.108.20.2 remote-as 109

Router A

router bgp 109neighbor 131.108.20.1 remote-as 109

BGP Troubleshooting


R2#show ip bgp summaryBGP router identifier 2.2.2.2, local AS number 1

BGP table version is 1, main routing table version 1

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State

1.1.1.1 4 1 0 0 0 0 0 never Active

3.3.3.3 4 2 0 0 0 0 0 never Idle

Peer Establishment Issues

• Both peers are having problems–State may change between active, idle and connect

R2#sh run | begin bgp

router bgp 1

bgp log-neighbor-changes

neighbor 1.1.1.1 remote-as 1


AS 1

AS 2

R1

IBGP

EBGP

1.1.1.1 2.2.2.2

3.3.3.3?

?

R2

R3


Peer Establishment—IBGP• Assume that IP connectivity has been checked

• Check TCP to find out what connections we are accepting

R2#show tcp brief all

TCB Local Address Foreign Address (state)

005F2934 *.179 3.3.3.3.* LISTEN

0063F3D4 *.179 1.1.1.1.* LISTEN

We are listening for TCP connections for port 179 for the

configured peering addresses only!

R2#debug ip tcp transactions

TCP special event debugging is on

R2#

TCP: sending RST, seq 0, ack 2500483296

TCP: sent RST to 4.4.4.4:26385 from 2.2.2.2:179

Remote is trying to open the session from 4.4.4.4 address …

AS 1

AS 2

R1

IBGP

EBGP1.1.1.1 2.2.2.2

3.3.3.3?

?

R2

R3


Peer Establishment—iBGP

What about us ?R2#debug ip bgp

BGP debugging is on

R2#

BGP: 1.1.1.1 open active, local address 4.4.4.5

BGP: 1.1.1.1 open failed: Connection refused by remote host

We are trying to open the session from 4.4.4.5 address…R2#sh ip route 1.1.1.1Routing entry for 1.1.1.1/32

Known via "static", distance 1, metric 0 (connected)* directly connected, via Serial1

Route metric is 0, traffic share count is 1

R2#show ip interface brief | include Serial1Serial1 4.4.4.5 YES manual up up

AS 1

AS 2

R1

IBGP

EBGP1.1.1.1 2.2.2.2

3.3.3.3?

?

R2

R3


Peer Establishment—IBGP

• Source address is the outgoing interface towards the destination but peering in this case is using loopback interfaces!

• Force both routers to source from the correct interface

• Use “update-source” to specify the loopback when loopback peering

R2#

router bgp 1


neighbor 1.1.1.1 update-source Loopback0



AS 1

AS 2

R1

IBGP

EBGP1.1.1.1 2.2.2.2

3.3.3.3?

?

R2

R3


Peer Establishment—Symptoms

• R1 is established now

• The EBGP session is still having trouble!

R2# sh ip bgp summary

BGP router identifier 2.2.2.2, local AS number 1

BGP table version is 1, main routing table version 1

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

1.1.1.1 4 1 7 7 1 0 0 00:00:24 3

3.3.3.3 4 2 0 0 0 0 0 never Idle

AS 1

AS 2

R1

IBGP

EBGP1.1.1.1 2.2.2.2

3.3.3.3?

?

R2

R3


R2#ping 3.3.3.3



!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/8 ms

Peer Establishment—EBGP

• Trying to load-balance over multiple links to the eBGP peer

• Verify IP connectivity–Check the routing table

–Use ping/trace to verify two way reachability

Routing Towards Destination Correct, but…

AS 1

AS 2

R1

IBGP

EBGP1.1.1.1 2.2.2.2

3.3.3.3?

?

R2

R3


Peer Establishment—EBGPR2# ping ip

Target IP address: 3.3.3.3

Repeat count [5]:

Datagram size [100]:

Timeout in seconds [2]:

Extended commands [n]: y

Source address or interface: 2.2.2.2

Type of service [0]:

Set DF bit in IP header? [no]:

Validate reply data? [no]:

Data pattern [0xABCD]:

Loose, Strict, Record, Timestamp, Verbose[none]:

Sweep range of sizes [n]:



.....

Success rate is 0 percent (0/5)

No Route Back from our Peer!



R2#sh ip bgp neigh 3.3.3.3

BGP neighbor is 3.3.3.3, remote AS 2, external link

BGP version 4, remote router ID 0.0.0.0

BGP state = Idle

Last read 00:00:04, hold time is 180, keepalive interval is 60 seconds

Received 0 messages, 0 notifications, 0 in queue

Sent 0 messages, 0 notifications, 0 in queue

Route refresh request: received 0, sent 0

Default minimum time between advertisement runs is 30 seconds

For address family: IPv4 Unicast

BGP table version 1, neighbor version 0

Index 2, Offset 0, Mask 0x4

0 accepted prefixes consume 0 bytes

Prefix advertised 0, suppressed 0, withdrawn 0

Connections established 0; dropped 0

Last reset never

External BGP neighbor not directly connected.

No active TCP connection

Neighbour Added Route but Still Having Problems?…



• eBGP peers are normally directly connected–By default, TTL is set to 1 for eBGP peers

– If not directly connected, specify ebgp-multihop

• At this point, the session should come up

router bgp 1


neighbor 3.3.3.3 ebgp-multihop 255




• Still having trouble!– Connectivity issues have already been checked and corrected

R2#show ip bgp summary

BGP router identifier 2.2.2.2, local AS number 1

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

3.3.3.3 4 2 10 26 0 0 0 never Active



• If an error is detected, a notification is sent and the sessionis closed

• R3 is configured incorrectly– Has “neighbour 2.2.2.2 remote-as 10”

– Should have “neighbour 2.2.2.2 remote-as 1”

• After R3 makes this correction the session comes up

R2#debug ip bgp events

14:06:37: BGP: 3.3.3.3 open active, local address 2.2.2.2

14:06:37: BGP: 3.3.3.3 went from Active to OpenSent

14:06:37: BGP: 3.3.3.3 sending OPEN, version 4

14:06:37: BGP: 3.3.3.3 received NOTIFICATION 2/2

(peer in wrong AS) 2 bytes 0001

14:06:37: BGP: 3.3.3.3 remote close, state CLOSEWAIT

14:06:37: BGP: service reset requests

14:06:37: BGP: 3.3.3.3 went from OpenSent to Idle

14:06:37: BGP: 3.3.3.3 closing


• *All examples are with “auto-summary”enabled

• Basic network statementR1# show run | begin bgp

network 6.0.0.0

• BGP is not originating the route???R1# show ip bgp | include 6.0.0.0

R1#

• Do we have a component route?R1# show ip route 6.0.0.0 255.0.0.0 longer

R1#

Route Origination—Example I


Route Origination—Example I

• As soon as the RIB has a component routeR1# show ip route 6.0.0.0 255.0.0.0 longer

6.0.0.0/32 is subnetted, 1 subnets

S 6.6.6.6 [1/0] via 20.100.1.6

• Bingo, BGP originates the route!!R1# show ip bgp | include 6.0.0.0

*> 6.0.0.0 0.0.0.0 0 32768 i


Route Origination—Example II

• Network statement with maskR1# show run | begin bgp

network 200.200.0.0 mask 255.255.252.0

• BGP is not originating the route???R1# show ip bgp | include 200.200.0.0

R1#

• Do we have the exact route?R1# show ip route 200.200.0.0 255.255.252.0

% Network not in table


Route Origination—Example II

• Nail down routes you want to originateR1#ip route 200.200.0.0 255.255.252.0 Null 0 200

• Check the RIBR1# show ip route 200.200.0.0 255.255.252.0


S 200.200.0.0 [1/0] via Null 0

• BGP originates the route!!R1# show ip bgp | include 200.200.0.0

*> 200.200.0.0/22 0.0.0.0 0 32768


Route Origination—Example III

• Trying to originate an aggregate routeR1#aggregate-address 7.7.0.0 255.255.0.0 summary-only

• The RIB has a component but BGP does not create the aggregate???R1# show ip route 7.7.0.0 255.255.0.0 longer


C 7.7.7.7 [1/0] is directly connected, Loopback 0

R1# show ip bgp | i 7.7.0.0

R1#


Route Origination—Example III• Remember, to have a BGP aggregate you need a

BGP component, not a RIB componentR1# show ip bgp 7.7.0.0 255.255.0.0 longer

R1#

• Once BGP has a component route we originatethe aggregatenetwork 7.7.7.7 mask 255.255.255.255

R1# show ip bgp 7.7.0.0 255.255.0.0 longer

*> 7.7.0.0/16 0.0.0.0 32768 i

s> 7.7.7.7/32 0.0.0.0 0 32768 i

• s means this component is suppressed due to the “summary-only” argument


Troubleshooting Tips

• “auto-summary” rules [default]– Network statement—must have component route (RIB)

– Network/Mask statement—must have exact route (RIB)

• “no auto-summary” rules– Always need an exact route (RIB)

• aggregate-address looks in the BGP table, not the RIB

• “show ip route x.x.x.x y.y.y.y longer”– Great for finding RIB component routes

• “show ip bgp x.x.x.x y.y.y.y longer”– Great for finding BGP component routes


Missing Route

• Automatic deny– AS_PATH contains our AS

– CLUSTER_LIST contains our CLUSTER_ID

– Martian NEXT_HOP

– Non-connected NEXT_HOP from directly connected eBGP peer

– NEXT_HOP belongs to us

– ORIGINATOR is us

• iBGP peers not in full mesh—BGP says we can not send a path from one iBGP peer to another iBGP peer


Missing Routes—Update Filters

• Determine which filters are applied to the BGP session

– show ip bgp neighbours x.x.x.x

– show run | include neighbour x.x.x.x

• Examine the route and pick out the relevant attributes

– show ip bgp x.x.x.x

• Compare the attributes against the filters



• Missing 156.1.0.0/16 in R1 (1.1.1.1)– Not received from R2 (142.102.10.2)

R1#show ip bgp neigh 2.2.2.2 routes

Total number of prefixes 0

R1 Is Not Receiving any Routes from R2 !



• Let’s take a look at R2

R2#show ip bgp neigh 1.1.1.1 advertised-routesNetwork Next Hop Metric LocPrf Weight Path

R2#show ip bgp 156.1.0.0BGP routing table entry for 156.1.0.0/16, version 1660

Paths: (1 available, best #1)Not advertised to any peer

Local0.0.0.0 from 0.0.0.0 (2.2.2.2)Origin IGP, metric 0, localpref 100, weight 32768, valid, sourced, local, best

Route Is Originated but Not Advertised!



• Time to check filters!

R2#show run | include neighbor 1.1.1.1


neighbor 1.1.1.1 filter-list 1 out

R2#sh ip as-path 1

AS path access list 1

permit ^$

Filter *Looks* Right !!!


R2#show ip bgp filter-list 1

R2#show ip bgp regexp ^$BGP table version is 1661, local router ID is 2.2.2.2Status codes: s suppressed, d damped, h history, * valid, > best, i - internal

Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path*> 156.1.0.0 0.0.0.0 0 32768 i


• But is it really what it looks like …

Nothing matches the filter-list!

Re-typing the regexp gives us the correct output


R2#show ip bgp regexp ^$

Nothing matches again! Let’s use the up arrow key and see where the cursor stops:

R2#show ip bgp regexp ^$End of line is at the cursor


• Let’s copy and paste the entire regexp line from the configuration

There is trailing white space at the end!!!

It is considered part of the regular expression and it changes its

meaning. Removing the whitespace fixes the issue



• “show ip community-list”–Displays the filter

• “show ip bgp community-list”–Displays BGP paths that match the filter

• “show ip prefix-list”–Displays the filter

–Prefix-list are generally easier to use than ACLs

• “show ip bgp prefix-list”–Displays BGP paths that match the filter



• “show route-map”–Displays the filter

• “show ip bgp route-map”–Displays BGP paths that match the filter

• “show access-list”–Displays the filter

• debug ip bgp update ACL–After going through the config, debug!

–Don’t forget the ACL


IPv6 NLRI in IPv4 - Problem

Router Arouter bgp 201

bgp router-id 192.168.30.1


!

address-family ipv6

neighbor 150.1.1.2 activate

network 2192:10::/48

!

BGP(1): 150.1.1.3 rcvd UPDATE w/ attr: nexthop ::FFFF:150.1.1.3, origin i, localpref 100, metric 0

BGP(1): 150.1.1.3 rcvd 2192:10::/48

BGP(1): no valid path for 2192:10::/48

BGP(1): 150.1.1.2 send UPDATE (format) 2192:10::/48, next ::FFFF:150.1.1.3, metric 0, path Local

Router A:

Router B:

AS 200150.10.0.0/16

2192:10::/48AS 201AS 301

A

C

D

B

150.1.1.1

150.1.1.2

2150:1:1::2150.1.1.3

2150:1:1::3


AS 200150.10.0.0/16

2192:10::/48AS 201AS 301

A

C

D

B

150.1.1.1

150.1.1.2

2150:1:1::2

150.1.1.3

2150:1:1::3




!

address-family ipv6

neighbor 150.1.1.2 activate

neighbor 150.1.1.2 route-map SETNH out

network 2192:10::/48

!

route-map SETNH permit 10

set ipv6 next-hop 2150:1:1::3

BGP(1): 2150:1:1::3 rcvd UPDATE w/ attr: nexthop 2150:1:1::3, origin i, localpref 100, metric 0

BGP(1): 2150:1:1::3 rcvd 2192:10::/48

BGP(1): Revise route installing 2192:10::/48 -> 2150:1:1::3 (::) to main IPv6 table

BGP(1): 150.1.1.2 send UPDATE (prepend, chgflags: 0x820) 2192:10::/48, next 2150:1:1::3, metric 0, path Local

Router A:

Router B:

IPv6 NLRI in IPv4 - Solution


AS 200150.10.0.0/16

192.10.0.0/24AS 201AS 301

A

C

D

B

150.1.1.2

2150:1:1::2

150.1.1.3

2150:1:1::3

IPv4 NLRI in IPv6(Global) - Problem



neighbor 2150:1:1::2 remote-as 301

!

address-family ipv4

neighbor 2150:1:1::2 activate

network 192.10.0.0

!

BGP(0): 2150:1:1::2 send UPDATE (format) 192.10.0.0/24, next 33.80.0.1, metric 0, path Local

BGP(0): 2150:1:1::3 rcvd UPDATE w/ attr: nexthop 33.80.0.1, origin i, localpref 100, metric 0

BGP(0): 2150:1:1::3 rcvd 192.10.0.0/24

BGP(0): no valid path for 192.10.0.0/24

Router A:

Router B:


AS 200150.10.0.0/16

192.10.0.0/24AS 201AS 301

A

C

D

B

150.1.1.1

150.1.1.2

2150:1:1::2

150.1.1.3

2150:1:1::3

IPv4 NLRI in IPv6(Global) - Solution




!

address-family ipv4


neighbor 2150:1:1::2 route-map SETNH out

network 192.10.0.0

!


set ip next-hop 150.1.1.3

BGP(0): 2150:1:1::2 send UPDATE (prepend, chgflags: 0x0) 192.10.0.0/24, next 150.1.1.3, metric 0, path Local

BGP(0): 2150:1:1::3 rcvd UPDATE w/ attr: nexthop 150.1.1.3, origin i, metric 0, path 10

BGP(0): 2150:1:1::3 rcvd 192.10.0.0/24

BGP(0): Revise route installing 1 of 1 routes for 192.10.0.0/24 -> 150.1.1.3(main) to main IP table

Router A:

Router B:


BGP(0): Can't advertise 192.10.0.0/24 to FE80::A8BB:CCFF:FE00:300 with NEXT_HOP 254.128.0.0

BGP(0): FE80::A8BB:CCFF:FE00:300 send UPDATE (format) 192.10.0.0/24, next 254.128.0.0, metric 0, path

Local

IPv4 NLRI in IPv6(Link Local) - Problem

AS 200150.10.0.0/16

192.10.0.0/24AS 201AS 301

A

C

D

B

150.1.1.1

150.1.1.2

2150:1:1::2

150.1.1.3

2150:1:1::3



neighbor FE80::A8BB:CCFF:FE00:300%E0 remote-as 301

!

address-family ipv4

neighbor FE80::A8BB:CCFF:FE00:300%E0 activate

network 192.10.0.0

!

Router A:

FE80::A8BB:CCFF:FE00:300

BGP(0): FE80::A8BB:CCFF:FE00:200 rcv UPDATE w/ attr: nexthop 254.128.0.0, origin i, metric 0, originator 0.0.0.0,

path 10, community , extended community BGP(0): FE80::A8BB:CCFF:FE00:200 rcv UPDATE about 192.10.0.0/24 -- DENIED due to: martian NEXTHOP;

Router B:



AS 200150.10.0.0/16

192.10.0.0/24AS 201AS 301

A

C

D

B

150.1.1.1

150.1.1.2

2150:1:1::2

150.1.1.3

2150:1:1::3

IPv4 NLRI in IPv6(Link Local) - Solution

BGP(0): FE80::A8BB:CCFF:FE00:300 send UPDATE (format) 192.10.0.0/24, next 150.1.1.2, metric 0, path Local

BGP(0): FE80::A8BB:CCFF:FE00:200 rcvd UPDATE w/ attr: nexthop 150.1.1.3, origin i, metric 0, path 10

BGP(0): FE80::A8BB:CCFF:FE00:200 rcvd 192.10.0.0/24

BGP(0): Revise route installing 1 of 1 routes for 192.10.0.0/24 -> 150.1.1.3(main) to main IP table

Router A:

Router B:




!

address-family ipv4


neighbor FE80::A8BB:CCFF:FE00:300 route-map SETNH out

network 192.10.0.0

!


set ip next-hop 150.1.1.3


BGP(1): Can't advertise 2192:10::/64 to FE80::A8BB:CCFF:FE00:200%Ethernet0/0 session 1 with NEXT_HOP FE80::A8BB:CCFF:FE00:100

BGP(1): FE80::A8BB:CCFF:FE00:200%Ethernet0/0 send UPDATE (format) 2192:10::/64, next ::, metric 0, path Local

AS 200150.10.0.0/16

AS 201AS 301

A

C

D

B

150.1.1.1

150.1.1.2

2150:1:1::2

150.1.1.3

2150:1:1::3

IPv6 NLRI in IPv6(Link Local) - Router A & B




!

address-family ipv6


network 2192:10::/48

!

Router A:


BGP(1): FE80::A8BB:CCFF:FE00:100%Ethernet0/0 rcvd UPDATE w/ attr: nexthop FE80::A8BB:CCFF:FE00:100

(FE80::A8BB:CCFF:FE00:100), origin i, metric 0, path 201BGP(1): FE80::A8BB:CCFF:FE00:100%Ethernet0/0 rcvd 2192:10::/64

BGP(1): Revise route installing 2192:10::/64 -> FE80::A8BB:CCFF:FE00:100 (FE80::A8BB:CCFF:FE00:100) to main IPv6 table

Router B:

2192:10::/48E



BGP(1): Can't advertise 2192:10::/64 to 2151:1:1::5 with NEXT_HOP FE80::A8BB:CCFF:FE00:100

BGP(1): 2151:1:1::5 send UPDATE (format) 2192:10::/64, next 2151:1:1::2, metric 0, path 10

AS 200150.10.0.0/16

AS 201AS 301

A

C

D

B2151:1:1::5

150.1.1.2

2150:1:1::2

150.1.1.3

2150:1:1::3

IPv6 NLRI in IPv6(Link Local) & iBGP Between Router B & E Using Global Unicast

Router Brouter bgp 301




!

address-family ipv6



!

Router B:


BGP(1): 2151:1:1::2 rcvd UPDATE w/ attr: nexthop 2151:1:1::2, origin i, localpref 100, metric 0, path 10

BGP(1): 2151:1:1::2 rcvd 2192:10::/64

BGP(1): Revise route installing 2192:10::/64 -> 2151:1:1::2 (::) to main IPv6 table

Router E:

2192:10::/48E



Summary

• Overview of OSPF, EIGRP and BGP

• Different troubleshooting commands and what to look for in those commands for these protocols

• Common issues in OSPF, EIGRP and multi protocol BGP networks; e.g., adjacency problems, routes missing from the routing table. Routes advertisement and origination issues and how to fix these issues.

• IPv6 related issues with these protocols and how to solve those issues

What We Learned?


More Information

• White Papers

• Web and Mailers

• Cisco Press

RTFB = “Read the Fine Book”


Give us your feedback and receive a

Cisco Live 2015 T-Shirt!

Complete your Overall Event Survey and 5 Session

Evaluations.

• Directly from your mobile device on the Cisco Live

Mobile App

• By visiting the Cisco Live Mobile Site

http://showcase.genie-connect.com/clmelbourne2015

• Visit any Cisco Live Internet Station located

throughout the venue

T-Shirts can be collected in the World of Solutions

on Friday 20 March 12:00pm - 2:00pm

Complete Your Online Session Evaluation

Learn online with Cisco Live! Visit us online after the conference for full

access to session videos and

presentations. www.CiscoLiveAPAC.com

http://showcase.genie-connect.com/clmelbourne2015

http://www.ciscoliveapac.com/

Thank you.

Troubleshooting Routing Protocols - alcatron.net Live 2015 Melbourne/Cisco...• OSPF not enabled on...

Documents

Transcript of Troubleshooting Routing Protocols - alcatron.net Live 2015 Melbourne/Cisco...• OSPF not enabled on...