SH> Date: Sun, 15 Feb 2004 16:50:02 +0000 SH> From: Sven Huster [ editted and reformatted for clarity ] SH> The core sends to R1, which believes the best path is via R2 SH> and sends it back to the core as that's the only way to reach SH> R2. Then the core again sends it to R1 and all the same SH> again. Yuck. SH> As this is a small network internally everything is routed SH> via static routes. Except for the smallest of networks, I try to avoid static routes. It's additional work and opportunity for error. Using BGP + TCP MD5 auth, OSPF auth, hardcoded ARP entries, per-port MAC address restrictions, prefix lists, route maps, etc., one can run a dynamic network and still keep security under control. SH> R1 and R2 have full BGP views from the transit providers as SH> well as partial view from the peers. Why not arrange the routers and switch in a single VLAN? (Or did I misunderstand your earlier ASCII-art diagram?) I usually use something like: 10.0.0.1/32 local sinkhole 10.0.0.2/28 virtual router (HSRP/VRRP; maybe XRRP now) 10.0.0.3/28 physical router #1 10.0.0.4/28 physical router #2 : : : : : : : 10.0.0.13/28 [routing] switch #2 10.0.0.14/28 [routing] switch #1 Let R1, R2, and R3 speak directly over ethernet without routing through core. If they already do, verify that you're setting nexthop correctly. Multihop routing sessions often can be made to work, but they're a tricky "house of cards". Remember, classic IP routing forwards to a { MAC addr | PVC | endpoint } based on destination IP addr. You can't do fancy rewriting at each hop; that's part of why PBR and label switching were invented. ;-) Note: I am _not_ suggesting PBR for this situation. SH> They [R1 and R2] run iBGP with R3 and the core. You have a partial mesh in which R1 and R2 do not exchange routes with each other? EBD> router bgp xxxx EBD> [no] bgp bestpath compare-routerid SH> All devices use the default settings in this respect. SH> R1-3 are Cisco routers, the core Extreme Alpine. Somewhere along the line Cisco changed the default from "bgp bestpath compare-routerid" to the converse. I forget when, although a quick Google search leads me to believe it was around 12.0/12.0S/12.0ST. I can't comment on Extreme. Again, though, I'm going out on a limb with this one. I'd bet on static routes, topology, and [lack of] IGP before BGP path selection algorithm. SH> It seems to be a temp problem, which we just figured out once Odd. SH> it went away based on netflow data and traffic dumps. So there SH> is no data available for this right now. If you catch any non-traceroute packets with expiring TTL, see if you can grab routing info from all the boxes involved. I'm confused how these devices are building their RIBs... Eddy -- EverQuick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _________________________________________________________________ DO NOT send mail to the following addresses : blacklist@brics.com -or- alfra@intc.net -or- curbjmp@intc.net Sending mail to spambait addresses is a great way to get blocked.