The last company I worked for moved to eBGP Multi-Hop where there were two connections to the same provider (same AS). This allowed them to utilize both links in both directions vs only one link in one direction and have failover. As you have mentioned link state detection gets a bit crazy with this. If you have a MetroE connection (for example) with multiple segments, this could be problematic. If your side of the link goes down, then you stop sending traffic to the provider, but the provider still tries to send traffic to you. If a segment in the middle goes down, then neither side stop sending traffic. Due to the fact that the BGP session is still up, and the interface on your router is still up, BGP sees the link as a valid path. However there is a fix for this. If your provider supports it that is. Ethernet OAM (Ethernet Operations, Administration, and Management) will allow you to monitor the connection on Layer 2 end to end and not switch to switch. If any part of the link breaks, OAM brings your and the other side of the link down, telling BGP that the link is no longer usable, therefore avoiding the issues above. If you are using a POS, MPLS, or other similar technology, then the issues talked about above are either less of an issue, or not an issue at all. The biggest problem with multi segment Ethernet links is that you need OAM to reliably run eBGP Multihop and OAM isn't supported by a lot of providers (mainly because it requires a newer software version). Hope this helps. On 9/2/10 5:30 AM, Graham Beneke wrote:

I have been asked to investigate moving an entire network to multi-hop on all the eBGP sessions. Basically all upstreams, downstreams and peers will eBGP with a route reflector located in the core. This RR will be some kind of quagga or similar box. The dev guys want to be able to poke at the BGP feeds directly and do *magic* that standard router aren't capable of.

My gut feel is that this is a bad idea. Besides anything else it makes sane link state detection very challenging - especially where we have multiple sessions with a peer.

Is their any BCP or operational experience that agrees or disagrees with my gut. ;-)

-- Steve King Senior Linux Engineer - Advance Internet, Inc. Cisco Certified Network Associate CompTIA Linux+ Certified Professional CompTIA A+ Certified Professional

On 02/09/2010 10:30, Graham Beneke wrote:

I have been asked to investigate moving an entire network to multi-hop on all the eBGP sessions. Basically all upstreams, downstreams and peers will eBGP with a route reflector located in the core. This RR will be some kind of quagga or similar box. The dev guys want to be able to poke at the BGP feeds directly and do *magic* that standard router aren't capable of.

My gut feel is that this is a bad idea. Besides anything else it makes sane link state detection very challenging - especially where we have multiple sessions with a peer.

Of course, this sort of thing is usually great fun and seems like a Very Good Idea At The Time. You get your cool configuration in place with lots of local hax and the network hums along. Then the developer who wrote the hax leaves because of something or another. And the person who configured the box leaves due to management politics, and then the Windows IT support person takes over, along with the smart person on the front-line tech support desk. Then you hit your first major security bug with your local route reflector and the vendor patch causes your configuration to break horribly. Then hilarity ensues. I've seen extreme local messing-around-with-systems at some companies. Hilarity ensued. But there is a silver lining to all this: all these companies learned from their stupidity and never did things like that again. At least, the ones which didn't go bust. As regards collapsing all your bgp requirements into a single BGP box, well, good luck with that. Can I recommend you call this box "spof.apolix.co.za"? It seems quite appropriate. [You could have another single box called "ospf.apolix.co.za", which dealt with all your ospf requirements... just a thought.] Incidentally, I presume your devs have found some way of patching quagga in memory so that every time they write a new local hack or need to fix the previous one, they don't have to bring down the entire network in order to bring it into production? That would bring the entire experiment to a new level of coolness. Anyway, I wish you well with this experiment in the future of your company's existence. Nick

On 2 Sep 2010, at 5:30 PM, Graham Beneke wrote:

I have been asked to investigate moving an entire network to multi-hop on all the eBGP sessions. Basically all upstreams, downstreams and peers will eBGP with a route reflector located in the core. This RR will be some kind of quagga or similar box. The dev guys want to be able to poke at the BGP feeds directly and do *magic* that standard router aren't capable of.

My gut feel is that this is a bad idea. Besides anything else it makes sane link state detection very challenging - especially where we have multiple sessions with a peer.

Is their any BCP or operational experience that agrees or disagrees with my gut. ;-)

-- Graham Beneke

Even in lab environments I would say that eBGP multihop is a bad idea. Production networks that try to make money should not embark on such a mission. Trust your gut :) If some box wants to take a feed of routes, it should just iBGP peer with your RRs. Truman