Multihop eBGP peering or VPN based eBGP peering
Hello all, Any idea why more companies don't offer eBGP peering / multi hop peering? Its very common for providers to offer single or double hop peering, so why not 5 or 10 hops? In many cases people find it logical to perform single or double hop peering, why is peering any greater always frowned upon. I understand the logic that you can't control the path beyond a point, however I still see numerous advantages. One obvious advantages one is, imagine you east coast data centre and you had a eBGP peering session with a west coast router, you'd be able to control ingress via the west coast. (aka routing around an region outage that is effecting ingress) For example during the last hurricane around New Jersey, numerous tier 1's were down towards the atlantic and every peer for the atlantic was effected. One could have just made the ingress via the west coast the logical route. Thoughts? Mike -- Michael McConnell WINK Streaming; email: michael@winkstreaming.com phone: +1 312 281-5433 x 7400 cell: +506 8706-2389 skype: wink-michael web: http://winkstreaming.com
Perhaps I am missing something from your advantage list, but why would you want to exchange routing information with a network to which you don't have a connection due to a local failure? I think you are attempting to abstract routing from the underlying physical infrastructure a bit too much. If the power is out in the carrier pop to which you are connected, they don't have a way to give you traffic so why would a multi-hop session help. BGP being down is rarely something that happens on its own, it is typically due to something far more physical (router failure, pop outage, circuit outage, etc). John -----Original Message----- From: Michael McConnell [mailto:michael@winkstreaming.com] Sent: Sunday, June 16, 2013 5:40 PM To: nanog@nanog.org Subject: Multihop eBGP peering or VPN based eBGP peering Hello all, Any idea why more companies don't offer eBGP peering / multi hop peering? Its very common for providers to offer single or double hop peering, so why not 5 or 10 hops? In many cases people find it logical to perform single or double hop peering, why is peering any greater always frowned upon. I understand the logic that you can't control the path beyond a point, however I still see numerous advantages. One obvious advantages one is, imagine you east coast data centre and you had a eBGP peering session with a west coast router, you'd be able to control ingress via the west coast. (aka routing around an region outage that is effecting ingress) For example during the last hurricane around New Jersey, numerous tier 1's were down towards the atlantic and every peer for the atlantic was effected. One could have just made the ingress via the west coast the logical route. Thoughts? Mike -- Michael McConnell WINK Streaming; email: michael@winkstreaming.com phone: +1 312 281-5433 x 7400 cell: +506 8706-2389 skype: wink-michael web: http://winkstreaming.com
Perhaps I am missing something from your advantage list, but why would you want to exchange routing information with a network to which you don't have a connection due to a local failure? I think you are attempting to abstract routing from the underlying physical infrastructure a bit too much. If the power is out in the carrier pop to which you are connected, they don't have a way to give you traffic so why would a multi-hop session help.
BGP being down is rarely something that happens on its own, it is typically due to something far more physical (router failure, pop outage, circuit outage, etc).
any time routing signaling comes from a source to which you can not directly deliver payload you will be in a load of grief when something goes wrong. abjure route servers. route reflectors should be in the data plane, ... in general, when layer N is not congruent with layer N-1 (and N+1), you have a recipe for exciting times. and well run ops is not supposed to be exciting. randy
route reflectors should be in the data plane, ... I believe in modern networks data-plane and control-plane(s) should be separated as it provides for great scalability and versatility the drawback of course is a more complex system to manage.
more complex systems scale poorly, break easily, and are hard to debug. oops! all my competitors should have such 'modern networks.' randy
-----Original Message----- From: Randy Bush [mailto:randy@psg.com] Sent: Monday, June 24, 2013 2:32 PM To: Adam Vitkovsky Cc: 'John van Oppen'; nanog@nanog.org Subject: Re: Multihop eBGP peering or VPN based eBGP peering
route reflectors should be in the data plane, ... I believe in modern networks data-plane and control-plane(s) should be separated as it provides for great scalability and versatility the drawback of course is a more complex system to manage.
more complex systems scale poorly, break easily, and are hard to debug. oops! all my competitors should have such 'modern networks.'
randy
Well in the context of RRs complex systems have virtually endless scalability, they are built with redundancy in mind so they don't break or don't break easily and well as far as the debug goes, I think it's a matter of how well are the Operations educated by the architects. adam
Any idea why more companies don't offer eBGP peering / multi hop
-----Original Message----- From: Michael McConnell [mailto:michael@winkstreaming.com] Sent: Sunday, June 16, 2013 7:40 PM To: nanog@nanog.org Subject: Multihop eBGP peering or VPN based eBGP peering peering? Its very common for providers to offer single or double hop peering, so why not 5 or 10 hops? In many cases people find it logical to perform single or double hop peering, why is >peering any greater always frowned upon. I understand the logic that you can't control the path beyond a point, however I still see numerous advantages. The norm has always been if you are peering with someone you have router in the location you are peering. Thus, direct connection!!! But I've seen folks do what you are describing but in terms of their own networks thru use of GRE Tunnels. The main point of peering is having better connectivity and dropping traffic directly or closest to its destination.
One obvious advantages one is, imagine you east coast data centre and you had a eBGP peering session with a west coast router, you'd be able to control ingress via the west coast. (aka routing around an region outage that is effecting ingress) For >example during the last hurricane around New Jersey, numerous tier 1's were down towards the atlantic and every peer for the atlantic was effected. One could have just made the ingress via the west coast the logical route.
I do see this advantage being an obvious workable logical one. However, large providers typically have their own network (layers 1-3) coast to coast if were talking USA. But in the case of the hurricane situation many were without power so you can have a router west coast and announce from that router but how will you get traffic back to east coast if that's your data center? You see you can have routers all over but if your data center (CDN) is without power you are done. Otis
On Jun 17, 2013, at 00:36 , "Otis L. Surratt, Jr." <otis@ocosa.com> wrote:
Any idea why more companies don't offer eBGP peering / multi hop peering? Its very common for providers to offer single or double hop peering, so why not 5 or 10 hops? In many cases people find it logical to perform single or double hop peering, why is >peering any greater always frowned upon. I understand the logic that you can't control the path beyond a point, however I still see numerous advantages.
The norm has always been if you are peering with someone you have router in the location you are peering. Thus, direct connection!!! But I've seen folks do what you are describing but in terms of their own networks thru use of GRE Tunnels. The main point of peering is having better connectivity and dropping traffic directly or closest to its destination.
First, inside your own network is not eBGP. iBGP has no hop limitation (well, 255). If you have you seen someone do eBGP "inside their own network", they were actually doing it between two separate networks they owned. If you saw someone do eBGP over a GRE tunnel, that is a direct connection, not multi-hop. [Cue discussion from last week about multiple islands in the same ASN.]
One obvious advantages one is, imagine you east coast data centre and you had a eBGP peering session with a west coast router, you'd be able to control ingress via the west coast. (aka routing around an region outage that is effecting ingress) For >example during the last hurricane around New Jersey, numerous tier 1's were down towards the atlantic and every peer for the atlantic was effected. One could have just made the ingress via the west coast the logical route.
I do see this advantage being an obvious workable logical one. However, large providers typically have their own network (layers 1-3) coast to coast if were talking USA. But in the case of the hurricane situation many were without power so you can have a router west coast and announce from that router but how will you get traffic back to east coast if that's your data center?
You see you can have routers all over but if your data center (CDN) is without power you are done.
I do not see an advantage here. You are on the east coast and you want to re-direct traffic to the west coast, so you announce a prefix to a west coast router and ask it to propagate that prefix to its peers. How do you guarantee that router has a route back to the east coast for that prefix? Remember, a prefix announcement is a promise to deliver traffic to that prefix. You are suggesting asking a router to make a promise when that router has no guarantee of reachability. In your hurricane example, perhaps the west coast router reaches that prefix through one of the down east coast routers? Now you have blackholed that prefix when a router in, say, Chicago or Dallas would have announced it properly and had reachability. If you want to control where a prefix ingresses another network, first you need a transit relationship with that network. Most modern transit networks have community-based signaling, allowing you to do what you suggest and more (e.g. "prepend to peer $X" or "do not announce to peer $Y"). -- TTFN, patrick
-----Original Message----- From: Patrick W. Gilmore [mailto:patrick@ianai.net] Sent: Monday, June 17, 2013 1:37 AM To: NANOG list Subject: Re: Multihop eBGP peering or VPN based eBGP peering On Jun 17, 2013, at 00:36 , "Otis L. Surratt, Jr." <otis@ocosa.com> wrote:
First, inside your own network is not eBGP. iBGP has no hop limitation (well, 255). If you have you seen someone do eBGP "inside their own network", they were actually doing it between two separate networks they owned.
If you saw someone do eBGP over a GRE tunnel, that is a direct connection, not multi-hop. [Cue discussion from last week about multiple islands in the same ASN.]
I know eBGP is not iBGP. And yes, it was two separate ASNs. The point was in my experience, providers will prefer direction connection (you can physically plug into switch port or router port) to establish an eBGP session with you versus a GRE Tunnel. And some do not allow it.
I do not see an advantage here.
You are on the east coast and you want to re-direct traffic to the west coast, so you announce a prefix to a west coast router and ask it to propagate that prefix to its peers. How do you guarantee that router has a route back to the east coast for that >prefix?
Remember, a prefix announcement is a promise to deliver traffic to that prefix. You are suggesting asking a router to make a promise when that router has no guarantee of reachability. In your hurricane example, perhaps the west coast router reaches >that prefix through one of the down east coast routers? Now you have blackholed that prefix when a router in, say, Chicago or Dallas would have announced it properly and had reachability.
If you want to control where a prefix ingresses another network, first you need a transit relationship with that network. Most modern transit networks have community-based signaling, allowing you to do what you suggest and more (e.g. "prepend to peer >$X" or "do not announce to peer $Y").
" You see you can have routers all over but if your data center (CDN) is without power you are done." Patrick, I was saying "done" in terms of being over, no more, not happening. :) But that was if you had all of the necessary network measures accounted for and content served via the east coast data center only. But I did forget to clarify (not CDN, or single served content). I was assuming the OP had the proper relationships network wise (transit/private/public peering) so it would work when you put it all together properly. I also, took into account the OP is a CDN so they probably have content distributed at a minimum in separate regions anyway so it wouldn't matter if the east coast DC is offline as the content is still reachable via the west coast data center which would null and void the OPs example. Like I explained, I was thinking the OP had the proper relationships to re-direct traffic to the west coast at that point that's why I said "I do see this advantage being an obvious workable logical one." I didn't say it was perfect but workable (work was still needed to complete it).
participants (6)
-
Adam Vitkovsky -
John van Oppen -
Michael McConnell -
Otis L. Surratt, Jr. -
Patrick W. Gilmore -
Randy Bush