RE: Confussion over multi-homing
There are LOTS of folks out there that filter on the /24 boundry that your announcement will make it through to via ISP B.
I wish that this were true. But, it's false enough that you can't bet a business on it (for various thresholds of risk-tolerance/pain-avoidance).
has *ANYBODY* done this? -- Network Working Group T. Bates Request for Comments: 2260 Cisco Systems Category: Informational Y. Rekhter Cisco Systems January 1998 Scalable Support for Multi-homed Multi-provider Connectivity <...> 5.2. Further improvements <...> An enterprise border router would maintain EBGP peering not just with the directly connected border router of an ISP, but with the border router(s) in one or more ISPs that have their border routers directly connected to the other border routers within the enterprise. We refer to such peering as "non-direct" EBGP. An ISP that maintains both direct and non-direct EBGP peering with a particular enterprise would advertise the same set of routes over both of these peerings. An enterprise border router that maintains either direct or non-direct peering with an ISP advertises to that ISP reachability to the address prefix that was allocated by that ISP to the enterprise. Within the ISP routes received over direct peering should be preferred over routes received over non-direct peering. Likewise, within the enterprise routes received over direct peering should be preferred over routes received over non-direct peering. Forwarding along a route received over non-direct peering should be accomplished via encapsulation [RFC1773]. As an illustration consider an enterprise connected to two ISPs, ISP-A and ISP-B. Denote the enterprise border router that connects the enterprise to ISP-A as E-BR-A, and the ISP-A border router that is connected to E-BR-A as ISP-BR-A; denote the enterprise border router that connects the enterprise to ISP-B as E-BR-B, and the ISP-B border router that is connected to E-BR-B as ISP-BR-B. Denote the address prefix that ISP-A allocated to the enterprise as Pref-A; denote the address prefix that ISP-B allocated to the enterprise as Pref-B. E-BR-A maintains direct EBGP peering with ISP-BR-A and advertises reachability to Pref-A over that peering. E-BR-A also maintain a non-direct EBGP peering with ISP-BR-B and advertises reachability to Pref-B over that peering. E-BR-B maintains direct EBGP peering with ISP-BR-B, and advertises reachability to Pref-B over that peering. E-BR-B also maintains a non-direct EBGP peering with ISP-BR-A, and advertises reachability to Pref-A over that peering. When connectivity between the enterprise and both of its ISPs (ISP-A and ISP-B is up, traffic destined to hosts whose addresses were assigned out of Pref-A would flow through ISP-A to ISP-BR-A to E-BR- A, and then into the enterprise. Likewise, traffic destined to hosts whose addresses were assigned out of Pref-B would flow through ISP-B to ISP-BR-B to E-BR-B, and then into the enterprise. Now consider what would happen when connectivity between ISP-BR-B and E-BR-B goes down. In this case traffic to hosts whose addresses were assigned out of Pref-A would be handled as before. But traffic to hosts whose addresses were assigned out of Pref-B would flow through ISP-B to ISP-BR-B, ISP-BR-B would encapsulate this traffic and send it to E- BR-A, where the traffic will get decapsulated and then be sent into the enterprise. Figure 2 below describes this approach graphically. +---------+ +---------+ ( ) ( ) ( ISP-A ) ( ISP-B ) ( ) ( ) +---------+ +---------+ | | +--------+ +--------+ |ISP-BR-A| |ISP-BR-B| +--------+ +--------+ | /+/ | /\ | Pref-B /+/ | || | /+/ \./ Pref-A| /+/ non- /.\ || | /+/ direct | | /+/ EBGP | +------+ +-------+ |E-BR-A|-----------|E-BR-B | +------+ IBGP +-------+ Figure 2: Reachability information advertised via non-direct EBGP Observe that with this scheme there is no additional routing information due to multi-homed enterprises that has to be carried in the "default-free" zone of the Internet. In addition this scheme doesn't degrade in the presence of ISPs that filter out routes based ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ on the length of their address prefixes. Note that the set of routers within an ISP that maintain non-direct peering with the border routers within an enterprise doesn't have to be restricted to the ISP's border routers that have direct peering with the enterprise's border routers. The non-direct peering could be maintained with any router within the ISP. Doing this could improve the overall robustness in the presence of failures within the ISP. -- thanks, -- dima.
There are problems with this approach as well. In fact, in the last paragraph of section 5.1 the RFC states "The approach described in this section may produce less than the full Internet-wide connectivity in the presence of ISPs that filter out routes based on the length of their address prefixes." RFC2260 essentially states how routers using BGP can be configured to detect failure of connectivity to a particular ISP and then advertise that address out to another ISP. However, if the ISP that is still up filters at a /20 you still end up with the same problem. Actually the are two issues that must be addressed by any multi-homing solution. 1) How does someone get to me? (outside connectivity in) 2) If I use more than one address pool, how do I NAT with the correct (meaning up) ip address (inside connectivity out) While RFC 2260 does not solve these problems, it does lead me to think that a solution is possible - give me a few days to work on this and I'll report back. Dmitri Krioukov wrote:
has *ANYBODY* done this?
-- Network Working Group T. Bates Request for Comments: 2260 Cisco Systems Category: Informational Y. Rekhter Cisco Systems January 1998
Scalable Support for Multi-homed Multi-provider Connectivity
<...>
5.2. Further improvements
<...>
An enterprise border router would maintain EBGP peering not just with the directly connected border router of an ISP, but with the border router(s) in one or more ISPs that have their border routers directly connected to the other border routers within the enterprise. We refer to such peering as "non-direct" EBGP.
An ISP that maintains both direct and non-direct EBGP peering with a particular enterprise would advertise the same set of routes over both of these peerings. An enterprise border router that maintains either direct or non-direct peering with an ISP advertises to that ISP reachability to the address prefix that was allocated by that ISP to the enterprise. Within the ISP routes received over direct peering should be preferred over routes received over non-direct peering. Likewise, within the enterprise routes received over direct peering should be preferred over routes received over non-direct peering.
Forwarding along a route received over non-direct peering should be accomplished via encapsulation [RFC1773].
As an illustration consider an enterprise connected to two ISPs, ISP-A and ISP-B. Denote the enterprise border router that connects the enterprise to ISP-A as E-BR-A, and the ISP-A border router that is connected to E-BR-A as ISP-BR-A; denote the enterprise border router that connects the enterprise to ISP-B as E-BR-B, and the ISP-B border router that is connected to E-BR-B as ISP-BR-B. Denote the address prefix that ISP-A allocated to the enterprise as Pref-A; denote the address prefix that ISP-B allocated to the enterprise as Pref-B. E-BR-A maintains direct EBGP peering with ISP-BR-A and advertises reachability to Pref-A over that peering. E-BR-A also maintain a non-direct EBGP peering with ISP-BR-B and advertises reachability to Pref-B over that peering. E-BR-B maintains direct EBGP peering with ISP-BR-B, and advertises reachability to Pref-B over that peering. E-BR-B also maintains a non-direct EBGP peering with ISP-BR-A, and advertises reachability to Pref-A over that peering.
When connectivity between the enterprise and both of its ISPs (ISP-A and ISP-B is up, traffic destined to hosts whose addresses were assigned out of Pref-A would flow through ISP-A to ISP-BR-A to E-BR- A, and then into the enterprise. Likewise, traffic destined to hosts whose addresses were assigned out of Pref-B would flow through ISP-B to ISP-BR-B to E-BR-B, and then into the enterprise. Now consider what would happen when connectivity between ISP-BR-B and E-BR-B goes down. In this case traffic to hosts whose addresses were assigned out of Pref-A would be handled as before. But traffic to hosts whose addresses were assigned out of Pref-B would flow through ISP-B to ISP-BR-B, ISP-BR-B would encapsulate this traffic and send it to E- BR-A, where the traffic will get decapsulated and then be sent into the enterprise. Figure 2 below describes this approach graphically.
+---------+ +---------+ ( ) ( ) ( ISP-A ) ( ISP-B ) ( ) ( ) +---------+ +---------+ | | +--------+ +--------+ |ISP-BR-A| |ISP-BR-B| +--------+ +--------+ | /+/ | /\ | Pref-B /+/ | || | /+/ \./ Pref-A| /+/ non- /.\ || | /+/ direct | | /+/ EBGP | +------+ +-------+ |E-BR-A|-----------|E-BR-B | +------+ IBGP +-------+
Figure 2: Reachability information advertised via non-direct EBGP
Observe that with this scheme there is no additional routing information due to multi-homed enterprises that has to be carried in the "default-free" zone of the Internet. In addition this scheme doesn't degrade in the presence of ISPs that filter out routes based ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ on the length of their address prefixes.
Note that the set of routers within an ISP that maintain non-direct peering with the border routers within an enterprise doesn't have to be restricted to the ISP's border routers that have direct peering with the enterprise's border routers. The non-direct peering could be maintained with any router within the ISP. Doing this could improve the overall robustness in the presence of failures within the ISP. --
thanks, -- dima.
-- David Lott VP of Operations MSN Communications (303) 347-8303
On Fri, 15 Sep 2000, David Lott wrote: (for some reason RFC2260 refers to RFC1773 as describing encapsulation over non-direct-ebgp-learned-routes, I assume that they meant GRE, RFC1701/2)
RFC2260 essentially states how routers using BGP can be configured to detect failure of connectivity to a particular ISP and then advertise that address out to another ISP. However, if the ISP that is still up filters at a /20 you still end up with the same problem. That's not a problem, since you have a contractual relationship with this ISP, you can thwack them with the money-stick, and tell them to blow a hole in the filter for you.
The real problems I see with it are following: 1. You need to get both upstreams to agree to run GRE on their edge router. That requires bigger money-stick than fixing their filtering :) 2. It only protects you from failure of a link from you to upstream, not from upstream losing their connectivity, power, or flapping like crazy and getting dampened. In my experience, latter happened more often than first. :)
Actually the are two issues that must be addressed by any multi-homing solution. 1) How does someone get to me? (outside connectivity in) 2) If I use more than one address pool, how do I NAT with the correct (meaning up) ip address (inside connectivity out)
While RFC 2260 does not solve these problems, it does lead me to think that a solution is possible - give me a few days to work on this and I'll report back. This RFC doesn't address cases where you use NAT. See this if you do: http://www.ieng.com/warp/public/cc/pd/iosw/ioft/ionetn/tech/emios_wp.htm It uses DNS trickery to accomplish it. You CAN use RFC2260 trickery with it (half of RFC2260 is included there), but you don't have to, if you don't care about broken connections when one of links fails.
-- Alex Pilosov | http://www.acecape.com/dsl CTO - Acecape, Inc. | AceDSL:The best ADSL in Bell Atlantic area 325 W 38 St. Suite 1005 | (Stealth Marketing Works! :) New York, NY 10018 |
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Alex Pilosov Sent: Friday, September 15, 2000 5:30 PM To: David Lott Cc: Dmitri Krioukov; nanog@merit.edu Subject: Re: Confussion over multi-homing
On Fri, 15 Sep 2000, David Lott wrote:
(for some reason RFC2260 refers to RFC1773 as describing encapsulation over non-direct-ebgp-learned-routes, I assume that they meant GRE, RFC1701/2)
RFC2260 essentially states how routers using BGP can be configured to detect failure of connectivity to a particular ISP and then advertise that address out to another ISP. However, if the ISP that is still up filters at a /20 you still end up with the same problem. That's not a problem, since you have a contractual relationship with this ISP, you can thwack them with the money-stick, and tell them to blow a hole in the filter for you.
The real problems I see with it are following: 1. You need to get both upstreams to agree to run GRE on their edge router. That requires bigger money-stick than fixing their filtering :)
:)
2. It only protects you from failure of a link from you to upstream, not from upstream losing their connectivity, power, or flapping like crazy and getting dampened. In my experience, latter happened more often than first. :)
note that "non-direct ebgp" peering on the picture can actually be between e-br-a and *any* router in isp-b, not necessarily isp-br-b. this way your real problem 2 is solved.
Actually the are two issues that must be addressed by any multi-homing solution. 1) How does someone get to me? (outside connectivity in) 2) If I use more than one address pool, how do I NAT with the correct (meaning up) ip address (inside connectivity out)
While RFC 2260 does not solve these problems, it does lead me to think that a solution is possible - give me a few days to work on this and I'll report back. This RFC doesn't address cases where you use NAT. See this if you do: http://www.ieng.com/warp/public/cc/pd/iosw/ioft/ionetn/tech/emios_wp.htm It uses DNS trickery to accomplish it. You CAN use RFC2260 trickery with it (half of RFC2260 is included there), but you don't have to, if you don't care about broken connections when one of links fails.
-- Alex Pilosov | http://www.acecape.com/dsl CTO - Acecape, Inc. | AceDSL:The best ADSL in Bell Atlantic area 325 W 38 St. Suite 1005 | (Stealth Marketing Works! :) New York, NY 10018 | -- dima.
On Fri, 15 Sep 2000, Dmitri Krioukov wrote:
2. It only protects you from failure of a link from you to upstream, not from upstream losing their connectivity, power, or flapping like crazy and getting dampened. In my experience, latter happened more often than first. :)
note that "non-direct ebgp" peering on the picture can actually be between e-br-a and *any* router in isp-b, not necessarily isp-br-b. this way your real problem 2 is solved.
Not really. If the 'internet defaultless core' routers drop the route to ISP-B, then you are still completely screwed. -alex
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Alex Pilosov Sent: Friday, September 15, 2000 6:31 PM To: Dmitri Krioukov Cc: Alex Pilosov; nanog@merit.edu; David Lott Subject: RE: Confussion over multi-homing
On Fri, 15 Sep 2000, Dmitri Krioukov wrote:
2. It only protects you from failure of a link from you to upstream, not from upstream losing their connectivity, power, or flapping like crazy and getting dampened. In my experience, latter happened more often than first. :)
note that "non-direct ebgp" peering on the picture can actually be between e-br-a and *any* router in isp-b, not necessarily isp-br-b. this way your real problem 2 is solved.
Not really. If the 'internet defaultless core' routers drop the route to ISP-B, then you are still completely screwed.
oh, yeah. also, if isp-b gets suddenly evaporated, then i'm screwed even more... :) anyway, i think it would be safe to conclude that the answer to my initial question would be "no".
-alex -- dima.
participants (4)
-
Alex Pilosov -
David Lott -
Dmitri Krioukov -
Roeland M.J. Meyer