At 19:09 -0400 7/23/97, netops wrote:
Does anyone know of any non-ISP companies that have decided to multi-home? Is this a major trend for non-ISP companies running mission-critical applications on the Internet?
So far, I only know of a couple, with PointCast being one of them.
Thanks, Lincoln Silver FlyCast Communications
Let me try to answer this somewhat broadly, and still stay within the NANOG charter. I believe it is an operational issue to consider any organization, whether ISP or not, that can affect internet routing. An enterprise that generates BGP announcements can affect such routing just as well as a service provider. A simple answer: yes, I see some form of complex Internet connectivity (I'm avoiding the term multi-homing due to ambiguity) in large enterprises (Fortune 500 roughly) and in enterprises for which Internet access is mission-critical, regardless of their overall size. Mergers and acquisitions, where the joined enterprises each had their own Internet access, often mean complex connectivity, at least for a transition period. Consolidation of separate divisional networks also creates this situation. I've found a very frequent case to be where a large enterprise decides that Internet access should be available corporate-wide, but their research labs have had Internet access for years -- and it works, as opposed to the new corporate connection that at best is untried. Different people use terms such as "multi-homing" in different ways. Let me propose a taxonomy I use in teaching design classes. I have had real networks that used all of these cases. 1. Single-homed. Enterprise generally does not have ASN; all its advertisements are made through its ISP. Uses default routes to the ISP. The customer is primarily concerned with protecting against link or router failures, rather than failures in the ISP routing system. 1.1 Single-homed, single-link. A single path to the ISP. 1.2 Single-homed, balanced link. Multiple parallel paths from a single customer router to an router. Protects against link failures. The single customer router constraint allows this router to do round-robin packet-level load balancing across the multiple links, for resiliency and possibly additional bandwidth. 1.3 Single-homed, multi-link. Separate paths from multiple customer routers to multiple ISP routers at different POPs. Default routes generated at each of the customer gateways are injected into the enterprise routing system, and the combination internal and external metrics are considered by internal routers in selecting the external gateway. [I generally recommend this to a customer who wants resiliency but wishes to avoid the complexity of BGP] Special Case 1.1+, 1.2+, 1.3+. While the customer is still single-homed, an AS upstream from the ISP has a routing policy that makes it necessary to distinguish routes originating in the customer from those originating in the ISP. In such cases, the enterprise may need to run BGP, or have the ISP run it on its behalf, to generate advertisements of the needed specificity. 2. Multi-homed. Enterprise connects to more than one ISP. May or may not use BGP. Wishes to protect against problems in the ISP routing system, and will accept additional complexity and router requirements to get this. May also have differing service agreements for Internet access for different divisions. 2.1 Multi-homed, primary/backup, single link. The enterprise connects to two or more ISPs from a single router, but has a strict policy that only one ISP at a time will be used for default. In an OSPF environment, this would be done by advertising defaults to both ISPs, but with different Type 2 external metrics. The primary ISP would have the lower metric. BGP is not necessary in this case. This easily can be extended to multi-link. 2.2 Multi-homed, differing internal policies. Assume OSPF interior routing. The main default for the enterprise comes from one or more ASBRs in Area 0, all routing to the same ISP. One or more organizations brought into the corporate network have pre-existing Internet access agreements with an ISP other than the corporate ISP, and wish to continue using this for their "divisional" Internet access. [I've seen this most often when a corporation decides to have general Internet access, but its research arm has long had its own Internet connectivity. Mergers and acquisitions also produce this case.] In this situation, an additional ASBR(s) are placed in the OSPF areas associated with the special-case, and this ASBR advertises default. Filters at the Area Border Router block the divisional ASBR's default from being advertised into Area 0, and the corporate default from being advertised into the division. 2.3 Multi-homed, "load shared" with primary/backup. [Thanks to Paul Ferguson for the distinction between load balancing and load sharing.] While there still is a primary/backup policy, there is an attempt to make active use of both the primary and backup providers. The enterprise runs BGP, but does not take full Internet routing. It takes partial routing from the backup provider, and prefers the backup provider path for destinations in the backup provider's AS, and perhaps directly connected to that AS. For all other destinations, the primary provider is the preferred default. A less preferred default is defined to the second ISP, but this default is advertised generally only if connectivity is lost to the primary ISP. 2.4 Multi-homed, global routing aware. Multiple customer router receive a full routing table, and, using appropriate filtering and aggregation, advertise different destinations (i.e., not just default) internally. This requires BGP, and, unless dealing with a limited number of special cases, requires significantly more resources inside the organization. 3. Transit. While we usually think of this in terms of ISPs, some enterprises may provide Internet connectivity to strategic partners. They do not offer Internet connectivity on a general basis. 3.1 Full iBGP mesh. Connectivity and performance requirements are such that a full iBGP mesh is practical. 3.2 Scalable IBGP required. The limits of iBGP full mesh have been reached, and confederations, route reflectors, etc., are needed for growth. -------- Howard Berkowitz PSC International/Protocol Interface (Cisco/Bay/Digital training partners) -- personal opinions, and other appropriate disclaimers