I had hoped to be able to stay out of this particular argument, but several messages, notably Alan Hannan's, seem to need some commentary. I hope people will forgive editing lapses; the amount of revision time I had for this is nearly zero. There are two somewhat-related issues at play here. The first is the issue of peering vs transit, which goes to the heart of how BGP is used. The second is the issue of peer vs. customer and boils down to "cui bono?". I shall address them separately. People interested more in "peering economics" may want to search forward for "II." I have some issues with how BGP is used traditionally, most notably with respect to how people tend to consider ASes and how policies are denoted. Rather than pick on how things are done now, I'd rather describe how I think about BGP and the directions it ought, in my opinion, to evolve. Firstly, the important thing to note is that historically there has never been a perfect definition for "peering" vs "transit", although many people (me, in particular) tried to make things simple by treating most peers in one particular way and everyone else in a completely different way. However, given the cost of international connectivity and the various things ICM has done for other fednets, particularly during the NSFNET transition, this was never universally applied. Moreover, with multihoming being all the rage, it would be a stupid engineering philosophy that built things with the belief that a clean and simple peering/transit distinction would be permanent. All this has driven and continues to drive the evolution of BGP and the tools to manipulate what's transmitted to and accept from BGP neighbours of all sorts in interesting ways. Therefore, there is perhaps only a very blurry, connotation-laden definition for "peering" and "transit". Secondly, I like to think of a RIB and the NLRI that is exchanged via BGP in terms of a set of reachable items which can be described in terms of geometry, and which, in fact, describe a partial topology of the Internet. This thinking lends itself more towards calculus rather than algebra, yet most of the tools for manipulating what enters and exits a RIB or what NLRI is exchanged tends to be described much more algebraically. Moreover, it is not too much of a stretch to consider a RIB as something along the lines of a relational database, particularly if one is used to doing all sorts of kinky filtering and attribute modification. The bright thing about thinking along these lines is that typically anything that can be expressed in terms of relational calculus can be expressed in terms of the relational algebra. I would therefore not argue that the tools available for dealing with BGP in a very general sense are broken, only that they are unweildy. Given the difficulty, in two well-known implementations, of introducing a policy of, "from this neighbour accept a set of prefixes in which all prefixes are at least 24 bits long, and in which all prefixes in the range of 206.0.0.0 to 223.255.255.255 are at least 19 bits long", and then debugging and modifying it afterwards, the algebraic approach tends to be painful. This kind of thing is fairly common, too; there are two common implementations of, "to this peer, send only my customers' routes", hinging on the definition of "only my customers' routes". The first, used at provider A, is to keep a list of all the ASes with which A peers, and to prevent any prefixes originating in or behind each of these prefixes from being reannounced. The second, which appears to be more common, is when provider B maintains an explicit list of all the ASes which are downstream from B, and announce only prefixes originating from those ASes. There are some variations on approach A and B, some of which deal with prefixes as well as or instead of ASes. There is also a third approach, which is newer, and similar to approach A. Provider C tags all NLRI received from its peers with a community attribute, and ensures that nothing with that community attribute is passed on to its peers. With respect to accepting inbound announcements there are also multiple approaches. One, favoured by A, is to tweak the list of all peers' ASes to prevent any peer from announcing prefixes that another peer would otherwise announce. A second approach, which appears to have been abandoned everywhere but at the "edges", was to accept from any particular peer only specific, exact prefixes. This approach has mutated into, "allow only this particular mapping of prefix to originating AS", with some modifications to perform some AS-based operations along the lines of the next approach. This next approach is to specify a list of AS paths or fractional AS paths which are "acceptable" from any particular peer, and is in some use by large international transit providers. Finally, there is the "I accept everything" approach, which essentially does no inbound filtering from peers, because peers are expected to be trustworthy. In each of these cases, a simple policy of send everything, or send only my stuff, is fairly tractable. However, implementing policies such as: to peer P send all my stuff, and also X's stuff, since I do mutual back-up with X from peer P, accept all their stuff, and also X's stuff especially when X is a large number of providers, becomes awkward. Moreover, policies such as: to peer Q send everything that they're paying for transit to becomes awkward when considering how to propagate Q's prefixes towards all the places they're paying for transit to, but not towards all the other prefixes. One of the things I have thought about from time to time is doing a merge of gated and quel from University Ingres or the like. I would far prefer to match against several things in the RIB simultaneously to build up a set of prefixes to announce, or to match against several things in a received announcement in order to determine whether to install a particular prefix into the RIB to begin with, possibly tweaking particular attributes. In fact, this is done rather regularly now, using route-maps in IOS, however it's ugly, particuarly as complexity increases, and one wants to do many operations on large numbers of sets of incoming NLRI, such as modifying local preferences to prefer longer AS paths which are to be preferred as backup or primary paths, rewriting MEDs, making exceptions in prefix-length filters, and the like. In a sense, this appraoches the core of a series of discussions elsewhere, revolving around "traditional" implementations of route-selection. I am a member of the Church of Explicit Selection, where the people controlling a particular set of routers should be able to make their own choices of what routes to use and what routes to propagate. The current crusade involves ripping apart MED, prefix length and AS path length as selectors, and making it possible to use these to modify a local metric (i.e., local preference). The traditionalists sometimes get hung up on what should be done in the case of ties, however, there are a couple of things I think there is broad agreement upon: -- prefix length should not be the ultimate determinant (The folks who have been having problems with another provider announcing subnets of their aggregates might like this...) -- AS paths should be little more than a trail of breadcrumbs used to prevent announcement loops, and AS path _length_ should not necessarily affect a routing decision -- AS paths could benefit from a per-AS metric which could help in the use of AS path _content_ to modify the local metric. (The idea is that one may want to attract traffic towards a particular AS1.AS2 pair or deflect it away from a particular AS3.AS4 pair) -- MEDs seem to be used in three ways: i. to attract traffic to one or the other box where there are multiple boxes in use by the same AS at an exchange point (defeating the use of lowest-router-ID as tiebreaker) ii. to attract traffic towards one exchange point or the other or to repel traffic away from one exchange point or the other iii. transmitting information about one's internal topology to a peer, so that the peer can adjust routing decision- making appropriately (There has been some debate about whether overloading attributes with multiple semantic behaviours is stupid or useful; I am of the former opinion, and think that each of these could usefully be its own attribute) Moreover, MED use has changed and likely will change again, and the current standard interpretation of MED is not quite right. Each of these changes is principally driven by the evolution of the way routes acquired by BGP are chosen and propagated, and reflect a move towards treating BGP routing information as a sort of relational database. If the tools for choosing and propagating BGP routing information likewise evolve towards a relational set theory, the distinction between "peer" and "transit" as they tend to be thought of these days could easily be thrown out the window. I would like this; I like a model wherein it becomes a combination of engineering sensibility and business decision-making about what is routed where, and that requires an easy to express a program that does integration and differentiation of sets of NLRI, based on large numbers of attributes, from individual ASes, fractional and complete AS paths, prefixes, mask length, announcing router, recent stability, internal and external topology, time of day and phase of moon, and uses these sets when generating a RIB and when propagating NLRI onwards, and other operations, such as forming aggregates. Current tools does not make this easy, even for people who know what they're doing. Finally, all of this leads me to conclude that on a purely technical level, the denotative difference between "peer" and "transit" is so obscured as to be meaningless, except that perhaps with current tools, "transit" is easy to configure and "peering" is not. I would prefer, therefore, not to think of these as technical terms at all, but rather economic ones. This of course leads to the next thing: II. Economically, the definition of "peering" vs "transit" is easy. The former is currently free, while the latter is not. Alternatively, the latter is what one sells to customers, the latter is what people would like to sell, but can't figure out how to price properly. I like considering the following simple equation: Value == Bandwidth + Reachability + Service Until recently, the bandwidth available at an exchange point was impossible to limit; an aggressive sender could swamp a particular provder's connection, as could an aggressive receiver. Moreover, as people have been discussing, without the assistance of an EP operator, there is no way to prevent people from sending traffic into your network without your permission, and EP operators don't seem to want to wade into the issues that have surfaced on the NANOG list today and yesterday. With some development work that stemmed out of a conversation with Fred Baker in 11.2 on its way, this problem is softened somewhat; with a crunchy IOS box, one can specify rate limits on subinterfaces. If traffic across a particular subinterface exceeds a threshold, the router will implement a dropping strategy. This has enormous utility not only to people selling fractional DS3s and the like, but also to people at LAN-based EPs. This should add to any LAN-based EP customer the one interesting feature of ATM-based EPs: the ability to select one's peers and to control how much bandwidth they can use. It also should allow one to _monitor_ how much bandwidth each peer is consuming. One can therefore control the Bandwidth variable in my simple equation, and do interesting things. One might consider that for purposes of traffic planning, 256kbps or 1.5Mbps might be made available for free to someone at an exchange point who is willing to accept a reduced level of Service compared to a more traditional customer. If a "peer" at an EP is using more than that amount of bandwidth, one might want to negotiate an exchange of money. For example if A and B meet at an EP and are running into the drop threshold because of the amount of traffic being exchanged, the one with the screaming customers may be asked to pay a certain amount of money per increment of additional bandwidth. This may always be B's customers, or it may flip-flop between A's customers and B's customers over time, and thus should be considered a business negotiation on both parties' part. Finally, at some point it may be observed that X is a large enough chunk of Y's total exchange-point traffic that migrating that traffic elsewhere, perhaps to a private peering, would appear to make engineering and possibly business sense. Now comes the fun bit: reachability. I would consider that if A and B are peering and B is paying some fee for some level of maximum bandwidth, that B is a customer of A whether one wants to use that term in marketing literature or not. In this case it seems much easier for A to treat B as any other customer, providing "transit" (that is, don't do much by way of filtering). Alternatively, if there is no money changing hands, it seems prudent for each of A or B to do some filtering to avoid providing "transit". The scope of that filtering is the heart of the long discussion of charging for routing which has been going on in various places for some time now, and would take too long for me to get into here. However, it's clear that reachability has value, and that it should be possible to determine a price point for the exchange of that reachability, in theory. However, I like the simple case, and figure it should solve some current heartburns in relatively short order. If big providers were to come up with a "product" wherein people wanting to talk at exchange points could easily be configured up as customers, with prices being determined by the maximum amount of bandwidth available before dropping begins and the amount of response-time one wants in the escalation path at the larger provider's NOC, this may solve the majority case of small provider frustrations with the Effectively No Peering At All policies of some of the larger providers. This appears to be a win for all parties (including Cisco, who gets to sell 75xx-es that can handle the bandwidth-limitation feature, and the various proposals and counter proposals wrt how to deal with the cost and price of reachability). The large parties get to make some income to at least offset the costs of maintaining network infrastructure at and around EPs, and can bring up extra connectivity without losing engineering control and oversight of the infrastructure and traffic flows, the EP operators get more interest in EPs since there will be less hesitation on the part of large parties to appear at them, the small providers get to buy cheaper, supported service while calling it "peering", and can control how their traffic moves around at EPs, and can complain if they are paying a particular amount of money for a service they are not getting because of capacity problems within large providers. I expect to see this sort of thing rolled out fairly soon, although I no longer know who will be the first to announce it formally. Informally, this type of service has been provided to some degree (modulo bandwidth controls) by a fair number of parties at EPs, notably CAIS. Sean.