Unless you are in "the swamp" - the old Class C, where I believe that they do accept /24's. Regards Marshall Eubanks Richard A Steenbergen wrote:

On Mon, Jul 15, 2002 at 05:10:28PM -0400, Ralph Doncaster wrote:

...

http://info.us.bb.verio.net/routing.html#PeerFilter

It seems if I were one of their customers they would accept my 66.11.168/23 announcement and re-announce it to their peers, but they won't accept it from any of their peers.

As a customer you pay them to announce your /23, as a peer you don't. Their line of logic is that if you are a peer of theirs you don't have to accept that /23 either.

...

Announcing a covering /20 along with the regional more specifics I have will only serve to increase the size of the routing table for most backbones, and lead to sub optimal routing in some cases since I'm announcing the more specifics due to geographical diversity.

Announce the /20 to your transit providers, and the more specifics with no-export. Verio's position is that they don't want to or need to hear your /23s unless you are a customer, and for the most part they are right.

-- T.M. Eubanks Multicast Technologies, Inc 10301 Democracy Lane, Suite 410 Fairfax, Virginia 22030 Phone : 703-293-9624 Fax : 703-293-9609 e-mail : tme@multicasttech.com http://www.multicasttech.com Test your network for multicast : http://www.multicasttech.com/mt/ Status of Multicast on the Web : http://www.multicasttech.com/status/index.html

Ralph, Welcome to the Internet. This has been the case for years, now. If you don't like it, you have a couple options. 1) You can go on hunger strike and chain yourself to the front door at NTT, demanding a change to the policy. 2) You can lobby Verio's peers to drop peering with them, unless they change their ways. 3) You can pay Verio to accept your routes. 4) You can live with it. May I suggest #4? I'm not a big fan of Verio's filtering policies, but as long as you announce the /20 as an aggregate, you'll be fine. - Daniel Golding

-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Ralph Doncaster Sent: Monday, July 15, 2002 6:37 PM To: nanog@merit.edu Subject: Re: verio arrogance

On Mon, 15 Jul 2002, Richard A Steenbergen wrote:

...

On Mon, Jul 15, 2002 at 05:10:28PM -0400, Ralph Doncaster wrote:

...

Announcing a covering /20 along with the regional more specifics I have will only serve to increase the size of the routing table for most backbones, and lead to sub optimal routing in some cases since I'm announcing the more specifics due to geographical diversity.

Announce the /20 to your transit providers, and the more specifics with no-export. Verio's position is that they don't want to or need to hear your /23s unless you are a customer, and for the most part they are right.

But I've broken my /20 into a /21 for Ottawa, a /22 for Toronto, a /23 for Montreal, and a /23 for expansion. I'm currently only getting transit in Toronto, but will have a second transit provider restored in Ottawa (I was using GT for a short while). While announcing the the /20 will my network is reachable for single-homed Verio customers, it won't provide the true best path that simply accepting the regional more specifics.

-Ralph

On Mon, Jul 15, 2002 at 05:55:51PM -0400, up@3.am wrote:

This is really old news...actually, I seem to recall that they would only accept /19 or shorter prefixes from former Class A & B space...I pressed Sprint for a /21 from the swamp (instead of the former Class A space /21 they initially assigned) because of Verio's policy, in fact. They must have softened the policy within the past year or so to /21 or shorter.

They tend to match the size of the smallest block assigned by the registries. Austin

That said, their current policy of refusing to accept de-aggregated prefixes from peers (while accepting such from paying customers) makes perfect sense, IMHO. Not arrogant, just a smart & reasonable business decision.

You could turn this around and ask what reasons there are to not filter the more specifics out from your peers... - kurtis -

That said, their current policy of refusing to accept de-aggregated prefixes from peers (while accepting such from paying customers) makes perfect sense, IMHO. Not arrogant, just a smart & reasonable business decision.

I have one downstream ISP customer that explicitly asked for "full BGP routes" to be written into the contract. Why Verio's customer's wouldn't want full routes makes no business sense to me. However a NANOG list subscriber was kind enough to help me get past Verio's NOC monkeys and get their filters updated to allow my announcements. -Ralph

In the referenced message, Ralph Doncaster said:

...

That said, their current policy of refusing to accept de-aggregated prefixes from peers (while accepting such from paying customers) makes perfect sense, IMHO. Not arrogant, just a smart & reasonable business decision.

I have one downstream ISP customer that explicitly asked for "full BGP routes" to be written into the contract. Why Verio's customer's wouldn't want full routes makes no business sense to me.

However a NANOG list subscriber was kind enough to help me get past Verio's NOC monkeys and get their filters updated to allow my announcements.

-Ralph

Accepting any route from anyone doesn't make much business sense to me. At least if you are interested in a quality network. If you'ld like, I'm sure multiple ISPs would be happy to send you all of their /32s. Verio's policy seems like a very responsible way to run a network. I'm saddened that more folks don't do filtering based upon RiR policy. Not announcing your largest aggregates is just plain stupid. If your peers are willing to accept more-specifics tagged no-export, with MEDs, then go for it, but the rest of us don't need them. I'm a little disappointed in Verio, if they really did decide to accept your unneccessarily deaggregated prefixes.

In the referenced message, Ralph Doncaster said:

...

I'm a little disappointed in Verio, if they really did decide to accept your unneccessarily deaggregated prefixes.

I'm a little disappointed you're wasting list bandwidth after this has been well discussed, and not a single post has offered a better technical alternative to de-aggregating my ARIN /20 (given my network topology).

-Ralph

Announce your largest aggregate, and announce more-specifics tagged no-export to those peers who agree to accept them? Upgrade the connectivity between your sites? (although I could have sworn I saw someone else make the above suggestions, already)

Say hypothetically it costs approximately $0.10 per route (in routing cpu/ram cost) per router. (average router cost $12,000 and 120,000 routes in the table). Lets also say hypothetically the average bgp speaking user has 3 bgp speaking routers. And lets say there are 25,000 AS's in use. By announcing as 4 blocks, instead of 1 aggregate, you are costing the bgp speaking community a total of $30,000. You are saving yourself money, at everyone else's expense. I know my math is not exactly correct, but it's an example to show why people get pissy about this sort of thing.. You aren't the biggest offender, but how should anyone draw an arbitrary line for "you are polluting too much" and "you are polluting, but to a reasonable extent". At this point, I have my whole network in NYC, so there isn't really any need for deaggregation. If/When my network expands to other cities, I will be planning things out to avoid deaggregating -- most likely with the deaggregate+no-export method. You could do a deaggregate+no-export method as well, even with your two different transit providers. You would just need to run ebgp-multihop to each of them from the opposite network, and announce your more-specifics there. Not a perfectly clean method, but at least it keeps your pollution local. --Phil -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Ralph Doncaster Sent: Friday, July 26, 2002 10:49 PM To: Stephen Griffin Cc: nanog@merit.edu Subject: Re: verio arrogance

...

I'm a little disappointed you're wasting list bandwidth after this has been well discussed, and not a single post has offered a better technical alternative to de-aggregating my ARIN /20 (given my network topology).

-Ralph

Announce your largest aggregate, and announce more-specifics tagged no-export to those peers who agree to accept them?

Which is worse than announcing just the more specifics to 2 different transit providers in 2 different cities.

Upgrade the connectivity between your sites? Technically sound, economically stupid. You offering to pony up the $5K/mth for an OC3 so I can have a redundant link between Ottawa and Toronto?

Besides the technical aspects of my network, I didn't see any proof that relaxed prefix filtering (and no I'm not saying accept /32's - the more specifics I got Verio to accept were /23's) would cause significant (i.e. >30%) routing table bloat when that was recently discussed. -Ralph

On Sat, Jul 27, 2002 at 09:14:35AM -0400, Ralph Doncaster wrote: [snip]

...

You could do a deaggregate+no-export method as well, even with your two different transit providers. You would just need to run ebgp-multihop to each of them from the opposite network, and announce your more-specifics there. Not a perfectly clean method, but at least it keeps your pollution local.

Then there is no ability for remote networks to choose the best path to my Toronto vs Ottawa networks (since the different transit providers would announce only the /20). Instead of using more router CPU/mem, this uses more network bandwidth than necessary (statistically speaking traffic has a 50% chance of going to the "wrong" transit provider). As well, for the ebgp-multihop to work wouldn't that require some extra static routes to be setup by my transit providers?

If you want to run seperate networks, run separate networks. Different ASes, the whole 9 yards; perhaps a re-reading of rfc1930 is in order? Back in the day, there was a promising local provider in my neck of the woods who has an AS per state for similar business reasons. They hit no filters, had no concerns, and when their business grew to the point of being able to clean things up, they did. No fuss, no muss and they're still in business with not chapter 11 in sight. Seems some folks would rather kick and scream. Perhaps people who weren't 'here' to work with and experience CIDR deployment don't think there's any harm in going aginst CIDR. Perhaps it is lack of experience in general engineering; one basic rule of thumb is to solve problems by avoiding the conditions which create them. By rushing headllong into activities that are -in even the most conservative terms- "debatable", you are inviting both known and unknown affects today and tomorrow. Using a reachbility protocol as an 'optimization' protocol for anything other than non-local affects (standardized well-known communities) is practice that is not guarenteed to work. I guess the point is, there's lots of "possible" activities in IP let alone BGP. If you presume that all which is technically possible is a good idea, then you are the only one responsible for the outcome. If you set yourself up for problems, especially ones that are known and trivially researchable, then don't gripe about it. Check who you're paying what. Google '"tragedy of the commons" internet route routing'. And to work to actully *solve* the problem, I'd suggest participating in PTOMAINE and the like. Rather than railing aginst current deployments, network operators, or the price of bits in CA your energy would be better spent nagging vendors to back drafts and to be ready to adopt new well-known-communities. Joe, thinking this belongs in a FAQ somewhere... -- Joe Provo Voice 508.486.7471 Director, Internet Planning & Design Fax 508.229.2375 Network Deployment & Management, RCN <joe.provo@rcn.com>

On Sat, 27 Jul 2002, Ralph Doncaster wrote:

...

If you want to run seperate networks, run separate networks. Different ASes, the whole 9 yards; perhaps a re-reading of rfc1930 is in order?

That brings us back to the discussion of PI space. If de-aggregating my /20 didn't work, then I'd either inefficiently use IP space in order to qualify for 2 /20's, or buy a defunct ISP or 2 to get a bunch of /24's in the 192-223 space.

Are you suggesting that either of those (which don't violate any RFCs) options are better than de-aggregating my /20?

The best solution is just as everybody here has suggested. Use the same provider for transit at both locations, announce your /20 normally, and your more specifics with no-export. Your response was something about "I guess you don't consider redundancy to be intelligent." What's stopping you from using the same two transit providers in both locations? Seems to me you don't value redundancy all that much. If you're willing to route through your small intra-AS link in case of local transit failure, yet you're not willing to route through it under normal circumstances, that would indicate to me that the link is not big enough for its purpose. I mean, at the end of the day, the argument boils down to you not wanting to foot the bill for 1) a fatter intra-AS link or 2) multiple transit providers at each location. It's ok if you want a bandaid, just don't try to tell anybody that your bandaid is actually a solid, best-practice solution. Andy xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Andy Dills 301-682-9972 Xecunet, LLC www.xecu.net xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Dialup * Webhosting * E-Commerce * High-Speed Access

On Sat, 27 Jul 2002, Ralph Doncaster wrote:

And your assumption about my Ottawa-Toronto link is wrong. I have a 100M point-to-point ethernet link between the cities. I have a 100M transit connection to Peer1 in Toronto, and have issued a letter of intent to a transit provider in Ottawa for a 100M link.

Ralph, if you have a 100m link between the two cities, why don't you use conditional announcements to only announce your /20 though Ottawa if your primary transit in Toronto goes down? Then, you only need to announce your /20 in Toronto, no need to deaggregate, and the whole issue is solved. Then, when you have the Ottawa 100m transit link up, you can announce your /20 to both transit providers all the time. Andy xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Andy Dills 301-682-9972 Xecunet, LLC www.xecu.net xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Dialup * Webhosting * E-Commerce * High-Speed Access

Setup a gre tunnel as your backup "path" for when the physical link goes down between Ottawa-Toronto if you can't afford a backup physical link. On Sat, 27 Jul 2002, Ralph Doncaster wrote:

...

On Sat, 27 Jul 2002, Ralph Doncaster wrote:

...

And your assumption about my Ottawa-Toronto link is wrong. I have a 100M point-to-point ethernet link between the cities. I have a 100M transit connection to Peer1 in Toronto, and have issued a letter of intent to a transit provider in Ottawa for a 100M link.

Ralph, if you have a 100m link between the two cities, why don't you use conditional announcements to only announce your /20 though Ottawa if your primary transit in Toronto goes down? Then, you only need to announce your /20 in Toronto, no need to deaggregate, and the whole issue is solved.

Then, when you have the Ottawa 100m transit link up, you can announce your /20 to both transit providers all the time.

That is roughly the intention. I also have to be able to announce the more specifics for when the Ottawa-Toronto link goes down. You could find in the archives my posts from a couple months back asking how to do this.

-Ralph

-- C. Jon Larsen Chief Technology Officer, Richweb.com (804.307.6939) SMTP: jlarsen@richweb.com (http://richweb.com/cjl_pgp_pub_key.txt) Richweb.com: Designing Open Source Internet Business Solutions since 1995 Building Safe, Secure, Reliable Cisco-Powered Networks since 1995

At 10:56 AM -0400 2002/07/27, Andy Dills wrote:

...

Are you suggesting that either of those (which don't violate any RFCs) options are better than de-aggregating my /20?

The best solution is just as everybody here has suggested. Use the same provider for transit at both locations, announce your /20 normally, and your more specifics with no-export.

I'm probably demonstrating my ignorance here (and my stupidity in stepping into a long-standing highly charged argument), but I'm completely missing something. For reasons of redundancy & reliability, even if you were to buy bandwidth in only one location, wouldn't you want to buy it from at least two different providers? If you buy bandwidth from two different providers at two different locations, this would seem to me to be a good way to provide backup in case on provider or one location goes Tango-Uniform, and you could always backhaul the bandwidth for the site/provider that is down. So, what am I missing? -- Brad Knowles, <brad.knowles@skynet.be> "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania.

At 3:51 PM -0400 2002/07/27, C. Jon Larsen wrote:

But with only 1 ISP link in each city (1 upstream) if he ever loses the link between the two cities, he has a problem, as there is no way to transfer traffic bound for city1 that enters city2's connection, and vice versa.

I think he has already explained that it is not possible for him to buy bandwidth from both providers in both cities. Therefore, your proposed solution is impossible.

Again, one needs to engineer ones network to work around one's own failures. I.e. ask or expect don't push routes into other people's tables because you are too cheap to buy a backup pipe, or too lazy to config a gre tunnel.

IIRC, he has also already explained that he already has a backup pipe between the two sites. However, because he can't buy bandwidth from both providers in both cities, this obviously is only part of the equation. -- Brad Knowles, <brad.knowles@skynet.be> "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania.

On Sat, 27 Jul 2002, Brad Knowles wrote:

At 10:56 AM -0400 2002/07/27, Andy Dills wrote: If you buy bandwidth from two different providers at two different locations, this would seem to me to be a good way to provide backup in case on provider or one location goes Tango-Uniform, and you could always backhaul the bandwidth for the site/provider that is down.

So, what am I missing?

The issue is how you want to influence inbound traffic. If you have the scenario you just described but want to keep as much traffic off your own intercity links as possible the only solution is to announce more specific networks that are heard globally. If you're ok with transporting a lot of intercity traffic between your locations, and just announce the same prefix everywhere, none of this applies and you've done your part to not pollute the DFZ. If you connect to the same transit(s) in both cities you can announce more specific networks with no-export set, keep most of your external traffic off your own network, and not cause the entire world to know about your more specific advertisements. Responsible yet lacking some redundancy: connect to the same single provider in both locations and announce more specific networks w/ no-export. Irresponsible yet gaining redundancy: connect to a different provider in each region and announce more specifics. no-export not an option, longer prefixes heard globally. Responsible and overall best: connect to the same 2+ providers in both locations and announce more specifics locally in each region/city/whatever with no-export. Your two or more transit providers have the more specific networks to bring the traffic to you across the last leg, but the rest of the world doesn't know/care about/have to deal with it. draft-ietf-ptomaine-bgp-redistribution-00.txt has some really good knobs to twist in order to engineer your traffic without making the rest of the world suffer. If/when it goes RFC getting people to actually make use of it will be painful, but atleast the horse is being taken to water. - Paul

On Sat, 27 Jul 2002, Brad Knowles wrote:

...

Responsible and overall best: connect to the same 2+ providers in both locations and announce more specifics locally in each region/city/whatever with no-export.

As said above, this isn't possible. I'd like to learn what could be done in this kind of situation that would allow the desired redundancy, while also being responsible.

I'm not a manager of a large network (where I might have this kind of problem myself), nor am I employed at a large ISP (where my customers might have this kind of problem). But I would like to learn.

It depends on the quality of intra-AS connectivity. In situations such as Ralph's, where intra-AS connectivity is strong, it has been shown that there is a technique to allow for the weaker transit link to be only used for backup: conditional BGP announcements and a gre tunnel over the transit links in both cities. Andy xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Andy Dills 301-682-9972 Xecunet, LLC www.xecu.net xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Dialup * Webhosting * E-Commerce * High-Speed Access

On Fri, Jul 26, 2002 at 10:49:21PM -0400, Ralph Doncaster wrote:

...

Announce your largest aggregate, and announce more-specifics tagged no-export to those peers who agree to accept them?

Which is worse than announcing just the more specifics to 2 different transit providers in 2 different cities.

Worse for those two transit providers, not the rest of the world.

...

Upgrade the connectivity between your sites? Technically sound, economically stupid. You offering to pony up the $5K/mth for an OC3 so I can have a redundant link between Ottawa and Toronto?

You are choosing to save money by poluting the global routing table. There may not be anything wrong with that, but don't be surprised when you hit providers who don't want or need to listen to your more specifics. Stop whining about it and fix your announcements.

Besides the technical aspects of my network, I didn't see any proof that relaxed prefix filtering (and no I'm not saying accept /32's - the more specifics I got Verio to accept were /23's) would cause significant (i.e. >30%) routing table bloat when that was recently discussed.

Verio carries something in the low 90k's, about a 20k savings (18%). -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)

On Sat, 27 Jul 2002, Ralph Doncaster wrote:

...

Worse for those two transit providers, not the rest of the world.

Why won't the rest of the world see extra hops and increased latency reaching my network (for the 50% of the time that the wrong transit provider is picked).

Because you could *gasp* be intelligent with your network design and do things like purchase transit from the same carriers in both your serving markets. Announce the /20 everywhere, announces your /23/whatever out the local connections for that city with no-export attached. The two carriers you *pay* for transit hold the more specifics to route the last leg to you, the rest of the world doesn't need or have to receive your deaggregation. This also gives added benefits like not slamming your inter-city circuits quite as hard when your link to one of your upstreams goes down. The problem here Ralph is that you see absolutely no problem with doing things on your network that have minimal benefit to you, yet have global impact. There's currently around 23-24k prefixes out there that are more specific announcements from the same origin AS as the aggregate. If all that junk would somehow magically get cleaned up the extra prefixes seen from multihomed networks announcing holepuncheed provider owned space would be _much less significant_, and RIR filters probably wouldn't even be necessary. You claim that your announcements have no global impact.. just like the thousands of others just like them, right?

The proposed "fix" is a big mess. My solution doesn't add to the global routing table since I'm renumbering out of a few /24's to the /20 I received from ARIN. I'm only de-aggregating to a /23, not /24. My solution provides optimal routing vs announcing the /20 globally.

Your network is the mess, not the solution. You've been given a /20 from ARIN, the responsible thing to do would be to do everything you can to make sure 90% of the internet sees that /20 and only that /20. The other 10% would be the people you pay money to for transit service who happily accept more specific prefixes because you are their customer.

The proposed "fix" will waste bandwidth, increase latency, and far more problematic to implement; how many NOC monkeys do you know that will be able to grasp the purpose of the 2 BGP sessions (one of them ebgp-multihop) let alone fix it if something goes wrong?

http://www.ietf.org/rfc/rfc2519.txt http://www.ietf.org/rfc/rfc1771.txt Typically one does not call another a "NOC monkey" unless they have demostrated to have a higher level understanding of the technologies at play. - Paul

On Sat, 27 Jul 2002, Ralph Doncaster wrote:

...

Because you could *gasp* be intelligent with your network design and do things like purchase transit from the same carriers in both your serving markets.

I guess you don't consider redundancy to be intelligent. I do. I guess you can call me stupid.

Carriers is a plural word.. How does that not accomplish redundancy again? - Paul

On Sat, Jul 27, 2002 at 11:17:57AM -0400, Ralph Doncaster wrote:

...

Carriers is a plural word.. How does that not accomplish redundancy again?

As I pointed out in my last post, I can't. And even if I could the economics of doing it don't make sense.

If economics don't matter, then the most intelligent network design would be a redundant OC192 mesh, but I don't know even one network that does that.

Ralph, I will give you some swamp space for your /24s if you never post about your 2621 powered OC192 mesh network-to-be again. If you want a solution that doesn't cost you any money, either a) get more IP space and carve up stuff into /21s, or b) go browsing through the list of allocated but unused swamp space and start announcing it. Otherwise, you will be filtered, no doubt gleefully from the members of this list. This is not the end of the world, they don't need your more specifics for the traffic to reach you. As long as your transit providers accept your more specifics from each other, announce the aggregate to whichever is primary, and more spcifics to other providers in other locations. The worst that can happen is you go Filterer->Transit1->Transit2 for some limited subset of the internet (but hey it doesn't cost you anything). Your economic problems are your own, if you were smart you would learn how to solve them within the rules of the game. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)