On Thu, 13 Oct 2005, Jeroen Massar wrote:

On Thu, 2005-10-13 at 11:05 -0700, Peter Lothberg wrote:

...

Is there anyone who can talk to it using IPv6 on the Nanog list?

(Time20.Stupi.SE, 2001:0440:1880:1000::0020)

As a certain "Tier 1" still uses a mesh of tunnels and uses Viagenie in Canada as their transit provider latency to the above IP is in the area of 300ms, going transatlantic twice. IPv4 latency is only 66ms though. I do hope that some "Tier 1's" get their act together and start doing native IPv6. I already once suggested upgrading their hardware to them ;)

I also presume you sent them a check and showed them the business case for the upgrade? No large provider is going to upgrade anything without a business reason. Oh, and some parts, critical parts even, of v6 are still 'broken'...

On Fri, 14 Oct 2005, JORDI PALET MARTINEZ wrote:

Of course, that's a business decision, but may be instead of getting a new check for the IPv6 service, not providing it, you will lost some checks from existing customers who demand dual stack ;-)

As ted and others have already said: "Show me the customers who are asking"... so far the numbers are startlingly low, too low to justify full builds by anyone large.

Business is also be competitive, and other carriers already have the service as a value added to the existing IPv4 customers.

Sure, and the decision to use their network I'd suspect hardly ever comes down to 'v6'. My point was, really, that the screaming crazy man saying: "I told them dudes to forklift their network" is hardly productive. Showing, if folks can't find it themselves, that there is a business case that would justify a few million dollar upgrade is... A few folks that have a deployment going are ahead of the curve, hopefully they can keep the parts they have running and upgrade away from the 7507 that is their current solution :) Hopefully other folks can make their beancounters understand that v6 is going to happen regardless of their wishes for it NOT to happen due to upgrade costs. Also, hoefully as old hardware is cycled out finally new and v6 capable hardware will take it's place :) -Chris

On Fri, Oct 14, 2005 at 10:21:47AM +0200, Daniel Roesen wrote:

On Fri, Oct 14, 2005 at 12:32:29AM +0000, Christopher L. Morrow wrote:

...

A few folks that have a deployment going are ahead of the curve, hopefully they can keep the parts they have running and upgrade away from the 7507 that is their current solution :)

The larger EU/US ISPs that have real deployments all use Junipers (for their IPv6), not Ciscos (with a few exceptions - Verio?). Don't know wether that's true for ASPAC folks too - can someone comment?

We're using both cisco and juniper in Verio for our IPv6 services and have been since it's launch (I think it was Oct/Nov 03). Running the dual-stack native service has been fairly straightforward. There are a few networks that are doing the dual-stack native thing. Others (eg: sprintv6 as seen in peoples traces to peters v6 clock) are doing tunneled infrastucture/overlay networks to support their IPv6 customers. This obviously has drawbacks that if you aggregate packets in a few locations (eg: asia, east-us, west-us, europe) for your tunneled stuff then have a full-mesh for their igp, the metrics don't always make sense. Combined with tunnels going long distances that don't reflect RTT and people still doing full transit to anyone they can tunnel with cause some interesting paths. One thing i find promising/good: Lots of people here sent their v6 traces to the list, so it's not just a few random geeks messing with v6 as much anymore, it's there. - jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.

On Fri, 14 Oct 2005 Michael.Dillon@btradianz.com wrote:

...

"I told them dudes to forklift their network" is hardly productive.

IPv6 is not a forklift upgrade.

agreed, it's a measured engineered decision hopefully. backed by financial and prudent engineering decisions. that wasn't the tone of the orignial comment though, which was: "Yea, I told them to just do it" which is tantamount to 'forklift your network you dummies'.

...

Showing, if folks can't find it themselves, that there is a business case that would justify a few million dollar upgrade is...

Again, it is cheaper to ease into IPv6 rather than waiting until it hutrts so bad that you have a business case for a million dollar spend. Start by making sure that your engineers get some IPv6 training

sure, most folks are doing this... (or atleast quite a few are) and at a certain time it's time to go from 'research' to 'production'. At that time it'd be prudent to address the financial reasons to 'go production', which is: "who's going to pay for this? What customer demand is there for this? uhm... why should I jeopardize my network for this now?"

...

Hopefully other folks can make their beancounters understand that v6 is going to happen regardless of their wishes for it NOT to happen due to upgrade costs.

The time to prepare is now.

agreed, some of the reason I asked a month ago about content providers :) also some of the reason I then said: "perhaps shim6 and multihoming is important to operators, better pay attention!" (I'm certainly not the only one saying these, I just don't want to drag other folks down with me :) )

When I read the Geoff Huston's response to Tony Hain's analysis here: http://www.cisco.com/en/US/about/ac123/ac147/archived_issues/ipj_8-3/ipv4.ht... I don't see him saying that we should do nothing. There seems to be general consensus among these guys that the time for action is now. If you want the IPv6 transition to be painless for your company, then you need to get planning and get IPv6 in your test labs today.

(and finding a business case that flies with management to actually do something higher in priority than all the other daily work)

On Fri, 14 Oct 2005, Chris Adams wrote:

Once upon a time, Christopher L. Morrow <christopher.morrow@mci.com> said:

...

agreed, it's a measured engineered decision hopefully. backed by financial and prudent engineering decisions. that wasn't the tone of the orignial comment though, which was: "Yea, I told them to just do it" which is tantamount to 'forklift your network you dummies'.

For some equipment, it still works out to "forklift your network". For example, our current dialup gear doesn't support IPv6 (and AFAIK no upgrades are available or planned to add it). There's no reason for us to replace our dialup gear; the only thing that fails on it is fans (and we can replace those easily enough with an hour's work of chassis dis/re-assembly). Dialup isn't going to go away in the near future either.

i suspect there is quite a large amount of gear (type not weight) that will never see v6 through the vendors but still support customers... speedstream anyone? cable-modem anyone? :( there are LOTS of things out there that don't know from v6 :( Thanks for another example though :)

On Fri, Oct 14, 2005 at 10:21:58PM -0500, Chris Adams wrote:

For some equipment, it still works out to "forklift your network". For example, our current dialup gear doesn't support IPv6 (and AFAIK no upgrades are available or planned to add it).

How does that hinder your backbone, leased line access and hosting deployment? Best regards, Daniel -- CLUE-RIPE -- Jabber: dr@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0

On Fri, Oct 14, 2005 at 12:32:29AM +0000, Christopher L. Morrow wrote:

As ted and others have already said: "Show me the customers who are asking"... so far the numbers are startlingly low, too low to justify full builds by anyone large.

Just wait for a popular adult-content-provider offering website-access for free via IPv6.. -- Sabri please do not throw salami pizza away

On Fri, 14 Oct 2005 15:13:44 +0200 Sabri Berisha <sabri@cluecentral.net> wrote:

On Fri, Oct 14, 2005 at 12:32:29AM +0000, Christopher L. Morrow wrote:

...

As ted and others have already said: "Show me the customers who are asking"... so far the numbers are startlingly low, too low to justify full builds by anyone large.

Just wait for a popular adult-content-provider offering website-access for free via IPv6..

Why ? Are you implying that there is unlimited free IPv6 bandwidth ? If not, why would they do that ? If so, I don't do porn, but I would be highly interested in free bandwidth. I suspect others would be too... Regards Marshall Eubanks

-- Sabri

please do not throw salami pizza away

Dear Sabri; On Fri, 14 Oct 2005 16:34:19 +0200 Sabri Berisha <sabri@cluecentral.net> wrote:

On Fri, Oct 14, 2005 at 10:17:51AM -0400, Marshall Eubanks wrote:

Dear Marshall,

...

...

Just wait for a popular adult-content-provider offering website-access for free via IPv6..

Why ? Are you implying that there is unlimited free IPv6 bandwidth ?

Nope.

...

If not, why would they do that ?

Imagine the following scenario:

"It's 2009, the world reaches the end of it's ipv4 supply. As large global networks are still struggling to implement ipv6 on their equipment, their customers are facing more and more problems to get additional IP-space from their RIR's and are forced to use ipv6. But due to the lack of planning, only a number of access-isp's have successfully deployed ipv6 on their networks and so we have shattered native ipv6 connectivity throughout the internet. To encourage the access- and carrierindustry, (adult)contentproviders in all continents decide to boost the demand for ipv6 connectivity and offer their services for free to ipv6 users, for a limited period of time."

Try as I may, I cannot imagine depending on the charity of porn content providers :) What I can imagine is the porn providers paying whatever is necessary for IPv4 address blocks. I would not want to get in a bidding war with them.

Why did the internet grow so fast in the 90's? The public was able to access the network and created the demand for more content. This content attracted more and more eyeballs, and thus more commercial activities were deployed, resulting in a exponential growth of the network.

Without eyeballs, contentproviders are not encouraged to deploy ipv6. Without content, eyeballproviders are not encouraged to deploy ipv6. It's a matter of time before one of them will be forced to end this circle and there is only one way to attract a large audience: giving a way your service for (nearly) free.

This sounds just like arguments heard in multicast discussions. I do give my content away for free, and AmericaFree.tv currently realizes about 10% of its (streaming) revenue from multicast (and presumably also realizes a similar percentage reduction in bandwidth costs from the same). If I thought that AFTV could increase revenues by another 10% by putting out IPv6 reflections of existing content, it would. If I thought that it could make money by putting out dedicated IPv6 content, it would do that too. I don't see any reason to expect either at present. I would be glad to be convinced otherwise.

-- Sabri

Regards Marshall

please do not throw salami pizza away

On Fri, 14 Oct 2005, Jeroen Massar wrote:

On Thu, 2005-10-13 at 22:55 +0000, Christopher L. Morrow wrote: <SNIP>

...

I also presume you sent them a check and showed them the business case for the upgrade? No large provider is going to upgrade anything without a business reason.

Current clients are already paying them at them moment are they not, as they apparently didn't reserve any funds for upgrades of their network, nor didn't take IPv6 along in the last 10 years of hardware cycles, thus clearly having played dumb for the last 10 years, how should their

silly me... I forgot that stable ipv6 code has been available for 10 years, forget my protest then.

...

Oh, and some parts, critical parts even, of v6 are still 'broken'...

Yep, there is no multihoming, but effectively, except for the BGP tricks that are currently being played in IPv4 there is nothing in IPv4 either. But one won't need to upgrade a Tier 1's hardware to support shim6, as

shim6 is: 1) not baked 2) not helpful for transit as's 3) not a reality

that will all be done at the end site and not at the "Tier 1" level, so that is just another bad excuse.

or bad assumptions on your part, it's perhaps a matter of perspective.

On Fri, 2005-10-14 at 10:57 -0400, Joe Abley wrote: <SNIP>

Are you suggesting that something else is required for ISPs above and beyond announcing PI space with BGP, or that shim6 (once baked and real) would present a threat to ISPs?

There is one situation which is not really covered here, one can of course announce multiple de-aggregates, but, these will be filtered. As such announcing them will only hurt one a lot, as the 'transits' that do carry them are mostly of bad quality. eg take the following situation: Big ISP, or a large corporate network, spread around the world. >200 customers and so, thus they can easily get a IPv6 prefix from their favourite RIR. Thus they get, say a /32. Now this ISP has a large webfarm in the US. They have a very small one in say, Taiwan. In IPv4, this would mean: chunk up your PA and simply announce them in /20's or whatever is comfortable for you. In IPv6 though, one is not supposed to announce chunks out of the /32, also when you do, as mentioned above, one gets bad routing. Anyway, you don't want your farm in Taiwan to attract all the local (complete asia?) traffic, which you have to ship over that same small link or your internal network in taiwan to the US again, while in IPv4 others would be doing that. In this case, which is basically "traffic engineering for endsites with a global prefix", one runs into the shim6 thing again.... For instance UUNET 'solved' this in a different way, they simply requested a 10 or so separate /32's. See GRH for the list. These chunks are still /32's thus only <n> of these /32's can exist in the global routing table. It would still be 'nicer' if they only had to use one prefix... Greets, Jeroen

On Fri, Oct 14, 2005 at 10:57:59AM -0400, Joe Abley wrote:

The big gap in the multi-homing story for v6 is for end sites, since those are specifically excluded by all the RIRs' policies on PI addressing right now. Shim6 is intended to be a solution for end sites.

But isn't a solution for many (most?) of them, EVEN if it would be universally implemented everywhere[tm]. This is a known problem, and it was decided to aim for a 80%[1] solution (80% of the problem space, not 80% of folks in need). There is still not half of an idea for a solution for the full problem in sight. Best regards, Daniel [1] number made up -- CLUE-RIPE -- Jabber: dr@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0

On Fri, Oct 14, 2005 at 11:50:33AM -0400, Joe Abley wrote:

I think it is far too early to judge how many end sites might find shim6 an acceptable solution, however -- I'd wait for some measurement and modelling before I made declarations about that,

You mean in some 5-10 years? When finally the many folks who even struggle to implement TCP properly manage to implement some one or even two (newest idea of the shim6 folks) shim layers into the stack and get that deployed widely? I don't see that. But I think the discussion is mood. IETF decided on their goal, and it's superfluous trying to change that. While watching shim6 we carry on hoping that we'll get IPv6 multihoming going in the conventional, proven, working, feature-complete way we're used to... until IETF perhaps at one point in time realize that they are designing a solution which misses the stated requirements of many folks actually operating networks - and start working on a solution which actually solves the preceived problem of scalability in a way operators look forward in deploying. And looking at the IPv6 allocation lists, I see that some of the folks' employers involved in shim6 developement actually have got their own allocations (and even leak more-specifics in geopgraphic distinct locations for traffic engineering). Looks like they couldn't convice even their own IT folks that shim6 or anything else will fix their problem (feature wise and/or timeline wise). Sorry for being so politically incorrect to spell out in open words what a lot of folks out there think. I'm wearing my asbestos anyway. :-) Best regards, Daniel -- CLUE-RIPE -- Jabber: dr@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0

On Sat, Oct 15, 2005 at 03:15:45AM +0000, Christopher L. Morrow wrote:

...

But I think the discussion is mood. IETF decided on their goal, and it's superfluous trying to change that. While watching shim6 we carry on hoping that we'll get IPv6 multihoming going in the conventional, proven, working, feature-complete way we're used to... until IETF

there is no hope in having operators explain to ietf that the current path is fruitless? certainly they can be made to see the light, yes?

Well, all this discussion and the set of requirements are nothing new. Quite the contrary. Lots and lots of talking was done, but still multi6 resulted in shim6. Where should one gain hope? They were constantly beaten with 6D Maglites, what does it take to see the light? Most folks have given up argueing I guess. I myself certainly did, at least in open fora. But I have also to admit that I'm shocked how few folks have the balls (or is it lazyness?) to express their opinion on IPv6 multihoming in the public, on the established fora for that stuff. See the recent threads about IPv6 PI / multihoming on ARIN PPML and other policy-making mailing lists. Almost zero feedback from enterprise / SME folks. That of course makes it much easier... "see, noone really complains! we must be going down the right road!".

...

And looking at the IPv6 allocation lists, I see that some of the folks' employers involved in shim6 developement actually have got their own allocations (and even leak more-specifics in geopgraphic distinct locations for traffic engineering). Looks like they couldn't convice even their own IT folks that shim6 or anything else will fix their problem (feature wise and/or timeline wise).

that is troubling, yes... 'hypocrisy' ?

Hm, perhaps more like OPP syndrome, no idea. You can very comfortably talk about ignoring requirements, when you have your own allocs in place, or know that you can easily pretend to be an ~ISP by sheer size of the company (you know, the "our IT department is the ISP for all other departments and spoke sites, and we have lots of them" standard trick). Frankly I don't have a clue what really lead the IETF to do the multi6 => shim6 move. Don't have any insight into the politics behind the curtains. Best regards, Daniel -- CLUE-RIPE -- Jabber: dr@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0

On Fri, Oct 14, 2005 at 09:52:19PM -0700, Tony Li wrote:

The alternative is a multihoming scheme that does not require a prefix per site. But that doesn't match the stated requirement of 'conventional', 'proven', 'working' [sic], 'feature-complete'.

Those weren't the "stated requirements" on an alternative multihoming scheme,, but only the attributes of conventional BGP multihoming. Please don't lay words into my mouth I didn't say.

The operational community needs to reach consensus on what its priorities are. We fought the CIDR wars to keep the routing subsystem working and the operational community were the primary backers of that. To not support scalable multihoming is to reverse that position entirely.

CIDR didn't have the big disadvantages to operators (at least non that I can identify, not having personally lived thru the CIDR migration). Operators DO support scalable multihoming, but it has to deliver what they want/need. HOW this can be achieved is the task of the IETF and the REAL challenge. shim6 is only "the easy way out". Best regards, Daniel -- CLUE-RIPE -- Jabber: dr@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0

Daniel,

...

The alternative is a multihoming scheme that does not require a prefix per site. But that doesn't match the stated requirement of 'conventional', 'proven', 'working' [sic], 'feature-complete'.

Those weren't the "stated requirements" on an alternative multihoming scheme,, but only the attributes of conventional BGP multihoming. Please don't lay words into my mouth I didn't say.

Those are exactly the words that you used in your message. I quote:

...

While watching shim6 we carry on hoping that we'll get IPv6 multihoming going in the conventional, proven, working, feature-complete way we're used to... until IETF perhaps at one point in time realize that they are designing a solution which misses the stated requirements of many folks actually operating networks -

...

The operational community needs to reach consensus on what its priorities are. We fought the CIDR wars to keep the routing subsystem working and the operational community were the primary backers of that. To not support scalable multihoming is to reverse that position entirely.

CIDR didn't have the big disadvantages to operators (at least non that I can identify, not having personally lived thru the CIDR migration).

No. It had big disadvantages to the end users. We asked them to suck it up in the name of having a scalable Internet. Now that we are proposing a technology to continue to help the providers scale, but that has disadvantages to the providers, we're seeing that the providers are not willing to sacrifice. Extremely disappointing.

Operators DO support scalable multihoming, but it has to deliver what they want/need. HOW this can be achieved is the task of the IETF and the REAL challenge. shim6 is only "the easy way out".

The IETF is responsible for providing real world engineering solutions to continue the growth of the Internet. When presented with the fundamentally conflicting requirements of supporting the Internet and fulfilling one faction's requests, they have chosen in favor of the Internet. If you'd like to suggest that they discover fundamentally new technology so that one can have their cake and eat it too, that's a fine thing, but that is the province of the IRTF. Engineering is the art of making tradeoffs, and that's what the IETF has done. I think that the provider community should examine the tradeoffs that have been made in much greater detail before condemning the result. The provider community has been well served by the IETF over the years and shim6 deserves at least a full and reasoned hearing before you throw the baby out with the bath-water. Tony

I don't want to speak for Daniel, nor other operators really, but a solution that doesn't allow an operator to traffic engineer internally or externally is just not workable. For the same reasons quoted in your other messages to me: "Increased reliance on the Internet"

There's nothing in any multi-prefix multihoming solution that prevents an operator from internal or external traffic engineering. There just isn't a single explicit prefix to manipulate. If, within any given routing domain, you choose to carry a longer prefix and manipulate it to whatever extent your vendor's BGP permits, you and your consenting adult peers are free to do so. Do not, however, expect the rest of us to carry your traffic engineering prefixes. We are not interested.

agreed, but it doesn't seem that, until recently atleast, there was much operator participation. Hopefully that's changing for the better :)

Hopefully, that will reach a point where the operators show up and participate at IETF, rather than the IETF coming to NANOG. Tony

On Sun, 16 Oct 2005, Christopher L. Morrow wrote:

I don't want to speak for Daniel, nor other operators really, but a solution that doesn't allow an operator to traffic engineer internally or externally is just not workable. For the same reasons quoted in your other messages to me: "Increased reliance on the Internet"

Well, people havn't been at all keen on solutions which would need fairly significant changes to how the operators do inter-AS routing (even if they would avoid shifting some aspects of routing to end-nodes). Given this high-resistance (rightly, wrongly, doesn't matter) to big changes in the transit parts of the internet, the only place then to do it is at the edges: have leaf-sites^Wnodes be more far active in how their packets are routed (by making deliberate use of the current provider aligned allocation<->topology transit internet). What kind of operator are you thinking of btw? End-node shouldn't bother operators of ISPs really (they'll only get the traffic that the end-node decided it wanted to send via them, which is exactly what you have today ;) ). It could bother operators of other kinds of sites though - but I'm hopeful though that the shim6 mechanisms will be malleable to site-multihoming, even if initially shim6 only concerns itself with end-hosts.

If the network isn't reliable due to suboptimal routing issues it can't survive :(

Just cause one network is unreliable does not mean that all the networks the end-node is connected to are unreliable. The end-node can try figure out which work and which don't and route accordingly. That's the whole point of shim6 ;). regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: I do not fear computers. I fear the lack of them. -- Isaac Asimov

FWIW, my current IPv6 assignment is PI to a degree (where P == my first hop IPv4 provider), I can change this "first hop IPv4" provider to any other provider within my country and still retain my IPv6 assignment.

it sounds as if you have the mythical separation of locator and identifier :-)/2. the problem is that there is likely to be a shortage of those locators. one problem with 6to4 is that having all traffic go through gateways will not scale well. to support v6-only folk, either the number of 6to4 gateways will need to approach the number of dfz routers, the dfz routers will run 6to4, or some combination thereof. randy

Paul Jakma wrote:

And 6to4 obviously won't fly for long after the 4 tank runs dry.

Hopefully it won't need to at that point as it is only intended as a transitional step. I like the simplicity of 6to4 and the way it preserves end-to-end addresses. If only there was a way to adapt it's stateless automatic tunneling to solve the IPv6 multihoming/PI problem. I took a quick hack at it and the result is interesting though far from perfect: http://kl.net/ipv6/pi-in-6.txt - Kevin

We do not think, that _it wil be IPv6_. IPv6 is a good example of _second_ system, and do not looks as _succesfull_ for now. And it is not definitely _LAST PROTOCOL_. It _can be_ IPv6, true. But it can be other protocol (or just workaround for IPv4, as we had CIDR and CLASSLESS) instead. ----- Original Message ----- From: "Gregory Edigarov" <greg@velcom.com> To: <nanog@nanog.org> Sent: Monday, October 17, 2005 3:42 AM Subject: Re: IPv6 news

Just my 5 cents to the topic:

Don't you all think that IPv6 would not be so neccessary for the very long time yet, if the IPv4 allocation scheme would be done right from the very very beginning? If the allocation policies would be something like the ones for ASn's. I.e. when you ask for IP space allocation you must be in the need to set your own routing policies. In any other cases you should use private network space with only one IP shown outside the network. Yes, this would be a headache for some appplications like IP telephony, but, I don't see any problems in making the _correct_ protocols so they could work through NAT.

As what I see now is that a very large address blocks are allocated to large companies, what companies do with them? Correct, they ae installing them as IP's of workstations, when, if IPs would be treated as a very valuable resource from the beggining, they would have to use at max /24 (well, may be 2 or three /24) for access routers.

When they are proposing /48 allocation scheme for IPv6 they must be out of their mind, because in case such allocation will be ineffect, IPv6 address space will end shortly too.

Again, IPv6 is creating more problems then solve. Better solution would be to freeze IPv4 allocation, then force big IPv4 users to return the addresses to the "public pool", and start allocation from the white piece of paper, doing the things right.

-- With best regards, GRED-RIPE

...

We do not think, that _it wil be IPv6_. IPv6 is a good example of _second_ system, and do not looks as _succesfull_ for now. And it is not definitely _LAST PROTOCOL_.

enter jim fleming (or those chinese guys, more recently) with ipv9

No, enter the National Science Foundation... http://www.nsf.gov/cise/geni/ Jim Fleming's idea wasn't all that crazy and some people are looking at similar partitioning schemes to make IPv6 multihoming practical. The IPv9 idea from China had nothing to do with IP, it was just a catchy marketing name for yet another domain naming scheme like RealNames. There really is serious, non-crazy research work going on to make a better replacement for the Internet Protocol. And Dave Clark, author of this interesting document on Internet routing: http://www.networksorcery.com/enp/ien/ien46.txt has recently been going around giving talks on fundamental re-architecting the Internet. At MIT he is a director of the CFP which is getting NSF GENI grant money to explore this: http://cfp.mit.edu/groups/internet/internet.html This is not the 1990's any more. ISO/CLNP has gone away. ATM has been embraced in MPLS. PSTN is being embraced in VoIP such as the British Telecom 21CN initiative. What was crazy yesterday, is thinkable today. In the end, IPv6 may be able to incorporate enough of these new ideas to continue as the last protocol. But we don't know that yet. --Michael Dillon

Michael.Dillon@btradianz.com wrote:

...

Another alternative is to force-align allocation and topology in some way /other/ than by "Providers" (geographical allocation in whatever hierarchy, IX allocation, whatever), such that networks were easily aggregatable. Lots of objections though (the "providers and geography don't align" one though is ultimately slightly bogus, because with non-provider-aligned allocation policies in place it would be in providers interests to align their peering to match the allocation policy).

I think we need a researcher to sit down and figure out exactly what this would look like in a sample city and a sample national provider.

This is one of those inversion situations where we are turning the existing model inside out. Some people may be familiar with the inversion of control in user interfaces that came about when Windows/Macintosh became the standard UI.

Here, the suggestion is that netblocks should be allocated to cities, not to providers. Within a city, providers would get a subset of the city address block to meet their local infrastructure needs. They would interconnect with each other a local exchange points to exchange local traffic as Paul Vixie is suggesting here: http://news.com.com/5208-1028-0.html?forumID=1&threadID=10554&messageID=77189&start=-1

Addresses from other cities would be viewed as a single aggregate for that city and these could be even further aggregated at some regional level such as Northwest, Southwest, Midwest, Southeast and Northeast.

Err... Sounds awfully like E.164. Why don't we use phone number instead of IP numbers? We all know how well carrier phone number routing and number portability works, don't we?

It's different than what we have now, but not extremely different. It is doable with IPv6 without any protocol changes because there is sufficient reserve address space available. It meets the concept of Internet as utility or mission-critical Internet because it mandates local interconnect. The customer point of view is that low latency and consistent latency is best and that mandates local interconnect.

I'm sorry, but your geographical approach is broken by design. -- Andre

Paul Jakma wrote:

On Mon, 17 Oct 2005, Andre Oppermann wrote:

...

We all know how well carrier phone number routing and number portability works, don't we?

EWORKSFORME (and everyone else here). Took a good bit of very firm pressure from ComReg, the telecoms wathdog/regulator here, to overcome negative reaction from the operators though.

We don't want them involved in Internet routing, do we?

(There's no such pressure which could be applied on IP operators, but same processes essentially could be applied at least for IP connectivity at national regulatory levels at least - trade /32's at INEX, the IX here and figure out billing. If only ComReg had the authority.. ;) ).

Do you have any idea how this works internally? Apparently not. Phone numbers are an interesting species. On a global level they are used for call routing. On a local level however it's not more than a DNS name mapping to some real on-net identifier. Unfortunatly anyone calling your ported mobile number from outside the mobile networks ends up with the number range holder (you former number range holder) who in turn has to forward the call to your current mobile operator. On a TDM network this works pretty OK as the quality parameters are standardized and fixed (64kbit transparent voice channel, call capacity, etc). Outgoing are not affected because the TDM network always sets up parallel in/out path's. The return channel for your outgoing call doesn't come back through your former mobile operator. Now compare this to the Internet and IP routing. See some little differences and diffculties here and there? Yea, I thought so. Conclusion: Applying the phone number portability to the Internet is broken by design. -- Andre

Paul Jakma wrote:

On Tue, 18 Oct 2005, Andre Oppermann wrote:

...

On a local level however it's not more than a DNS name mapping to some real on-net identifier.

Within a telco?

No. Within a region. Normally area codes are a region. Sometimes entire country codes are a region in this sense. Depends on the size of the region/country though. In some cases there is even more than one area code for the same region. For every area code there is a 'default' carrier. Usually this is the incumbant. They've got the obligation to forward inbound calls to the true carrier of that particular number holder. However this only works if the target carrier has some kind of interconnect with said default carrier. On top of this forwarding doesn't come for free. Depending on my call volume with that area I have to look into direct termination of ported numbers. Usually the regulator or some party designated by the regulator runs a number portability registry with the true carrier information for every number. If I want to optimize my call routing I have to periodically synchronize the call routing tables on my switches with that registry. In a very competitive area this lead to 30-50% of all numbers being ported and thus showing up in my routing table. As we know from the Internet DFZ the routing table becomes very large. Fortunatly for the TDM network I have to do the routing table lookup only once when setting up the call, not for every voice sample. Thus I pay the lookup cost only once for x minutes of voice traffic. Very DNS like. And because of the area code hierarchy even more so. That's why number portability is normally only offered within the same area code or region. So you can't take your NY fixed line phone number to LA. Unless of course you have someone picking up the call in NY and transporting it to you in LA. For non-US mobile networks the number portability works the same. The mobile network(s) have got their own area codes.

There's a myriad of ways to do it afaict. The case I know best though involved calls inbound to an operator specific prefix (there are a set of 4 or so major telco peering exchanges in Ireland, where the domestic and transit telcos /must/ be avilable for peering). The operator used custom software to map specific numbers to X.25 "addresses" (I forget the exact X.25 jargon) on their own network to deliver the calls to various locations on our network.

You can forget that X.25 stuff. It's only used for SS7 message routing and doesn't have anything to do with call routing as such. The telco peering points is just a technicality. It's there just for optimization. Most regulators have set up an "easy interconnection" policy to prevent your favorite incumbant from offering 'peering' only on lands end.

...

Outgoing are not affected because the TDM network always sets up parallel in/out path's. The return channel for your outgoing call doesn't come back through your former mobile operator.

I didn't know that, but sounds exactly what you want.

Sure. However this is the main difference between the TDM network and the Internet. Due to this fact many things work on the phone network like carrier pre-selection, phone number portability, etc., that do not work on an IP network.

...

Now compare this to the Internet and IP routing. See some little differences and diffculties here and there? Yea, I thought so.

There are huge differences in the details, obviously. The basic concepts though are at least interesting to consider, if not directly applicable to IP (technically at least - operationally/politically is another question):

1. Providers servicing these prefixes must peer and exchange the prefix information

On the phone network the prefix information is not dynamically exchanged. There are number portability registries whose data you can download every night or so and then dump it into your own switch or IN platform.

2. Providers must be prepared to carry other providers traffic into the area

Only one of them. The 'default' carrier. There are many phone networks and carriers carrier who do not have 100% coverage.

2a. The providers within the area have to figure out how to bill for the difference of this traffic.

No. Usually the tariff is set by the regulator based on some fixed interconnection charge and network element usage.

...

Conclusion: Applying the phone number portability to the Internet is broken by design.

Right, cause phone number portability is up and running for several sets of prefixes in various regions across the world[1], so there's definitely nothing we can learn from them. ;)

Well, we can learn from them that circuit switched networks are different than packet switched networks. Beyond that not much.

1. Does the US have number portability anywhere? If so, that would be a /huge/ region, and very interesting to examine to see how they manage it.

See above for an explanation. To summarize the differences between PSTN and Internet routing: o PSTN ports numbers only within regions/area codes o PSTN routes the return path along the forward path (symetric) o PSTN calls have pre-determined characteristics and performance (64kbit) o PSTN has static routing with periodic sync from porting database o PSTN pays the routing table lookup only once when doing call setup o PSTN call forwarding and peering is not free or zero settlement -- Andre

On Tue, 18 Oct 2005, Paul Jakma wrote:

If you want to focus on the differences between IP and POTS/GSM, sure, they're completely different. However, the point is to examine the abstract model for how telcos manage to achieve number portability without global-scope exchange of subscriber information and see what, if any, techniques could apply to IP.

Eg, given some arbitrary area: - RIR assigns a prefix to that area - For that area, for the set of ISPs providing service in that area (the area-ISP set) which are all peered with each other (eg at some IX in or near the area concerned), each ISP: - announces the area prefix as far and wide as they can (doing so will be an advantage for settlement with the other area-ISP set ISPs) - exchanges very very specific routes of: area-site -> AS with the other area-ISP set ISPs (if they peer locally, they can keep these very specific routes local too) - keep track of how much traffic to the area-prefix is handed off to other area-ISP set ISPs (and to which, obviously), and how much is received. - periodically, for every other area-ISP, reconcile traffic handed off / received and either send your or wait for their invoice as appropriate. Fraught with some difficulties obviously. (Politics of settlement, particularly when there is no benevolant entity to arbitrate and/or impose - before you ever get to the question of how to define an "area"). If it seems too difficult and the status quo is preferred - no worries, the hosts will figure out some kind of indirection. Bit less efficient than if ISPs would route natively/locally, but hey it won't require any difficult decisions and co-ordination in the ISP community. And maybe that'd be for the best. ;) regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: Nowlan's Theory: He who hesitates is not only lost, but several miles from the next freeway exit.

On Tue, 18 Oct 2005, Andre Oppermann wrote:

Again, this fails with the asymmetric nature of IP routing.

The assymetric nature is plus-point. It means the traffic out of the area goes out via the "correct" provider (ie the one whose customer it is).

On top it fails on bandwidth issues. What if super-cheap pron hoster X is in that area doing streaming full-res HDTV to it's suckers?

It goes via the ISP(s) which "super cheap hoster X" pays for transit.

I bet some participants in your service area face some serious link saturation issues. None of the participants have any control or estimates over the traffic that is and will be passing through them.

Yep.

Traffic flows will just happen there. Forget capacity planning. You'd have a hard time finding ISP's interested in that.

Maybe. Look at it the other way though, it's a business opportunity - you can make money by attracting as much area-destined external traffic as possible and handing it off to correct intra-area ISP for that subscriber. The more the better, it's a potential revenue source. It's in your interest to be able to carry all the external traffic into the area that you can get. regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: Profanity is the one language all programmers know best.

On Tue, 18 Oct 2005, Andre Oppermann wrote:

you care whether it is Sprint, Level(3) or Cogent? Apparently you don't. With your proposed you don't have much/any influence on the way your packets take.

They might take much better routes actually than is possible today. regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: The early worm gets the bird.

On Tue, 18 Oct 2005, Andre Oppermann wrote:

Yea, but only by chance, not by design. ;-)

Nope, by design. Routing would generally be better. The entire area would effectively be multihomed to the set of area-ISPs. There'd be some downsides too, eg where a provider attracting traffic for the prefix has some failure internally and for some reason doesn't withdraw the area-aggregate to ASes wholly external to the area. regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: To give of yourself, you must first know yourself.

Paul Jakma wrote:

On Tue, 18 Oct 2005, Andre Oppermann wrote:

...

As we know from the Internet DFZ the routing table becomes very large.

However, it can be confined to that arbitrary area.

Yes, but it's a very cumbersum process. You have to track this stuff for all regions and countries. They all vary how they do it. For example your ComReg publishes a couple of tables now and then with new/changed information. (Look for ComReg 04/35, 03/143R, etc.)

...

You can forget that X.25 stuff. It's only used for SS7 message routing and doesn't have anything to do with call routing as such.

Ah, it was used for everything in that network actually - but that was a very very specialised telco network. (And they had started moving to IP when I last worked with them.)

SS7 over IP is quite popular these days. However call routing != SS7 message routing.

...

Sure. However this is the main difference between the TDM network and the Internet. Due to this fact many things work on the phone network like carrier pre-selection, phone number portability, etc., that do not work on an IP network.

I'm not source how assymetric paths affect portability etc. Also, IP is well capable of that, and makes life easier.

IP routing is not symmetric whereas circuit switching is. In a case of individual IP address portability the return traffic always goes back to the ISP who has that particular prefix. No matter who 'opened' the connection. If I port my static dial-up IP to my new super FTTH ISP then suddenly up to 100Mbit of return traffic have to pass from Dial-Up ISP to FTTH ISP. I'd say this screws the Dial-Up ISP pretty royally. And you too because he most likely doesn't have that much capacity.

...

On the phone network the prefix information is not dynamically exchanged.

Uhm, sure it is.

Nope, it's not. Can you name a phone prefix routing protocol?

...

There are number portability registries whose data you can download every night or so and then dump it into your own switch or IN platform.

The number portability registries can be updated infrequently, yes.

The telco prefix routing information however most definitely *is* routed dynamically. Maybe you don't have to participate in this routing (your not a telco?), but between the telcos - most definitely ;).

(If not, we were scammed for a fortune for dynamically routed redundancy of calls across a set of exchanges ;) ).

That works differently. In the PSTN you always have multiple routes to a destination. If you have a direct trunk between two CO's then it will fill that first. When the direct trunk is full, the local switch has got an overflow route towards a neighboring or higher switch. It can have multiple overflow routes with different priorities. You can replace full trunk with dead trunk to get your redundancy. However there is no dynamic call routing as we know it from BGP or OSPF. At least not directly. Some switch vendors have developed call optimization software which runs in some sort of central intelligence center in the network and tries to optimize the trunk usage and priorities based on statistical and historical data.

...

...

2a. The providers within the area have to figure out how to bill for the difference of this traffic.

No. Usually the tariff is set by the regulator based on some fixed interconnection charge and network element usage.

How they figure it out (with or without a regulator) doesn't matter. It just has to be figured out. We don't have IP regulators, so for IP providers would have to figure it out all by themselves obviously. ;)

That'd be the stumbling block I suspect.

The stumbling block is that all IP packets return to the prefix holder (the old ISP) and the end-user bandwidth is not fixed.

...

To summarize the differences between PSTN and Internet routing:

o PSTN ports numbers only within regions/area codes

We're discussing what would be possible with area (rather than provider) assigned IP addresses. Ie, this is as possible for IP as PSTN, if $RIR decides to make some allocations in this way.

$RIR making allocations that way is not sufficient. It would need regulatory backing to enforce IP address portability. Every established carrier is not very interested in porting IP addresses to competitors.

...

o PSTN routes the return path along the forward path (symetric)

I thought you said it didn't? No matter, IP is assymmetric.

IP is asymmetric and PSTN is symmetric. There you have the first major problem with IP in this szenario.

...

o PSTN calls have pre-determined characteristics and performance (64kbit)

No bearing on routing.

Very much so. See my Dial-Up vs. FTTH ISP example.

...

o PSTN has static routing with periodic sync from porting database

The important point is that information to describe number->provider is exchanged betweeen providers in the area only. Whether it's done by dynamic protocols, email or post is an irrelevant detail, all that matters is that we have a way to do same in IP (we do: BGP).

The differences are far greater. See my description of call routing above.

...

o PSTN call forwarding and peering is not free or zero settlement

Indeed, I thought I had emphasised that working out the billing would be a major component of area-allocated IP. ;)

As you can see in the Dial-Up vs. FTTH ISP case as new ISP you don't have a chance to differentiate yourself through better routing or performance or QOS or whatever from anyone else. If the performance at the old ISP was lousy before it is lousy after porting the IP address because it's still the shitty bandwidth of that old ISP. -- Andre

Paul Jakma wrote:

On Tue, 18 Oct 2005, Andre Oppermann wrote:

...

SS7 over IP is quite popular these days. However call routing != SS7 message routing.

By call-routing you mean the actual circuit switching of each call? I don't mean that, I mean the number routing, which SS7 /does/ do - you referred to it as being more analogous to DNS iirc in operation.

...

Nope, it's not. Can you name a phone prefix routing protocol?

Ehm, SS7 ;).

You might call it DNS-like because it's request based, but it still provides routing information.

SS7 is not a routing protocol. SS7 is a transport stack like what we refer commonly to as TCP/IP. There are a number of protocols that run atop the basic SS7 transport network. Some protocols are datagram oriented and some are session oriented. For call handling ISUP or derivates are used. The main difference to the IP world is the limited scope in the PSTN. A PSTN switch consults its (static) number routing table for the trunk to forward the call on. Then it contacts the switch at the other end of the tunk and hands over further forwarding to it. If that doesn't work out the circuit gets shut down backward switch to switch. For special numbers like 800 and 900 there is another protocol called IN (Intelligent Network) which is nothing else than DNS. Each special number has a 'real' number in its shadow. The originating switch requests the real number through IN and then the normal call forward happens. IN may deliver different numbers based on the location of the orginating switch or daytime or any other criteria you may think of. A little bit like Akamai if you wish. However there is nothing akin BGP or OSPF in the SS7 suite of protocols. All the forwarding/trunk tables are computed offline for each switch and then stored on the switches by some bulk data transfer. Variantions are emerging with extended IN platforms where you have one more central databases of forwarding information but that is just a large geographically distributed switch then. -- Andre

Michael.Dillon@btradianz.com wrote:

...

...

Right, cause phone number portability is up and running for several

sets

...

...

of prefixes in various regions across the world[1], so there's definitely nothing we can learn from them. ;)

Well, we can learn from them that circuit switched networks are different than packet switched networks. Beyond that not much.

I disagree. There are many parallels and in many ways the telephony operators are struggling with the same kinds of problems that we are. NANPA has forecast that the North American number plan will be exhausted within 20 years. Just like the IPv4 address space.

Their plan is to extend the number plan by two digits using 4-digit area codes and 4 digit central office codes. Rather like IPv6's extended address length. The new digits will be introduced at the same time so that everyone will dial an extra digit at the end of their existing area code, and another extra digit at the beginning of their central office code. Today you would dial (703)227-0660 to reach ARIN's help desk. After the change you would dial (7030)0227-0660. Full details here: http://www.atis.org/inc/docs/finaldocs/020107029.doc

NANPA's website points to more information. http://www.nanpa.com/index.html

There is also a North American Numbering Council that meets regularly and has several working groups. http://www.nanc-chair.org/docs/documents.html

It is foolish to regard people outside the IP networking industry as inferior. Good ideas can come from anywhere and we can often understand our own area of interest much better by comparing and contrasting with other similar areas of interest.

There is a major difference between phone numbers and IP addresses which makes direct comparisons harder. Phone numbers are more like Domain names (+email addresses behind them) than IP addresses. People use phone numbers the same way they use domain names. They remember them and use them to access other people, or companies. I haven't seen many billboards with IP addresses on them lately. Nobody cares about the actual IP address. Only the computer does at the time of the DNS lookup. So an IP address is only used as underlying transport vehicle of data. For the enduser it doesn't have any direct significance. A phone number has significance to the end user and has a hybrid function as underlying routing element to varying degrees too. The entire problemset with IP address portability comes from two issues: Ease of ISP changes and redundant connectivity. The former could theoretically be solved with with better procedures and methods for host address assignment. However it still requires some labor intensive transition period and the IP addresses are much tangled with other things like DNS and so on. The second issue is IP architecture specific. The PSTN, due to its symmetric nature, doesn't have the redundancy problem to the same extent as the Internet. For the IP prefix however you have to participate in the global routing system to survive link losses. Without any shim6 or SCTP stuff that is. Again, phone numbers and their portability can and should not be compared with the IP address portability issues. They're very different animals. -- Andre

On Wed, 19 Oct 2005, Michael.Dillon@btradianz.com wrote:

...

Again, phone numbers and their portability can and should not be compared with the IP address portability issues. They're very different animals.

That's your elephant. My elephant looks different.

Survey says... BZZZZZT. Read about SS7 LNP implementation before speaking, please. They are very different creatures. Something that resembles telephony LNP will not scale to the quantity of micro-streams currently used by WWW applications. The reason it works (FSVO "works") for telephony is because, unlike TCP streams, telephone circuits are comparatively large streams with much longer keepalive times. -- -- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>

On Oct 19, 2005, at 9:39 AM, Michael.Dillon@btradianz.com wrote:

Why is it that whenever people suggest that the IP networking world can learn from the experience of the telephony world, some people assume that the proposal is to imitate the telephony world in every detail?

Seems to me to be a species of the broader attitude among some that any analogy or reference to anything outside of the IP world is inherently flawed. On this view, there is no such thing as a good analogy or comparison. All are equally bad and misleading. Full stop. Perhaps an ungenerous observation, but this is exactly the sort of attitude that one would expect of narrow experts in the field who have no knowledge whether and how it might resemble or relate to anything else... Private editorial ends. TV

1. Does the US have number portability anywhere? If so, that would be a /huge/ region, and very interesting to examine to see how they manage it.

In the USA this is called LNP (Local Number Portability). This article has a couple of pages of history and then a technical overview of how LNP works. http://scholar.lib.vt.edu/ejournals/JOTS/Winter-Spring-2001/pdf/obermier.pdf This document explains the architecture of LNP in today's phone network: http://www.verisign.com/stellent/groups/public/documents/white_paper/001950.... However, LNP is not as simple as most laypeople think. It has other applications than simply consumer convenience. For instance, disaster recovery http://www.neustar.com/pressroom/datasheets/DisasterRecPress.pdf Read this description of number pooling http://www.verisign.com/stellent/groups/public/documents/white_paper/001949.... and reflect on how similar this seems to injecting longer prefixes into BGP (hole punching) to support moving a customer from another network. LNP and routing are the same problem. The details of the solutions differ because the technology environment and constraints differ. But you will never understand IP routing unless you understand how non-IP networks solve these same problems. That's why some people use RIP to teach routing even though it is considered bad practice to run RIP on any network in this day and age. People need to learn routing theory separately from "How to configure BGP on your brand-X boxes". --Michael Dillon

...

I think we need a researcher to sit down and figure out exactly what this would look like in a sample city and a sample national provider.

especially: http://arneill-py.sacramento.ca.us/ipv6mh/geov6.txt

will be quite of your liking.

Not at all. This proposal is all about allocating addresses based on country boundaries and I reject this model. The Internet is a network of cities, not countries. The national boundaries are completely random in technical terms, but the cities are not random. The cities are where the people are, where the railways and roads are, where the channels of trade and communication begin and end.

Which indeed seems quite reasonable. The problem with this is: 'who is paying for which traffic and to whom'

Customers pay for all the traffic on their link and they pay their money to their Internet access provider. But that is beside the point.

Notez bien, though this solves multihoming, it doesn't solve relocation, thus if your company moves it has to renumber, and renumbering is no fun,

Not true. If a company moves across the city and connects to a different access provider, they don't have to renumber. After all, they are still in the same city. It just means that inside that city, the providers will carry an additional longer prefix in their routing tables. But the global view will be unchanged. In fact, the smaller global table allows for much more detail to be carried locally, given the same constraints of RAM and processing power in routers. --Michael Dillon

On 17-okt-2005, at 14:18, Jeroen Massar wrote:

...

...

Another alternative is to force-align allocation and topology in some way /other/ than by "Providers" (geographical allocation in whatever hierarchy, IX allocation, whatever), such that networks were easily aggregatable. Lots of objections though (the "providers and geography don't align" one though is ultimately slightly bogus, because with non-provider-aligned allocation policies in place it would be in providers interests to align their peering to match the allocation policy).

The current assumption is that all aggregation happens on ISP. Replacing that with the assumption that all aggregation will happen on geography isn't all that useful. The important thing here is that you can aggregate on pretty much anything: hair color, router vendor, market capitalization, you name it. In the end, you always aggregate on the way the addresses are given out, which may or may not be meaningful. Aggregating on provider is the most powerful because the aggregate leads you fairly directly to the place where you need to go as long as the destination is single homed. But suppose at some point we end up with a routing table consisting of 10 million PI blocks from multihomers and some unimportant stuff that disappears in the error margin (i.e., those 5000 IPv6 /20s for huge ISPs). Also suppose that it's possible to build a reasonably cost effective router that handles 1M routes, but this router technology doesn't scale to the next order of magnitude. The simple solution is to build a big router that actually consists of 11 small ones: 10 sub-routers that each hold one tenth of the global routing table, and an 11th sub-router that distributes packets to the sub-router that holds the right part of the global routing table. So sub-router 1 has the part of the global IPv6 routing table that falls within 2000::/6, sub-router 2 has 2400::/6, sub-router 3 2800::/6 and so on. So we're aggregating here, but not really "on" something. This has the unpleasant side effect that we now have to spend 11 times more money to keep a 10 times larger routing table. Alternatively, we can trade hardware costs for bandwidth, by having 10 routers that are present in the network anyway each handle part of the global routing table. So a router in Boston would handle everything under 2000::/6, a router in Chicago 2400::/6, one in Seattle 2800::/6 and so on. Obviously this isn't great if you're in Boston and your address is 2800::1, but it doesn't require additional hardware. This scheme can be optimized by aligning addressing and geography to a reasonable degree. So if you're in Boston, you'd get 2000::1 rather than 2800::1. But that doesn't magically shrink the routing table to one route per city. In the case of Boston, it's likely that the source and destination ISPs for a certain packet don't interconnect within the city itself. So someone sitting in New York probably won't see much difference: he or she still has to carry all the routes for multihomers in Boston. Some of these will point to her own customers in Boston, some to peers in New York, others to peers in DC, and so on. However, as distance increases the difference between "this packet needs to go to a customer in Boston", "this packet needs to go to a peer in New York" and "this packet needs to go to a peer in DC" becomes meaningless, so it's possible to replace a large number of individual routes by a single city or region aggregate. So even without magic interconnection dust, aggregation based on geographical addressing can have benefits. However, it has several limitations. An important one is that early exit routing is replaced by late exit routing. Also, when someone multihomes by connecting to ISPs in Miami and Tokyo you don't get to aggregate. But worst case, you just don't get to aggregate, either because people multihome in weird ways, for traffic engineering reasons or because of lack of interconnection (however as interconnects become really sparse the savings go up again) so you're no worse off than today. But if and when the routing tables explode and routers can't keep up, having geographical addressing in place for multihoming allows for a plan B that we don't have today.

...

I think we need a researcher to sit down and figure out exactly what this would look like in a sample city and a sample national provider.

There has been quite some research on it, there where ideas, there was even talk of a vendor going to implement it, but it never happened. It won't work because of cash reasons (read: telco/transit don't want it)

I'm not familiar with that... Do you have a reference?

For your 'city data' check: http://unstats.un.org/unsd/demographic/default.htm

or for pre-processed files: http://arneill-py.sacramento.ca.us/ipv6mh/ under "Geographical data".

Note that this page hasn't been updated in more than two years. When Michel started this initiative the IETF multihoming in IPv6 (multi6) working group was pretty much dead and it certainly wasn't considering any input. However, our efforts resulted in the wg coming back to life again, considering input, rejecting most of it, and start work on a solution in a new wg: shim6. (Paul Jakma wrote something to the effect that I am involved with shim6 so that says something about other options. It doesn't, as far as I'm concerned. But shim6 is a worthy pursuit in its own right.) For anyone who wants to read the latest version of all of this (still two years old, though): http://www.muada.com/drafts/draft-van-beijnum- multi6-isp-int-aggr-01.txt

especially: http://arneill-py.sacramento.ca.us/ipv6mh/geov6.txt

Which indeed seems quite reasonable. The problem with this is: 'who is paying for which traffic and to whom'

Just because I press the "up" button for the elevator doesn't mean I'm going to the top floor. Still, having one button for "down" and one for "up" rather than having a different one for each floor seems to work well for this initial part. Once you get inside the elevator you still have to pick a floor, of course. Iljitsch

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Yo Michael! On Mon, 17 Oct 2005, Michael.Dillon@btradianz.com wrote:

Here, the suggestion is that netblocks should be allocated to cities, not to providers. Within a city, providers would get a subset of the city address block to meet their local infrastructure needs. They would interconnect with each other a local exchange points to exchange local traffic

And who is going to force the ISPs to interconnect at the city level? For competitive reasons there is no peering in my city. The nearest peering points are several hundred miles away, in different directions, and even those are not shared in common by the local ISPs. RGDS GARY - --------------------------------------------------------------------------- Gary E. Miller Rellim 20340 Empire Blvd, Suite E-3, Bend, OR 97701 gem@rellim.com Tel:+1(541)382-8588 Fax: +1(541)382-8676 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDU9eq8KZibdeR3qURAnBfAKC4ZBCUrGq9HgW80FJxIGqfbR7mowCgi4GD ykujmYnq/FPv6MA1nKdf49A= =aUG/ -----END PGP SIGNATURE-----

On Mon, Oct 17, 2005 at 11:39:37AM +0100, Michael.Dillon@btradianz.com wrote:

Here, the suggestion is that netblocks should be allocated to cities, not to providers. Within

I am a multihomed customer and my ISPs are in two different cities. What are my IP addresses going to be? This situation happens all the time, by the way. In fact, the closer you get to smaller, consumer grade connectivity, the more you will see backhauling below the network layer in providers' networks that will make this happen. -Phil

I reread this and still don't see how geographical ip address allocation is going to work if typical customer connections are network-centric and any large area has number of competitive access providers (unless you're fine with multiple providers announcing aggregate summary in anycast fashion). The only way I see that geographical addressing might have some advantage is if the area is covered by large monopoly that connects everyone else there (and its not to say that such situation does not exist in some countries, but I don't think this situation should be encouraged and forced to continue forever with network allocation policies). On Tue, 18 Oct 2005 Michael.Dillon@btradianz.com wrote:

...

...

Here, the suggestion is that netblocks should be allocated to cities, not to providers. Within

I am a multihomed customer and my ISPs are in two different cities. What are my IP addresses going to be?

Your assumptions are flawed. I never suggested that there would be a flag day. I never suggested that geotopological addressing would work everywhere or solve all problems. I never suggested that we should turn off the existing provider aggregatable IP address allocations.

I just suggested an alternative way of issuing addresses so that they are geotopologically aggregatable, not provider aggregatable. There are sufficient reserved addresses in the IPv6 address space to do this. We could start issuing geotopological netblocks and try it for 5 years or so to see whether it works better or not.

In any case, you are located in Montreal which is such a major city that I expect any ISP selling service (geotopologically) in Montreal would use Montreal address space even if they backhauled at layer two to some other city.

However, there will likely be lots of situations where people in small towns roughly equidistant from two cities will choose to multihome with links to separate cities. This will either have to be done using provider-aggregated addresses or by using addresses from one of the cities with a longer prefix inside that city's routing table to direct the traffic to the neighboring city. If this is suboptimal, it won't be by much considering that these are neighboring cities.

I'm not suggesting any change to IPv6 stacks or to routing protocols. I'm just suggesting that we could allocate the same IPv6 addresses to operators in a way that allows geotopological aggregation rather than the existing provider aggregation. Combine this with local traffic exchange in every city and you have a more robust Internet with lower overall latency that will run with a smaller global routing table.

I know that some individual operators, such as the one I work for, have very robust IP networks with low overall latency. But when we talk about the Internet, then we include all the private interconnects and public exchange points and tromboning of traffic due to peering "issues", etc.

--Michael Dillon

-- William Leibzon Elan Networks william@elan.net

I reread this and still don't see how geographical ip address allocation is going to work if typical customer connections are network-centric and any large area has number of competitive access providers

Inside the city, you see lots of longer prefixes from that city's netblock. Outside the city you see only the single aggregate prefix.

The only way I see that geographical addressing might have some advantage is if the area is covered by large monopoly that connects everyone else there

Monopoly? Not necessary. Yes, you need to have universal exchange of local traffic in the city but that can happen through private interconnects and multiple exchange points. No need for a monopoly. The major change is that providers which participate in geotopological addressing would have to interconnect with *ALL* other such providers in that city. This would mean more use of public exchange points. Also, I think it makes sense to have a second regional layer of aggregation where you group neighboring cities that have a lot of traffic with each other. I think this would result in no more than 20-30 regions per continent. --Michael Dillon

On Fri, 14 Oct 2005, Tony Li wrote:

...

...

But I think the discussion is mood. IETF decided on their goal, and it's superfluous trying to change that. While watching shim6 we carry on hoping that we'll get IPv6 multihoming going in the conventional, proven, working, feature-complete way we're used to... until IETF

there is no hope in having operators explain to ietf that the current path is fruitless? certainly they can be made to see the light, yes?

Doubtful. The IETF was operating under the impression that having a scalable routing subsystem was paramount. Do you think operators can be made to see that light?

They've been asking for that as well I think. I certainly don't want to have 1M+ routes for JUST the Internet to worry about anytime soon, I'd hate to see over 300k for real Internet routes anytime soon :( Much of today's hardware doesn't seem so happy around that number :( Operators and IETF need to hit a middle ground. Perhaps that middle ground is a mix of these 2 things? Routing hardware will certainly scale to larger than today's table, other factors are driving that, will it scale to 1B prefixes or 1T prefixes? I'm not able to speculate on that... I can see an immediate need to get over 500k though, and not from ipv6 nor lack of shim, and I'm probably shooting low.

Implementing IPv6 multihoming the "conventional" way guarantees that we end up with one prefix per site, and as the need for multihoming reaches deeper into the population, the growth rate of the routing table would surpass even the growth rate of the Internet itself.

I'm not sure I agree that the end state is 100% multihoming. I can certainly agree that more multihoming is coming. Many more people are pushing for multihoming today than in previous years, apparently telco instability (financial not technical) is/has driven this :) (among other things I'm sure) Again, I don't know what the end state size is, I agree it's bigger than today, I think it's probably smaller than infinity.

The alternative is a multihoming scheme that does not require a prefix per site. But that doesn't match the stated requirement of 'conventional', 'proven', 'working' [sic], 'feature-complete'.

whichever solution is the end solution there needs to be the capability to solve today's problems in tomorrow's world. The current proposal I think doesn't capture all of today's problems. If the set of missing solutions were going to magically go away then alls good... I can't see link failure (not complete failure of the path, partial path failure) or congestion or suboptimal paths going away though. (just to name a few) -Chris

They've been asking for that as well I think. I certainly don't want to have 1M+ routes for JUST the Internet to worry about anytime soon, I'd hate to see over 300k for real Internet routes anytime soon :( Much of today's hardware doesn't seem so happy around that number :( Operators and IETF need to hit a middle ground.

There are 437 cities of 1 million or more population. There are roughly 5,000 cities of over 100,000 population. And there are 3,047,000 named communities in the world. Seems to me that the number of routes in the global routing table should logically be closer to 5,000 than to 3,000,000.

I'm not sure I agree that the end state is 100% multihoming. I can certainly agree that more multihoming is coming. Many more people are pushing for multihoming today than in previous years, apparently telco instability (financial not technical) is/has driven this :) (among other things I'm sure)

I agree that the end state is *NOT* 100% multihoming. It is too complex for most people and there is no business justification for it. But an awful lot of business customers will be able to justify multihoming. That is part and parcel of the "mission critical" Internet. --Michael Dillon

On Mon, Oct 17, 2005 at 12:08:38PM +0100, Michael.Dillon@btradianz.com <Michael.Dillon@btradianz.com> wrote a message of 28 lines which said:

There are 437 cities of 1 million or more population. There are roughly 5,000 cities of over 100,000 population. And there are 3,047,000 named communities in the world.

Seems to me that the number of routes in the global routing table should logically be closer to 5,000 than to 3,000,000.

If there is an exchange point per city over 100,000 (the route goes to the IXP and then to the actual provider)... Otherwise, there is a flaw in your calculation.

On Mon, 17 Oct 2005 Michael.Dillon@btradianz.com wrote:

I agree that the end state is *NOT* 100% multihoming. It is too complex for most people and there is no business justification for it. But an awful lot of business customers will be able to justify multihoming. That is part and parcel of the "mission critical" Internet.

" The year is 2011. Last week my DSL provider died for 6 hours and my family was unable to get to the Internet. A friend suggests I try Suparoute11 (tm), I download the program and install on my Xbox 5. A few seconds later it uses my Xbox's 802.66 wireless port to contact several other people running the program within a few blocks of my house. Options are displayed on my screen, one guy's hub is offering a "backup 10Mb/s home link, tunneled and advertised to TIX" across a large cable provider. My son tells me that is what I want so I setup a payment of $5 per month to him. In 10 minutes from start to finish my house's /54 is "multi-homed", whatever that means. " -- Simon J. Lyall | Very Busy | Web: http://www.darkmere.gen.nz/ "To stay awake all night adds a day to your life" - Stilgar | eMT.

man, 17,.10.2005 kl. 14.22 +0200, skrev Jeroen Massar:

On Tue, 2005-10-18 at 01:07 +1300, Simon Lyall wrote:

...

My son tells me that is what I want so I setup a payment of $5 per month to him. In 10 minutes from start to finish my house's /54 is "multi-homed", whatever that means.

You cover the payment part, partially. But now the fun parts:

This story may be a lot closer to reality in 2011 than you seem able to accept. It's a mere illusion given limitations in current (2005) routing-protocols, but those protocols may also be nothing more than a distant memory by 2011.

- which prefix

Why shouldn't it be his own?

- where did you get the prefix

Maybe it's assigned to him?

- who says you are you and thus that you are allowed to use that prefix (someone has a working PKI available? :)

The allocating entity should be able to authorise that

- routing table size?

Who says everybody has to know about everybody else all the time. Maybe it's enough to know where to go when you need to get there. (more in other thread).

- is the other end of the possible communication going to be able to reach this prefix?

ditto

- will everybody accept you as being the endpoint?

Why not. As long as it is unique.

- what about a broken prefix 'upstream' (you just said your primary link goes down, thus it might be a misconfig 'upstream')

Yeah. There will probably be many failure modes in future networks too.

- insert various other "2011" stories...

... and get some vision about the future. Don't assume the world will stop. //Per

there is no hope in having operators explain to ietf that the current path is fruitless? certainly they can be made to see the light, yes?

you have not spent much time with the ivtf, have you? randy

On Sun, 16 Oct 2005, Susan Harris wrote:

...

...

there is no hope in having operators explain to ietf that the current path is fruitless? certainly they can be made to see the light, yes?

you have not spent much time with the ivtf, have you?

Actually Chris has been extremely active in the IETF - his draft on current/desired router filtering capabilities is something that's been needed for a while (draft-morrow-filter-caps-01.)

doh, supposedly it's a WG document now, but honestly I just have a big mouth, George Jones provided most of the content and guidance... I just tried to hang myself with the xml authoring 'tools' :( I do hope to gain more ietf experience though... (I may bring a hard hat this time around)

On Oct 14, 2005, at 12:10 PM, Daniel Roesen wrote:

designing a solution which misses the stated requirements of many folks actually operating networks

So far it's missing some of the stated requirements (reasons for multihoming) listed in the charter... well I was going to cut-n-paste like I did to my email to shim6 dated Oct 4 2005, but it seems to have been removed in an update... so I'll cut-n-paste from that email: For the purposes of redundancy, load sharing, operational policy or cost, a site may be multi-homed, with the site's network having connections to multiple IP service providers. So the IETF identified 4 reasons to multihome. Of those 4, shim6 ignores at least 2 of them (operational policy and cost), and so far as I can see glosses over load sharing. I'd actually redefine load sharing to load (im)balancing, but that may just be pedantics. I don't recall seeing any followup to my question, and it's not showing up in the list archive so maybe there's something wrong with my list subscription (although I have had responses to other postings... which are also not in the archive)

On 15-Oct-2005, at 15:29, Tony Li wrote:

...

So the IETF identified 4 reasons to multihome. Of those 4, shim6 ignores at least 2 of them (operational policy and cost), and so far as I can see glosses over load sharing.

If you have a solution that satisfies all requirements, you should contribute it. Shim6 is indeed a partial solution to the stated requirements. There was no tractable solution found to all requirements, and to not solve any of the issues was seen as basically fatal.

Yes. It may be worth noting that the "requirements" you're talking about were very deliberately published in a document which professes to contain "goals" and was intended to avoid any mention of the "r" word (although I see we missed one in the title of section 3.2 :-) The draft that led to RFC 3582 was originally a requirements document. As you say, there was no confidence that there would be any proposals which would meet all the items in the document if they had been wrapped in MUSTs and SHOULDs. As the abstract says: This document outlines a set of goals for proposed new IPv6 site- multihoming architectures. It is recognised that this set of goals is ambitious and that some goals may conflict with others. The solution or solutions adopted may only be able to satisfy some of the goals presented here. Joe

I don't have an acceptable solution... however, I am getting tired of shim6 being pushed as *the* solution to site rehoming, when at best it's an end node rehoming solution.

Well, sorry. When we explored site multihoming (not rehoming) in the ways that you seem to suggest, it was effectively a set of coordinated NAT boxes around the periphery of the site. That was rejected quite quickly. Tony

drc@virtualized.org (David Conrad) writes:

On Oct 15, 2005, at 3:27 PM, Tony Li wrote:

...

When we explored site multihoming (not rehoming) in the ways that you seem to suggest, it was effectively a set of coordinated NAT boxes around the periphery of the site. That was rejected quite quickly.

What were the reasons for rejection?

i wasn't there for that meeting. but when similar things were proposed at other meetings, somebody always said "no! we have to have end-to-end, and if we'd wanted nat-around-every-net we'd've stuck with IPv4." -- Paul Vixie

...

but when similar things were proposed at other meetings, somebody always said "no! we have to have end- to-end, and if we'd wanted nat-around-every-net we'd've stuck with IPv4."

Is VJ compression considered a violation of the "end-to-end" principle?

Or perhaps I misunderstand (yet again).

Paul is correct. Things that looked like NAT were rejected because "NAT is evil". Shifting the NAT to end system removed the objection to NAT, tho it's not entirely clear why. Shifting NAT to the end system also happened to simplify the entire solution as well. VJ compression should not be considered a violation of the "end-to- end" principle, as it is a per-link hack and performs a function that CANNOT be performed in the end systems. However, I'm not entirely sure that this is relevant. NAT is not, strictly speaking, a violation of the end-to-end principle. It certainly is rather ugly and awkward from an architectural perspective, but it is a function that is not otherwise required in the end host, so placing it into the network does not violate the letter of the principle. Perhaps this is yet another case where people misunderstand the principle itself and are invoking it to give a name to their (well placed) architectural distaste. Tony

Doesn't NAT, or more specifically the most commonly used, NAPT, create hard state within the network, which then makes it violate the end-to-end argument ? Also, because it has to understand transport and application layer protocols, to be able to translate embedded addresses, doesn't this also make it violate end-to-end ? I've understood the fundamental benefit of following the end-to-end argument is that you end up with a application agnostic network, which therefore doesn't create future constraints on which applications can then be used over that network. In an end-to-end "compliant" network, any new transport layer protocols, such as SCTP or DCCP, and new user applications, only require an upgrade of the end or edge node software, which can be performed in an incremental, per edge node as needed basis. In other words, there isn't any whole of network upgrade cost or functionality deployment delay to support new applications, which was the drawback of application specific networks, such as the traditional POTS network.

Have I somehow misunderstood the intent or benefits of the end-to-end argument ?

Mark, This is probably the most common misunderstanding of the end-to-end principle out there. Someone else can dig up the quote, but basically, the principle says that the network should not replicate functionality that the hosts already have to perform. You have to look at X.25's hop-by-hop data windows to truly grok this point. Many people pick this up and twist it into ~the network has to be application agnostic~ and then use this against NATs or firewalls, which is simply a misuse of the principle. Really, this is a separate principle in and of its own right. It's not one that I subscribe to, but that's a different conversation... Regards, Tony

On Sun, Oct 16, 2005 at 01:45:40AM -0700, Tony Li wrote:

...

Doesn't NAT, or more specifically the most commonly used, NAPT, create hard state within the network, which then makes it violate the end-to-end argument ? Also, because it has to understand transport and application layer protocols, to be able to translate embedded addresses, doesn't this also make it violate end-to-end ? I've understood the fundamental benefit of following the end-to-end argument is that you end up with a application agnostic network, which therefore doesn't create future constraints on which applications can then be used over that network. In an end-to-end "compliant" network, any new transport layer protocols, such as SCTP or DCCP, and new user applications, only require an upgrade of the end or edge node software, which can be performed in an incremental, per edge node as needed basis. In other words, there isn't any whole of network upgrade cost or functionality deployment delay to support new applications, which was the drawback of application specific networks, such as the traditional POTS network.

Have I somehow misunderstood the intent or benefits of the end-to-end argument ?

Mark,

This is probably the most common misunderstanding of the end-to-end principle out there. Someone else can dig up the quote, but basically, the principle says that the network should not replicate functionality that the hosts already have to perform. You have to look at X.25's hop-by-hop data windows to truly grok this point.

Many people pick this up and twist it into ~the network has to be application agnostic~ and then use this against NATs or firewalls, which is simply a misuse of the principle. Really, this is a separate principle in and of its own right. It's not one that I subscribe to, but that's a different conversation...

Maybe its time to pull out some of Noel's work on both topics. Reasonable introductions to both the e2e principle and locator/id split topics can be found on http://users.exis.net/~jnc/tech/end_end.html and http://users.exis.net/~jnc/tech/endpoints.txt respectively. Dave

Tony, On Oct 15, 2005, at 11:26 PM, Tony Li wrote:

Paul is correct. Things that looked like NAT were rejected because "NAT is evil".

Religion is so much fun.

Shifting the NAT to end system removed the objection to NAT, tho it's not entirely clear why. Shifting NAT to the end system also happened to simplify the entire solution as well.

Except for the part about having to rewrite all existing implementations to take full advantage of the technology.

VJ compression should not be considered a violation of the "end-to- end" principle, as it is a per-link hack and performs a function that CANNOT be performed in the end systems. However, I'm not entirely sure that this is relevant.

Well, if you NAT the destination identifier into a routing locator when a packet traverses the source edge/core boundary and NAT the locator back into the original destination identifier when you get to the core/destination edge boundary, it might be relevant. The advantages I see of such an approach would be: - no need to modify existing IPv6 stacks in any way - identifiers do not need to be assigned according to network topology (they could, in fact, be allocated according to national political boundaries, geographic boundaries, or randomly for that matter). They wouldn't even necessarily have to be IPv6 addresses just so long as they could be mapped and unmapped into the appropriate locators (e.g., they could even be, oh say, IPv4 addresses). - locators could change arbitrarily without affecting end-to-end sessions in any way - the core/destination edge NAT could have arbitrarily many locators associated with it - the source edge/core NAT could determine which of the locators associated with a destination it wanted to use Of course, the locator/identifier mapping is where things might get a bit complicated. What would be needed would be a globally distributed lookup technology that could take in an identifier and return one or more locators. It would have to be very fast since the mapping would be occurring for every packet, implying a need for caching and some mechanism to insure cache coherency, perhaps something as simple as a cache entry time to live if you make the assumption that the mappings either don't change very frequently and/ or stale mappings could be dealt with. You'd also probably want some way to verify that the mappings weren't mucked with by miscreants. This sounds strangely familiar... Obviously, some of the disadvantages of such an approach would be that it would require both ends to play and end users wouldn't be able to traceroute. I'm sure there are many other disadvantages as well. However, if an approach like this would be technically feasible (and I'm not entirely sure it would be), I suspect it would get deployed _much_ faster than an approach that requires every network stack to be modified. Again. Particularly given the number of folks who care about multi-homing are so small relative to the number of folks on the Internet. Can two evils make a good? :-) Rgds, -drc (speaking only for myself, of course)

Certainly does. Apparently this or a similar idea was suggested back in 1997, and is the root origin of the 64 bits for host address space, according to Christian Huitema, in his IPv6 book - http://www.huitema.net/ipv6.asp.

A google search found the draft :

"GSE - An Alternate Addressing Architecture for IPv6" M. O'Dell, INTERNET DRAFT, 1997

http://www.caida.org/outreach/bib/networking/entries/odell97GSE.xml

Note that GSE is in no way a NAT, so is very different than David's proposal. GSE also has a direct impact on all implementations (e.g., only use the identifier bits in the TCP pseudo-header, so that is also an all- implementations change. Further, that is a flag day, worldwide, even for non-multi-homed sites. Tony

...

Shifting the NAT to end system removed the objection to NAT, tho it's not entirely clear why. Shifting NAT to the end system also happened to simplify the entire solution as well.

Except for the part about having to rewrite all existing implementations to take full advantage of the technology.

That was inevitable from the start. A real locator/identifier separation requires a rewrite. Any system that provided site-wide source address control was going to require a rewrite. The bigger issue is that given that rewrite was inevitable, why didn't we have more design freedom. Religion *is* so much fun.

Obviously, some of the disadvantages of such an approach would be that it would require both ends to play and end users wouldn't be able to traceroute. I'm sure there are many other disadvantages as well. However, if an approach like this would be technically feasible (and I'm not entirely sure it would be), I suspect it would get deployed _much_ faster than an approach that requires every network stack to be modified. Again. Particularly given the number of folks who care about multi-homing are so small relative to the number of folks on the Internet.

David, I should point out that if only a small number of folks care about multihoming, then only a small number of folks need to change their stacks. And even in your solution, there would need to be some changes to the end host if you want to support exit point selection, or carry alternate locators in the transport. It's just a mess. I think that we all can agree that a real locator/ identifier split is the correct architectural direction, but that's simply not politically tractable. If the real message that the provider community is trying to send is that they want this, and not IPv6 as it stands today, then that's the message that should be sent, without reference to shim6. Tony

How is a split between locator / identifier any different logicaly from the existing ipv4 source routing?

IPv4 source routing, as it exists today, is an extremely limited mechanism for specifying waypoints along the path to the destination. This is completely orthogonal to a real identifier/locator split, which would divide what we know of as the 'address' into two separate spaces, one which says "where" the node is, topologically, and one which says "who" the node is. One might use the identifier in the TCP pseudo-header, but not the locator, for one example, immediately allowing both mobility and multi-homing. Tony

On Sun, 16 Oct 2005, Tony Li wrote:

This is completely orthogonal to a real identifier/locator split, which would divide what we know of as the 'address' into two separate spaces, one which says "where" the node is, topologically, and one which says "who" the node is.

Hmm, no idea whether it's a good idea or not, but from POV of scaling while it might make 'where' scaleable, you still have to find a way to tie "who" to "where". Some might say we already have this split though, DNS.

Tony

regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: Robotic tape changer mistook operator's tie for a backup tape.

On Mon, 17 Oct 2005, Tony Li wrote:

True. Even better, you get to change this binding (mobility) or have multiple bindings (multihoming).

Indeed.

True enough, but unfortunately, it's not done in a way that we can make use of the identifier in the routing subsystem or in the transport protocols.

Well, if the idea is that the global routing subsystem should not have to burdened with the overwhelming details of "who"->"where", then this mightn't matter much - all routing needs to know is "where". The transport protocols, well they generally act on behalf of something which can do the lookup and supply transport with right address, as long the DNS server does not require "who"->"where" indirection ;). The other way of course is to carry "who"->"where" as routes in our current routing system. Then we just have to figure out how to confine things topologically. Would require changes in how providers peer though, but it is possible. And has been done in other networks, eg GSM in Ireland. We have provider-assigned, but provider-mobile prefixes. Just as with IP multihoming there was much protest that it couldn't be done, would be too problematic, would be too burdensome. However the regulator told the operators "I don't care, you have till X to figure it out and implement". The operators did figure out, presumably including how to do billing for the differentials of any traffic carried for customers who had moved to other providers... The rest of the world has no clue that a large set of Irish GSM telephone numbers are essentially "/32 routed" between Irish providers. ;) A possibility anyway (but whether it's the least worst way - i don't know ;) ). It does though keep operators fully involved in all aspects of routing. Otherwise end-hosts will just work around the 'dumb' providers themselves, if there's no solution operators like. (Not a bad thing either really).

Tony

regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: Money isn't everything -- but it's a long way ahead of what comes next. -- Sir Edmond Stockdale

# >> True enough, but unfortunately, it's not done in a way that we can make # >> use of the identifier in the routing subsystem or in the transport # >> protocols. # > # > The transport protocols, well they generally act on behalf of something # > which can do the lookup and supply transport with right address, as long # > the DNS server does not require "who"->"where" indirection ;). # # The transport protocols unfortunately need the identifier in the packet to # demux connections. the idea of a "transport protocol" comes from the OSI Reference Model which might not be the best conceptual fabric for re-thinking Internet routing. we know it's a "distributed system" and we know that various waypoints will or will not have "state", but i don't think we know that there will always be a "layer" that does what the "transport protocol" does in the OSIRM. i mention this because padlipsky's mantra about maps and territories came into my head just now as i was listening to folks talk about what the "transport protocol" had to have or had to provide. there's only a "transport protocol" if we decide to keep thinking in ISORM terms. and with that, i do indeed wonder if this has stopped being operational and if so, whether nanog wants to overlap THIS much with the irtf? refs: http://www.amazon.com/exec/obidos/tg/detail/-/0132681110/103-3252601-1266225

On Sun, 16 Oct 2005, Joe Maimon wrote:

Tony Li wrote:

...

It's just a mess. I think that we all can agree that a real locator/ identifier split is the correct architectural direction, but that's simply not politically tractable. If the real message that the provider community is trying to send is that they want this, and not IPv6 as it stands today, then that's the message that should be sent, without reference to shim6.

Tony

How is a split between locator / identifier any different logicaly from the existing ipv4 source routing?

I thought that got dead ended?

Or is a table lookup going to be needed?

Wont all those tables need to be in the exact (or close to) same place as the current routing tables?

Appreciate any enlightenment.

For example, if your goal was to have TCP-like sessions between identifiers survive network events without globally propagating full network topology information about your site (the gripe against classic IPv4 BGP) you could have multiple locators associated with any single identifier sort of like the same way you can have multiple A records for a domain name. If the location layer session times out then it would try the other locators listed (pick a method of selection) and if it suceeded would resume the session transparent to the identifier layer. Design the timeout and retransmit algorithm and parameters to achieve the convergence times of your choice. You would need a new protocol stack on the hosts at both ends of connections. By common convention classic TCP hosts could be told to use one of the locators (a transition hack, or just run the protocols in parallel). No change would be required to the network, and existing TCP could continue to be supported (no flag day). Of course support of this new protocol would be limited to the clients and servers that chose to implement it, however this is no less than the change required for IPv6 which some hoped would solve the multihoming problem (possibly defined as scalably supporting network topology change without sessions being interrupted). Mike. +----------------- H U R R I C A N E - E L E C T R I C -----------------+ | Mike Leber Direct Internet Connections Voice 510 580 4100 | | Hurricane Electric Web Hosting Colocation Fax 510 580 4151 | | mleber@he.net http://www.he.net | +-----------------------------------------------------------------------+

On Sun, 16 Oct 2005, Joe Maimon wrote:

Mike Leber wrote:

...

On Sun, 16 Oct 2005, Joe Maimon wrote: For example, if your goal was to have TCP-like sessions between identifiers survive network events without globally propagating full network topology information about your site (the gripe against classic IPv4 BGP) you could have multiple locators associated with any single identifier sort of like the same way you can have multiple A records for a domain name.

Real world shows that that doesnt work very well. Multiple A records is not usuable practicaly speaking for anything other than load balancing, today.

You are missing the point. Currently multihomed sites have multiple path entries in the routing table for a specific multihomed prefix. Instead of having multiple paths, you would have multiple location records in DNS. (Which are A records and any possible reordering by round robbin is not part of the actual path selection algorithm and which are associated with indentifiers though a standard to be designed as part of the new protocol.) The process of how you select which one to use (and what knobs you have for tuning) is a design decision, the same way it was with BGP and OSPF.

...

If the location layer session times out then it would try the other locators listed (pick a method of selection) and if it suceeded would resume the session transparent to the identifier layer. Design the timeout and retransmit algorithm and parameters to achieve the convergence times of your choice.

DNS is a good example of something that was designed that way, but few people rely on common implementations actualy performing it properly.

BGP and OSPF have timeouts and select other paths. They give you the convergence you have now in the event of router failure. For example, a BGP session with a peer accross a public exchange has to time out, your interface is still up. Yes, you have to engineer the protocol for the convergence time you want. Define the goal and figure out the algorithm to achieve it.

...

You would need a new protocol stack on the hosts at both ends of connections. By common convention classic TCP hosts could be told to use one of the locators (a transition hack, or just run the protocols in parallel). No change would be required to the network, and existing TCP could continue to be supported (no flag day).

Appears to me thats what shim6 is (cursory reading + nanog discussions)

Perhaps a shim6 advocate will explain the differences. Does shim6 provide separate identifiers from locators? Does shim6 require new protocol stacks on the hosts at both ends of a session? (If not then the source is not making its own path selection decisions.)

...

Of course support of this new protocol would be limited to the clients and servers that chose to implement it, however this is no less than the change required for IPv6 which some hoped would solve the multihoming problem (possibly defined as scalably supporting network topology change without sessions being interrupted).

Long story short, seperating endpoint/locator does nothing to allow multiple paths to a single IP6 address/prefix to scale.

Um, this is equivalent to saying "it doesn't work because I say so". How doesn't it work? For example you could claim (and then try to defend your claim): * It can't possibly converge quick enough because the genius that went into BGP and OSPF was lost and can never be found again. * (ok seriously) It can't converge quick enough because the timeout would have to be X and based on a guestimate of network topology entropy that would result in Y percent more traffic as each host tries to reestablish locator sessions. (Well, then define what percentage of sessions you think get interruped and support your claim.) * You throw away real topology information and rely on latency (or whatever), and using latency doesn't work because it doesn't allow traffic engineering according to policy. (Who said you have to give everybody the same set of locators? Paul might say ewwww. FWIW, if you want the ability to tell different peers different answers like with BGP you will need the ability to give different answers with the new protocol.) Mike. +----------------- H U R R I C A N E - E L E C T R I C -----------------+ | Mike Leber Direct Internet Connections Voice 510 580 4100 | | Hurricane Electric Web Hosting Colocation Fax 510 580 4151 | | mleber@he.net http://www.he.net | +-----------------------------------------------------------------------+

On Sun, 16 Oct 2005, Mikael Abrahamsson wrote:

Think in the future, do we really want routers that'll handle millions of prefixes and hundreds of thousands of AS numbers, just because people want resiliance? If this can be solved on the end-user layer instead, it's more

you are getting these anyway, thank network convergence for that... or curse it, your call. things like 2547 'vpn' and the like are driving prefix numbers up regardless of what the Internet is doing. Hardware will be required to handle million(s) of prefixes sooner than large scale v6 deployment IMHO. Note, just cause it's there (or will be) doesn't mean I'm advocating that solution for v6, I just had questions (and thought others might as well) about shim6 or the direction of 'site multihoming' in v6... (not even specificly shim6)

On Mon, 17 Oct 2005, Mikael Abrahamsson wrote:

On Mon, 17 Oct 2005, Christopher L. Morrow wrote:

...

you are getting these anyway, thank network convergence for that... or curse it, your call. things like 2547 'vpn' and the like are driving prefix numbers up regardless of what the Internet is doing. Hardware will be required to handle million(s) of prefixes sooner than large scale v6 deployment IMHO.

Both MPLS and any tunneled VPN over IP means the core won't have to know about all those prefixes (think aggregation of addresses regionally in the IP case and outer label in the MPLS case).

'core' doesn't matter so much, somewhere there has to be this knowledge... Perhaps you'll get lucky with some 'edge' devices not having to know about every destination, but I think that might be more rare than you'd like.

So if you're building a 100G capable platform that'll do IPv6 and MPLS, how much difference would it be if you only had to support 16000 labels and 16000 IPv6 prefixes, rather than 2 million?

not sure, I'm a chemical engineer :) Seriously though, is the break 1-2M or is it 10-20m? (doubling or orders of magnitude?)

Then of course I guess the argument can be made to put everything on MPLS to avoid the core knowing about anything but outer labels.

oh yes, mpls everywhere! wait... did I say that? yuck.

On Mon, 17 Oct 2005, Per Heldal wrote:

man, 17,.10.2005 kl. 07.17 +0200, skrev Mikael Abrahamsson:

...

Both MPLS and any tunneled VPN over IP means the core won't have to know about all those prefixes (think aggregation of addresses regionally in the IP case and outer label in the MPLS case).

Hope you don't imply NAT and private addresses like it is usually associated with VPN in the IPv4 world ;)

No, no NAT and RFC1918 implied, even though it might be part of it.

...

Then of course I guess the argument can be made to put everything on MPLS to avoid the core knowing about anything but outer labels.

<flame>MPLS on its own won't solve anything. Although MPLS has its uses, it smells too much like another desperate attempt from the telco-heads in the ITU crowd to make a packet-switched network look and behave like a circuit-switched network.</flame>

Why? The initial argument for MPLS was that it would solve the core problem and put intelligence at the edge. You would have a core that only needed to know about hundreds of nodes instead of 100.000:nds of nodes.

Growth can't be eliminated. In the future network you'll have routers that may know a lot about their "local region" of the network but have to rely on nodes that are several hops (even AS-hops) away to pass the packets to more remote destinations. These trust-relationships have to

Yes, that is what's being proposed. Know your internal nodes, announce single big prefix externally. With ISPs only having a single prefix and no "single customer" prefixes, routing table can be kept low. Redundancy can be solved with for instance shim6.

alternative. Without improved routing protocols, all we can do is to pray that the development of routing hardware in terms of memory and processing capability outpaces the growth of the routing table.

We have done this for 15 years or so, what good has it brought us? Yes, TCAM size etc has been fairly good in keeping up with routing table size, but at quite high cost.

Initiatives like shim6 that changes the behaviour of leaf-nodes are only a supplement and won't replace the need for true multi-homing for end-sites. Here we have to adapt to business needs, and businesses have

Why? What problem does multihoming with single prefix solve that a fully working shim6 doesn't? What is the argument that the "internet" needs to know about a lot of end-users, instead of the end-user knowing that each end user might have n number of IP addresses and that there are n^2 combinations to send packets? Convergence time in the real world today is in the minutes, with shim6 it would for the end user be much quicker to "route around" the problem. Shouldn't be any problem to have failover in the subsecond timeframe, even thought that might need some kind of hello mechanism that is suboptimal because it sends traffic not carrying any data.

single provider. Besides, shim6 doesn't eliminate the need for a mechanism to locate any globally unique address. What if there's

I thought DNS solved that?

suddenly 10M LIR's, or otherwise a trend towards a market with very small providers each handling only a small number of customers? Who gets to decide who may peer with whom, or decide which providers will be denied the ability to build redundant connectivity with multiple upstreams?

It costs money to maintain a LIR which limits the number of LIRs economically viable in the world. -- Mikael Abrahamsson email: swmike@swm.pp.se

On Mon, 17 Oct 2005, Per Heldal wrote:

Well, let's try to turn the problem on its head and see if thats clearer; Imagine an internet where only your closest neighbors know you exist. The rest of the internet knows nothing about you, except there are mechanisms that let them "track you down" when necessary. That is very different from today's full-routing-table.

Yes, it's true that it's different, but is it better?

It does not provide 100% provider-indepence to begin with. Depending on who you ask that alone is a show-stopper.

Well, the reason for people wanting to stick to their "own" IP adresses are administrative and technical. If we solve that then hopefully, it wont be such a big hassle to renumber to go to another provider. Also, if everybody got their equal size subnet delegation from each ISP then it shouldnt be that much of a problem to run two "networks" side-by-side by using the subnet part of the delegation equal to both networks, but keep the prefix separate. If you switch providers you change the prefix part. This means we need new mechanisms to handle this, but I feel that's better than doing the routing mistake again.

The internet shouldn't need to know anything about individual users to begin with, provided there are mechanisms avilable track them down. By that I mean that algorithms to locate end-nodes may include mechanisms to "interrogate" a large number of nodes to find the desired location as opposed to looking it up in a locally stored database (routing-table).

So what is it you're proposing? I understand what shim6 tries to do (since it basically keeps most of todays mechanisms) but I do not understand your proposal. Could you please elaborate?

I thought DNS only provided a name for an address ;) How does DNS tell us that e.g. 193.10.6.6 is part of a subnet belonging to AS2838 and how to get there?

Should end users really care for that level of routing information? Also, your proposal seems to indicate that we need something that sounds like a proxy server that actually do know more about the internet and who needs to keep state, this doesn't sound scalable? -- Mikael Abrahamsson email: swmike@swm.pp.se

man, 17,.10.2005 kl. 07.25 -0700, skrev Fred Baker:

is that anything like using, in Cisco terms, a "fast-switching cache" vs a "FIB"?

I'll bite as I wrote the paragraph you're quoting; Actually, hanging on to the old concepts may be more confusing than trying to look at it in completely new ways. Imagine a situation with no access to any means of direct communication (phone etc). You've got a message to deliver to some person, and have no idea where to find that person. Chances are there's a group of people nearby you can ask. They may know how to find the one you're looking for. If not they may know others they can ask on your behalf. Several iterations later the person is located and you've established a path through which you can pass the information you wanted. Translated into cisco terms this mean that the FIB is just a partial routing database, enough to start the search and otherwise handle communications in the neighborhood (no more than X router-hops, maybe AS-hops away). When the destination is located you keep that information for a while in case there are more packets going to the same place, similar to what you do with traditional route-cache.

On Oct 17, 2005, at 6:47 AM, Mikael Abrahamsson wrote:

...

...

Well, let's try to turn the problem on its head and see if thats clearer; Imagine an internet where only your closest neighbors know you exist. The rest of the internet knows nothing about you, except there are mechanisms that let them "track you down" when necessary. That is very different from today's full-routing-table.

mon, 17,.10.2005 kl. 11.29 -0700, Fred Baker:

OK. What you just described is akin to an enterprise network with a default route. It's also akin to the way DNS works.

No default, just one or more *potential* routes. Your input is appreciated, and yes I'm very much aware that many people who maintain solutions that assume full/total control of the entire routing-table will be screaming bloody murder if that is going to change. Further details about future inter-domain-routing concepts belong in other fora (e.g. ietf's inter-domain-routing wg). The long-term operational impact is that the current inter-domain-routing concepts (BGP etc) don't scale indefinitely and will have to be changed some time in the future. Thus expect the size of the routing-table to be eliminated from the list of limiting factors, or that the bar is considerably raised. --- Note that I'm not saying that nothing should be done to preserve and optimise the use of the resources that are available today just because there will be something better available in a distant future. I'm in favor of the most restrictive allocation policies in place today. The development of the internet has for a long time been based on finding better ways to use available resources (CIDR anyone). To me a natural next-step in that process is for RIR's to start reclaiming unused v4 address-blocks, or at least start collect data to document that space is not being used (if they're not already doing so). E.g prevously announced address-blocks that has disappeared from the global routing-table for more than X months should go back to the RIR-pool (X<=6). //Per

Fred,

If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion.

There is a fundamental difference between a one-time reduction in the table and a fundamental dissipation of the forces that cause it to bloat in the first place. Simply reducing the table as a one-off only buys you linearly more time. Eliminating the drivers for bloat buys you technology generations. If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again. Regards, Tony

...

There is a fundamental difference between a one-time reduction in the table and a fundamental dissipation of the forces that cause it to bloat in the first place. Simply reducing the table as a one-off only buys you linearly more time. Eliminating the drivers for bloat buys you technology generations.

If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again.

That's the goal here? To ensure we'll never have another protocol transition? I hope you realize what a flawed statement that is.

tony probably did not think about it because that's not what he said at all. he was speaking of routing table bloat, not transitions. and he was spot on. randy

Daniel,

...

If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again.

That's the goal here? To ensure we'll never have another protocol transition? I hope you realize what a flawed statement that is. We can't see into the future. However, assuming that IPv6 is the Last Protocol seems a bit short sighted. If we get 20 years out of IPv6, that will be just peachy.

I see that as a worthy goal and no, I don't see that as flawed. While we certainly cannot guarantee that v6 will be the last protocol, we should certainly be designing for it to be the best that we can possibly make it. Just how many times do you think that we will replace all implementations? This change is simply fundamental to the way the Internet works. There is almost as much pain associated with this change as if we were to change the electric outlet voltage in every single country to a mutually incompatible standard. Can you imagine power companies making that change and then telling consumers to expect another such change in 20 years? To not even *attempt* to avoid future all-systems changes is nothing short of negligent, IMHO.

Of course, if we can't get PI address space for enterprises and real multihoming, there won't be any real IPv6 deployment. Lots of (possibly illegitimate) IPv4 trading and NAT, but not IPv6.

These aren't nice to haves. Even if it shortens the life of IPv6, that is an acceptable trade-off.

IPv6 is not the Last Protocol.

If you do get PI space for multihoming, then by definition, it cannot be the last protocol. In fact, it will have cemented v6's lifetime as just 10 more years. Tony

Wasn't Noel Chiappa Nimrods "father" ? He explained his philosophy to me in an interview a decade ago as well as why he believed that BGP was not sustainable. yet here we are still chugging along meanwhile back to your operational flows ;-) ============================================================= The COOK Report on Internet Protocol, 431 Greenway Ave, Ewing, NJ 08618 USA 609 882-2572 (PSTN) 415 651-4147 (Lingo) cook@cookreport.com Subscription info: http://cookreport.com/subscriptions.shtml IMS and an Internet Economic & Business Model at: http://cookreport.com/14.09.shtml ============================================================= On Oct 17, 2005, at 5:58 PM, Fred Baker wrote:

now, the proposal put forward lo these many moons ago to avoid any possibility of a routing change was, as I recall, Nimrod, and the Nimrod architecture called for variable length addresses in the network layer protocol and the use of a flow label (as in "IPv6 flow label") as a short-form address in some senses akin to a virtual circuit ID.

tony.li@tony.li (Tony Li) writes:

Specifically, the IAB should call for a halt to IPv6 deployment until consensus is reached on a scalable routing architecture. I realize that this is painful, but continuing to deploy is simply creating a v6 mortgage that we cannot afford to pay off.

well, maybe so. but IAB could never make deployment start, and IAB cannot make deployment stop. deployment happens on its own terms. leadership has a built in time delay and a couple layers of indirection between intent and action and results. if you want IAB to lead somehow, give them some runway. -- Paul Vixie

works for me - I did say I'd like to change the routing protocol - but I think the routing protocol can be changed asynchronously, and will have to. On Oct 17, 2005, at 1:51 PM, Tony Li wrote:

Fred,

...

If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion.

There is a fundamental difference between a one-time reduction in the table and a fundamental dissipation of the forces that cause it to bloat in the first place. Simply reducing the table as a one- off only buys you linearly more time. Eliminating the drivers for bloat buys you technology generations.

If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again.

Regards, Tony

At 04:51 PM 10/17/2005, Tony Li wrote:

Fred,

...

If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion.

There is a fundamental difference between a one-time reduction in the table and a fundamental dissipation of the forces that cause it to bloat in the first place. Simply reducing the table as a one-off only buys you linearly more time. Eliminating the drivers for bloat buys you technology generations.

If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again.

But wasn't that the rationale for originally putting the kitchen sink into IPv6, rather than fixing the address length issue? I think we missed a lot of opportunities. Extended addressing may well have been possible to integrate in the mid 1990's ahead of much of the massive Internet expansion. Too late. We're 10 years on, and talking about whether there will need to be more than one massive pain of migration, because the kitchen sink didn't take into account multihoming. Now we're talking about a solution that appear to be an even worse Rube Goldberg than token ring source-route bridging. Moore will likely have to continue to produce the solution.

...

Moore will likely have to continue to produce the solution.

What happens if he can't? Silicon technology *is* topping out. What happens to v6 if every single household and business on the planet decides to multihome?

I often wonder what would happen if IETF and NANOG were to collectively swear off reductio ad absurdum scenarios. For extra credit, the reader is invited to compare and contrast the relative probabilities of "pervasive end-site multihoming" and "IETF and NANOG learning what a 'false dichotomy' is". Traditional multihoming techniques work today and buy us a decade or more _until the v6 routing table is the same size as today's v4 table_. That's a decade to make Pad's seminal work (cited by Vixie earlier; it's nice to have it back in print!) required reading for IETF participation and smack some sense into the acolytes of Rube Goldberg whose influence on shim6 is all too apparent. ---Rob

we agree that at least initially every prefix allocated should belong to a different AS (eg, no AS gets more than one); the fly in that is whether there is an ISP somewhere that is so truly large that it needs two super-sized blocks. I don't know if such exists, but one hopes it is very much the exception. The question is "does every AS get a prefix". Under current rules, most AS's assigned to edge networks to support multihoming will not get a prefix. I personally think that's probably the wrong answer (eg, you and I seem to agree on PI space for networks that would warrant an AS number does to size, connectivity, and use of BGP to manage their borders), but it is the current answer. On Oct 17, 2005, at 2:06 PM, Per Heldal wrote:

...

The RIRs have been trying pretty hard to make IPv6 allocations be one prefix per ISP, with truly large edge networks being treated as functionally equivalent to an ISP (PI addressing without admitting it is being done). Make the bald assertion that this is equal to one prefix per AS (they're not the same statement at all, but the number of currently assigned AS numbers exceeds the number of prefixes under discussion, so in my mind it makes a reasonable thumb-in-the-wind- guesstimate), that is a reduction of the routing table size by an order of magnitude.

I wouldn't even characterise that as being bald. Initial allocations of more than one prefix per AS should not be allowed. Further; initial allocations should differentiate between network of various sizes into separate address-blocks to simplify and promote strict prefix-filtering policies. Large networks may make arrangements with their neighbors to honor more specifics, but that shouldn't mean that the rest of the world should accept those.

Thus spake "Fred Baker" <fred@cisco.com>

The RIRs have been trying pretty hard to make IPv6 allocations be one prefix per ISP, with truly large edge networks being treated as functionally equivalent to an ISP (PI addressing without admitting it is being done). Make the bald assertion that this is equal to one prefix per AS (they're not the same statement at all, but the number of currently assigned AS numbers exceeds the number of prefixes under discussion, so in my mind it makes a reasonable thumb-in-the-wind- guesstimate), that is a reduction of the routing table size by an order of magnitude.

If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion.

If we reduce the average number of prefixes per AS by an order of magnitude, IMHO the result will be that there will be an order of magnitude growth in the number of ASes. We're just going to trade one problem for another. What we need is an interdomain routing system that can either (a) drastically reduce the incremental cost of additional prefixes in the DFZ, or (b) move the exist cost out of the DFZ to the people who want to multihome. Both probably mean ditching BGP4 and moving to some sort of inter-AS MPLS scheme, but it will never see the light of day unless it allows leaving hosts and intra-site routing intact (i.e. hop-by-hop routing and a single prefix per site). This last is why shim6 is DOA. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin

E.g prevously announced address-blocks that has disappeared from the global routing-table for more than X months should go back to the RIR-pool (X<=6).

In RFC 2050 section 3 a) the organization has no intention of connecting to the Internet-either now or in the future-but it still requires a globally unique IP address. The organization should consider using reserved addresses from RFC1918. If it is determined this is not possible, they can be issued unique (if not Internet routable) IP addresses. Seems to me that the Internet routing table contents past and present are irrelevant. Note also that the so-called Internet routing table contents vary depending on where you look at it. In any case, at the ARIN meeting in LA there will be an Open Policy Hour for people to suggest things that the RIRs should do. More info here: http://www.arin.net/ARIN-XVI/agenda/tuesday.html#policy --Michael Dillon

Obviously if the RIRs contacted the folks responsible for a given block and were provided justification for its continued allocation, then it should not be reclaimed. On the other hand, folks sitting on several class Bs and not using them could have their blocks reclaimed trivially; ditto for companies that no longer exist. The last is certainly doable without much risk of

controversy.

This is exactly what the Internic did many years ago. I, like many other people, had registered a .com domain name at no cost. Then suddenly one day, the Internic said that I had to pay an annual subscription fee for this domain name. Like many others, I paid my fees. There were a few complaints about this but by and large people accepted the idea that you had to MAINTAIN a business relationship with the domain name registry in order to continue using a domain name. For some reason, this concept of MAINTAINING a business relationship with the registry, has not caught on in the Regional IP Registry arena. Of course, a large number of IP address users do pay annual membership subscription fees to ARIN (or other RIRs) but not all. And the RIRs seem unwilling to withdraw services (in-addr.arpa) from those who do not MAINTAIN a business relationship.

However, one of the articles referred to recently in this thread (I forget which) showed that even if we reclaimed all of the address space that is

currently unannounced (in use or not), we'd buy ourselves a trivially short extension of the IPv4 address space exhaustion date. Considering the cost of performing the task, doing so seems rather pointless; our time would be better spent getting IPv6 deployed and either reengineering the routing system or switching to geo addresses.

From the viewpoint of avoiding an addressing crisis and avoiding a v6 transition crisis, your advice is sound. However, from the viewpoint of having a sensibly functioning RIR system, I think we still need to deal with two issues. One is that the holders of IP address allocations should be required to maintain a business relationship with the RIR or lose the right to use those IP addresses. The other is

Probably this article from the Cisco IP Journal which has been mentioned a few times in the past week. http://www.cisco.com/en/US/about/ac123/ac147/archived_issues/ipj_8-3/ipv4.ht... that the RIRs need to fix the whole debacle that is "whois" and "routing registries". There should be no need for 3rd party bogon lists. The RIR's should publish an authoritative registry, rooted in IANA, that covers the entire IPv4 and IPv6 address spaces. An operator faced with receiving a new BGP announcement should be able to query such a registry and find out whether or not the address block is allocated, who it is allocated to, and whether that party intends to announce that exact block size from that exact AS number. --Michael Dillon

On Mon, Oct 17, 2005 at 09:03:45AM -1000, Randy Bush wrote:

...

Imagine a situation with no access to any means of direct communication (phone etc). You've got a message to deliver to some person, and have no idea where to find that person. Chances are there's a group of people nearby you can ask. They may know how to find the one you're looking for. If not they may know others they can ask on your behalf. Several iterations later the person is located and you've established a path through which you can pass the information you wanted.

Translated into cisco terms this mean that the FIB is just a partial routing database, enough to start the search and otherwise handle communications in the neighborhood (no more than X router-hops, maybe AS-hops away). When the destination is located you keep that information for a while in case there are more packets going to the same place, similar to what you do with traditional route-cache.

check out "The Landmark Hierarchy: A New Hierarchy for Routing in Very Large Networks"; Paul Tsuchiya; 1989.

great stuff... i have a hardcopy. is it online yet? --bill (checking citesear...)

randy

--bill (checking citesear...)

does that only yield rare papers :-) and citeseer does not have the paper, only a few cites to it randy