Re: shim6 @ NANOG (forwarded note from John Payne)
[Crossposted to shim6 and NANOG lists, please don't make me regret this... Replies are probably best sent to just one list for people who don't subscribe to both.] On 27-feb-2006, at 22:13, Jason Schiller (schiller@uu.net) wrote:
Is it the consensus of the shim6 working group that the full suite of TE capabilities should not be a requirement? Or is this just the opinion of a few vocal people?
I don't think I'm going out on a limb when I say that there is consensus that we need good enough traffic engineering from the start. (Where "start" means deployment, not necessarily the publication of the first RFC.) I think basic balancing of both incoming and outgoing traffic over the available links is both assumed to be part of what we need to have and implementable without too much trouble. Push back by transit ASes is harder. This is what I mean by that: A --- B / \ X Y \ / C --- D C's link to D may be low capacity or expensive, so D would prefer it if X would send traffic to Y over another route if possible. C can make this happen in BGP by prepending its AS one or more times so X will see the following AS paths: A B Y C C C D Y All else being equal, X will choose the path over A to reach Y. The simple answer here is that if the multihomed site receives a BGP feed just like today (except that it's a read only feed) and thus makes outgoing path selection decisions just like today, transit ASes have exactly the same tools as they have today. But presumably, if shim6 takes off many smaller sites that aren't comfortable with BGP will multihome also, so this push back won't work as well anymore. Creating a new way to accomplish this result is probably possible, but not entirely non-trivial, and probably something we wouldn't want to deliver on day one. Thoughts? Another capability that would be hard to replicate with shim6 is selective announcement. Today, many transit ASes allow multihomed sites to influence the way their prefix is propagated to neighbors of the transit AS. For instance, in the picture above X may decide that the link between C and D is of low quality, and set a community on the prefix it sends to C that tells C either that it should perform AS path prepending on X's prefix ONLY towards D and not towards other neighbors of C, or even not announce the prefix at all. We would need considerable extra mechanisms to replicate this capability, and maybe it can't even be fully replicated at all. So how critical is this capability?
On Tue, 28 Feb 2006, Iljitsch van Beijnum wrote:
A --- B / \ X Y \ / C --- D
C's link to D may be low capacity or expensive, so D would prefer it if X would send traffic to Y over another route if possible. C can make this happen in BGP by prepending its AS one or more times so X will see the following AS paths:
A B Y C C C D Y
All else being equal, X will choose the path over A to reach Y.
There's plenty of route mangler technologies out there that provide overriding BGP information to borders that trumps path length. "All else" is often not as equal as you seem to expect. It's time to wake up and smell the intelligent routing trend. The usefulness of prepending is rapidly dwindling. Don't try to push it as a future-compatible solution; it is not. Prepending is not a tool; it is a hack that has outlived its usefulness.
Another capability that would be hard to replicate with shim6 is selective announcement.
Now, selective announcement is something completely different -- but it's still a historical hack for lack of better mechanisms in BGP[34]. If the route isn't there at all, it won't be selected in today's world. But also consider this: - C does not advertise the prefix for Y, but it does have the next superprefix for Y (and C is "transit", so the superprefix must be considered valid); - X's link to A dies. So X will still try to push packets over C to reach Y, and per the existence of the superprefix on C, that route should[!] be valid. Don't think this will forever be a rare circumstance, either. The route mangling technologies I mentioned above are now starting to offer the ability for traffic to go out a "transit" neighbor so long as some containing prefix is advertised (even if it's not the most specific). Traffic engineering is happening on both ends of the BGP mesh *today*, so you should present any proposed solution in that context. -- -- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>
On 28-feb-2006, at 16:34, Todd Vierling wrote:
A B Y C C C D Y
All else being equal, X will choose the path over A to reach Y.
There's plenty of route mangler technologies out there that provide overriding BGP information to borders that trumps path length. "All else" is often not as equal as you seem to expect.
It's time to wake up and smell the intelligent routing trend. The usefulness of prepending is rapidly dwindling. Don't try to push it as a future-compatible solution; it is not. Prepending is not a tool; it is a hack that has outlived its usefulness.
In my experience, if anything, AS path prepending is TOO effictive: just one prepend can make a 60/40 split that you're trying to get to 50/50 into 25/75 instead. So I agree that it's not as useful as it used to be, but I blamed this on the flattening of the AS interconnection hierarchy. But maybe it's the routing/TE boxes that are responsible.
Another capability that would be hard to replicate with shim6 is selective announcement.
Now, selective announcement is something completely different -- but it's still a historical hack for lack of better mechanisms in BGP[34]. If the route isn't there at all, it won't be selected in today's world.
Right. That would be hard to accomplish with shim6.
But also consider this:
- C does not advertise the prefix for Y, but it does have the next superprefix for Y (and C is "transit", so the superprefix must be considered valid);
- X's link to A dies.
So X will still try to push packets over C to reach Y, and per the existence of the superprefix on C, that route should[!] be valid.
This kind of thing is, as far as I can see, pretty much impossible to replicate in shim6. Mind you, even if we end up with PI in IPv6, it's unlikely that you get to do this with IPv6 because the address space and the provider aggregates are so large, that deagregating becomes a hazard rather than a nuisance. Deaggregating a /32 into /48 makes for upto 65536 additional routes, which is a third of the current IPv4 routing table (and several dozen times the current IPv6 routing table). So I think most people will use strict prefix length filters to avoid this. At least, after it has happened for the first time.
Don't think this will forever be a rare circumstance, either. The route mangling technologies I mentioned above are now starting to offer the ability for traffic to go out a "transit" neighbor so long as some containing prefix is advertised (even if it's not the most specific).
Traffic engineering is happening on both ends of the BGP mesh *today*, so you should present any proposed solution in that context.
I'm not too worried about what happens on both ends: since both ends implement the shim protocol and the two ends communicate with each other, we can build in whatever is required. The challenges are: - getting site wide policies into the individual hosts or apply side wide policies in middleboxes in a secure way - come up with a reasonable way to have information "in the middle" taken into account And we have to figure out which capabilities must be present as a mandatory part of the specification on day one, and which can be optional and/or added later. (Ideally, all TE is kept outside of the base spec because modularity makes everything easier, but some stuff is only useful if it's everywhere so it either has to be mandatory or forget it, and other stuff is so important that we need it from day one.)
On Feb 28, 2006, at 6:31 AM, Iljitsch van Beijnum wrote:
[Crossposted to shim6 and NANOG lists, please don't make me regret this... Replies are probably best sent to just one list for people who don't subscribe to both.]
On 27-feb-2006, at 22:13, Jason Schiller (schiller@uu.net) wrote:
Is it the consensus of the shim6 working group that the full suite of TE capabilities should not be a requirement? Or is this just the opinion of a few vocal people?
I don't think I'm going out on a limb when I say that there is consensus that we need good enough traffic engineering from the start. (Where "start" means deployment, not necessarily the publication of the first RFC.)
I think basic balancing of both incoming and outgoing traffic over the available links is both assumed to be part of what we need to have and implementable without too much trouble.
Some problems/issues that are solved by current IPv4 TE practices that we are currently using, that we can't do easily in Shim6: 1) Prepending/tagging routes to influence the amount of inbound we receive from certain providers 2) Announcing more specifics to some peers/transit to influence which POP certain traffic is received 3) Announcing less specifics (total aggregate announcement) to "backup" transit provider/connections that we don't want to receive traffic on unless something is really really wrong 4) Being able to do 1-3 in realtime, in one place, without waiting for DNS caching or connections to expire 5) Being able to make routing/policy changes without having to rely on the owners/administrators of the machines/sites/domains themselves to do the right thing. (i.e. untrusted/not-maintained-by-us systems/ networks on our network) 6) Anycast? 7) During what will be a very lengthy dual-stack transitional period, having to do TE in two entirely different ways. BGP+Prepending +Selective-announcements along side Shim6 doesn't really sound like fun to me. We can't treat bits as bits, we have to consider if they're IPv4 bits or IPv6 bits, and engineer them differently, even though they're sharing the same lines and are probably going to have a 1:1 addressing relationship between IPv4 and IPv6 services. On top of those, even if shim6 accomplishes the failover and reliability goals, I can't see how shim6 is going to make path decisions as optimal as IPv4/BGP/etc. My last IPv6 experiment proved that if we're going to provide IPv6, it has to be as fast to the end user as IPv4 is, or users will switch off their IPv6 stack entirely. If an end user is running a dual stack system, sees slow performance a non-optimal path being chosen via shim6, they'll turn IPv6 off so they can reach the IPv4 version of the site. Anything we do has to ensure that IPv6 has AT LEAST the same visible performance to the end user, or they're not going to be willing switchers. I'm not saying that shim6 is going to CAUSE routing problems, but a lot of thought is being given to localprefs, MEDs, prepending, and bunch of other strategies to select the best path for a given destination. NSPs have designed their routing policies (hopefully) to take the best path whenever possible, and BGP allows for those decisions to change in relatime. Shim6 is capable of picking a valid route, but can't see enough into the network to select "best". It works if you want to maintain reliability, but not if you're multihoming to increase performance, not just stability. Shim6 is great for a lot of people. I know that not everyone wants to run BGP just to handle multiple connections. But, Shim6 isn't a replacement for what a lot of us are doing now.
On 28-Feb-2006, at 11:09, Kevin Day wrote:
Some problems/issues that are solved by current IPv4 TE practices that we are currently using, that we can't do easily in Shim6:
Just to be clear, are you speaking from the perspective of an access provider, or of an enterprise? Joe
--- Joe Abley <jabley@isc.org> wrote:
On 28-Feb-2006, at 11:09, Kevin Day wrote:
Some problems/issues that are solved by current IPv4 TE practices that we are currently using, that we can't do easily in Shim6:
Just to be clear, are you speaking from the perspective of an access provider, or of an enterprise?
It's good to clarify that those are quite different requirement sets. One thing which Shim6 does not provide easily is the ability for an enterprise to have policy decisions made in a very limited number of places - for instance, a customer has two Internet pipes to two different providers to their DMZ. Right now, that means that BGP gets spoken by two routers (maybe four at most), and all external policy decisions happen there. By moving the decision-making to the hosts, it's possible to have different decisions being made on each of the 85 webservers being served by those two Internet pipes. "But each of the servers is optimizing the path for its own traffic" Correct, but what if there are other policy goals? I.e. "don't use pipe 2 unless pipe 1 is full/down, because it's more expensive" "only send low-jitter traffic to pipe-2" Whatever mechanism is selected, it needs to support an intermediate-system-based routing decision algorithm, not just an end-system-based approach. -David David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On Feb 28, 2006, at 10:28 AM, Joe Abley wrote:
On 28-Feb-2006, at 11:09, Kevin Day wrote:
Some problems/issues that are solved by current IPv4 TE practices that we are currently using, that we can't do easily in Shim6:
Just to be clear, are you speaking from the perspective of an access provider, or of an enterprise?
In my case, we'd be best described as "content provider". As in: Our primary business does not include providing access to others We multihome extensively, and have multiple POPs scattered around If it weren't for some branching out into unrelated areas, we wouldn't have qualified for IPv6 PI space, and most others like us wouldn't at all. I mean nothing but respect for the work you guys have put into shim6. I realize there are significant problems in scaling the current architecture much higher. My only objection really is this line of thinking: If you're not huge(providing access to hundreds of networks, or can demonstrate a huge number of devices), you're not getting PI space. If you don't get PI space, you're not going to announce your PA space anywhere, your ISP's announcement of their /32 handles that for you. If you're using PA space and you want to multihome, shim6 is how you're going to do it. I'm not saying shim6 is flawed beyond anyone being able to use it. I can see many scenarios where it would work great. However, I'm really wary of it becoming the de facto standard for how *everyone* multihomes if they're under a certain size. I'm just bringing up my objections now, so that it's really clear that shim6 doesn't provide what a lot of us smaller networks are doing now in IPv4 land. -- Kevin
On 28-Feb-2006, at 11:52, Kevin Day wrote:
I'm not saying shim6 is flawed beyond anyone being able to use it. I can see many scenarios where it would work great. However, I'm really wary of it becoming the de facto standard for how *everyone* multihomes if they're under a certain size. I'm just bringing up my objections now, so that it's really clear that shim6 doesn't provide what a lot of us smaller networks are doing now in IPv4 land.
These are important things to point out, and I'd encourage you to say them on the shim6 list too. There are ideas floating around about extending the shim6 such that the protocol between hosts can be mediated by middleboxes, such that site policies can be imposed upon the more opportunistic actions of the end stations. These ideas would have far more currency if it could be shown that they help to meet requirements of operators which are otherwise not addressed. It seems to me that hosting companies who do not provide access (and hence who don't qualify for PI space under the current harmonised RIR v6 policies) ought to have a lot to say about this, more so than enterprises in some respects (e.g. due to the impact of shim6 state on load balancers and servers). Joe
On 28-feb-2006, at 17:09, Kevin Day wrote:
Some problems/issues that are solved by current IPv4 TE practices that we are currently using, that we can't do easily in Shim6:
Well, you can't do anything with shim6 because it doesn't exist yet. That's the good part: if you speak up now, you can get capabilities added before the spec is finished.
1) Prepending/tagging routes to influence the amount of inbound we receive from certain providers
Should be doable with a DNS SRV record like mechanism. Don't worry too much about this one.
2) Announcing more specifics to some peers/transit to influence which POP certain traffic is received
Actually you could still do that with shim6: whatever happens between you and your ISP is your business and doesn't inflate the global routing table. In practice, you'd probably have different /48 blocks for different POPs to begin with so for stuff where you can differentiate on destination address, you can very easily get the traffic to the place where you want it to be.
3) Announcing less specifics (total aggregate announcement) to "backup" transit provider/connections that we don't want to receive traffic on unless something is really really wrong
This is something that is incompatible with shim6. So if we want to retain this functionality, we have to go back to what you're really trying to do and then come up with a new, shim6-compatible way of doing it.
4) Being able to do 1-3 in realtime, in one place, without waiting for DNS caching or connections to expire
How fast is real time? And are we just talking about changing preferences here, or about what happens when there are outages?
5) Being able to make routing/policy changes without having to rely on the owners/administrators of the machines/sites/domains themselves to do the right thing. (i.e. untrusted/not-maintained-by- us systems/networks on our network)
If you're a multihomed hosting company you would want to do TE for your entire POP, but you wouldn't necessarily be able to change information in the DNS for all the hosts/services that your customers run. Is that what you mean?
6) Anycast?
I don't think shim6 applies to interdomain anycast. (Which is a hack anyway.)
7) During what will be a very lengthy dual-stack transitional period, having to do TE in two entirely different ways. BGP +Prepending+Selective-announcements along side Shim6 doesn't really sound like fun to me. We can't treat bits as bits, we have to consider if they're IPv4 bits or IPv6 bits, and engineer them differently, even though they're sharing the same lines and are probably going to have a 1:1 addressing relationship between IPv4 and IPv6 services.
:-) This is a result of the transition to IPv6, regardless of shim6.
On top of those, even if shim6 accomplishes the failover and reliability goals, I can't see how shim6 is going to make path decisions as optimal as IPv4/BGP/etc.
Really??? The way I see it, BGP decisions today are mediocre at best. If anything, I would expect things to get better with shim6.
My last IPv6 experiment proved that if we're going to provide IPv6, it has to be as fast to the end user as IPv4 is, or users will switch off their IPv6 stack entirely. If an end user is running a dual stack system, sees slow performance a non-optimal path being chosen via shim6, they'll turn IPv6 off so they can reach the IPv4 version of the site. Anything we do has to ensure that IPv6 has AT LEAST the same visible performance to the end user, or they're not going to be willing switchers.
Tell it to the people who still do IPv6 routing the way they did in 1999... It's not much fun to go from one part of Europe to another through Japan. Fortunately, this is getting better all the time, but we're not there yet. But also orthogonal to IPv6.
On Feb 28, 2006, at 2:22 PM, Iljitsch van Beijnum wrote:
Should be doable with a DNS SRV record like mechanism. Don't worry too much about this one.
Where does the assumption that the network operators control the DNS for the end hosts come from?
On 28-feb-2006, at 23:15, John Payne wrote:
Should be doable with a DNS SRV record like mechanism. Don't worry too much about this one.
Where does the assumption that the network operators control the DNS for the end hosts come from?
...or in another way. Don't worry too much about this one.
On Feb 28, 2006, at 4:21 PM, Iljitsch van Beijnum wrote:
On 28-feb-2006, at 23:15, John Payne wrote:
Should be doable with a DNS SRV record like mechanism. Don't worry too much about this one.
Where does the assumption that the network operators control the DNS for the end hosts come from?
...or in another way. Don't worry too much about this one.
Well, make sure you're taking into account ALL of these situations, as they all exist currently: 1) We run the servers, DNS and connectivity for a website. Should be the easy case. 2) We run the DNS and connectivity for the site, but do not control the server at all. (No root access to the server, must rely on the customer to follow instructions to setup, can't be asking them to make changes.) 3) We run the server and connectivity, but do not have control of DNS. (Customer is using their registrar's DNS services) 4) We provide connectivity only. (Colocation. We have no control over DNS or what goes on inside the server) 5) We provide DNS services to an entire domain, and have no involvement in the actual connectivity of any services on the site. (EasyDNS, etc) How can I, as a hypothetical hosting company, manage traffic engineering under all of these situations with shim6? If we do not control the server itself, we're completely reliant on customers to "do the right thing". We can't ask them to change things on their end for traffic engineering(we change it too much, and it's not their problem). We can't trust that they won't modify their hosts' behavior in ways that would suit them. If you're saying we don't need to rely on the server side at all to DTRT, the solution either has to come in on the DNS side (which we also don't always control, and takes too long to update) or additional functionality added to the router/firewall/load balancer/ something. I can't imagine that going over well with hosting/content companies either. No matter how you look at this, the routing policy and routing decisions need to be made somewhere. There isn't any one point where a hosting company can do this where it's guaranteed they have control of it. If you're suggesting that this be changed, that's further raising the bar for IPv6 deployment. If people have to change their business models around a new addressing scheme, it's not going to be a very willing move. -- Kevin
On 2/28/06 5:21 PM, "Iljitsch van Beijnum" <iljitsch@muada.com> wrote:
On 28-feb-2006, at 23:15, John Payne wrote:
Should be doable with a DNS SRV record like mechanism. Don't worry too much about this one.
Where does the assumption that the network operators control the DNS for the end hosts come from?
...or in another way. Don't worry too much about this one.
Unacceptable. This is the whole problem with shim6 - the IETF telling us to "sit back and enjoy it, because your vendors know what's best". This attitude combined with Shim6's (many) limitations speed it toward irrelevance. -- Daniel Golding
On 28-Feb-2006, at 23:37, Daniel Golding wrote:
Unacceptable. This is the whole problem with shim6 - the IETF telling us to "sit back and enjoy it, because your vendors know what's best".
Actually, I think the problem with shim6 is that there are far too few operators involved in designing it. This has evidently led to a widespread perception of an ivory tower with a moat around it.
This attitude combined with Shim6's (many) limitations speed it toward irrelevance.
To gain real relevance it needs to be deployed; to be deployed, it needs to be embraced by enterprise operators and content providers. If these operators dismiss it out of hand on principal, and refuse to actually find out whether the general approach is able to solve problems or not, then irrelevance does indeed seem inevitable. However, the only alternative on the table is a v6 swamp. How about some actual technical complaints about shim6? The jerking knees become tedious to watch, after a while. Joe
On Mar 1, 2006, at 1:00 AM, Joe Abley wrote:
On 28-Feb-2006, at 23:37, Daniel Golding wrote:
Unacceptable. This is the whole problem with shim6 - the IETF telling us to "sit back and enjoy it, because your vendors know what's best".
Actually, I think the problem with shim6 is that there are far too few operators involved in designing it. This has evidently led to a widespread perception of an ivory tower with a moat around it.
One man's perception is another man's reality. ;-)
If these operators dismiss it out of hand on principal, and refuse to actually find out whether the general approach is able to solve problems or not, then irrelevance does indeed seem inevitable. However, the only alternative on the table is a v6 swamp.
Would that really be so bad? I keep being bonked on the head by this thing called Moore's law. I think until you slay the daemon of default global reachability (which is counter to everything IP), draining the swamp is an exercise in futility. Controlling the flooding OTOH is a creative posture.
On 1-Mar-2006, at 01:06, Christian Kuhtz wrote:
However, the only alternative on the table is a v6 swamp.
Would that really be so bad? I keep being bonked on the head by this thing called Moore's law.
I don't know that anybody can tell how bad it might be. It'd be a shame if it turned out to be really bad, and we had no fallback plan. Shim6 also has some features which aren't possible with the swamp -- for example, it allows *everybody* to multi-home, down to people whose entire infrastructure consists of an individual device, and to do so in a scaleable way.
I think until you slay the daemon of default global reachability (which is counter to everything IP), draining the swamp is an exercise in futility. Controlling the flooding OTOH is a creative posture.
The point is that in v6, there is not yet a swamp to drain. Joe
On Mar 1, 2006, at 1:52 AM, Joe Abley wrote:
Shim6 also has some features which aren't possible with the swamp -- for example, it allows *everybody* to multi-home, down to people whose entire infrastructure consists of an individual device, and to do so in a scaleable way.
Only if *everybody* has a shim6 capable stack...
On 1-Mar-2006, at 10:33, John Payne wrote:
On Mar 1, 2006, at 1:52 AM, Joe Abley wrote:
Shim6 also has some features which aren't possible with the swamp -- for example, it allows *everybody* to multi-home, down to people whose entire infrastructure consists of an individual device, and to do so in a scaleable way.
Only if *everybody* has a shim6 capable stack...
Not quite -- the practical usefulness of the multi-homing increases with the deployment of shim6-capable stacks. You could imagine a threshold of server and host upgrades which would provide useful multi-homing a good proportion of the time without universal deployment. If Linux and the currently-supported variants of Windows were to be updated to support shim6, and we waited through three or four widely- publicised security vulnerabilities which required OS/kernel upgrades, perhaps that would be sufficient deployment for the benefits of shim6 to be felt, most of the time. My hands are waving again, of course. I feel fairly certain I have exceeded some kind of unenforced posting threshold to this list in the last twelve hours. I will try hard to be quiet for a while, now :-) Joe
Hello; On Mar 1, 2006, at 10:45 AM, Joe Abley wrote:
On 1-Mar-2006, at 10:33, John Payne wrote:
On Mar 1, 2006, at 1:52 AM, Joe Abley wrote:
Shim6 also has some features which aren't possible with the swamp -- for example, it allows *everybody* to multi-home, down to people whose entire infrastructure consists of an individual device, and to do so in a scaleable way.
Only if *everybody* has a shim6 capable stack...
Not quite -- the practical usefulness of the multi-homing increases with the deployment of shim6-capable stacks. You could imagine a threshold of server and host upgrades which would provide useful multi-homing a good proportion of the time without universal deployment.
If Linux and the currently-supported variants of Windows were to be updated to support shim6, and we waited through three or four widely-publicised security vulnerabilities which required OS/kernel upgrades, perhaps that would be sufficient deployment for the benefits of shim6 to be felt, most of the time. My hands are waving again, of course.
I have to object to this here; your hands are not waving nearly hard enough. This was exactly the same mistake that was made with IGMPv3, which IIRC was finalized around the time of the Adelaide IETF (i.e., almost exactly 6 years ago). 1.) It took about 4 years for Windows variants with IGMPv3 support to dominate the Windows logs in my web servers. (By dominate, I mean > 80% of hits from windows machines.) In February of this year, Windows 98 (non IGMPv3 capable) was still 2% of the total web hits, compared to 0.56 % for all flavors of Linux and 0.03% for all flavors of BSD. 2.) The Mac (8% of web hits in February 2006), still doesn't have it. 3.) So IGMPv3 deployment in hosts _at this instant_ is almost certainly less than 90%. 4.) Partially as a result, SSM deployment is still miniscule. That's after 6 years. I would be surprised if Shim6 going into actual deployed boxes was any faster. So, if Shim6 was finalized today, which it won't be, in 2010 we might have 70% deployment and in 2012 we might have 90% deployment. I actually think that 2012 would be a more realistic date for 70% deployment of Shim6, given the lack of running code and a finalized protocol now. In my opinion, that doesn't imply that Shim6 should be abandoned. But it does mean IMHO that regarding it as a means to spur IPv6 deployment is just not realistic. Regards Marshall Eubanks
I feel fairly certain I have exceeded some kind of unenforced posting threshold to this list in the last twelve hours. I will try hard to be quiet for a while, now :-)
Joe
Marshall,
That's after 6 years.
I would be surprised if Shim6 going into actual deployed boxes was any faster. So, if Shim6 was finalized today, which it won't be, in 2010 we might have 70% deployment and in 2012 we might have 90% deployment.
I actually think that 2012 would be a more realistic date for 70% deployment of Shim6, given the lack of running code and a finalized protocol now.
In my opinion, that doesn't imply that Shim6 should be abandoned. But it does mean IMHO that regarding it as a means to spur IPv6 deployment is just not realistic.
Sorry, but I'm just not buying the analogy. The market drivers for IGMP are somewhat smaller than they are for IPv6. Yes, it would take a couple of years for Shim6 to be implemented and depending on where we hit Redmond's release cycle, actually penetrate a significant number of hosts. 6 years is probably long, and definitely long if we get a confluence of panic about the death of v4 plus a strong endorsement about Shim6 from the IETF. Consider that the IETF *could* conceivably require every compliant v6 implementation to include it. I grant that that's unlikely and some lesser endorsement is probably more reasonable, but I don't think that you should underestimate the capability of the IETF/ISP/vendor/host community to act a bit more quickly, if there is sufficient motivation. I suggest that we compromise, split the difference and swag it at 4 years. Tony
--- Tony Li <tony.li@tony.li> wrote:
Consider that the IETF *could* conceivably require every compliant v6 implementation to include it.
God Forbid. I somehow don't want my core routers deciding to speak shim6... David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Thus spake "Tony Li" <tony.li@tony.li>
Marshall,
That's after 6 years.
I would be surprised if Shim6 going into actual deployed boxes was any faster. So, if Shim6 was finalized today, which it won't be, in 2010 we might have 70% deployment and in 2012 we might have 90% deployment.
I actually think that 2012 would be a more realistic date for 70% deployment of Shim6, given the lack of running code and a finalized protocol now.
In my opinion, that doesn't imply that Shim6 should be abandoned. But it does mean IMHO that regarding it as a means to spur IPv6 deployment is just not realistic.
Sorry, but I'm just not buying the analogy. The market drivers for IGMP are somewhat smaller than they are for IPv6.
That depends on your perspective. There's a compelling need for usable multicast in many environments, and so far there's nobody (in the US) with a compelling need for IPv6, much less shim6.
Yes, it would take a couple of years for Shim6 to be implemented and depending on where we hit Redmond's release cycle, actually penetrate a significant number of hosts.
Shim6 needs to be finalized first, then someone has to convince MS to implement it. I'd put that, conservatively, at 4 years.
6 years is probably long, and definitely long if we get a confluence of panic about the death of v4 plus a strong endorsement about Shim6 from the IETF.
The most dire predictions of v4's death have it at least 12-15 years away. To companies worried about next quarter's profits, you might as well be talking about global warming.
Consider that the IETF *could* conceivably require every compliant v6 implementation to include it. I grant that that's unlikely and some lesser endorsement is probably more reasonable, but I don't think that you should underestimate the capability of the IETF/ISP/vendor/ host community to act a bit more quickly, if there is sufficient motivation.
Without any enforcement powers, an IETF "requirement" is pretty useless. Those vendors that care will merely see one more complicated thing they have to add to their IPv6 stack and put it off adding IPv6 even longer.
I suggest that we compromise, split the difference and swag it at 4 years.
His was a minimum; I'd put the likely number at 4-6 years after shim6 is finally published (itself no fixed date), and potentially much longer if middlebox support is added (and without which shim6 will certainly never see the light of day). S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
On 3-Mar-2006, at 11:48, Stephen Sprunk wrote:
That depends on your perspective. There's a compelling need for usable multicast in many environments, and so far there's nobody (in the US) with a compelling need for IPv6, much less shim6.
If there's such a compelling need for native multicast, why has it seen such limited deployment, and why is it available to such a tiny proportion of the Internet? Joe
Thus spake "Joe Abley" <jabley@isc.org>
On 3-Mar-2006, at 11:48, Stephen Sprunk wrote:
That depends on your perspective. There's a compelling need for usable multicast in many environments, and so far there's nobody (in the US) with a compelling need for IPv6, much less shim6.
If there's such a compelling need for native multicast, why has it seen such limited deployment, and why is it available to such a tiny proportion of the Internet?
Just because it's not widely available on the public network doesn't mean that it's not widely available on private networks connected to the public one. There are tens of millions of users out there with access to Cisco IP/TV, Real, etc. over multicast, not to mention custom business apps (particularly common in the securities world) that use multicast. They're self-contained, though, so you don't see the packets/users or even know they're out there. I'm not terribly surprised the public Internet doesn't have real mcast yet; the cost to build replicating unicast servers is paid by content sources while the cost to deploy PIM SSM is paid by another, and as such the cheaper alternative doesn't necessarily win. In a private network, one org can see the total costs for both and pick whichever one makes more sense. If anything, it's in ISPs interests to keep things unicast since there's more bits to bill for. At least until someone figures out how to bill for the traffic exiting the network at the other end (and that still leaves a problem for peering). S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
Hello; By everything I can tell, it's roughly about 10% for global deployment, see http://www.multicasttech.com/status and is becoming basically BCP in the enterprise IPTV environment (which are currently all "walled gardens"). (Note that this deployment is effectively entirely ASM / IGMP v2.) By any measure, multicast deployment is much larger than IPv6 deployment at present, and it is growing. I will be glad to argue the point to any length you might desire. Of course, the deployment did not exactly take the form that was anticipated in 2000... Regards Marshall On Mar 3, 2006, at 3:42 PM, Joe Abley wrote:
On 3-Mar-2006, at 11:48, Stephen Sprunk wrote:
That depends on your perspective. There's a compelling need for usable multicast in many environments, and so far there's nobody (in the US) with a compelling need for IPv6, much less shim6.
If there's such a compelling need for native multicast, why has it seen such limited deployment, and why is it available to such a tiny proportion of the Internet?
Joe
On Fri, Mar 03, 2006 at 04:02:00PM -0500, Marshall Eubanks wrote:
By everything I can tell, it's roughly about 10% for global deployment, see
10% by one measure. That is perhaps the most positive spin on what we can see. All that tells us is the AS view. To some extent netblocks, but that is really still based on origin AS also. It says nothing about which interfaces inside and behind an AS have PIM enabled and that is what really matters. I suspect real multicast reachability and deployment is quite a bit lower than 10%. I have no idea how to go about measuring it though, but 10% would have to be the best we could do if all hidden interfaces were enabed.
By any measure, multicast deployment is much larger than IPv6 deployment at present, and it is growing. I will be glad to argue the point to any length you might desire.
You know me Marshall, I care about multicast, but I'm not sure I'd go around comparing multicast to IPv6, that doesn't help. :-) John
On 3 mar 2006, at 22.02, Marshall Eubanks wrote:
By any measure, multicast deployment is much larger than IPv6 deployment at present, and it is growing. I will be glad to argue the point to any length you might desire.
There are also operators that are deploying IPv6 just so that they can do multicast to their end-users without have to figure out how to run it through NAT. As was discussed at the last SANOG in some of the workshops. - kurtis -
JA> Date: Fri, 3 Mar 2006 15:42:25 -0500 JA> From: Joe Abley JA> If there's such a compelling need for native multicast, why has it JA> seen such limited deployment, and why is it available to such a tiny JA> proportion of the Internet? One could ask the same of long-prefix PI availability and announcement. Lack of demand is not the only answer; one must also examine technical and policy constraints. Taking your statement a step further, though, you have a very good point: A smart approach is to analyze end-user wishes and demands, transform the "wish list" into engineering requirements, and take it from there. (e.g., just what is the global table asymptotic limit?) Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
On 3 mar 2006, at 04.13, Marshall Eubanks wrote:
I would be surprised if Shim6 going into actual deployed boxes was any faster. So, if Shim6 was finalized today, which it won't be, in 2010 we might have 70% deployment and in 2012 we might have 90% deployment.
OTOH Teredo, which isn't even a standard is in more or less all Windows XP boxes.... - kurtis -
On Sat, 4 Mar 2006 13:59:18 +0100 Kurt Erik Lindqvist <kurtis@kurtis.pp.se> wrote:
On 3 mar 2006, at 04.13, Marshall Eubanks wrote:
I would be surprised if Shim6 going into actual deployed boxes was any faster. So, if Shim6 was finalized today, which it won't be, in 2010 we might have 70% deployment and in 2012 we might have 90% deployment.
OTOH Teredo, which isn't even a standard is in more or less all Windows XP boxes....
Teredo is described in RFC 4380; it's a Proposed Standard. I should note that Microsoft really believes in IPv6. I wonder what that means for its future....
On Mar 4, 2006, at 11:43 AM, Steven M. Bellovin wrote:
On 3 mar 2006, at 04.13, Marshall Eubanks wrote:
I would be surprised if Shim6 going into actual deployed boxes was any faster. So, if Shim6 was finalized today, which it won't be, in 2010 we might have 70% deployment and in 2012 we might have 90% deployment.
OTOH Teredo, which isn't even a standard is in more or less all Windows XP boxes....
Teredo is described in RFC 4380; it's a Proposed Standard.
I should note that Microsoft really believes in IPv6. I wonder what that means for its future....
Let's hope it means that they'll eventually come up with nicer ways to configure IPv6 stacks in Windows flavors than they have this far...
On 4 mar 2006, at 17.43, Steven M. Bellovin wrote:
On Sat, 4 Mar 2006 13:59:18 +0100 Kurt Erik Lindqvist <kurtis@kurtis.pp.se> wrote:
On 3 mar 2006, at 04.13, Marshall Eubanks wrote:
I would be surprised if Shim6 going into actual deployed boxes was any faster. So, if Shim6 was finalized today, which it won't be, in 2010 we might have 70% deployment and in 2012 we might have 90% deployment.
OTOH Teredo, which isn't even a standard is in more or less all Windows XP boxes....
Teredo is described in RFC 4380; it's a Proposed Standard.
Duh. Given I was part of the discussion of which prefixes to use in the document I should have remembered that... Still it was deployed way before that. - kurtis -
On Sat, 4 Mar 2006 13:59:18 +0100, "Kurt Erik Lindqvist" <kurtis@kurtis.pp.se> said:
On 3 mar 2006, at 04.13, Marshall Eubanks wrote:
I would be surprised if Shim6 going into actual deployed boxes was any faster. So, if Shim6 was finalized today, which it won't be, in 2010 we might have 70% deployment and in 2012 we might have 90% deployment.
OTOH Teredo, which isn't even a standard is in more or less all Windows XP boxes....
It illustrates an important difference in motivation. Is the primary goal to solve a problem or to make a standard? Remember, quite a few important technologies were implemented, tested and even in production-use for a long time before standardisation (e.g. ssh). //per -- Per Heldal http://heldal.eml.cc/
On Wed, Mar 01, 2006 at 10:33:51AM -0500, John Payne wrote:
On Mar 1, 2006, at 1:52 AM, Joe Abley wrote:
Shim6 also has some features which aren't possible with the swamp -- for example, it allows *everybody* to multi-home, down to people whose entire infrastructure consists of an individual device, and to do so in a scaleable way.
Only if *everybody* has a shim6 capable stack...
and nobody does! extrapolations and visions of a brave new world are just that. kind of like the Boeing/Airbus mockups that have lounges, gyms&showers and restrants onboard their 747 and A380 aircraft. and attractive flight attendants talking about shoes... :) yeah it -might- happen, but... -- bill (particularly grumpy & cynical tonight)
How about some actual technical complaints about shim6?
good question. to give such discussion a base, could you point us to the documents which describe how to deploy it in the two most common situation operators see o a large multi-homed enterprise customer o a small to medium multi-homed tier-n isp thanks. randy
On 1-Mar-2006, at 01:09, Randy Bush wrote:
How about some actual technical complaints about shim6?
good question. to give such discussion a base, could you point us to the documents which describe how to deploy it in the two most common situation operators see o a large multi-homed enterprise customer
There are no documents describing deployment. Probably there should be. The general approach is presumably well-known (for those for whom it is not, go browse around <http://www.ietf.org/html.charters/shim6- charter.html>, and perhaps in particular <http://www.ietf.org/ internet-drafts/draft-ietf-shim6-proto-03.txt>. Deployment in an enterprise is a matter of: (a) deploying hosts with shim6-capable stacks within the enterprise; (b) arranging for those hosts to receive addresses in each PA assignment made by each transit provider (multiple PA addresses per interface), e.g. using dhcp6; (c) optionally, perhaps, installing shim6 middleware at some suitable place between host and border in order to impose site policy or modulate locator selection by the hosts. In the event that one provider goes away, the internal address assignment infrastructure doesn't need to participate in the traditional handwave magic IPv6 renumbering protocol; shim6-capable hosts talking to other shim6-capable hosts will switch locators based on observed failure of the dead transit provider's addresses to work; transport-layer sessions are hence preserved. You will note I have glossed over several hundred minor details (and several hundred more not-so-minor ones). The protocols are not yet published; there is no known implementation.
o a small to medium multi-homed tier-n isp
A small-to-medium, multi-homed, tier-n ISP can get PI space from their RIR, and don't need to worry about shim6 at all. Ditto larger ISPs, up to and including the largest. Individual ISP customers (e.g. residential users, small/home office users) can multi-home in the same way as hosts within an enterprise network. For residential users, for example, step (b) above might be achieved by installing two NICs, and attaching one to the cable modem and the other to the DSL modem; step (c) would be unnecessary. Content providers have a different set of problems, since a server with N simultaneously-active clients, each with an average of M available locators needs to deal with N*M worth of state, which is presumably M times worse than the situation today. For very large content providers, aggregating very large numbers of simultaneous clients through load balancers or other middleboxes, this is quite possibly not something that is going to be a simple matter of upgrading to a shim6-capable firmware release. Joe
On Mar 1, 2006, at 12:47 AM, Joe Abley wrote:
o a small to medium multi-homed tier-n isp
A small-to-medium, multi-homed, tier-n ISP can get PI space from their RIR, and don't need to worry about shim6 at all. Ditto larger ISPs, up to and including the largest.
If you include "Web hosting company" in your definition of ISP, that's not true. Unless you're providing connectivity to 200 or more networks, you can't get a /32. If all of your use is internal(fully managed hosting) or aren't selling leased lines or anything, you are not considered an LIR by the current IPv6 policies. Even the proposed ARIN 2006-4 assignment policy for "end sites" doesn't help a lot of small to mid sized hosting companies. For that, to just get a /48, you need to already have a /19 or larger, and be using 80% of that. That's 6553 IPs being utilized. If you're running a managed hosting company (name based vhosts) and deploying 1 IP per web server, you're pretty huge before you've hit 6553 devices. Even assuming 20% of that is wasted, you're still talking about more than 5000 servers. 40 1U servers per rack, you need to have 125 racks of packed to the gills servers before you'd qualify for PI space. That excludes every definition I have of "small-to-medium" in the hosting arena. You don't get PI space, and Shim6 is looking like your only alternative for multihoming.
Content providers have a different set of problems, since a server with N simultaneously-active clients, each with an average of M available locators needs to deal with N*M worth of state, which is presumably M times worse than the situation today.
For very large content providers, aggregating very large numbers of simultaneous clients through load balancers or other middleboxes, this is quite possibly not something that is going to be a simple matter of upgrading to a shim6-capable firmware release.
Yes, and content providers have other issues as well when it comes to IPv6 policy... I'm betting only the top 1 or 2 CDN/content providers out there qualify for a /32. Many content providers set up multiple non-interconnected POPs in different geographical locations. The only way this can be accomplished is by making separate announcements in each POP for each space. This means either being able to deaggregate, or to get a block for each POP. I don't know of *ANY* that are deploying 5000+ servers per POP.
Actually, I think the problem with shim6 is that there are far too few operators involved in designing it. This has evidently led to a widespread perception of an ivory tower with a moat around it.
I think the issue was... When I first heard of shim6, I thought "Oooh, that's really clever. A lot of small businesses/enterprises will use that, they don't need to deal with BGP, adding a new provider is just a drop in." Then when we got to deploying IPv6 the discovery of "Oh, wait, they expect EVERYONE who uses PA space to do this? That's not cool." was a negative reaction.
To gain real relevance it needs to be deployed; to be deployed, it needs to be embraced by enterprise operators and content providers.
If these operators dismiss it out of hand on principal, and refuse to actually find out whether the general approach is able to solve problems or not, then irrelevance does indeed seem inevitable. However, the only alternative on the table is a v6 swamp.
How about some actual technical complaints about shim6?
I'm just one guy, one ASN, and one content/hosting network. But I can tell you that to switch to using shim6 instead of BGP speaking would be a complete overhaul of how we do things. Putting routing decisions in the control of servers we don't operate scares me. I wouldn't rely on 90% of our customers to get this right unless it was completely idiot proof. Even if it was, I don't see how we can trust that users aren't messing with things to "game the system" somehow. We deal with long lived TCP sessions (hours/days). I don't see how routing updates can happen that won't result in a disconnect/ reconnect, which isn't acceptable. With current BGP technologies, if I need to move traffic off a transit port, I can do so without relying on all of our servers to know anything about it, the move is instant, and non-disruptive. Shim6 requires a keepalive to expire for the end nodes to realize something is broken, then re-negotiate the remaining routing decisions. With BGP, I can see if one of my transit links goes down directly, and compensate before users start getting impatient. We have peering arrangements with about 120 ASNs. How do we mix BGP IPv6 peering and Shim6 for transit? So far it looks like Shim6 is going to rely on DNS. The DNS caching issue is a real problem. We need changes to happen faster than DNS caching will allow. Our network is complicated. We have a /21 that's split into 4 /23s. One for each non-interconnected POP. We only advertise the /23 for each POP out to transit, but we give peers access to our entire network wherever they peer with us and we pay to haul/tunnel it around. How do we even do this without PI space, let alone through shim6? For quite the foreseeable future, we'd be running IPv4 and IPv6 at the same time, over the same transit connections. We'd have to TE our IPv6 bits completely differently than our IPv4 bits, even though we'd be billed for the aggregate usage of both. Automated tools for tweaking total usage per transit port is hard enough in BGP. Having to tweak both BGP and some external shim6 method of TE when the goal is a common aggregate number is going to be a very difficult issue. Some of our applications are extremely sensitive to jitter/latency. We've spent ages tweaking route-maps manually (and through automated continual tweaking) to make sure we avoid any congested links. We also rely on BGP communities by our providers to give us some more information when it comes to route decisions. (If NSP A tells me through communities that they peer directly with someone, where NSP B is crossing the country, then hitting another NSP before the Origin ASN, we prefer NSP A). I don't see how information like this, or tweaking to that level is even possible with Shim6. BGP works well for applications like this because each network the traffic passes through can add its own hints (Communities, prepending, etc) to the route, that lots of us use. We'd still be relying on PA space. No matter how great dhcp6 is, there will be significant renumbering pain when providers are changed. Static ACLs, firewall rules, etc. If you're including customer machines in the renumbering, many simply won't do it. Putting the logic behind traffic engineering and routing decisions into thousands of boxes seems a step backwards from putting the decision on our border/edges. Many more places where things can break. If we want to do things in a non-standard way, every box has to support it. If there are refinements to Shim6 later, we're forced with either not using them, or forcing our customers to upgrade their OS. How do we deal with "backup connections"? I.e. connections that are only used if all others are down. Right now we advertise only a supernet out to our "backup transit" provider, and the more specifics to our main providers. (Yes, I realize this isn't perfect, but it works fine for us.) Please don't get me wrong, I think Shim6 is great for a lot of people. Being able to let ANYONE multihome with no impact on the world is great. BUT, there needs to be a fallback to the BGP/IPv4-ish way for people who need the "power user" set of tools, or there is going to be a huge pushback from a lot of groups when asked to switch to ipv6. This fallback has to be available to anyone who can justify the need, not just "anyone bigger than X size". -- Kevin
On 1-Mar-2006, at 02:56, Kevin Day wrote:
On Mar 1, 2006, at 12:47 AM, Joe Abley wrote:
o a small to medium multi-homed tier-n isp
A small-to-medium, multi-homed, tier-n ISP can get PI space from their RIR, and don't need to worry about shim6 at all. Ditto larger ISPs, up to and including the largest.
If you include "Web hosting company" in your definition of ISP, that's not true.
Right. I wasn't; I listed them separately. It's important to note that even if you are a hosting company who *does* qualify for PI v6 space, you still need shim6-capable servers, if you want to make them optimally available to multi-homed, shim6- capable hosts. The difference PI makes is in the distribution of addresses to servers (the servers only need a single set).
You don't get PI space, and Shim6 is looking like your only alternative for multihoming.
Right. For a hosting company with multiple PA netblocks, shim6 is the option on the table.
Many content providers set up multiple non-interconnected POPs in different geographical locations. The only way this can be accomplished is by making separate announcements in each POP for each space. This means either being able to deaggregate, or to get a block for each POP. I don't know of *ANY* that are deploying 5000 + servers per POP.
Right. With shim6, getting a block per POP is trivial, since they are all PA assignments from transit providers.
I'm just one guy, one ASN, and one content/hosting network. But I can tell you that to switch to using shim6 instead of BGP speaking would be a complete overhaul of how we do things.
You are not alone in fearing change.
Putting routing decisions in the control of servers we don't operate scares me. I wouldn't rely on 90% of our customers to get this right unless it was completely idiot proof. Even if it was, I don't see how we can trust that users aren't messing with things to "game the system" somehow.
This is the kind of feedback that the shim6 architects need. There is talk at present of whether the protocol needs to be able to accommodate a site-policy middlebox function to enforce site policy in the event that host behaviour needs to be controlled. The scope of that policy mediation function depends strongly on people like you saying "at a high level, this is the kind of decision I am not happy with the hosts making".
We deal with long lived TCP sessions (hours/days). I don't see how routing updates can happen that won't result in a disconnect/ reconnect, which isn't acceptable.
One of the primary objectives of shim6 is to provide session survivability over re-homing events. Since routing protocols are not used to manage re-homing, the speed at which a session can recover from a topological event depends on the operation of the shim6 protocol between client and server. It seems reasonable to say that in some cases shim6 re-homing transitions will be faster than the equivalent routing transition in v4; in other cases it will be shorter. Depends on the network, and how enthusiastically you flap, perhaps. The experience of people who provide services involving long-held TCP sessions is exactly the kind of thing that the shim6 architects need to hear about.
We have peering arrangements with about 120 ASNs. How do we mix BGP IPv6 peering and Shim6 for transit?
You advertise all your PA netblocks to all your peers.
So far it looks like Shim6 is going to rely on DNS. The DNS caching issue is a real problem. We need changes to happen faster than DNS caching will allow.
Well, not quite. If you change a transit provider, then you need to remove a set of AAAA records from the servers you operate, and substitute a new set. The time taken for this change to propagate in the DNS is non-zero, assuming you use reasonable TTLs. This is your point above, I think. With shim6-capable clients and servers, the dark period during which the changes propagate is handled by an address selection/retry algorithm in the client (for new sessions) and by the shim6 protocol doing failure detection and selecting a new locator (for established sessions). Once the DNS change has propagated, the address selection and shim6 band-aids are no longer required, and clients have an accurate set of information. Renumbering for hosting providers can be a monstrous pain in the neck, especially for hosting providers who rely on third parties (or, horrors, their customers) to maintain the zone files within which services are named. Some hosting providers of my acquaintance insist on customer zones being redelegated to the hosting providers' nameservers, so that any renumbering that needs to happen can be coordinated by the hosting provider directly. Hosting providers who don't do this, and who use PA addresses with shim6 to multi-home, are definitely going to face some challenges.
Our network is complicated. We have a /21 that's split into 4 /23s. One for each non-interconnected POP. We only advertise the /23 for each POP out to transit, but we give peers access to our entire network wherever they peer with us and we pay to haul/tunnel it around. How do we even do this without PI space, let alone through shim6?
You avoid it completely, and use PA space in every POP. You can still announce PA space from other POPs to peers, if you want to retain your tunnels.
For quite the foreseeable future, we'd be running IPv4 and IPv6 at the same time, over the same transit connections. We'd have to TE our IPv6 bits completely differently than our IPv4 bits, even though we'd be billed for the aggregate usage of both. Automated tools for tweaking total usage per transit port is hard enough in BGP. Having to tweak both BGP and some external shim6 method of TE when the goal is a common aggregate number is going to be a very difficult issue.
Yep. Difficult and expensive.
Some of our applications are extremely sensitive to jitter/latency. We've spent ages tweaking route-maps manually (and through automated continual tweaking) to make sure we avoid any congested links. [...]
The site-policy middleware that I alluded to earlier seems like the analogous place to specify this policy. Such a facility might actually give you more control than you have now -- tweaking BGP attributes to accomplish this kind of thing is often like a game of whack-a-mole; if you were able to control the route taken by traffic in both directions by influencing the locator selection for each and every session, you'd have far greater, and more fine-grained, control over your external traffic than BGP/swamp-abuse gives you currently. Your specific requirements in this regard (the high-level objectives that you currently meet using BGP) would no doubt be gratefully received on the shim6 list.
We'd still be relying on PA space. No matter how great dhcp6 is, there will be significant renumbering pain when providers are changed. Static ACLs, firewall rules, etc. If you're including customer machines in the renumbering, many simply won't do it.
Agreed, renumbering is a pain. Dhcp6 sounds like a scary thing to use with servers. Customers suck. Change in operational practices will be required. Lest I sound too much like a foam-at-the-mouth shim6 advocate, I think it would be perfectly fine if, in the final analysis, the conclusion was that shim6 and PA/renumbering was not an option for hosting providers. A reasoned technical argument which came to that conclusion would provide a solid basis for the RIRs to modify their allocation policies such that hosting providers could use PI space instead. As perhaps the recent attempt to change the v6 PI policy indicates, the chances of making changes without such a reasoned argument are slim. However, I think it's possible that shim6, incorporating some facility for a site to manage the locator selection of the hosts, could actually make some things easier for hosting providers. There might even be reasons to like it :-) Joe
--- Joe Abley <jabley@isc.org> wrote:
I'm just one guy, one ASN, and one content/hosting network. But I can tell you that to switch to using shim6 instead of BGP speaking would be a complete overhaul of how we do things.
You are not alone in fearing change.
It isn't fearing change to ask the question "it's not broken today, why should I fix it?"
This is the kind of feedback that the shim6 architects need. There is talk at present of whether the protocol needs to be able to accommodate a site-policy middlebox function to enforce site policy in the event that host behaviour needs to be controlled. The scope of that policy mediation function depends strongly on people like you saying "at a high level, this is the kind of decision I am not happy with the hosts making".
Resounding YES - I specifically DON'T want end-hosts to be able to make these decisions, but need to be able to multihome.
We deal with long lived TCP sessions (hours/days). I don't see how routing updates can happen that won't result in a disconnect/ reconnect, which isn't acceptable.
One of the primary objectives of shim6 is to provide session survivability over re-homing events. Since routing protocols are not used to manage re-homing, the speed at which a session can recover from a topological event depends on the operation of the shim6 protocol between client and server.
It seems reasonable to say that in some cases shim6 re-homing transitions will be faster than the equivalent routing transition in v4; in other cases it will be shorter. Depends on the network, and how enthusiastically you flap, perhaps.
A - X - Y - B \ | \ | / W - Z A and B are hosts, W-Z are ISPs On what basis would you say that in the event of a network outage in Y, communication between A and B will be faster than the routing transition?
The experience of people who provide services involving long-held TCP sessions is exactly the kind of thing that the shim6 architects need to hear about.
We have peering arrangements with about 120 ASNs. How do we mix BGP IPv6 peering and Shim6 for transit?
You advertise all your PA netblocks to all your peers.
And maintain 120 different context tables on each host? ouch. I'm guessing that server vendors are going to be quite happy with this.
You avoid it completely, and use PA space in every POP. You can still announce PA space from other POPs to peers, if you want to retain your tunnels.
Wait a second - doesn't that deaggregation bring back the "lots of small routes" business which the whole v6 hierarchical addressing model was supposed to fix? If we're in the world of deaggregates anyway, why not just ditch the addressing model instead of accepting its limitations in this way? -David David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On 1-Mar-2006, at 11:55, David Barak wrote:
--- Joe Abley <jabley@isc.org> wrote:
I'm just one guy, one ASN, and one content/hosting network. But I can tell you that to switch to using shim6 instead of BGP speaking would be a complete overhaul of how we do things.
You are not alone in fearing change.
It isn't fearing change to ask the question "it's not broken today, why should I fix it?"
What's broken today is that there's no mechanism available for people who don't qualify for v6 PI space to multi-home. That's what shim6 is trying to fix.
It seems reasonable to say that in some cases shim6 re-homing transitions will be faster than the equivalent routing transition in v4; in other cases it will be shorter. Depends on the network, and how enthusiastically you flap, perhaps.
A - X - Y - B \ | \ | / W - Z
A and B are hosts, W-Z are ISPs
On what basis would you say that in the event of a network outage in Y, communication between A and B will be faster than the routing transition?
That's an example of a simple topology where the routing system might be expected to reconverge rapidly. However, it's not hard to find examples in today's v4 Internet where reconvergence following a re-homing event can take 30 to 60 seconds to occur. In the case where such an event includes some interface flapping, it's not that uncommon to see paths suppressed due to dampening for 20-30 minutes. I would expect (in some future, hypothetical implementation of shim6) that the default failure detection timers to start rotating through the locator set far sooner than 30-60 seconds.
We have peering arrangements with about 120 ASNs. How do we mix BGP IPv6 peering and Shim6 for transit?
You advertise all your PA netblocks to all your peers.
And maintain 120 different context tables on each host?
No; maintain one address per PA netblock on each host.
You avoid it completely, and use PA space in every POP. You can still announce PA space from other POPs to peers, if you want to retain your tunnels.
Wait a second - doesn't that deaggregation bring back the "lots of small routes" business which the whole v6 hierarchical addressing model was supposed to fix? If we're in the world of deaggregates anyway, why not just ditch the addressing model instead of accepting its limitations in this way?
There's a vast difference in impact on the state held in the core between deaggregating towards direct peers, and deaggregating towards transit providers and having the deaggregated swamp propagated globally. Joe
Thus spake "Joe Abley" <jabley@isc.org>
On 1-Mar-2006, at 11:55, David Barak wrote:
It isn't fearing change to ask the question "it's not broken today, why should I fix it?"
What's broken today is that there's no mechanism available for people who don't qualify for v6 PI space to multi-home. That's what shim6 is trying to fix.
Shim6 is an answer to "what kind of multihoming can we offer to sites without PI space?"; it is yet to be seen if anyone cares about the answer to that question. The question that folks with money are asking is "how do I ensure that any random user can get reliable access to my website", and that's a question that the IETF is, in general, uninterested in.
However, it's not hard to find examples in today's v4 Internet where reconvergence following a re-homing event can take 30 to 60 seconds to occur. In the case where such an event includes some interface flapping, it's not that uncommon to see paths suppressed due to dampening for 20-30 minutes.
That may be acceptable compared to the general limitations of PA space. Folks have learned to deal with the limitations of BGP-based redundancy; asking them to give those benefits up without substantially greater benefits is foolhardy.
I would expect (in some future, hypothetical implementation of shim6) that the default failure detection timers to start rotating through the locator set far sooner than 30-60 seconds.
If we ever see shim6 (or its equivalent) widely deployed... So far, we don't even have simple IPv6 on even a noticeable fraction of end nodes. Any solution which requires upgrading all the end nodes is a non-starter, and the IETF needs to wake up to that fact. It's taken over a _decade_ for simple IPv6 to make it into host stacks, and it's still not viable yet. No host-dependent upgrade will matter to the Internet over the long run.
No; maintain one address per PA netblock on each host.
And so, if I have 6 upstream providers, every one of my hosts has to keep track of the outbound policy I want for each? How exactly am I supposed to keep track of that? Even the outbound policy for a single host (aka firewall) is beyond most organizations' capabilities today... Why is it even remotely rational that a corporate admin trust 100k+ hosts infested with worms, virii, spam, malware, etc. to handle multihoming decisions? Especially when we don't even have a sample of working code today? I don't even trust the <5 PCs I have at home to make those kind of decisions, much less every PC in my corporate network...
There's a vast difference in impact on the state held in the core between deaggregating towards direct peers, and deaggregating towards transit providers and having the deaggregated swamp propagated globally.
Obviously, folks differ in their definition of "swamp". I'd love a world where $large orgs could connect to N providers and not have to figure out the vagaries of BGP, but the reality is that if a large customer depends on the Internet for their financial health connectivity, the only answer today (with either v4 or v6) is PI space. Now, some may take that as a sign the IETF needs to figure out how to handle 10^6 BGP prefixes... I'm not sure we'll be there for a few years with IPv6, but sooner or later we will, and someone needs to figure out what the Internet is going to look like at that point. If the IETF isn't interested, some group of vendors will, if for no other reason than that's what will be needed for the vendors to sell routers in a few years. Is it any surprise that $vendor is pushing how many millions of routes they can handle in the FIB today? IPv6 is just a convenient placeholder for all the problems that today's ISPs are ignoring about today's Internet. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
Man, I hope I never become as cynical as you. On 2-mrt-2006, at 11:09, Stephen Sprunk wrote:
Why is it even remotely rational that a corporate admin trust 100k+ hosts infested with worms, virii, spam, malware, etc. to handle multihoming decisions?
They trust those hosts to do congestion control too, which is even more important.
Especially when we don't even have a sample of working code today?
The IAB goes out of its way to solicit input on ongoing work, and now you whine about lack of working code?
Now, some may take that as a sign the IETF needs to figure out how to handle 10^6 BGP prefixes... I'm not sure we'll be there for a few years with IPv6, but sooner or later we will, and someone needs to figure out what the Internet is going to look like at that point.
It won't look good. ISPs will have to buy much more expensive routers. At some point, people will start to filter out routes that they feel they can live without and universal reachability will be a thing of the past. It will be just like NAT: every individual problem will be solvable, but as an industry, or even a society, we'll be wasting enormous amounts of time, energy and money just because we didn't want to bite the bullet earlier on.
On Thu, Mar 02, 2006 at 03:51:43PM +0100, Iljitsch van Beijnum wrote:
Now, some may take that as a sign the IETF needs to figure out how to handle 10^6 BGP prefixes... I'm not sure we'll be there for a few years with IPv6, but sooner or later we will, and someone needs to figure out what the Internet is going to look like at that point.
It won't look good. ISPs will have to buy much more expensive routers. At some point, people will start to filter out routes that they feel they can live without and universal reachability will be a thing of the past.
But don't we filter out routes we feel we can live without *right now* without the world ending? I mean, who accepts prefixes longer than /24 these days anyway? We've all decided that we "can live without" any network smaller than 254 hosts and it hasn't made a lick of difference to universal reachability. What's to stop someone who wants to carry around less prefixes from saying, "Bugg'rit, I'm not going to accept anything smaller than a /18"? - mark -- Mark Newton Email: newton@internode.com.au (W) Network Engineer Email: newton@atdot.dotat.org (H) Internode Systems Pty Ltd Desk: +61-8-82282999 "Network Man" - Anagram of "Mark Newton" Mobile: +61-416-202-223
On 2-mrt-2006, at 16:20, Mark Newton wrote:
Now, some may take that as a sign the IETF needs to figure out how to handle 10^6 BGP prefixes... I'm not sure we'll be there for a few years with IPv6, but sooner or later we will, and someone needs to figure out what the Internet is going to look like at that point.
It won't look good. ISPs will have to buy much more expensive routers. At some point, people will start to filter out routes that they feel they can live without and universal reachability will be a thing of the past.
But don't we filter out routes we feel we can live without *right now* without the world ending?
Did I say that the world would end? All of this is important stuff but fortunately not THAT important...
I mean, who accepts prefixes longer than /24 these days anyway? We've all decided that we "can live without" any network smaller than 254 hosts and it hasn't made a lick of difference to universal reachability.
...because everyone who can't get a /24 will find another way to connect to the internet. Now suppose that today we all filter at /24 but tomorrow we start...
What's to stop someone who wants to carry around less prefixes from saying, "Bugg'rit, I'm not going to accept anything smaller than a /18"?
This will break connectivity to MANY parts of the internet. And yes, you can do this if you want. Your customers may not like the new policy, though. A more realistic scenario would be to go from /24 to / 23, which will make your routing tables a lot smaller but not break _too_ much. If enough people start doing this, people who have a /24 will have to renumber into a larger block or move to PA space. And two years later it's /22 and so on. At this point, the people at the short end of that stick may start to think that shim6 wasn't so bad after all. Note that in IPv6 all of this will play out differently because you basically have /32s for most ISPs (some shorter ones as well of course, but no longer ones for ISPs) So filtering on prefix length (other than as a safety mechanism against accidental deaggregation) won't be useful. Today you can deaggregate a /16 into 256 /24s, but deaggregating a /32 into 65536 /48s is of course more problematic. But maybe in IPv6 people will filter out stuff from other regional registries. Especially when they discover multihoming in Asia.
Mark Newton wrote:
I mean, who accepts prefixes longer than /24 these days anyway? We've all decided that we "can live without" any network smaller than 254 hosts and it hasn't made a lick of difference to universal reachability. What's to stop someone who wants to carry around less prefixes from saying, "Bugg'rit, I'm not going to accept anything smaller than a /18"?
Hopefully, customers. Furthermore, such a policy will also do little to encourage IPv4 conservation. We're already in a situation where for each routing policy, folk are recommended to use /20 or smaller prefixes (per routing policy) when applying for PI, despite the fact that a /23 might suit their multi-homed, end-site network, in order to help beat-the-filters. -a
Thus spake "Iljitsch van Beijnum" <iljitsch@muada.com>
Man, I hope I never become as cynical as you.
A pessimist is never disappointed.
On 2-mrt-2006, at 11:09, Stephen Sprunk wrote:
Why is it even remotely rational that a corporate admin trust 100k+ hosts infested with worms, virii, spam, malware, etc. to handle multihoming decisions?
They trust those hosts to do congestion control too, which is even more important.
No, they don't. That's why nearly every enterprise has deployed intradomain QoS of some sort. Nearly everyone doing VoIP has to use QoS to prevent hosts (with "congestion control") from messing with their voice traffic. Others have had to deploy it to prevent non-mission-critical (or even prohibited) apps from interfering with mission-critical stuff. I had one customer that had to implement QoS on their entire WAN just to keep Outlook and web access from starving out their serial-over-X.25-over-IP business application. The people who pay for the network want to have control over it.
Especially when we don't even have a sample of working code today?
The IAB goes out of its way to solicit input on ongoing work, and now you whine about lack of working code?
I'm not whining (at least I don't think so), but I think it's very premature to talk about shim6 as the solution to IPv6 multihoming when it's not a deployable solution or even a fully specified one.
Now, some may take that as a sign the IETF needs to figure out how to handle 10^6 BGP prefixes... I'm not sure we'll be there for a few years with IPv6, but sooner or later we will, and someone needs to figure out what the Internet is going to look like at that point.
It won't look good. ISPs will have to buy much more expensive routers. At some point, people will start to filter out routes that they feel they can live without and universal reachability will be a thing of the past.
That's one possible end case. The other is that all of this is a tempest in a teapot and the growth of IPv6 PI routes will continue to be non-dominant just as PI is with IPv4. As others have noted, one prefix per ASN (which is the goal of IPv6 PI policy) is nowhere near enough to create a problem unless there's a serious explosion in ASN assignment. The policies for IPv4 are pretty darn lax, so if we don't have a problem today, why do people think we'll have a problem with stricter policies on the IPv6 side? And I'm the cynic...
It will be just like NAT: every individual problem will be solvable, but as an industry, or even a society, we'll be wasting enormous amounts of time, energy and money just because we didn't want to bite the bullet earlier on.
People pay what they perceive to be the lowest cost to themselves; so far, NAT has that honor. I'm more confident that we'll find an answer to the IDR problem sooner than we'll convince people to act in the good of the community at their own expense. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
I'm more confident that we'll find an answer to the IDR problem sooner than we'll convince people to act in the good of the community at their own expense.
The solution to the IDR problem is to have a scalable routing architecture. Unfortunately, that involves change from the status quo, and thus altruistic action. The alternative, of course, is to wait for IDR to implode and let the finger-pointing begin. Tony
Thus spake "Tony Li" <tony.li@tony.li>
I'm more confident that we'll find an answer to the IDR problem sooner than we'll convince people to act in the good of the community at their own expense.
The solution to the IDR problem is to have a scalable routing architecture. Unfortunately, that involves change from the status quo, and thus altruistic action.
Not if/when folks understand that the implosion is imminent and the only way to preserve their business is to build a better routing architecture. Only when self-interest and altruism are coincident is the latter consistently achieved.
The alternative, of course, is to wait for IDR to implode and let the finger-pointing begin.
... which is what I expect to happen. A few folks will see it coming, design a fix, and everyone will deploy it overnight when they discover they have no other choice. Isn't that about what happened with CIDR, in a nutshell? S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
SS> Date: Fri, 3 Mar 2006 20:05:36 -0600 SS> From: Stephen Sprunk SS> > Unfortunately, that involves change from the status quo, and thus SS> > altruistic action. SS> SS> Only when self-interest and altruism are coincident is the latter SS> consistently achieved. Witness BCP38, spam/worm cleanup, prefix deaggregation. If the past is any indication of the future... SS> > The alternative, of course, is to wait for IDR to implode and let the SS> > finger-pointing begin. SS> SS> ... which is what I expect to happen. A few folks will see it coming, SS> design a fix, and everyone will deploy it overnight when they discover they SS> have no other choice. Isn't that about what happened with CIDR, in a SS> nutshell? Those who do not study (or remember) history... Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
The alternative, of course, is to wait for IDR to implode and let the finger-pointing begin.
... which is what I expect to happen. A few folks will see it coming, design a fix, and everyone will deploy it overnight when they discover they have no other choice. Isn't that about what happened with CIDR, in a nutshell?
Actually, no. For the most part, folks realized that the swamp didn't scale and they were willing to deploy the fix well before things would have actually imploded. Furthermore, the fix and the associated angst were far less painful and time consuming to deploy than an entirely new architecture will be. In short, they were better 'netizens' in that they could see a distant wall coming and act decisively well before panic set in. Tony
On 4-mrt-2006, at 3:05, Stephen Sprunk wrote:
The alternative, of course, is to wait for IDR to implode and let the finger-pointing begin.
... which is what I expect to happen. A few folks will see it coming, design a fix, and everyone will deploy it overnight when they discover they have no other choice. Isn't that about what happened with CIDR, in a nutshell?
We got lucky with CIDR because even though all default free routers had to be upgraded in a short time, it really wasn't that painful. Ok, I wasn't there, but what I mean is that the problem was solved by aggregating already deployed address space, which isn't going to fly if excessive PI makes IDR implode in the future. I've been in multi6, two multi6 design teams and shim6 for nearly five years, and I've seen many of the smartest people in the IETF community join in. I can tell you this: the only scalable solutions on the horizon are: - moving multihoming related state out of the DFZ (this is what shim6 does) - remove the requirement that every DFZ router carries every prefix, which can't be done as long as PI blocks sit at the top of the addressing hierarchy There are many aspects to current IDR that can stand to be improvemed, but at the end of the day that doesn't shrink your FIB by orders of magnitude. The closest thing to a magic, pain-free solution would be to allocate PI blocks such that it's possible to aggregate them together and ignore the more specifics for far away regions of the world, so that in 2030 you don't have to carry 60000 Chinese PI blocks world wide that all sit behind the same Great Firewall anyway, but no, that doesn't make sense because how can I multihome to ISPs in Shainghai and Toronto then, this will never work.
On Mar 4, 2006, at 2:21 AM, Iljitsch van Beijnum wrote:
... which is what I expect to happen. A few folks will see it coming, design a fix, and everyone will deploy it overnight when they discover they have no other choice. Isn't that about what happened with CIDR, in a nutshell?
We got lucky with CIDR because even though all default free routers had to be upgraded in a short time, it really wasn't that painful.
Isn't that an excellent argument against shim6 though? In IPv4, something unanticipated occurred by the original developers (the need for CIDR), and everyone said "Oh, thank god that all we have to do is upgrade DFZ routers." Eventually all hosts got CIDR routing, but it didn't break anything if you didn't. In IPv6/shim6 what happens if shim6 has an unanticipated bottleneck, security issue, or scaleability problem? Everyone, everywhere, has to upgrade at some point. This means that upgrade/workaround has to backwards compatible indefinitely, since it isn't going to be possible to force all the world's servers, desktops and devices to upgrade by some flag day. If it requires an OS update to add a feature, you can't rely on having 90% penetration within YEARS of it being released. After XP Service Pack 2 was released, only 24% had upgraded after 9 months. According to one of our website's stats, we still see 5% of our users on Windows 98, and 13% on Windows 2000. Getting systems not controlled by the networking department of an organization upgraded, when it's for reasons that are not easily visible to the end user, will be extraordinarily difficult to start with. Adding shim6 at all to hosts will be one fight. Any upgrades or changes later to add features will be another. I don't think (many) operators are really dreading having to eventually and slowly upgrade their DFZ routers to support 32 bit ASNs. It'll require an OS upgrade, probably an upgrade to a few tools (netflow parsers, etc), but nothing customer impacting. There will be a crossover period where the software will be available but no 32-bit ASNs will be visible, until one day some unlucky sap gets AS65536 and guinea-pigs the internet. It's a networking problem being solved by the guys who run the network, and it can get done in a reasonable timeframe, without bothering end users, other departments or touching boxes outside the network admin's control. In an enterprise environment, you're not talking the DBA of your Oracle box into installing service packs, upgrades or new software just because you want to use a new routing feature that wasn't around when their OS was released. I know of several enterprises still running NT 4.0 and Mac OS 9 boxes for some legacy applications. A parallel to that is going to still be happening in the next 4-8 years when shim6 would have to prove itself. There are going to be enterprises, end users, database servers, and embedded devices that have IPv6 support because they shipped with it (XP, Vista, OS X, Solaris, etc), but don't have shim6 support. The services either get an expensive upgrade or lose out on multihoming. The real "injustice" about this is that it's creating two classes of organizations on the internet. One that's meets the guidelines to get PI space, multihomes with BGP, and can run their whole network (including shim6less legacy devices) with full redundancy, even when talking to shim6 unaware clients. Another(most likely smaller) that can't meet the rules to get PI space, is forced to spend money upgrading hardware and software to shim6 compatible solution or face a lower reliability than their bigger competitors. Someone earlier brought up that a move to shim6, or not being able to get PI space was similar to the move to name based vhosting(HTTP/1.1 shared IP hosting). It is, somewhat. It was developed to allow hosting companies to preserve address space, instead of assigning one IP address per hostname. (Again, however, this could be done slowly, without forcing end users to do anything.) ARIN's policy is that you should use name based hosting wherever possible. However, if you're not using name based hosting(blowing through many IPs that could be consolidated if you didn't), all that's asked is that you justify why you can't/won't do it that way on your application for PI space. If IPv6 PI space worked the same way - If you could justify why shim6 isn't sufficient for your network, and receive PI space... I'd have zero objections to anything shim6 wanted to do. Let those who want to use it use it, and let those who can't do what they need to do use the existing alternatives. People aren't going to frivolously pay the yearly fees for an ASN and address space unless they truly believe they need it, and it would completely level the playing field. Small businesses won't have an unfair disadvantage to their larger competitors who get PI space. Let shim6 (or whatever takes its place for IPv6 multihoming) prove itself and seem like a more attractive solution than paying for PI space, and you have a winner. No matter how much you think shim6 is the way to do it, you're not going to have willing users of it unless it's just as good as what they're doing now. Unless we start now working on getting people moved to IPv6, the pain of running out of IPv4 before IPv6 has reached critical mass is going to be much much worse than a long term problem of IPv6 route size. The question of IPv6 migration and IPv6 route size are *two different problems*. Solve them independently or neither will get solved. We can't try to force our views of how the internet should work on networks when we've already got a fight on our hands just convincing them to deploy IPv6 at all. Nothing at all precludes allowing people to start deploying multihomed IPv6 networks using the traditional system, and presenting alternatives when they've proven themselves. If shim6 truly is the perfect match for people who need to multihome, new networks will deploy it natively, and small companies will accept it as a way to stop paying a yearly fee for PI/ASN allocations. If you're not so sure that shim6 meets the "it's good enough that people will use it willingly" goal, it's not going to work to mandate its use by policy. Right now, shim6 isn't even completed. There are networks who could be convinced to start deploying IPv6 today, if they had a multihoming solution. Let them do this! The sooner we get people publishing AAAA records, the smoother the transition is going to be for everyone. I really don't want to see the state of the internet if IPv4 is exhausted and IPv6 is still being shot down because of policy debates. (Again, my organization already has a /32, I'm not out anything with the current policies, if anything I'm helping my competition by arguing this point) -- Kevin
On 4-mrt-2006, at 14:07, Kevin Day wrote:
We got lucky with CIDR because even though all default free routers had to be upgraded in a short time, it really wasn't that painful.
[Because there was no need to renumber]
Isn't that an excellent argument against shim6 though?
In IPv4, something unanticipated occurred by the original developers (the need for CIDR), and everyone said "Oh, thank god that all we have to do is upgrade DFZ routers."
You are absolutely right that having to upgrade not only all hosts in a multihomed site, but also all the hosts they communicate with is an important weakness of shim6. We looked very hard at ways to do this type of multihoming that would work if only the hosts in the multihomed site were updated, but that just wouldn't fly.
In IPv6/shim6 what happens if shim6 has an unanticipated bottleneck, security issue, or scaleability problem? Everyone, everywhere, has to upgrade at some point. This means that upgrade/ workaround has to backwards compatible indefinitely, since it isn't going to be possible to force all the world's servers, desktops and devices to upgrade by some flag day.
That's why it's extremely important to get it right the first time. On the other hand, the fact that shim6 works between just two hosts without having to upgrade the whole internet first makes it a lot easier to test and work out the kinks.
If it requires an OS update to add a feature, you can't rely on having 90% penetration within YEARS of it being released. After XP Service Pack 2 was released, only 24% had upgraded after 9 months. According to one of our website's stats, we still see 5% of our users on Windows 98, and 13% on Windows 2000.
Yes, this is an issue. If we have to wait for a major release or even a service pack, that will take some time. But OS vendors have software update mechanisms in place so they could send out shim6 code in between. But again, it cuts both ways: if only two people run shim6 code, those two people gain shim6 benefits immediately.
Getting systems not controlled by the networking department of an organization upgraded, when it's for reasons that are not easily visible to the end user, will be extraordinarily difficult to start with. Adding shim6 at all to hosts will be one fight. Any upgrades or changes later to add features will be another.
One thing I'll take away from these discussions is that we should redouble our efforts to support shim6 in middleboxes as an alternative for doing it in individual hosts, so deployment can be easier.
In an enterprise environment, you're not talking the DBA of your Oracle box into installing service packs, upgrades or new software just because you want to use a new routing feature that wasn't around when their OS was released. I know of several enterprises still running NT 4.0 and Mac OS 9 boxes for some legacy applications.
Ah, but in the world of shim6 "legacy" takes on a whole new meaning, because it relates to today's IPv6 which the OSes you mention don't even implement. (-: And don't forget that in enterprises, most boxes don't talk directly (or at least not much) to the outside world.
The real "injustice" about this is that it's creating two classes of organizations on the internet. One that's meets the guidelines to get PI space, multihomes with BGP, and can run their whole network(including shim6less legacy devices) with full redundancy, even when talking to shim6 unaware clients. Another(most likely smaller) that can't meet the rules to get PI space, is forced to spend money upgrading hardware and software to shim6 compatible solution or face a lower reliability than their bigger competitors.
And that's exactly why it's so hard to come up with a good PI policy: you can't just impose an arbitrary limit, because that would be anti- competitive.
Someone earlier brought up that a move to shim6, or not being able to get PI space was similar to the move to name based vhosting(HTTP/ 1.1 shared IP hosting). It is, somewhat. It was developed to allow hosting companies to preserve address space, instead of assigning one IP address per hostname. (Again, however, this could be done slowly, without forcing end users to do anything.)
Tthis isn't that good an analogy. With name based virtual hosting, the server either is name based or IP based. If you run name based, old HTTP 1.0 clients won't be served the content they're looking for. So people running servers had to wait until a large enough percentage of users ran clients that supported HTTP 1.1 (or HTTP 1.0 with the host: variable). Fortunately, there was a browser war on at that time so people upgraded their web browser software relatively often, but it still took a few years before name based virtual hosting became viable. Shim6 is completely backward compatible. If either end doesn't support the protocol, everything still works, but without multihoming benefits of course. So everyone can enable shim6 as soon as their OS supports it with no ill effects.
If you could justify why shim6 isn't sufficient for your network, and receive PI space... I'd have zero objections to anything shim6 wanted to do.
Actually that's not the worst idea ever. The main problem would be coming objective criteria that determine shim6-insufficientness and of course the bar must be sufficiently high that we don't have to worry about excessive numbers of PI prefixes in the IPv6 global routing table.
Unless we start now working on getting people moved to IPv6, the pain of running out of IPv4 before IPv6 has reached critical mass is going to be much much worse than a long term problem of IPv6 route size.
I disagree. You assume that IPv6 will be able to gain critical mass before IPv4 addresses run out. I don't think that will happen, because of the chicken/egg problem. "Running out" is a relative term. John Klensin says we've effictively already run out because IPv4 addresses are too hard to get for some applications. That may be true but people aren't turning to IPv6 (yet) to run those applications. My prediction is that we'll see interesting things happen when the remaining IPv4 address suppy < 3 * addresses used per year. That will probably happen around the end of this decade. At that point, there is likely to be hoarding and/or the allocation policies will become stricter, and people will start to think about a future where it's no longer possible to get IPv4 addresses. At this point, there will still be time to migrate. If we screw up the routing table real good on the other hand, we're in trouble immediately and it will be both expensive to do nothing and to fix it.
The question of IPv6 migration and IPv6 route size are *two different problems*. Solve them independently or neither will get solved. We can't try to force our views of how the internet should work on networks when we've already got a fight on our hands just convincing them to deploy IPv6 at all.
I see this differently: as long as people are postponing deployment, we have the opportunity to improve IPv6 without too much trouble. So not having significant deployment isn't such a bad thing, as long as it's clear that IPv6 is inevitable. As long as we're debating whether IPv6 will be deployed at all we're wasting time. In another year or maybe two that debate will probably be over, though.
On 3/4/06, Iljitsch van Beijnum <iljitsch@muada.com> wrote:
On 4-mrt-2006, at 14:07, Kevin Day wrote:
We got lucky with CIDR because even though all default free routers had to be upgraded in a short time, it really wasn't that painful.
[Because there was no need to renumber]
Isn't that an excellent argument against shim6 though?
In IPv4, something unanticipated occurred by the original developers (the need for CIDR), and everyone said "Oh, thank god that all we have to do is upgrade DFZ routers."
You are absolutely right that having to upgrade not only all hosts in a multihomed site, but also all the hosts they communicate with is an important weakness of shim6. We looked very hard at ways to do this type of multihoming that would work if only the hosts in the multihomed site were updated, but that just wouldn't fly.
And given that any network big enough to get their own PI /32 has *zero* incentive to install/support shim6 means that all those smaller networks that are pushed to install shim6 are going to see *zero* benefit when they try to reach the major sites on the internet. What benefit does shim6 bring, if only the little guys are using it? This dog won't hunt. Move on to something useful. Yes, this is an issue. If we have to wait for a major release or even
a service pack, that will take some time. But OS vendors have software update mechanisms in place so they could send out shim6 code in between.
And no major company supports/allows automated software update mechanisms to run on their production machines--it adds too much of an element of randomness to an environment that has to be as much as possible deterministic in its behaviour. But again, it cuts both ways: if only two people run shim6 code,
those two people gain shim6 benefits immediately.
Cool. So let individuals make a choice to install it if they want. But that's a choice they make, and should not be part of a mandated IP allocation policy, because otherwise we're codifying a split between "big" companies and everyone else. The companies that can justify /32 allocations _aren't_ going to install shim6; they already have their multihoming options (for the most part) covered--so the little guys who install shim6 to "multihome" are going to discover it doesn't do diddly squat for helping them reach any major sites on the internet during an outage of one of their providers. You haven't preserved end-to-end connectivity this way, you've just waved a pretty picture in front of the smaller company's face to make them think they'll have the benefits of multihoming when they really don't.
Getting systems not controlled by the networking department of an
organization upgraded, when it's for reasons that are not easily visible to the end user, will be extraordinarily difficult to start with. Adding shim6 at all to hosts will be one fight. Any upgrades or changes later to add features will be another.
One thing I'll take away from these discussions is that we should redouble our efforts to support shim6 in middleboxes as an alternative for doing it in individual hosts, so deployment can be easier.
Won't matter. shim6 on a middle box still won't be able to re-route to the majority of the large sites on the Internet during an outage on one of the upstream providers given that the large content players and large network providers aren't going to be installing shim6 on their servers and load balancers.
The real "injustice" about this is that it's creating two classes
of organizations on the internet. One that's meets the guidelines to get PI space, multihomes with BGP, and can run their whole network(including shim6less legacy devices) with full redundancy, even when talking to shim6 unaware clients. Another(most likely smaller) that can't meet the rules to get PI space, is forced to spend money upgrading hardware and software to shim6 compatible solution or face a lower reliability than their bigger competitors.
And that's exactly why it's so hard to come up with a good PI policy: you can't just impose an arbitrary limit, because that would be anti- competitive.
You failed to note that the smaller company, *even after spending money upgrading hardware and software to shim6 compatible solution* won't achieve the same reliability as their bigger competitors. (see above if you missed it). shim6 is _more_ anti-competitive than extending the existing IP allocation policies from v4 into v6, and is therefore not going to garner the support of the companies that actually spend money to create this thing we call the Internet. And without money behind it, the effort is a non-starter.
Someone earlier brought up that a move to shim6, or not being able
to get PI space was similar to the move to name based vhosting(HTTP/ 1.1 shared IP hosting). It is, somewhat. It was developed to allow hosting companies to preserve address space, instead of assigning one IP address per hostname. (Again, however, this could be done slowly, without forcing end users to do anything.)
Tthis isn't that good an analogy. With name based virtual hosting, the server either is name based or IP based. If you run name based, old HTTP 1.0 clients won't be served the content they're looking for. So people running servers had to wait until a large enough percentage of users ran clients that supported HTTP 1.1 (or HTTP 1.0 with the host: variable). Fortunately, there was a browser war on at that time so people upgraded their web browser software relatively often, but it still took a few years before name based virtual hosting became viable.
Shim6 is completely backward compatible. If either end doesn't support the protocol, everything still works, but without multihoming benefits of course. So everyone can enable shim6 as soon as their OS supports it with no ill effects.
But the smaller sites who enable shim6 don't gain any benefit when talking to the large sites on the internet--so they've gone through a lot of pain and effort for very little actual benefit, since they still aren't usefully multihomed. There's just no real benefit to shim6 unless you require *EVERY* site to support it; and I can tell you that the large content sites will simply stay on v4 rather than install the complexity that is shim6 on their production webservers.
If you could justify why shim6 isn't sufficient for your network,
and receive PI space... I'd have zero objections to anything shim6 wanted to do.
Actually that's not the worst idea ever. The main problem would be coming objective criteria that determine shim6-insufficientness and of course the bar must be sufficiently high that we don't have to worry about excessive numbers of PI prefixes in the IPv6 global routing table.
Unless we start now working on getting people moved to IPv6, the pain of running out of IPv4 before IPv6 has reached critical mass is going to be much much worse than a long term problem of IPv6 route size.
I disagree. You assume that IPv6 will be able to gain critical mass before IPv4 addresses run out. I don't think that will happen, because of the chicken/egg problem. "Running out" is a relative term. John Klensin says we've effictively already run out because IPv4 addresses are too hard to get for some applications. That may be true but people aren't turning to IPv6 (yet) to run those applications. My prediction is that we'll see interesting things happen when the remaining IPv4 address suppy < 3 * addresses used per year. That will probably happen around the end of this decade. At that point, there is likely to be hoarding and/or the allocation policies will become stricter, and people will start to think about a future where it's no longer possible to get IPv4 addresses. At this point, there will still be time to migrate.
Consolidation will likely occur; those that need address space will find that buying less-fortunate companies in order to swallow their address space will become a normal, understood part of their business planning cycle. Competition will decrease, and the shift towards larger and larger companies will ensue, as smaller players gobble each other up in order to become large enough such that any needed migration to IPv6 can happen directly onto a PI /32. If we persist on following this path, we'll simply end up in a world where the large entities control the resources, and the barriers for entry turn out to be the very ones we set up in our own well-meaning bumbling. If we screw up the routing table real good on the other hand, we're
in trouble immediately and it will be both expensive to do nothing and to fix it.
I have more faith in our ability to deal with route table growth than I do in our ability to come up with a viable instantiation of shim6.
The question of IPv6 migration and IPv6 route size are *two
different problems*. Solve them independently or neither will get solved. We can't try to force our views of how the internet should work on networks when we've already got a fight on our hands just convincing them to deploy IPv6 at all.
I see this differently: as long as people are postponing deployment, we have the opportunity to improve IPv6 without too much trouble. So not having significant deployment isn't such a bad thing, as long as it's clear that IPv6 is inevitable. As long as we're debating whether IPv6 will be deployed at all we're wasting time. In another year or maybe two that debate will probably be over, though.
IPv6 may be inevitable; but the way shim6 is pushing allocation policies, it will be in a world in which only big players multihome, and everyone else must buy from a big player and won't get to multihome. Yes, people will wave the shim6 flag around to make small startups think they can multihome and pretend to be a big player, but at the first outage, the little guy will discover his multihoming is a facade, and that none of the major sites on the Internet that he wants to talk to are interested in playing his shim6 games with his end hosts--and his customers will quickly realize that any independance from the upstream networks is all smoke and mirrors, and not worth the paper such claims may be printed upon. If that's the direction we're heading, let's just stop beating around the bush and say it plainly: Shim6 is just a handwaving panacea to make the smaller enterprises shut up and stop obstructing v6 deployment for the short term so that we can get more critical mass on the v6 networks and maybe justify getting some of the large players to start making useful material available via v6 which might finally show a few dollars of real revenue flowing due to v6 deployments. But it's insulting to keep pretending that shim6 is going to offer any level of real multihoming-style reliability benefit for the smaller players when talking to engineers. Save it for the marketing literature for the customers. Matt
Hello; On Mar 4, 2006, at 4:31 PM, Matthew Petach wrote:
On 3/4/06, Iljitsch van Beijnum <iljitsch@muada.com> wrote:
On 4-mrt-2006, at 14:07, Kevin Day wrote:
We got lucky with CIDR because even though all default free routers had to be upgraded in a short time, it really wasn't that painful.
[Because there was no need to renumber]
Isn't that an excellent argument against shim6 though?
In IPv4, something unanticipated occurred by the original developers (the need for CIDR), and everyone said "Oh, thank god that all we have to do is upgrade DFZ routers."
You are absolutely right that having to upgrade not only all hosts in a multihomed site, but also all the hosts they communicate with is an important weakness of shim6. We looked very hard at ways to do this type of multihoming that would work if only the hosts in the multihomed site were updated, but that just wouldn't fly.
And given that any network big enough to get their own PI /32 has *zero* incentive to install/support shim6 means that all those smaller networks that are pushed to install shim6 are going to see *zero* benefit when they try to reach the major sites on the internet.
What benefit does shim6 bring, if only the little guys are using it?
This dog won't hunt. Move on to something useful.
Yes, this is an issue. If we have to wait for a major release or even a service pack, that will take some time. But OS vendors have software update mechanisms in place so they could send out shim6 code in between.
And no major company supports/allows automated software update mechanisms to run on their production machines--it adds too much of an element of randomness to an environment that has to be as much as possible deterministic in its behaviour.
But again, it cuts both ways: if only two people run shim6 code, those two people gain shim6 benefits immediately.
Cool. So let individuals make a choice to install it if they want. But that's a choice they make, and should not be part of a mandated IP allocation policy, because otherwise we're codifying a split between "big" companies and everyone else. The companies that can justify /32 allocations _aren't_ going to install shim6; they already have their multihoming options (for the most part) covered--so the little guys who install shim6 to "multihome" are going to discover it doesn't do diddly squat for helping them reach any major sites on the internet during an outage of one of their providers. You haven't preserved end-to-end connectivity this way, you've just waved a pretty picture in front of the smaller company's face to make them think they'll have the benefits of multihoming when they really don't.
Even guys in the middle may not. If I am streaming for profit, and if I am comfortable with my bandwidth suppliers (or if I am multihoming one way or another), then why should I spend a dime installing and running shim6 ? If you (an end user in a SOHO with 2 connections say) go down for 5 minutes once per day, that's too bad, but the ad revenue I would lose would be tiny, and you wouldn't blame me for the outage, you would blame your provider. Also, qn outage may hurt you with 100% probability, but it will only hurt me if it happens while you are using my services, which will probably be < 100% of the time. Again, the downside to me from your outages is very small. So, my feeling is that most content providers would be very slow to adopt this.
Getting systems not controlled by the networking department of an organization upgraded, when it's for reasons that are not easily visible to the end user, will be extraordinarily difficult to start with. Adding shim6 at all to hosts will be one fight. Any upgrades or changes later to add features will be another.
One thing I'll take away from these discussions is that we should redouble our efforts to support shim6 in middleboxes as an alternative for doing it in individual hosts, so deployment can be easier.
Won't matter. shim6 on a middle box still won't be able to re- route to the majority of the large sites on the Internet during an outage on one of the upstream providers given that the large content players and large network providers aren't going to be installing shim6 on their servers and load balancers.
I could see a business built around integrating shim6 into a proxy, such that an end user with two connections runs shim6 to get to an outside proxy (which could be very multihomed) and the proxy gets the packets to ordinary (non shim6) sites. This is under the model that my home office may need shim6 for multi- homing, but AOL or Google will not, and thus won't be supporting it. So I wouldn't count it out even if deployment is << 100%. But that is of course not a reason to have it drive address allocation policies.
The real "injustice" about this is that it's creating two classes of organizations on the internet. One that's meets the guidelines to get PI space, multihomes with BGP, and can run their whole network(including shim6less legacy devices) with full redundancy, even when talking to shim6 unaware clients. Another(most likely smaller) that can't meet the rules to get PI space, is forced to spend money upgrading hardware and software to shim6 compatible solution or face a lower reliability than their bigger competitors.
And that's exactly why it's so hard to come up with a good PI policy: you can't just impose an arbitrary limit, because that would be anti- competitive.
You failed to note that the smaller company, *even after spending money upgrading hardware and software to shim6 compatible solution* won't achieve the same reliability as their bigger competitors. (see above if you missed it).
shim6 is _more_ anti-competitive than extending the existing IP allocation policies from v4 into v6, and is therefore not going to garner the support of the companies that actually spend money to create this thing we call the Internet. And without money behind it, the effort is a non-starter.
Someone earlier brought up that a move to shim6, or not being able to get PI space was similar to the move to name based vhosting(HTTP/ 1.1 shared IP hosting). It is, somewhat. It was developed to allow hosting companies to preserve address space, instead of assigning one IP address per hostname. (Again, however, this could be done slowly, without forcing end users to do anything.)
Tthis isn't that good an analogy. With name based virtual hosting, the server either is name based or IP based. If you run name based, old HTTP 1.0 clients won't be served the content they're looking for. So people running servers had to wait until a large enough percentage of users ran clients that supported HTTP 1.1 (or HTTP 1.0 with the host: variable). Fortunately, there was a browser war on at that time so people upgraded their web browser software relatively often, but it still took a few years before name based virtual hosting became viable.
Shim6 is completely backward compatible. If either end doesn't support the protocol, everything still works, but without multihoming benefits of course. So everyone can enable shim6 as soon as their OS supports it with no ill effects.
But the smaller sites who enable shim6 don't gain any benefit when talking to the large sites on the internet--so they've gone through a lot of pain and effort for very little actual benefit, since they still aren't usefully multihomed. There's just no real benefit to shim6 unless you require *EVERY* site to support it; and I can tell you that the large content sites will simply stay on v4 rather than install the complexity that is shim6 on their production webservers.
If you could justify why shim6 isn't sufficient for your network, and receive PI space... I'd have zero objections to anything shim6 wanted to do.
Actually that's not the worst idea ever. The main problem would be coming objective criteria that determine shim6-insufficientness and of course the bar must be sufficiently high that we don't have to worry about excessive numbers of PI prefixes in the IPv6 global routing table.
Unless we start now working on getting people moved to IPv6, the pain of running out of IPv4 before IPv6 has reached critical mass is going to be much much worse than a long term problem of IPv6 route size.
I disagree. You assume that IPv6 will be able to gain critical mass before IPv4 addresses run out. I don't think that will happen, because of the chicken/egg problem. "Running out" is a relative term. John Klensin says we've effictively already run out because IPv4 addresses are too hard to get for some applications. That may be true but people aren't turning to IPv6 (yet) to run those applications. My prediction is that we'll see interesting things happen when the remaining IPv4 address suppy < 3 * addresses used per year. That will probably happen around the end of this decade. At that point, there is likely to be hoarding and/or the allocation policies will become stricter, and people will start to think about a future where it's no longer possible to get IPv4 addresses. At this point, there will still be time to migrate.
Consolidation will likely occur; those that need address space will find that buying less-fortunate companies in order to swallow their address space will become a normal, understood part of their business planning cycle. Competition will decrease, and the shift towards larger and larger companies will ensue, as smaller players gobble each other up in order to become large enough such that any needed migration to IPv6 can happen directly onto a PI /32.
If we persist on following this path, we'll simply end up in a world where the large entities control the resources, and the barriers for entry turn out to be the very ones we set up in our own well-meaning bumbling.
If we screw up the routing table real good on the other hand, we're in trouble immediately and it will be both expensive to do nothing and to fix it.
I have more faith in our ability to deal with route table growth than I do in our ability to come up with a viable instantiation of shim6.
The question of IPv6 migration and IPv6 route size are *two different problems*. Solve them independently or neither will get solved. We can't try to force our views of how the internet should work on networks when we've already got a fight on our hands just convincing them to deploy IPv6 at all.
I see this differently: as long as people are postponing deployment, we have the opportunity to improve IPv6 without too much trouble. So not having significant deployment isn't such a bad thing, as long as it's clear that IPv6 is inevitable. As long as we're debating whether IPv6 will be deployed at all we're wasting time. In another year or maybe two that debate will probably be over, though.
IPv6 may be inevitable; but the way shim6 is pushing allocation policies, it will be in a world in which only big players multihome, and everyone else must buy from a big player and won't get to multihome. Yes, people will
That assumes that people will be able to make such an allocation policy stick. I don't think they can; at least, not for every region in the world, as the aggregated power of the "little guys" is considerable. And, if one RIR adopts a more reasonable routing policy, in the modern world it would be easy to create shell corporations to get blocks for use elsewhere. As evidence for that, look at a few of recent companies to show up on BGP (from the ones that showed up here this week) : Howard Hughes Medical Institute Great American Insurance Company CARNIVAL CRUISE LINES Major League Baseball Advanced Media, LP Flagship Customs Services BP Pte Limited, Energy Company, Singapore Vonage Australia Xtiva Financial Systems, Inc. (Of the "new" ASN this week, only 2 are obviously ISP's, one is in Cambodia and the other is National LambdaRail. This is typical in the last few years.) Do you really think that you will be able to prevent these institutions from getting PI space, when they decide that they need to ?
wave the shim6 flag around to make small startups think they can multihome and pretend to be a big player, but at the first outage, the little guy will discover his multihoming is a facade, and that none of the major sites on the Internet that he wants to talk to are interested in playing his shim6 games with his end hosts--and his customers will quickly realize that any independance from the upstream networks is all smoke and mirrors, and not worth the paper such claims may be printed upon.
If that's the direction we're heading, let's just stop beating around the bush and say it plainly: Shim6 is just a handwaving panacea to make the smaller enterprises shut up and stop obstructing v6 deployment for the short term so that we can get more critical mass on the v6 networks and maybe justify getting some of the large players to start making useful material available via v6 which might finally show a few dollars of real revenue flowing due to v6 deployments.
If you want to spur v6 deployment, treat it like a business. Give incentives to use it. Consider this : today, as of Noon EST, there were 178,214 announced (IPv4) prefixes here, from only 21,517 ASN. 97,196 of these prefix blocks were /24's - on average, every ASN has 4.5 /24's. That is the swamp in operation. So, if we gave every active ASN a contiguous IPv6 block, and moved everyone over to IPv6, we would REDUCE the size of the routing table by a factor of 8.28. That would gain several years of growth before the routing table is the size it is now. That would mean the better part of a decade of even rapid growth before the routing table size would be a worry (as routers will also continue to grow in capabilities). As none of us can predict what will be going on 8-10 years from now, this would have actually solved the routing table growth problem for the foreseeable future. So, hand out PI /48's to everyone who has an active ASN. Hand out PI / 47's or whatever, to anyone with an ASN who can justify the larger size using the usual justification procedures. Don't hand these out in contiguous blocks, though. One simple procedure would just be to hand out the first /48 from, say, a /38 and reserve the rest of the /38 for future growth of that ASN. Note that even with this wasteful procedure, you would need almost 3 million years at the current rate of usage to hand out 1/2 of the available /38's. I am sure that better procedures could be arrived at, it doesn't matter much, as long as organizational growth does not have to mean multiple, non-contiguous address blocks. As part of this generous policy, make assignees sign a statement that they understand that they cannot deaggregate their address blocks. Now, legally, I doubt that this would be really enforceable, but it would be a useful thing to point to when they ask why they are being filtered out if they start announcing 2048 /48s. Charge a nominal amount for this boon. Easy, hassle free, address space would start getting people at least thinking about v6 deployment. From what I know about organizations, that would lead to some additional deployment, to justify the original investment, no matter how small. With luck, that would snowball into actual usage. I will probably be flamed for writing this, but if there is any support for it, I will write it up as an ARIN proposal. My understanding is that this was the original intent behind 2005-1.
But it's insulting to keep pretending that shim6 is going to offer any level of real multihoming-style reliability benefit for the smaller players when talking to engineers. Save it for the marketing literature for the customers.
Matt
I would agree. Worse, it makes some people think that there are ulterior motives to the pretense. Regards Marshall
ME> Date: Sat, 4 Mar 2006 19:01:14 -0500 ME> From: Marshall Eubanks ME> So, if we gave every active ASN a contiguous IPv6 block, and moved ME> everyone over to IPv6, we would REDUCE the size of the routing table ME> by a factor of 8.28. That would gain several years of growth before ME> the routing table is the size it is now. Exactly. Fragmentation doesn't help... ME> Don't hand these out in contiguous blocks, though. One simple ME> procedure would just be to hand out the first /48 from, say, a /38 ME> and reserve the rest of the /38 for future growth of that ASN. ...and was/is caused by stride-n allocation policies where "n" is too small. (Would any sane software developer use strictly skip lists and unsorted arrays?) Exponential problems need logarithmic solutions. ME> I am sure that better procedures could be arrived at "Allocate from the middle of the largest contiguous block. Align as appropriate." Exponential problems need logarithmic solutions. ME> With luck, that would snowball into actual usage. It depends how forward-thinking people are. A carrot now to avoid a stick later would appear enticing... Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
On 4-Mar-2006, at 16:31, Matthew Petach wrote:
And given that any network big enough to get their own PI /32 has *zero* incentive to install/support shim6 means that all those smaller networks that are pushed to install shim6 are going to see *zero* benefit when they try to reach the major sites on the internet.
No support in big networks is required, beyond the presence of shim6 in server stacks. The assumption is, therefore, that if there has been sufficient deployment of shim6 in client stacks for this to matter, the chances are that the servers that those clients want to talk to have already enjoyed similar upgrades. In the companies I have been involved in which do hosting, my observation has been that the servers are generally upgraded and patched far more vigourously than machines belonging to clients. If that non-scientific observation holds any water, then this suggests that the issue of shim6 support in servers which are being used by shim6-capable clients will look after itself. Joe
On Mar 4, 2006, at 7:06 PM, Joe Abley wrote:
No support in big networks is required, beyond the presence of shim6 in server stacks.
Why do you say this? Enterprises who multihome need their client machines (tens and hundreds of thousands of them) to be able to take advantage of multihoming, as well. It's a requirement, not a luxury. [Note that I do not address the blurring of client and server roles which is happening even now, and which will almost certainly become more prevalent over the anticipated lifetime of the success protocol to IPv4.] This fundamental misconception of the requirements of large enterprise customers should be an indicator to proponents of shim6, among others, that they do not have a good grasp of the day-to-day operational and business realities faced by large enterprises. This lack of understanding has led to such fundamental misconceptions as a belief that large enterprises can accept frequent renumbering within their organizations based upon changing business relationships with their SPs (they cannot, see RFC 4192 for some of the reasons why not), as well as underestimating the importance of multihoming for client computers as well as servers. shim6 is simply not viable for large enterprises, who are the customers who require multihoming. I would argue that it's not really viable for smaller organizations either, due to the complexity it adds to the troubleshooting matrix for support staff. I hope that the operational community will turn to more fruitful lines of enquiry regarding IPv6 multihoming. ---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice Everything has been said. But nobody listens. -- Roger Shattuck
On 4-Mar-2006, at 23:48, Roland Dobbins wrote:
On Mar 4, 2006, at 7:06 PM, Joe Abley wrote:
No support in big networks is required, beyond the presence of shim6 in server stacks.
Why do you say this? Enterprises who multihome need their client machines (tens and hundreds of thousands of them) to be able to take advantage of multihoming, as well. It's a requirement, not a luxury.
You're missing the context; by "big networks" I was referring to those who are able to obtain PI addresses, following from: On 4-Mar-2006, at 16:31, Matthew Petach wrote:
And given that any network big enough to get their own PI /32 has *zero* incentive to install/support shim6 [...]
[I didn't mean to imply that enterprise networks weren't big or complicated.] Joe
On 5-mrt-2006, at 5:48, Roland Dobbins wrote:
This fundamental misconception of the requirements of large enterprise customers should be an indicator to proponents of shim6, among others, that they do not have a good grasp of the day-to-day operational and business realities faced by large enterprises. This lack of understanding has led to such fundamental misconceptions as a belief that large enterprises can accept frequent renumbering within their organizations based upon changing business relationships with their SPs (they cannot, see RFC 4192 for some of the reasons why not)
Ok, let me show you a trick I learned recently. I'm going to agree with you (although like everything else, the "need" for stable addresses can be translated into money, at some point it makes sense to renumber), and tell you I have the solution that fixes all of this, doesn't cost any money, no new code, nothing. Then after this, you'll see that nobody replies to this message and we simply go on arguing about the relative evil of PI in IPv6 vs that of shim6. The solution is routing based on geography: give every city of ~ 250k people a /32, and give people who multihome in or near that city a / 48 out of that /32. Since such a /48 will be easy to get, the global routing table will fill up with these /48s relatively quickly, and at some point it will start to look attractive to filter some of them out in part of an AS. This can be done without trouble by adding a few /32 aggregates that point towards the part of the AS where the / 48s that are filtered here are still known. Since every AS still has all the /48s somewhere within the AS, this works without strange requirements such as free transit or interconnection in every city. So what do we need to get this off the ground? New allocation policies. As long as the number of geo PI prefixes is smaller than what comfortably fits inside routers that's all. If the global routing table continues to grow transit ASes will have to choose between buying more expensive routers or adding complexity by implementing geographic aggregation using BGP filters. Obviously strange ways of multihoming (towards ISPs on different continents, for instance) or strange ways of interconnecting (such as two European networks interconnecting in Chicago) aren't very compatible with geographic aggregation, but who cares about 10% exceptions as long as you can aggregate the other 90%. Enterprises will have to choose between individual geo /48s in every location or becoming their own ISP with a /32 and backhaul traffic between locations themselves.
(oh how I'm going to regret jumping into this conversation at point 'here' not at the beginning :( ) On Sun, 5 Mar 2006, Iljitsch van Beijnum wrote:
On 5-mrt-2006, at 5:48, Roland Dobbins wrote:
This fundamental misconception of the requirements of large enterprise customers should be an indicator to proponents of shim6, among others, that they do not have a good grasp of the day-to-day operational and business realities faced by large enterprises. This lack of understanding has led to such fundamental misconceptions as a belief that large enterprises can accept frequent renumbering within their organizations based upon changing business relationships with their SPs (they cannot, see RFC 4192 for some of the reasons why not)
The solution is routing based on geography: give every city of ~ 250k people a /32, and give people who multihome in or near that city a / 48 out of that /32. Since such a /48 will be easy to get, the global
can the v6 table/space work with 6B /48's? Assume that in the end-state each 'person' is multi-homed: 802.11 + cell + cable + gprs. Also though a /48 is quite large for a single person there will be something (most likely) to interconnect each family member in a household most likely as well... or there might be :) who knows. Additionally, how many /32's are there? Does scaling that to each municipality make sense? the arbitrary (I suspect) 250k person limit will quickly devolve into 'any municipality' with much less restriction on it.
routing table will fill up with these /48s relatively quickly, and at some point it will start to look attractive to filter some of them out in part of an AS. This can be done without trouble by adding a few /32 aggregates that point towards the part of the AS where the /
For a single provider situation this might work, MIGHT. For a multi-provider world, like the one we live in today, it seems doubtful that this will scale... Unless we implement 'default to the left' or something of that sort (pass default to the guy on your left, hope he knows where west-kalamzoo's /32 is in the network)
48s that are filtered here are still known. Since every AS still has all the /48s somewhere within the AS, this works without strange requirements such as free transit or interconnection in every city.
I think I'm missing how this would work... if I don't have a route to you you don't exist. I have to have a default to 'somewhere' or a route, it seems fairly simple to me... see 'default to the left' above (also not a good solution, but :) )
So what do we need to get this off the ground? New allocation policies. As long as the number of geo PI prefixes is smaller than what comfortably fits inside routers that's all. If the global routing table continues to grow transit ASes will have to choose between buying more expensive routers or adding complexity by implementing geographic aggregation using BGP filters.
inside a single ASN aggregation may be workable. On the entire network it's going to be much more complex :(
Obviously strange ways of multihoming (towards ISPs on different continents, for instance) or strange ways of interconnecting (such as
ah, like pipex or vnsl or ... name your extra-us provider of choice. From the 'non-NAnog' perspective, what about US carriers in ASPAC that multihome to several pacific island nations at once?
two European networks interconnecting in Chicago) aren't very compatible with geographic aggregation, but who cares about 10% exceptions as long as you can aggregate the other 90%. Enterprises
is it 10%? did you measure this? or did Geoff? or Randy? or KC?
will have to choose between individual geo /48s in every location or becoming their own ISP with a /32 and backhaul traffic between locations themselves.
On 5-mrt-2006, at 17:37, Christopher L. Morrow wrote:
The solution is routing based on geography: give every city of ~ 250k people a /32, and give people who multihome in or near that city a / 48 out of that /32.
can the v6 table/space work with 6B /48's?
It can if you distribute those 6G prefixes over 6k - 64k routers, so each only has to carry 100k - 1M of those prefixes. Our approximate design goal was 1 multihomers per 10 people in the year 2050. That would be around one billion mulithomers. Today we don't see much more than 1 in 25000 in parts of the world where multihoming is popular, and in most places it isn't.
Additionally, how many /32's are there? Does scaling that to each municipality make sense? the arbitrary (I suspect) 250k person limit will quickly devolve into 'any municipality' with much less restriction on it.
The idea was having 64k /32s holding 64k /48s each. That means you'll end up with a /32 for each area with around 250k people. Having aggregation below that level doesn't look worthwhile and the number of municipal /32s would become uncomfortably large. Of course people living in small towns aren't left in the cold, it's just that they have to share a /32 with other small towns, the surrounding rural area or fall under a big city close by.
48s that are filtered here are still known. Since every AS still has all the /48s somewhere within the AS, this works without strange requirements such as free transit or interconnection in every city.
I think I'm missing how this would work... if I don't have a route to you you don't exist.
That's the beauty of aggregation. As an example, your AS could have US routes split into two sets: east of the Mississippi and west of the Mississippi. If you then want to send a packet from Boston to a multihomer in San Diego, the routers in Boston don't carry the /48 for that San Diego multihomer. But there is an aggregate for San Diego, so the packet is sent on its way with help from the aggregate. When the packet hits the first router west of the Mississippi, it is forwarded as per the actual /48.
inside a single ASN aggregation may be workable. On the entire network it's going to be much more complex :(
As long as each AS carries its customer routers everywhere and announces them to other ASes everywhere, there is no need to coordinate, because the outward behavior of each AS is as before. The only difference is that internally, things happen slightly differently.
Obviously strange ways of multihoming (towards ISPs on different continents, for instance) or strange ways of interconnecting (such as
ah, like pipex or vnsl or ... name your extra-us provider of choice. From the 'non-NAnog' perspective, what about US carriers in ASPAC that multihome to several pacific island nations at once?
Not sure what you mean, but suppose there is an island in the pacific where several US carriers land, but they don't interconnect there. Also suppose that carrier 1 links this island to LA and carrier 2 to Palo Alto. A packet from a customer of carrier 1 to a customer of carrier 2 will flow to LA because the /32 for the island points to LA (not to the island itself because there is no interconnection there). Carrier 2 doesn't have the full set of more specifics for the island in LA (they have those in Palo Alto) but they DO have all their customer routes in LA, so carrier 1 hears the /48 for carrier 2's customer in LA and hands the packet off to carrier 2, which transports it to Palo Alto and on to the island. Return packets will flow from carrier 2 to carrier 1 in Palo Alto.
two European networks interconnecting in Chicago) aren't very compatible with geographic aggregation, but who cares about 10% exceptions as long as you can aggregate the other 90%. Enterprises
is it 10%? did you measure this?
No. The only thing that we can possibly measure is what's in place today, and that's not representative for a situation where we have more than a million multihomers. However, it is to be expected that when we reach the point where there are that many multihomers, there will be some reason to the way multihomers connect to their ISPs. I would expect that the vast majority connects to local or regional ISPs. Even if they don't, it's not going to be that 5% of Paris multihomers connects to Montana, another 5% to Cape Town, 5% to Aruba and so on. If they do at the point that geographical aggregation takes off, they'll find that their traffic takes detours, i.e. a packet from Montana to Paris will first flow to Paris as per the aggregate, then go to the multihomer's ISP, then to Montana where the multihomer connects and finally back to Paris. So they'll have an incentive to multihome to ISPs closer to home but doing that wouldn't be too painful because they don't have to renumber if they used a Paris prefix from the start.
On 4-mrt-2006, at 22:31, Matthew Petach wrote:
And given that any network big enough to get their own PI /32 has *zero* incentive to install/support shim6 means that all those smaller networks that are pushed to install shim6 are going to see *zero* benefit when they try to reach the major sites on the internet.
A case can be made that some big sites sitting high and dry wouldn't care enough about their customers to do shim6 with them if they're multihomed using shim6, but that's not the same thing as there being zero benefit. Especially in a competitive market place: what if Hotmail runs shim6 so that multihomed Hotmail users can keep sending mail even when one ISP fails, while Gmail doesn't? You assume that all the big sites will get PI /32s and that this will allow them to multihome the way they want. I'm sure _some_ big sites will get a /32 or /48 or other PI block, but it remains to be seen whether all do. And having a single block isn't enough if you connect to the net in various locations but don't want to backhaul traffic between those locations yourself.
Yes, this is an issue. If we have to wait for a major release or even a service pack, that will take some time. But OS vendors have software update mechanisms in place so they could send out shim6 code in between.
And no major company supports/allows automated software update mechanisms
Dit I use the word "automated"?
But again, it cuts both ways: if only two people run shim6 code, those two people gain shim6 benefits immediately.
Cool. So let individuals make a choice to install it if they want. But that's a choice they make, and should not be part of a mandated IP allocation policy
Nobody is forced to implement shim6 if they don't want. But not liking shim6 doesn't buy you PI in IPv6.
shim6 is _more_ anti-competitive than extending the existing IP allocation policies from v4 into v6, and is therefore not going to garner the support of the companies that actually spend money to create this thing we call the Internet. And without money behind it, the effort is a non-starter.
2 million prefixes in a router that supports 1 million is also a non- starter. Insisting that shim6 isn't going to work is a waste of time, because it doesn't do anything to make shim6 better so it could work or create alternatives, it just adds FUD.
I have more faith in our ability to deal with route table growth than I do in our ability to come up with a viable instantiation of shim6.
Engineering based on faith is insane. Even with today's science we have balconies falling off of appartment buildings and roofs collapsing when it rains or snows a bit harder than usual, so a little caution here and there isn't too much to ask for.
You are absolutely right that having to upgrade not only all hosts in a multihomed site, but also all the hosts they communicate with is an important weakness of shim6. We looked very hard at ways to do this type of multihoming that would work if only the hosts in the multihomed site were updated, but that just wouldn't fly.
It flies if you look at changing the routing paradigm instead of pushing routing decisions out of the routers and off to the hosts. Source Routing is a technology that most of the internet figured out is problematic years ago. Making source routing more complicated and calling it something else doesn't make it less of a bad idea.
In IPv6/shim6 what happens if shim6 has an unanticipated bottleneck, security issue, or scaleability problem? Everyone, everywhere, has to upgrade at some point. This means that upgrade/ workaround has to backwards compatible indefinitely, since it isn't going to be possible to force all the world's servers, desktops and devices to upgrade by some flag day.
That's why it's extremely important to get it right the first time. On the other hand, the fact that shim6 works between just two hosts without having to upgrade the whole internet first makes it a lot easier to test and work out the kinks.
Sure, it's really easy to test shim6 between two hosts without involving the internet. I'll buy that. I'm not sure what the benefit of that is supposed to be to the average end user, but, I can accept that as a reason that developers of shim6 might like it.
But again, it cuts both ways: if only two people run shim6 code, those two people gain shim6 benefits immediately.
Only to the extent that they are talking to each other. If I deploy shim6, but, the top 5000 web sites that my users need to talk to do not, then, there's no benefit whatsoever to shim6 to the majority of my users.
One thing I'll take away from these discussions is that we should redouble our efforts to support shim6 in middleboxes as an alternative for doing it in individual hosts, so deployment can be easier.
That's a small step in the right direction. Looking at the possibility to change the fundamental routing paradigm for interdomain routing is probably a better possibility. Of course, these options are not technically mutually exclusive, but, I think the latter will be actually easier to deploy and yield greater benefit.
The real "injustice" about this is that it's creating two classes of organizations on the internet. One that's meets the guidelines to get PI space, multihomes with BGP, and can run their whole network(including shim6less legacy devices) with full redundancy, even when talking to shim6 unaware clients. Another(most likely smaller) that can't meet the rules to get PI space, is forced to spend money upgrading hardware and software to shim6 compatible solution or face a lower reliability than their bigger competitors.
And that's exactly why it's so hard to come up with a good PI policy: you can't just impose an arbitrary limit, because that would be anti- competitive.
A good PI policy is easy. Coming up with a scalable routing solution that supports good PI policy is hard. That's what we should be working on. A good PI policy is "If you need PI space, you get it." What we are trying to come up with in the meantime is a PI policy that will meet the needs of the majority of users while somewhat constraining growth in the routing table until that problem can be solved. (At least that's my intent with 2005-1)
Someone earlier brought up that a move to shim6, or not being able to get PI space was similar to the move to name based vhosting(HTTP/ 1.1 shared IP hosting). It is, somewhat. It was developed to allow hosting companies to preserve address space, instead of assigning one IP address per hostname. (Again, however, this could be done slowly, without forcing end users to do anything.)
Yeah, and it worked right up to the point that SSL became improtant and then it pretty much went away as a practical matter.
Tthis isn't that good an analogy. With name based virtual hosting, the server either is name based or IP based. If you run name based, old HTTP 1.0 clients won't be served the content they're looking for. So people running servers had to wait until a large enough percentage of users ran clients that supported HTTP 1.1 (or HTTP 1.0 with the host: variable). Fortunately, there was a browser war on at that time so people upgraded their web browser software relatively often, but it still took a few years before name based virtual hosting became viable.
And you think it won't be years before shim6 provides tangible benefits? You're dreaming.
Shim6 is completely backward compatible. If either end doesn't support the protocol, everything still works, but without multihoming benefits of course. So everyone can enable shim6 as soon as their OS supports it with no ill effects.
There were ways of handling this for NBVH initially too.
If you could justify why shim6 isn't sufficient for your network, and receive PI space... I'd have zero objections to anything shim6 wanted to do.
Actually that's not the worst idea ever. The main problem would be coming objective criteria that determine shim6-insufficientness and of course the bar must be sufficiently high that we don't have to worry about excessive numbers of PI prefixes in the IPv6 global routing table.
I would argue that you are putting the cart before the horse in this case. The solution must be sufficiently adequate that users do not feel the need to create routing table growth, or, the solution must be considered insufficient.
Unless we start now working on getting people moved to IPv6, the pain of running out of IPv4 before IPv6 has reached critical mass is going to be much much worse than a long term problem of IPv6 route size.
I disagree. You assume that IPv6 will be able to gain critical mass before IPv4 addresses run out. I don't think that will happen, because of the chicken/egg problem. "Running out" is a relative term. John Klensin says we've effictively already run out because IPv4 addresses are too hard to get for some applications. That may be true but people aren't turning to IPv6 (yet) to run those applications. My prediction is that we'll see interesting things happen when the remaining IPv4 address suppy < 3 * addresses used per year. That will probably happen around the end of this decade. At that point, there is likely to be hoarding and/or the allocation policies will become stricter, and people will start to think about a future where it's no longer possible to get IPv4 addresses. At this point, there will still be time to migrate.
IPv4 addresses are not hard to get. Portable IPv4 addresses are relatively harder to get than they should be, but, Portable IPv6 addresses are even harder to get than portable IPv4 addresses. People aren't turning to IPv6 because as things stand right now, it's harder to deploy and presents even less value than IPv4. The lack of PI space availability in IPv6 is one of the reasons for this. The lack of ISPs providing native v6 routing is another. The higher cost of IPv6 address space is a third. So far, from a business perspective, IPv6 is all cost and no benefit compared to IPv4.
If we screw up the routing table real good on the other hand, we're in trouble immediately and it will be both expensive to do nothing and to fix it.
I don't think it will be as expensive as you think to fix it. I think if we start working on a new routing paradigm today in order to support IDR based on AS PATH instead of Prefix, we would realistically see this in deployable workable code within 3-5 years. I think that we would see reasonable deployment in less than 10 years. I think that if we have this routing paradigm ready before the routing table starts to "overflow", we will see an accelerated adoption of this routing paradigm, much like the deployment of CIDR. Further, this routing paradigm would only need to be deployed on DFZ routers, which is a much smaller subset of more frequently updated hosts than the end-nodes.
The question of IPv6 migration and IPv6 route size are *two different problems*. Solve them independently or neither will get solved. We can't try to force our views of how the internet should work on networks when we've already got a fight on our hands just convincing them to deploy IPv6 at all.
I see this differently: as long as people are postponing deployment, we have the opportunity to improve IPv6 without too much trouble. So not having significant deployment isn't such a bad thing, as long as it's clear that IPv6 is inevitable. As long as we're debating whether IPv6 will be deployed at all we're wasting time. In another year or maybe two that debate will probably be over, though.
It is not clear that IPv6 is inevitable. In fact, at this point, it's not even clear that IPv6 has any long term viability at all. As it stands right now, for most situations, IPv4 with NAT is more desirable than IPv6. For servers, IPv4 remains more desirable until addresses start becoming prohibitively expensive (I think we are at least 15 years from that point). A lot can happen in 15 years, but, unless we start addressing the real end user requirements in IPv6 instead of continuing to focus on ways to extend CIDR and keep it alive (am I the only one who remembers that CIDR was originally viewed as a gross hack that was temporarily necessary until we came up with a real solution in IPv6?) I think that IPv6 will remain less than desirable. So, I accept that eventually, IPv6 may become unavoidable, but, I'd rather see it become desirable. Owen -- If it wasn't crypto-signed, it probably didn't come from me.
On 5-Mar-2006, at 14:16, Owen DeLong wrote:
It flies if you look at changing the routing paradigm instead of pushing routing decisions out of the routers and off to the hosts. Source Routing is a technology that most of the internet figured out is problematic years ago. Making source routing more complicated and calling it something else doesn't make it less of a bad idea.
Calling shim6 source-routing when it's not in order to give it an aura of evil is similarly unproductive :-)
I don't think it will be as expensive as you think to fix it. I think if we start working on a new routing paradigm today in order to support IDR based on AS PATH instead of Prefix, we would realistically see this in deployable workable code within 3-5 years.
I'm confused by statements such as these. Was it not the lack of any scalable routing solution after many years of trying that led people to resort to endpoint mobility in end systems, à la shim6? Joe
Thus spake "Joe Abley" <jabley@isc.org>
Was it not the lack of any scalable routing solution after many years of trying that led people to resort to endpoint mobility in end systems, à la shim6?
Who exactly has been trying to find scalable routing solutions? IPv6 advocates have been pushing a no-PI model for over a decade because that's what ISPs told them to do. When they found end users didn't like that, they went off and developed what has become shim6 as a poor apology. There has never been any significant work done on replacing CIDR with something that scales better. Every such proposal I've seen has been ignored or brushed aside by folks who've been doing CIDR for their entire careers and refuse to even consider that anything else might be better. All this time, energy, and thought spent on shim6 would have been better spent on a scalable IDR solution. Luckily, we still have another decade or so to come up with something. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
On 5-Mar-2006, at 17:03, Stephen Sprunk wrote:
All this time, energy, and thought spent on shim6 would have been better spent on a scalable IDR solution. Luckily, we still have another decade or so to come up with something.
So the answer to the lack of a routing solution to multi-homing in v6 is to proceed as if we do have one, and hope that one appears at some point in the future before the lack of one causes too much pain? Very little time has been spent on shim6 so far. Far more time before that was spent on multi6, which considered many different approaches to multi-homing. Note that I'm not saying that the IETF process in general (and the multi6/shim6 process in particular) have been without dysfunction; however, I think your characterisation that "there has never been any significant work done on replacing CIDR with something that scales better" is a little misleading. Joe
On Mar 5, 2006, at 2:51 PM, Joe Abley wrote:
Very little time has been spent on shim6 so far. Far more time before that was spent on multi6, which considered many different approaches to multi-homing.
Spending time in and of itself has no value, you're entirely correct. Spending time to come up with a solution which does not engage in problem-shifting routing onto the nodes, fixed geographic allocations (another nonstarter for reasons which have been elucidated previously), and other schemes which ignore customer requirements and correspond to operational and business realities - does- have value, and that's what's being proposed. ---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice Everything has been said. But nobody listens. -- Roger Shattuck
On 6-mrt-2006, at 3:52, Roland Dobbins wrote:
fixed geographic allocations (another nonstarter for reasons which have been elucidated previously)
What I hear is "any type of geography can't work because network topology != geography". That's like saying cars can't work because they can't drive over water which covers 70% of the earth's surface. Early proposals for doing any geographic stuff were fatally flawed but there is enough correlation between geography and topology to allow for useful savings. Even if it's only at the continent level that would allow for about an 80% reduction of routing tables in the future when other continents reach the same level of multihoming as North America and Europe.
--On March 6, 2006 12:46:51 PM +0100 Iljitsch van Beijnum <iljitsch@muada.com> wrote:
On 6-mrt-2006, at 3:52, Roland Dobbins wrote:
fixed geographic allocations (another nonstarter for reasons which have been elucidated previously)
What I hear is "any type of geography can't work because network topology != geography". That's like saying cars can't work because they can't drive over water which covers 70% of the earth's surface.
No, it's more like saying "Cars which can't operate off of freeways won't work" because there are a lot of places freeways don't go. Hmmm... Come to think of it, I haven't seen anyone selling a car which won't operate off of a freeway.
Early proposals for doing any geographic stuff were fatally flawed but there is enough correlation between geography and topology to allow for useful savings. Even if it's only at the continent level that would allow for about an 80% reduction of routing tables in the future when other continents reach the same level of multihoming as North America and Europe.
I've got no opposition to issuing addresses based on some geotop. design, simply because on the off chance it does provide useful aggregation, why not. OTOH, I haven't seen anyone propose geotop allocation as a policy in the ARIN region (hint to those pushing for it). Owen -- If it wasn't crypto-signed, it probably didn't come from me.
On 6-mrt-2006, at 22:08, Owen DeLong wrote:
What I hear is "any type of geography can't work because network topology != geography". That's like saying cars can't work because they can't drive over water which covers 70% of the earth's surface.
No, it's more like saying "Cars which can't operate off of freeways won't work" because there are a lot of places freeways don't go. Hmmm... Come to think of it, I haven't seen anyone selling a car which won't operate off of a freeway.
If we slightly open this up to "vehicles on wheels" and "long distance infrastructure created specially for said vehicles" trains would qualify...
I've got no opposition to issuing addresses based on some geotop. design, simply because on the off chance it does provide useful aggregation, why not.
Exactly, that's all I ask.
OTOH, I haven't seen anyone propose geotop allocation as a policy in the ARIN region (hint to those pushing for it).
Hm, I would rather do this globally but maybe this is the way to go...
On Tue, 7 Mar 2006, Iljitsch van Beijnum wrote:
Hm, I would rather do this globally but maybe this is the way to go...
Geo-aggregation is something that stands its best chance of being implemented locally: - the 'players' involved will be fewer - requirements for a workable policy will be easier to work out (and fewer conflicts between requirements) - there may be existing 'arbiter of last resort' (particularly at national levels) to resolve the inevitable conflicts. Ie this may be best done even /below/ the RIR level (though, RIR agreement would be needed). Still though, hard to see it happening without some acceptance by operators. regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: The public is an old woman. Let her maunder and mumble. -- Thomas Carlyle
Paul Jakma wrote:
On Tue, 7 Mar 2006, Iljitsch van Beijnum wrote:
Hm, I would rather do this globally but maybe this is the way to go...
Geo-aggregation is something that stands its best chance of being implemented locally:
While I agree that any aggregation would happen locally, the overall allocation policy for a consistent geo approach needs to be done globally.
- the 'players' involved will be fewer - requirements for a workable policy will be easier to work out (and fewer conflicts between requirements) - there may be existing 'arbiter of last resort' (particularly at national levels) to resolve the inevitable conflicts.
Ie this may be best done even /below/ the RIR level (though, RIR agreement would be needed).
Still though, hard to see it happening without some acceptance by operators.
You are mixing issues here. A policy for assigning PI space is what ARIN can deal with. A deployment agreement about aggregating a collection of PI assignments is what operators can deal with. What needs to happen is to define a global mechanism for PI assignments such that it is possible to do aggregations when it becomes necessary. Any of the geo approaches allows aggregation of a high-density pocket without requiring aggregation of all pockets globally. In particular the approach I have been pursuing allows the definition of any aggregate to evolve and track population shifts over time. Tony
On Tue, 7 Mar 2006, Tony Hain wrote:
While I agree that any aggregation would happen locally, the overall allocation policy for a consistent geo approach needs to be done globally.
Ideally, yes. Failling that, it's still possible for it to be done unilaterally at a regional level, there would still be benefits. I.e. "globally agreed policy" need not be a blocking dependency.
You are mixing issues here.
Quite possibly ;).
A policy for assigning PI space is what ARIN can deal with. A deployment agreement about aggregating a collection of PI assignments is what operators can deal with.
Sure. However, imagine if $RIR can not agree on such a policy, it then could still be done within $REGION (in the $RIRs service area), presuming $RIR can at least agree to delegate the required address space (even if it can not agree on a policy). I agree though it would be better if $RIR would drive policy formulation, and even if better if the RIRs could jointly agree on such polic{y,ies}.
What needs to happen is to define a global mechanism for PI assignments such that it is possible to do aggregations when it becomes necessary. Any of the geo approaches allows aggregation of a high-density pocket without requiring aggregation of all pockets globally. In particular the approach I have been pursuing allows the definition of any aggregate to evolve and track population shifts over time.
Do you have any pointers to online material? Sounds very interesting. regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: The truth about a man lies first and foremost in what he hides. -- Andre Malraux
--On March 7, 2006 4:29:28 PM +0100 Iljitsch van Beijnum <iljitsch@muada.com> wrote:
On 6-mrt-2006, at 22:08, Owen DeLong wrote:
What I hear is "any type of geography can't work because network topology != geography". That's like saying cars can't work because they can't drive over water which covers 70% of the earth's surface.
No, it's more like saying "Cars which can't operate off of freeways won't work" because there are a lot of places freeways don't go. Hmmm... Come to think of it, I haven't seen anyone selling a car which won't operate off of a freeway.
If we slightly open this up to "vehicles on wheels" and "long distance infrastructure created specially for said vehicles" trains would qualify...
True, and, a good case in point. A relatively small percentage of the US population finds trains routinely useful. An even smaller percentage (infinitessimal, actually) finds them useful enough to not have a car.
I've got no opposition to issuing addresses based on some geotop. design, simply because on the off chance it does provide useful aggregation, why not.
Exactly, that's all I ask.
OTOH, I haven't seen anyone propose geotop allocation as a policy in the ARIN region (hint to those pushing for it).
Hm, I would rather do this globally but maybe this is the way to go...
The only way to achieve global policy is to achieve a similar policy in each RIR and then get them to agree on a globally consistent one together. This is by design because it is a process which allows each region to have full input into the process without the stakeholders in any region being steamrolled by the needs of another region. Owen
At 1:08 PM -0800 3/6/06, Owen DeLong wrote:
I've got no opposition to issuing addresses based on some geotop. design, simply because on the off chance it does provide useful aggregation, why not. OTOH, I haven't seen anyone propose geotop allocation as a policy in the ARIN region (hint to those pushing for it).
Does anyone have statistics for the present prefix mobility experiment in the US with phone number portability? It would be interesting to know what percent of personal and business numbers are now routed permanently outside their original NPA assignment area... If one presumes a modest movement rate across the entire population of businesses, and locality for some percent of those moves (which may be hidden from global visibility due to regional interconnects/exchanges), would the resulting global routing table really be any larger then the current AS allocation count? It certainly would result in a lot of happy businesses to have a PI allocation from a local LIR, even if portability wasn't assured if they relocated to another state. /John p.s. personal thoughts only, designed entirely to encourage discussion... :-)
JC> Date: Tue, 7 Mar 2006 13:38:50 -0500 JC> From: John Curran JC> Does anyone have statistics for the present prefix mobility experiment JC> in the US with phone number portability? It would be interesting to JC> know what percent of personal and business numbers are now routed JC> permanently outside their original NPA assignment area... NPA or NXX? With ILECs required to interconnect with CLECs, it seems the distinction between NPA and NXX is significant. Of course, this brings up the issue of cellular telephony. Perhaps we should see what knowledge may be gleaned from cell infrastructure. Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
--On March 7, 2006 1:38:50 PM -0500 John Curran <jcurran@istaff.org> wrote:
At 1:08 PM -0800 3/6/06, Owen DeLong wrote:
I've got no opposition to issuing addresses based on some geotop. design, simply because on the off chance it does provide useful aggregation, why not. OTOH, I haven't seen anyone propose geotop allocation as a policy in the ARIN region (hint to those pushing for it).
Does anyone have statistics for the present prefix mobility experiment in the US with phone number portability? It would be interesting to know what percent of personal and business numbers are now routed permanently outside their original NPA assignment area...
Almost impossible to get statistics on this because of the influx of portable VOIP devices and cellular phones. However, if you're just talking about at the SS7 level, then, realistically, LNP doesn't really provide NPA level portability of numbers. At least I don't know of any telcos allowing you to take your 408 number to the 212 area when you move.
If one presumes a modest movement rate across the entire population of businesses, and locality for some percent of those moves (which may be hidden from global visibility due to regional interconnects/exchanges), would the resulting global routing table really be any larger then the current AS allocation count? It certainly would result in a lot of happy businesses to have a PI allocation from a local LIR, even if portability wasn't assured if they relocated to another state.
Interesting question. I wonder if CAIDA has any statistics which could provide illumination on this question.
/John
p.s. personal thoughts only, designed entirely to encourage discussion... :-)
Stephen Sprunk wrote:
Who exactly has been trying to find scalable routing solutions?
Well, for the last decade or so, there's been a small group of us who have been working towards a new routing architecture. Primary influences in my mind are Chiappa, O'Dell, Hain, Hinden, Nordmark, Atkinson, Fuller, Huston, Rekhter and Meyer. Apologies to any folks that feel I've incorrectly included or excluded them. ;-)
IPv6 advocates have been pushing a no-PI model for over a decade because that's what ISPs told them to do.
More accurately, the routing community has been trying to avoid PI addressing simply because of the scalability (and thus cost) concerns.
When they found end users didn't like that, they went off and developed what has become shim6 as a poor apology. There has never been any significant work done on replacing CIDR with something that scales better.
From my perspective, we have now explored the dominant quadrants of the solution space and various factions have vociferously denounced all
More accurately, that work (GSE/8+8) was slapped down politically without due technical consideration. Note that replacing CIDR isn't exactly the point. The point is to have something that scales. Where CIDR can't cope is exactly when we come to multihoming. When multihoming was a rare exception, the small number of PI prefixes remains tolerable. However, over time, the continued growth in multihoming, even solely as a function of the growth of the Internet will come to dominate the cost structure of the routing subsystem. The only alternative to a PI-like architecture is to provide multihomed sites with multiple prefixes, none of which need to be globally disseminated. Making this multiple prefix architecture work was the charter of the multi6 group. This was constrained in interesting ways, as both NAT box solutions were considered politically unacceptable, as was changing the core functionality of the v6 stack (i.e., redefining the TCP pseudoheader). Given these constraints, it was somewhat unsurprising that NAT got pushed into the host. possible solutions. You'll pardon me if some of us are feeling just a tad frustrated.
Every such proposal I've seen has been ignored or brushed aside by folks who've been doing CIDR for their entire careers and refuse to even consider that anything else might be better.
More accurately, the folks that have been CIDR advocates 'for their entire careers' are exactly the same folks who have been advocating shifting to something else, but have been rejected by other political elements when trying to propose actual architectural change. Further, those same CIDR advocates have been, and continue to be, in such political disfavor that they are effectively powerless anyhow. It hardly seems like their rejection could count for much.
All this time, energy, and thought spent on shim6 would have been better spent on a scalable IDR solution.
On that, we can agree. However, my feeling is that fully exploring the solution space is an unfortunate necessity before the community is willing to accept changes to the fundamental v6 architecture.
Luckily, we still have another decade or so to come up with something.
Unfortunately, that's not entirely true. If the RIR's begin wholesale PI assignment, then we start down the road of re-constituting the v4 routing architecture, locking in additional cost and complexity with each PI prefix. All such prefixes will be indefinitely grandfathered, so even if something new does come along, we will continue to pay for the overhead forever. Regards, Tony
What Tony said, especially about what happened to 8+8. A lot of the grounds for rejection were security, but there wasn't a single security person on the committee. In my opinion, most of the arguments just didn't hold up. --Steven M. Bellovin, http://www.cs.columbia.edu/~smb
On 6-mrt-2006, at 2:34, Steven M. Bellovin wrote:
What Tony said, especially about what happened to 8+8. A lot of the grounds for rejection were security, but there wasn't a single security person on the committee. In my opinion, most of the arguments just didn't hold up.
[RB = routing bits, IB = identity bits] So when I send you an 8+8 packet where [RB=me+IB=www.paypal.com] how do you know that this is bad while if Paypal sends you a packet with [RB=paypal+IB=www.paypal.com] that's good? Also, how does 8+8 accomplish failover? Original 8+8/GSE is incomplete. If you add the necessary extra stuff and think about backward compatibility for a while, you end up with something that's extremely close to shim6. If we add source address rewriting to shim6 (which is certainly doable) the family resemblence becomes even clearer.
Thus spake "Tony Li" <tony.li@tony.li>
Stephen Sprunk wrote:
Who exactly has been trying to find scalable routing solutions?
Well, for the last decade or so, there's been a small group of us who have been working towards a new routing architecture. Primary influences in my mind are Chiappa, O'Dell, Hain, Hinden, Nordmark, Atkinson, Fuller, Huston, Rekhter and Meyer. Apologies to any folks that feel I've incorrectly included or excluded them. ;-)
And my apologies for not recognizing the work that y'all have done; my point was that none of this seems to be officially supported by the IETF, and thus hasn't borne much fruit. I've seen a few proposals by folks listed above, and it seems to be old drafts (or even napkins) that get trotted out when this discussion comes up. If there's work actively going on today, it's not well publicized.
IPv6 advocates have been pushing a no-PI model for over a decade because that's what ISPs told them to do.
More accurately, the routing community has been trying to avoid PI addressing simply because of the scalability (and thus cost) concerns.
s/routing/ISP/ and I'd agree with that. The IETF has virtually no enterprise representation, and those folks (a) have a lot more routers than the ISPs, and (b) pay the ISPs' bills. I agree that PI has scaling/cost problems, but so far all of the alternatives presented by the IETF present worse problems _in the eyes of the people that pay the bills_. That doesn't mean the latter are right, but their views should not be taken lightly.
When they found end users didn't like that, they went off and developed what has become shim6 as a poor apology. There has never been any significant work done on replacing CIDR with something that scales better.
More accurately, that work (GSE/8+8) was slapped down politically without due technical consideration.
Correction noted.
Note that replacing CIDR isn't exactly the point. The point is to have something that scales. Where CIDR can't cope is exactly when we come to multihoming. When multihoming was a rare exception, the small number of PI prefixes remains tolerable. However, over time, the continued growth in multihoming, even solely as a function of the growth of the Internet will come to dominate the cost structure of the routing subsystem.
I'm not sure I agree with that. The ISPs out there have tens of prefixes each even in v6 land (and hundreds in v4 land), whereas the goal is to have one per end site. Until the number of multihomed end sites exceeds the number of ISPs by several orders of magnitude, the impact on the routing table will be non-dominant though certainly also non-trivial. Also, consider how easy it is to do PI-based multihoming in v4: all you need is a couple pipes (or tunnels), an ASN, and enough hosts to justify a /24. If you believed all the chicken littles, this would leave us with millions of v4 PI routes and the DFZ would be in ruins, yet only a few hundred people have taken ARIN up on that offer. In short, implementation of PI-based multihoming has ground almost to a halt even under today's liberal policies. Now, given the floodgates are open for v4 and all we see is a trickle of water, why is everyone running around screaming that the sky will fall if we do something similar for v6? Do we have any evidence at all that multihoming growth will outpace Moore and this whole debate is even relevant?
The only alternative to a PI-like architecture is to provide multihomed sites with multiple prefixes, none of which need to be globally disseminated. Making this multiple prefix architecture work was the charter of the multi6 group. This was constrained in interesting ways, as both NAT box solutions were considered politically unacceptable, as was changing the core functionality of the v6 stack (i.e., redefining the TCP pseudoheader). Given these constraints, it was somewhat unsurprising that NAT got pushed into the host.
From my perspective, we have now explored the dominant quadrants of the solution space and various factions have vociferously denounced all possible solutions. You'll pardon me if some of us are feeling just a tad frustrated.
I think we're all a bit frustrated at this point. However, I think we haven't adequately explored several ideas that allow PI space for all that need it yet don't require carrying all those routes in every DFZ router or schemes that do away with our current idea of the DFZ entirely. The solution space is a lot bigger than the few corners that we've explored over and over.
Every such proposal I've seen has been ignored or brushed aside by folks who've been doing CIDR for their entire careers and refuse to even consider that anything else might be better.
More accurately, the folks that have been CIDR advocates 'for their entire careers' are exactly the same folks who have been advocating shifting to something else, but have been rejected by other political elements when trying to propose actual architectural change. Further, those same CIDR advocates have been, and continue to be, in such political disfavor that they are effectively powerless anyhow. It hardly seems like their rejection could count for much.
CIDR is in political disfavor? Then explain how the IETF's golden solution for multihoming perfectly toes the CIDR line of not allowing end sites to have PI space, forcing them to use multiple CIDR-style PA prefixes in parallel. Or did you mean that the folks that came up with CIDR are now personally out of favor? That'd be a shame, since that was good work at the time and we need every bright mind we can get to make the next step past CIDR.
All this time, energy, and thought spent on shim6 would have been better spent on a scalable IDR solution.
On that, we can agree. However, my feeling is that fully exploring the solution space is an unfortunate necessity before the community is willing to accept changes to the fundamental v6 architecture.
IMHO, it's 5+ years too late to change _anything_ about how end hosts process IPv6 packets. If we need to do that, it's time to scrap v6 and start working on IPv9 (or whatever the next unassigned number is), with the understanding it won't be deployed for another 15+ years and we'll need to keep IPv4 working that long.
Luckily, we still have another decade or so to come up with something.
Unfortunately, that's not entirely true. If the RIR's begin wholesale PI assignment, then we start down the road of re-constituting the v4 routing architecture, locking in additional cost and complexity with each PI prefix. All such prefixes will be indefinitely grandfathered, so even if something new does come along, we will continue to pay for the overhead forever.
If the new solution is sufficiently attractive, then folks using PI-based multihoming will switch. If nobody wants to switch and/or there's still demand for PI space, that is a very strong sign that the proposed solution is not acceptable in the first place. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
Stephen,
Thus spake "Tony Li" <tony.li@tony.li>
Stephen Sprunk wrote:
Who exactly has been trying to find scalable routing solutions?
Well, for the last decade or so, there's been a small group of us who have been working towards a new routing architecture. Primary influences in my mind are Chiappa, O'Dell, Hain, Hinden, Nordmark, Atkinson, Fuller, Huston, Rekhter and Meyer. Apologies to any folks that feel I've incorrectly included or excluded them. ;-)
And my apologies for not recognizing the work that y'all have done; my point was that none of this seems to be officially supported by the IETF, and thus hasn't borne much fruit. I've seen a few proposals by folks listed above, and it seems to be old drafts (or even napkins) that get trotted out when this discussion comes up. If there's work actively going on today, it's not well publicized.
Speaking for myself, I can say that many of us are trying to raise the profile of exactly these issues in the IETF. As you might imagine, that's not exactly a "path of least resistance", given the history/baggage. My position, however, is that the past is something we can learn from, but not be controlled by (a perhaps metaphorical way of saying "lets get on with it"). I'm optimistic that we're getting traction, but as I said, history and inertia are pushing the other way (not that that's going to stop us, but it does slow things down at times).
IPv6 advocates have been pushing a no-PI model for over a decade because that's what ISPs told them to do.
More accurately, the routing community has been trying to avoid PI addressing simply because of the scalability (and thus cost) concerns.
s/routing/ISP/ and I'd agree with that. The IETF has virtually no enterprise representation, and those folks (a) have a lot more routers than the ISPs, and (b) pay the ISPs' bills.
Agreed, and representation is a big problem. I'm trying to work that too. The first thing we tried was the NANOG/APRICOT BOFs (and I think we're going to try it at RIPE too, waiting to hear from the PC on that one). So maybe these were not as successful as I would have hoped, but hey, you have to start somewhere (on the theory that the best way to finish anything is to start).
I agree that PI has scaling/cost problems, but so far all of the alternatives presented by the IETF present worse problems _in the eyes of the people that pay the bills_. That doesn't mean the latter are right, but their views should not be taken lightly.
When they found end users didn't like that, they went off and developed what has become shim6 as a poor apology. There has never been any significant work done on replacing CIDR with something that scales better.
More accurately, that work (GSE/8+8) was slapped down politically without due technical consideration.
Correction noted.
Note that replacing CIDR isn't exactly the point. The point is to have something that scales. Where CIDR can't cope is exactly when we come to multihoming. When multihoming was a rare exception, the small number of PI prefixes remains tolerable. However, over time, the continued growth in multihoming, even solely as a function of the growth of the Internet will come to dominate the cost structure of the routing subsystem.
I'm not sure I agree with that. The ISPs out there have tens of prefixes each even in v6 land (and hundreds in v4 land), whereas the goal is to have one per end site. Until the number of multihomed end sites exceeds the number of ISPs by several orders of magnitude, the impact on the routing table will be non-dominant though certainly also non-trivial.
Also, consider how easy it is to do PI-based multihoming in v4: all you need is a couple pipes (or tunnels), an ASN, and enough hosts to justify a /24. If you believed all the chicken littles, this would leave us with millions of v4 PI routes and the DFZ would be in ruins, yet only a few hundred people have taken ARIN up on that offer. In short, implementation of PI-based multihoming has ground almost to a halt even under today's liberal policies.
Now, given the floodgates are open for v4 and all we see is a trickle of water, why is everyone running around screaming that the sky will fall if we do something similar for v6? Do we have any evidence at all that multihoming growth will outpace Moore and this whole debate is even relevant?
The only alternative to a PI-like architecture is to provide multihomed sites with multiple prefixes, none of which need to be globally disseminated. Making this multiple prefix architecture work was the charter of the multi6 group. This was constrained in interesting ways, as both NAT box solutions were considered politically unacceptable, as was changing the core functionality of the v6 stack (i.e., redefining the TCP pseudoheader). Given these constraints, it was somewhat unsurprising that NAT got pushed into the host.
From my perspective, we have now explored the dominant quadrants of the solution space and various factions have vociferously denounced all possible solutions. You'll pardon me if some of us are feeling just a tad frustrated.
I think we're all a bit frustrated at this point.
Agreed, we all are. And BTW, I'm also on the inside, and I think you can see how that might be frustrating. But things can (and will) change. Again, at the very least I am optimistic.
However, I think we haven't adequately explored several ideas that allow PI space for all that need it yet don't require carrying all those routes in every DFZ router or schemes that do away with our current idea of the DFZ entirely. The solution space is a lot bigger than the few corners that we've explored over and over.
Every such proposal I've seen has been ignored or brushed aside by folks who've been doing CIDR for their entire careers and refuse to even consider that anything else might be better.
More accurately, the folks that have been CIDR advocates 'for their entire careers' are exactly the same folks who have been advocating shifting to something else, but have been rejected by other political elements when trying to propose actual architectural change. Further, those same CIDR advocates have been, and continue to be, in such political disfavor that they are effectively powerless anyhow. It hardly seems like their rejection could count for much.
CIDR is in political disfavor? Then explain how the IETF's golden solution for multihoming perfectly toes the CIDR line of not allowing end sites to have PI space, forcing them to use multiple CIDR-style PA prefixes in parallel.
It doesn't. The current thinking seems to be a combination of PI+SHIM6 until some other solution is found. Even if one agrees with this approach, one must first start trying to find such an "other solution", and second, understand how we're going to drain the (potentially massive) swamp created by PI allocations (if that is even going to be possible, given technical and nontechnical issues such as grandfathering, etc).
Or did you mean that the folks that came up with CIDR are now personally out of favor? That'd be a shame, since that was good work at the time and we need every bright mind we can get to make the next step past CIDR.
All this time, energy, and thought spent on shim6 would have been better spent on a scalable IDR solution.
On that, we can agree. However, my feeling is that fully exploring the solution space is an unfortunate necessity before the community is willing to accept changes to the fundamental v6 architecture.
IMHO, it's 5+ years too late to change _anything_ about how end hosts process IPv6 packets. If we need to do that, it's time to scrap v6 and start working on IPv9 (or whatever the next unassigned number is), with the understanding it won't be deployed for another 15+ years and we'll need to keep IPv4 working that long.
Continuing with the "you need to start somewhere" theme...
Luckily, we still have another decade or so to come up with something.
Unfortunately, that's not entirely true. If the RIR's begin wholesale PI assignment, then we start down the road of re-constituting the v4 routing architecture, locking in additional cost and complexity with each PI prefix. All such prefixes will be indefinitely grandfathered, so even if something new does come along, we will continue to pay for the overhead forever.
If the new solution is sufficiently attractive, then folks using PI-based multihoming will switch. If nobody wants to switch and/or there's still demand for PI space, that is a very strong sign that the proposed solution is not acceptable in the first place.
Exactly. Dave
--On March 5, 2006 3:28:05 PM -0500 Joe Abley <jabley@isc.org> wrote:
On 5-Mar-2006, at 14:16, Owen DeLong wrote:
It flies if you look at changing the routing paradigm instead of pushing routing decisions out of the routers and off to the hosts. Source Routing is a technology that most of the internet figured out is problematic years ago. Making source routing more complicated and calling it something else doesn't make it less of a bad idea.
Calling shim6 source-routing when it's not in order to give it an aura of evil is similarly unproductive :-)
Sorry, I guess we'll agree to disagree on this, but, I see very little difference between shim6 and LSR other than the mechanism of implementation (shim6 requires a bit more overhead).
I don't think it will be as expensive as you think to fix it. I think if we start working on a new routing paradigm today in order to support IDR based on AS PATH instead of Prefix, we would realistically see this in deployable workable code within 3-5 years.
I'm confused by statements such as these.
Was it not the lack of any scalable routing solution after many years of trying that led people to resort to endpoint mobility in end systems, à la shim6?
I haven't seen any concrete proposals presented around the idea of IDR based on something other than prefix. Everything I've seen leading up to shim6 was about ways to continue to use prefixes and, to me, shim6 is just another answer to the wrong question... "How can we help scale prefix based routing?". The right question still hasn't been asked by most people in my opinion... "What can we use for routing instead of prefixes that will scale better?" As much as I agree the internet is not the PSTN, this is one place where we have a lot to learn from SS7. No, SS7 is not perfect... Far from it, but, there are lessons to be learned that are applicable to the internet, and, separating the end system identifier from the routing function is one we still seem determined to avoid for reasons passing my understanding. Owen
Joe
-- If it wasn't crypto-signed, it probably didn't come from me.
On Mar 5, 2006, at 6:59 PM, Owen DeLong wrote:
Far from it, but, there are lessons to be learned that are applicable to the internet, and, separating the end system identifier from the routing function is one we still seem determined to avoid for reasons passing my understanding.
And this is the real answer, of course. There were two fundamental design decisions made back in the Olden Days which continue to exert a strong and in many cases quite negative sway over this entire set of inter-related issues: 1. Utilizing the endpoint identifier in the routing function, as Vince Fuller and you (among others) have stated, and 2. The ships-in-the-night nature of the TCP/IP protocol stack. This latter design decision is a big part of the reason TCP/IP has been so successful to date; however, we find more and more kludgey, brittle hacks to try and provide some sort of linkages for purposes of enforcing policy, etc. The irony is that these attempts largely stem from the unforeseen side-effects of #1, and also contribute to a reinforcing feedback loop which further locks us into #1. Given the manifold difficulties we're facing today as a result of these two design decisions (#2 is a 'hidden' reason behind untold amounts of capex and opex being spent in frustratingly nonproductive ways), perhaps it is time to consider declaring the 'Limited- Deployment IPv6 Proof-of-Concept Experiment' to be a success, take the lessons learned (there are a lot more unresolved and potentially problematic issues than those mentioned in this thread) into account and get started on IPv8. ---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice Everything has been said. But nobody listens. -- Roger Shattuck
[ It's been pointed out that, due to various historical reasons, IPv8 might not be the best choice of version-number to use in this context. So, IPv10 can serve for purposes of discussion, in its stead. ] On Mar 5, 2006, at 7:19 PM, Roland Dobbins wrote:
On Mar 5, 2006, at 6:59 PM, Owen DeLong wrote:
Far from it, but, there are lessons to be learned that are applicable to the internet, and, separating the end system identifier from the routing function is one we still seem determined to avoid for reasons passing my understanding.
And this is the real answer, of course.
There were two fundamental design decisions made back in the Olden Days which continue to exert a strong and in many cases quite negative sway over this entire set of inter-related issues:
1. Utilizing the endpoint identifier in the routing function, as Vince Fuller and you (among others) have stated, and
2. The ships-in-the-night nature of the TCP/IP protocol stack. This latter design decision is a big part of the reason TCP/IP has been so successful to date; however, we find more and more kludgey, brittle hacks to try and provide some sort of linkages for purposes of enforcing policy, etc. The irony is that these attempts largely stem from the unforeseen side-effects of #1, and also contribute to a reinforcing feedback loop which further locks us into #1.
Given the manifold difficulties we're facing today as a result of these two design decisions (#2 is a 'hidden' reason behind untold amounts of capex and opex being spent in frustratingly nonproductive ways), perhaps it is time to consider declaring the 'Limited-Deployment IPv6 Proof-of-Concept Experiment' to be a success, take the lessons learned (there are a lot more unresolved and potentially problematic issues than those mentioned in this thread) into account and get started on IPv8.
---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice
Everything has been said. But nobody listens.
-- Roger Shattuck
---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice Everything has been said. But nobody listens. -- Roger Shattuck
oh yeah... IPX - that works a treat. (who was it that said "Its Deja Vu all over again") --bill
[ It's been pointed out that, due to various historical reasons, IPv8 might not be the best choice of version-number to use in this context. So, IPv10 can serve for purposes of discussion, in its stead. ]
On Mar 5, 2006, at 7:19 PM, Roland Dobbins wrote:
On Mar 5, 2006, at 6:59 PM, Owen DeLong wrote:
Far from it, but, there are lessons to be learned that are applicable to the internet, and, separating the end system identifier from the routing function is one we still seem determined to avoid for reasons passing my understanding.
And this is the real answer, of course.
There were two fundamental design decisions made back in the Olden Days which continue to exert a strong and in many cases quite negative sway over this entire set of inter-related issues:
1. Utilizing the endpoint identifier in the routing function, as Vince Fuller and you (among others) have stated, and
2. The ships-in-the-night nature of the TCP/IP protocol stack. This latter design decision is a big part of the reason TCP/IP has been so successful to date; however, we find more and more kludgey, brittle hacks to try and provide some sort of linkages for purposes of enforcing policy, etc. The irony is that these attempts largely stem from the unforeseen side-effects of #1, and also contribute to a reinforcing feedback loop which further locks us into #1.
Given the manifold difficulties we're facing today as a result of these two design decisions (#2 is a 'hidden' reason behind untold amounts of capex and opex being spent in frustratingly nonproductive ways), perhaps it is time to consider declaring the 'Limited-Deployment IPv6 Proof-of-Concept Experiment' to be a success, take the lessons learned (there are a lot more unresolved and potentially problematic issues than those mentioned in this thread) into account and get started on IPv8.
---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice
Everything has been said. But nobody listens.
-- Roger Shattuck
---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice
Everything has been said. But nobody listens.
-- Roger Shattuck
I disagree with your understanding of the "limited deployment ...". There is much more commercial deployment and traffic that what you realize. Because some ISPs didn't deployed yet IPv6 doesn't meant is a failure. The deployment of any new protocol take time, and actually I will say that IPv6 has taken the right time to ensure a smooth transition. Precisely because that, most people is not noticing that some applications are already using IPv6, and we will see this much more in the next 12-18 months or so. So yes, is happening, and is a success. Regards, Jordi
De: Roland Dobbins <rdobbins@cisco.com> Responder a: <owner-nanog@merit.edu> Fecha: Sun, 5 Mar 2006 19:19:46 -0800 Para: <nanog@nanog.org> Asunto: Time for IPv8? (was Re: shim6 @ NANOG)
On Mar 5, 2006, at 6:59 PM, Owen DeLong wrote:
Far from it, but, there are lessons to be learned that are applicable to the internet, and, separating the end system identifier from the routing function is one we still seem determined to avoid for reasons passing my understanding.
And this is the real answer, of course.
There were two fundamental design decisions made back in the Olden Days which continue to exert a strong and in many cases quite negative sway over this entire set of inter-related issues:
1. Utilizing the endpoint identifier in the routing function, as Vince Fuller and you (among others) have stated, and
2. The ships-in-the-night nature of the TCP/IP protocol stack. This latter design decision is a big part of the reason TCP/IP has been so successful to date; however, we find more and more kludgey, brittle hacks to try and provide some sort of linkages for purposes of enforcing policy, etc. The irony is that these attempts largely stem from the unforeseen side-effects of #1, and also contribute to a reinforcing feedback loop which further locks us into #1.
Given the manifold difficulties we're facing today as a result of these two design decisions (#2 is a 'hidden' reason behind untold amounts of capex and opex being spent in frustratingly nonproductive ways), perhaps it is time to consider declaring the 'Limited- Deployment IPv6 Proof-of-Concept Experiment' to be a success, take the lessons learned (there are a lot more unresolved and potentially problematic issues than those mentioned in this thread) into account and get started on IPv8.
---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice
Everything has been said. But nobody listens.
-- Roger Shattuck
********************************************** The IPv6 Portal: http://www.ipv6tf.org Barcelona 2005 Global IPv6 Summit Slides available at: http://www.ipv6-es.com This electronic message contains information which may be privileged or confidential. The information is intended to be for the use of the individual(s) named above. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, including attached files, is prohibited.
On Sun, 5 Mar 2006, Roland Dobbins wrote:
Given the manifold difficulties we're facing today as a result of these two design decisions (#2 is a 'hidden' reason behind untold amounts of capex and opex being spent in frustratingly nonproductive ways), perhaps it is time to consider declaring the 'Limited-Deployment IPv6 Proof-of-Concept Experiment' to be a success, take the lessons learned (there are a lot more unresolved and potentially problematic issues than those mentioned in this thread) into account and get started on IPv8.
If you want to redesign IPv[4|6] into IPvNG that deals with all known issues - more power to you. But do you really think we can have fully working prototype and set of RFCs from WG by the time we ran out of IPv4 addresses? One of the lessons learned from IPv6 is actually that it takes long time to agree on changes to TCP/IP stack and even longer before those changes begin to appear on end-devices. Another lesson is that any new type of "address" also has associated allocation policy issues and to get consensus on that in entire world is also very difficult and takes just as much time. -- William Leibzon Elan Networks william@elan.net
On Sat, 4 Mar 2006 20:17:26 +0100, "Iljitsch van Beijnum" <iljitsch@muada.com> said:
On 4-mrt-2006, at 14:07, Kevin Day wrote:
[snip]
Unless we start now working on getting people moved to IPv6, the pain of running out of IPv4 before IPv6 has reached critical mass is going to be much much worse than a long term problem of IPv6 route size.
I disagree. You assume that IPv6 will be able to gain critical mass before IPv4 addresses run out. I don't think that will happen, because of the chicken/egg problem. "Running out" is a relative term. John Klensin says we've effictively already run out because IPv4 addresses are too hard to get for some applications. That may be true but people aren't turning to IPv6 (yet) to run those applications. My prediction is that we'll see interesting things happen when the remaining IPv4 address suppy < 3 * addresses used per year. That will probably happen around the end of this decade. At that point, there is likely to be hoarding and/or the allocation policies will become stricter, and people will start to think about a future where it's no longer possible to get IPv4 addresses. At this point, there will still be time to migrate.
Doesn't the above disagreement indicate that IPv6 is incomplete until a workable locator/id-split is implemented? If so, why bother with operational policies and deployment beyond what is of experimental nature necessary to facilitate further development? //per -- Per Heldal http://heldal.eml.cc/
I can tell you this: the only scalable solutions on the horizon are:
- moving multihoming related state out of the DFZ (this is what shim6 does)
This is what geo-topological addressing does.
- remove the requirement that every DFZ router carries every prefix, which can't be done as long as PI blocks sit at the top of the addressing hierarchy
Geotop addressing does this also because only a few aggregates are in the DFZ. The detail is elsewhere.
The closest thing to a magic, pain-free solution would be to allocate PI blocks such that it's possible to aggregate them together and ignore the more specifics for far away regions of the world, so that in 2030 you don't have to carry 60000 Chinese PI blocks world wide that all sit behind the same Great Firewall anyway,
Exactly! And this doesn't need to be done in a mandatory way. It can be done so that large providers can continue to use provider-aggregatable addresses. Geotop addressing is one of those 80-20 solutions where the largest 20% of providers mostly use classic IPv6 address but the other 80% of smaller multihomers use geotopologically aggregatable addresses. --Michael Dillon
Shim6 is an answer to "what kind of multihoming can we offer to sites without PI space?
as far as i can tell, s/sites/hosts/. to make it work for sites, as yet unspecified middleware (adding even more complexity and thus further reducing reliability (and margins)) will be needed if there is any concern for security and/or site management; and, aside from the home user, there always are. randy
Stephen Sprunk wrote:
Shim6 is an answer to "what kind of multihoming can we offer to sites without PI space?"; it is yet to be seen if anyone cares about the answer to that question. This argument is circular. The only real way to test demand is to offer a service and see if customers bite.
Eliot
Thus spake "Eliot Lear" <lear@cisco.com>
Stephen Sprunk wrote:
Shim6 is an answer to "what kind of multihoming can we offer to sites without PI space?"; it is yet to be seen if anyone cares about the answer to that question.
This argument is circular. The only real way to test demand is to offer a service and see if customers bite.
I'm not a fan of "build it and they will come" engineering. One might first talk to customers and see if your proposal is laughed at, at least. So far, that's the most charitable reaction I've seen to shim6. It's also not encouraging that many of the folks working on shim6 happen to have PI blocks themselves (despite not qualifying as LIRs); I'm also not a fan of "it's good enough for everyone else, but not good enough for me." S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
Stephen,
I'm not a fan of "build it and they will come" engineering. I suppose a reasonable question one could ask is this: who's the customer? Is the customer the ISP? I tend to actually it's the end enterprise. But that's just me.
Eliot
The scope of that policy mediation function depends strongly on people like you saying "at a high level, this is the kind of decision I am not happy with the hosts making".
Resounding YES - I specifically DON'T want end-hosts to be able to make these decisions, but need to be able to multihome.
When I see comments like this I wonder whether people understand what shim6 is all about. First of all, these aren't YOUR hosts. They belong to somebody else. If you are an access provider then these hosts belong to a customer that is paying you to carry packets. This customer also pays another ISP for the same service and the hosts are making decisions about whether to use your service or your competitors. If you are a hosting provider, then these hosts, owned by a third party, are making decisions about whether to send you packets through one or another AS. Is there something inherently wrong with independent organizations deciding where to send their packets? --Michael Dillon P.S. I don't believe that shim6 will ever succeed.
--- Michael.Dillon@btradianz.com wrote:
Resounding YES - I specifically DON'T want end-hosts to be able to make these decisions, but need to be able to multihome.
When I see comments like this I wonder whether people understand what shim6 is all about. First of all, these aren't YOUR hosts. They belong to somebody else. If you are an access provider then these hosts belong to a customer that is paying you to carry packets. This customer also pays another ISP for the same service and the hosts are making decisions about whether to use your service or your competitors.
If you are a hosting provider, then these hosts, owned by a third party, are making decisions about whether to send you packets through one or another AS.
Is there something inherently wrong with independent organizations deciding where to send their packets?
That's not the case I'm discussing - I'm talking about the multihomed enterprise. From an access provider point of view, Shim6 is no worse/better than the various TE-fu devices which end customers use - it makes predicting load a bit more difficult, but it's just bits to be passed. From an enterprise POV I want two or three decision points which I need to monitor and manage, not 10,000.
P.S. I don't believe that shim6 will ever succeed.
Neither do I. David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Date: Thu, 2 Mar 2006 10:07:33 +0000 From: Michael.Dillon@...
[ snip ]
Is there something inherently wrong with independent organizations deciding where to send their packets?
1. Many a transit seems to think so. 2. Is there an inherent need? 3. Is this DPA+sourceroute cocktail the best method? Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
On 3/2/06 7:57 AM, "Edward B. DREGER" <eddy+public+spam@noc.everquick.net> wrote:
Date: Thu, 2 Mar 2006 10:07:33 +0000 From: Michael.Dillon@...
[ snip ]
Is there something inherently wrong with independent organizations deciding where to send their packets?
1. Many a transit seems to think so. 2. Is there an inherent need? 3. Is this DPA+sourceroute cocktail the best method?
What Eddy said and also: The designers of shim6 seem to live in a different network security world than I do. Even assuming that shim6 ever gets deployed, which is pretty close to complete fantasy, the threat of a massive TE botnet being used to control large amounts of Internet traffic is a serious threat to Internet stability. Right now, DDoS attacks from Botnets are bad enough. Think about what happens when they have source routing control. Shim6 is a non-starter. A critical mass of host OS's will not get their software upgraded to support this in the next 5 years - there isn't running code ANYWHERE. Time to stop screwing around. There is a tremendous amount of effort being wasted here arguing against it and even more so in the IETF, where time being wasted on shim6 could be better spent on a new IDR paradigm. Where is the IETF leadership?
Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
-- Daniel Golding
At 11:27 AM 3/2/2006 -0500, Daniel Golding wrote:
There is a tremendous amount of effort being wasted here arguing against it and even more so in the IETF, where time being wasted on shim6 could be better spent on a new IDR paradigm.
Where is the IETF leadership?
I think the IRTF is the right place to discuss the new IDR paradigm. http://www.irtf.org/charter?gtype=rg&group=rrg "This research group is chartered to explore routing problems that are important to the development of the Internet but are not yet mature enough for engineering work within the IETF. The group will work closely with the IESG Routing Area Directors to ensure the free flow of information in both directions and avoid duplication of work with the various IETF working groups. The first steps taken in the RRG consisted of conducting a survey the state of the art in routing research, the needs of the user community (ISPs, router vendors, etc.) and the history of routing requirements. While this work will be ongoing as the state of research and of needs changes, the next step involves taking a number of the problem brought out in the analysis and focusing on these issues. .... The current set of sub-groups includes: Future Domain Routing Scalability" gus
Right now, DDoS attacks from Botnets are bad enough.
DDos!?!?!? What about the $1 billion dollars of clickbot fraud that the advertising industry is presently struggling with? Has anyone ever put a figure on the cost of DDos per year?
Where is the IETF leadership?
Not on the NANOG list... --Michael Dillon
On Mar 2, 2006, at 4:07 AM, Michael.Dillon@btradianz.com wrote:
ome.
When I see comments like this I wonder whether people understand what shim6 is all about. First of all, these aren't YOUR hosts. They belong to somebody else. If you are an access provider then these hosts belong to a customer that is paying you to carry packets. This customer also pays another ISP for the same service and the hosts are making decisions about whether to use your service or your competitors.
If you are a hosting provider, then these hosts, owned by a third party, are making decisions about whether to send you packets through one or another AS.
Is there something inherently wrong with independent organizations deciding where to send their packets?
The problem is when the *hosting company* or *ISP* is multihomed and using shim6. The customers aren't straddling two hosting companies, they're using a hosting company who is using shim6. Take us as a slightly exaggerated example(using totally made up bandwidth and prices, to protect NDAs). We have several boxes on our network that we do not control, we don't even have a login on the server. In one POP we have three transit providers. NSP A gives us 10Gbps of bandwidth, and charges us $50/mbps. NSP B is on a GigE, but we only have a 500mbps commit. B charges us $75/mbps, but $150/mbps if we go over our commit. NSP C is also on a GigE, but we only have a 100mbps commit, charges us $200/mbps, and $500/mbps if we go over our commit. I don't want a customer to touch NSP C, except for a very tiny number of routes where A and B aren't so great. I want to use NSP B as close to, but not going over, our commit as possible. I want everything else to go over NSP A. If any of the three transit connections go down, all the rules change temporarily (but hopefully not for long enough that we get dinged for 95th-percentile) Putting the routing decisions in the hands of the servers(that we do not control) requires that we somehow impart this routing policy on our customers, make them keep it up to date when we change things, and somehow enforce that they don't break the policy. If a customer sees that forcing traffic to go through NSP C results in a faster connection for him, they may tweak/break the selection process of shim6(or just ignore our policy instructions) and cost us lots of money. We may learn from one of our providers that they lost an OC48 in our city, and can't handle our full traffic so we need to back off immediately. Or we can know in advance that a connection is about to go down, and want to preemptively route around it before things get blackholed before the routers notice. On very high traffic days, we may make 10+ manual changes to our BGP policies to balance outbound and inbound traffic, to keep levels under their commits while still utilizing as much of our commit as possible. We have automated tools that make slight tweaks every 5 minutes. How can information that changes this frequently, and involves a very large dataset (several full tables of routes) get propagated to hundreds/thousands of hosts in a reasonable timeframe? Are we reinventing BGP as an IGP to send route data to shim6? :) And do we want to blow that much ram keeping a full routing table on each server? Even compressed to only list exceptions to a default route, my list of exceptions is still huge. The same problems exist, on a smaller level, on enterprise networks. Routing policies can be complex, requiring information that isn't currently visible to end hosts, that changes frequently, and can be very costly if anyone ignores the policy. Under current BGP-style decisions-at-the-edges networking, it's impossible for an end user or server to ignore routing policy. With shim6, the end nodes ARE the routing policy. There's a lot more to many network's decision making process of "how to select the best route" that can't be measured with RTT or received TTLs, or anything else the end nodes can see. Even outside the case of enterprise/hosting environments, transit providers already send route preference data to their customers. As a transit provider I'm able to depref/prepend/tag/etc routes to customers that we'd rather they not use (but are free to ignore). Under shim6, it's not really possible for your upstreams to tell you "My connection to this network is degraded at the moment, use it only as a last resort", where as with BGP they can prepend those routes a dozen times or flag it with a community and you won't use it unless you have to. Under host-based routing, all end nodes have to be made aware of this information. Something like shim6 works great for small or medium businesses where they don't care about this sort of thing, their routing policies only change when they add/drop a provider, and they don't have thousands of customers with root access on their boxes trying to game the system. I just don't think it's a solution for everyone.
Putting the routing decisions in the hands of the servers(that we do not control) requires that we somehow impart this routing policy on our customers, make them keep it up to date when we change things, and somehow enforce that they don't break the policy.
The same problems exist, on a smaller level, on enterprise networks. Routing policies can be complex, requiring information that isn't currently visible to end hosts, that changes frequently, and can be very costly if anyone ignores the policy. Under current BGP-style decisions-at-the-edges networking, it's impossible for an end user or server to ignore routing policy.
Clearly, it would be extremely unwise for an ISP or an enterprise to rely on shim6 for multihoming. Fortunately they won't have to do this because the BGP multihoming option will be available.
I just don't think it's a solution for everyone.
It may never be a solution for anyone. --Michael Dillon
On 2-mrt-2006, at 14:49, Michael.Dillon@btradianz.com wrote:
Clearly, it would be extremely unwise for an ISP or an enterprise to rely on shim6 for multihoming. Fortunately they won't have to do this because the BGP multihoming option will be available.
I guess you have a better crystal ball than I do. One thing is very certain: today, a lot of people who have their own PI or even PA block with IPv4, don't qualify for one with IPv6. While it's certainly possible that the rules will be changed such that more people can get an IPv6 PI or PA block, it is EXTREMELY unlikely that this will become as easy as with IPv4. Ergo: some people who multihome with BGP in IPv4 today won't be able to do the same with IPv6. And if you manage to get a PI or PA block you will very likely find that deaggregating won't work nearly as well with IPv6 as it does with IPv4. So learn to love shim6 or help create something better. Complaining isn't going to solve anything.
So learn to love shim6 or help create something better. Complaining isn't going to solve anything.
I am helping to create something better by supporting the IPv6 PI address policy on ppml@arin.net Go here http://www.arin.net/mailing_lists/index.html to subscribe or to read the archives. To read the proposed policy that is under discussion go here http://www.arin.net/policy/proposals/2006_4.html This policy will be voted on at one of the Public Policy days (April 10th and 11th) at the next ARIN meeting in Montreal. --Michael Dillon
Please consider also 2005-1 at http://www.arin.net/policy/proposals/2005_1.html Owen
--On March 2, 2006 3:15:59 PM +0100 Iljitsch van Beijnum <iljitsch@muada.com> wrote:
On 2-mrt-2006, at 14:49, Michael.Dillon@btradianz.com wrote:
Clearly, it would be extremely unwise for an ISP or an enterprise to rely on shim6 for multihoming. Fortunately they won't have to do this because the BGP multihoming option will be available.
I guess you have a better crystal ball than I do.
One thing is very certain: today, a lot of people who have their own PI or even PA block with IPv4, don't qualify for one with IPv6. While it's certainly possible that the rules will be changed such that more people can get an IPv6 PI or PA block, it is EXTREMELY unlikely that this will become as easy as with IPv4.
Possibly, but, if that is true, then, to that extent, it will delay or prevent the adoption of IPv6 by those people.
Ergo: some people who multihome with BGP in IPv4 today won't be able to do the same with IPv6. And if you manage to get a PI or PA block you will very likely find that deaggregating won't work nearly as well with IPv6 as it does with IPv4.
And why would those people consider migrating to IPv6?
So learn to love shim6 or help create something better. Complaining isn't going to solve anything.
I'm trying to create something better. I doubt many people in the operational community will ever learn to love shim6. Owen -- If it wasn't crypto-signed, it probably didn't come from me.
On 2-mrt-2006, at 22:27, Owen DeLong wrote:
One thing is very certain: today, a lot of people who have their own PI or even PA block with IPv4, don't qualify for one with IPv6. While it's certainly possible that the rules will be changed such that more people can get an IPv6 PI or PA block, it is EXTREMELY unlikely that this will become as easy as with IPv4.
Possibly, but, if that is true, then, to that extent, it will delay or prevent the adoption of IPv6 by those people.
I agree that this is a possibility. So I guess we'll have to choose between an IPv6 that's better than IPv4 but people don't want it, or an IPv6 that people want but it has the same inherent problems as IPv4. Hm...
Ergo: some people who multihome with BGP in IPv4 today won't be able to do the same with IPv6. And if you manage to get a PI or PA block you will very likely find that deaggregating won't work nearly as well with IPv6 as it does with IPv4.
And why would those people consider migrating to IPv6?
Because they can't get IPv4 addresses or so many other people use IPv6 (because _they_ can't get IPv4 addresses) that communicating with them natively is important. But today there are still enough IPv4 addresses (I just checked: we still have 1444.12 million addresses or 86.08 /8s) so that won't happen for a few more years.
So learn to love shim6 or help create something better. Complaining isn't going to solve anything.
I'm trying to create something better. I doubt many people in the operational community will ever learn to love shim6.
Stranger things have happened. Some people actually like NAT...
On Thu, Mar 02, 2006 at 10:44:01PM +0100, Iljitsch van Beijnum wrote:
And why would those people consider migrating to IPv6?
Because they can't get IPv4 addresses or so many other people use IPv6 (because _they_ can't get IPv4 addresses) that communicating with them natively is important. But today there are still enough IPv4 addresses (I just checked: we still have 1444.12 million addresses or 86.08 /8s) so that won't happen for a few more years.
You've probably seen Geoff Huston's comments about this; I tend to agree with him here. When IPv4 space is exhausted, the sky won't fall; We'll simply work out an address space management policy which is different from the one we have right now. Right now we can hand them out to anyone who demonstrates a need for them. When they run out we'll need to be able to reallocate address blocks which have already been handed out from orgs who perhaps don't need them as much as they thought they did to orgs which need them more. Sounds like a marketplace to me. How much do you think a /24 is worth? How many microseconds do you think it'll take for members of each RIR to debate the policy changes needed to alter their rules to permit trading of IPv4 resource allocations once IANA says, "No!" for the first time? That'll be interesting, because it'll place a cost on not migrating to IPv6: If an ISP wishes to grow its business it'll need to have sufficient resources to buy the address space it needs. We'll also have a reasonably good idea of what it'd cost to perform an IPv6 migration as we gather feedback from orgs who have actually done it. My guess is that we'll keep using IPv4 until the cost of growing businesses with the old address space exceeds the cost of migrating to the new one. One thing that Geoff hasn't been cynical enough to put forward is the idea that orgs with lots of valuable, monetized address space may very well end up lobbying the IAB and RIRs to erect new cost structures around green-fields IPv6 allocations as well, to make sure that the profit-providing marketplace survives for as long as possible by making the IPv6 migration process as expensive and inconvenient as possible. What will happen when the MCI's of the world discover that the race to $0 for IP transit prices has created a world in which they make more money by selling their IPv4 addresses than they make by selling Internet access? Will we see them coming out as a strong supporter of restrictive RIR policies and IPv6 technologies which don't work as a way of artificially boosting the price of IPv6? It's going to be a fun ride :-) - mark -- Mark Newton Email: newton@internode.com.au (W) Network Engineer Email: newton@atdot.dotat.org (H) Internode Systems Pty Ltd Desk: +61-8-82282999 "Network Man" - Anagram of "Mark Newton" Mobile: +61-416-202-223
On 3-mrt-2006, at 0:22, Mark Newton wrote:
You've probably seen Geoff Huston's comments about this; I tend to agree with him here.
Geoff tends to make lots of comments, it's hard to either agree or disagree with them all. :-)
When IPv4 space is exhausted, the sky won't fall; We'll simply work out an address space management policy which is different from the one we have right now.
Of course. But don't underestimate how address-hungry some people/ organizations/places are. For instance, Comcast got 12.5 million addresses last year.
Right now we can hand them out to anyone who demonstrates a need for them. When they run out we'll need to be able to reallocate address blocks which have already been handed out from orgs who perhaps don't need them as much as they thought they did to orgs which need them more.
Sounds like a marketplace to me. How much do you think a /24 is worth? How many microseconds do you think it'll take for members of each RIR to debate the policy changes needed to alter their rules to permit trading of IPv4 resource allocations once IANA says, "No!" for the first time?
This is what I wrote about this a couple of months ago: http:// ablog.apress.com/?p=835 An interesting aspect about address trading is that some organizations have huge amounts of address space which didn't cost them anything, or at least not significantly more than what smaller blocks of address space cost others. Having them pocket the proceeds strikes me as rather unfair, and also counter productive because it encourages hoarding. Maybe a system where ARIN and other RIRs buy back addresses for a price per bit prefix length rather than per address makes sense.
We'll also have a reasonably good idea of what it'd cost to perform an IPv6 migration as we gather feedback from orgs who have actually done it.
I don't think the cost is too relevant (and hard to calculate because a lot of it is training and other not easily quantified expenditures), what counts is what it buys you. I ran a web bug for a non-networking related page in Dutch for a while and some 0.16% of all requests were done over IPv6. (That's 1 in 666.) So even if it's free, deploying IPv6 today isn't all that useful. But when you're the last one running IPv4, you'll really want to move over to IPv6, even if it's very expensive.
Thus spake "Iljitsch van Beijnum" <iljitsch@muada.com>
On 3-mrt-2006, at 0:22, Mark Newton wrote:
Right now we can hand them out to anyone who demonstrates a need for them. When they run out we'll need to be able to reallocate address blocks which have already been handed out from orgs who perhaps don't need them as much as they thought they did to orgs which need them more.
Sounds like a marketplace to me. How much do you think a /24 is worth? How many microseconds do you think it'll take for members of each RIR to debate the policy changes needed to alter their rules to permit trading of IPv4 resource allocations once IANA says, "No!" for the first time?
This is what I wrote about this a couple of months ago: http:// ablog.apress.com/?p=835
An interesting aspect about address trading is that some organizations have huge amounts of address space which didn't cost them anything, or at least not significantly more than what smaller blocks of address space cost others. Having them pocket the proceeds strikes me as rather unfair, and also counter productive because it encourages hoarding. Maybe a system where ARIN and other RIRs buy back addresses for a price per bit prefix length rather than per address makes sense.
Keep in mind that current RIR allocations/assignments are effectively leases (though the RIRs deny that fact) and, like any landlord, they can refuse to renew a lease or increase the rent at any point. There might be some interesting political battles when it comes to legacy allocations which are currently rent-free, but those tenants will find themselves woefully outnumbered when that day comes.
We'll also have a reasonably good idea of what it'd cost to perform an IPv6 migration as we gather feedback from orgs who have actually done it.
I don't think the cost is too relevant (and hard to calculate because a lot of it is training and other not easily quantified expenditures), what counts is what it buys you. I ran a web bug for a non-networking related page in Dutch for a while and some 0.16% of all requests were done over IPv6. (That's 1 in 666.) So even if it's free, deploying IPv6 today isn't all that useful. But when you're the last one running IPv4, you'll really want to move over to IPv6, even if it's very expensive.
Ah, but why? As long as IPv4 has similar or better performance characteristics to IPv6, why would anyone _need_ to migrate? Add to that the near certainty that vendors will create NAT devices that will allow an entire v4 enterprise to reach the v6 Internet... S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
On 3/3/06 11:04 AM, "Stephen Sprunk" <stephen@sprunk.org> wrote:
Keep in mind that current RIR allocations/assignments are effectively leases (though the RIRs deny that fact) and, like any landlord, they can refuse to renew a lease or increase the rent at any point.
There might be some interesting political battles when it comes to legacy allocations which are currently rent-free, but those tenants will find themselves woefully outnumbered when that day comes.
Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
Leases are actually a bad thing, from an address exhaustion point of view. Its like a country where the government owns all the land, but people have been farming it for generations. They can't sell it. If an address trading scheme evolves, address block holders will need clear title granted them by the RIRs. That would make an IP address market, moderated through the RIRs as clearing houses, tenable. Sadly, many of the folks who are involved with ARIN are sadly short sighted in this regard. They dismiss both the idea of an address market upon v4 exhaustion and the idea of clear title to address blocks. While I can't state unequivocally that this is the answer, it does seem to merit further study. -- Daniel Golding
Sadly, many of the folks who are involved with ARIN are sadly short sighted in this regard. They dismiss both the idea of an address market upon v4 exhaustion and the idea of clear title to address blocks.
I can imagine a similar scenario in the boardrooms of Exxon et al. A young executive suggests that gasoline prices should be raised to $20 per gallon because reserves are dropping. The seasoned executives glance nervously at the unknown Russian oil reserves and the huge Canadian oilsands reserves and wonder what would happen to that plan if huge new supplies suddenly entered the market. Let's face it, IPv6 is close enough to IPv4 that any attempt to put a price on IPv4 addresses will simply cause a massive migration to free and plentiful IPv6 addresses. --Michael Dillon
On Mon, 6 Mar 2006 Michael.Dillon@btradianz.com wrote:
Let's face it, IPv6 is close enough to IPv4 that any attempt to put a price on IPv4 addresses will simply cause a massive migration to free and plentiful IPv6 addresses.
Let's say we put a price of $1 per year per IP address you want allocated to you. For the people really using their IP addresses according to current policy, this is nothing. For the people with historic allocations (/8 for instance), they would really have to think if it's worth $16m to keep that /8. Most likely scenario is that this would free up a lot of IPv4 address space. If this causes migration to IPv6, well, so be it. I seriously doubt it, only thing I think would happen is that we would free up IPv4 space and it would live longer. -- Mikael Abrahamsson email: swmike@swm.pp.se
On Mon, 6 Mar 2006, Mikael Abrahamsson wrote:
Let's say we put a price of $1 per year per IP address you want allocated to you. For the people really using their IP addresses according to current policy, this is nothing. For the people with historic allocations (/8 for instance), they would really have to think if it's worth $16m to keep that /8. Most likely scenario is that this would free up a lot of IPv4 address space.
If this causes migration to IPv6, well, so be it. I seriously doubt it, only thing I think would happen is that we would free up IPv4 space and it would live longer.
I'd love to see something like that (even better: charging by what you advertise).. but unfortunately, I don't think it'd happen, and if it did, I guess the main folks benefiting would be lawyers.. :) -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
On Mar 6, 2006, at 4:32 AM, Michael.Dillon@btradianz.com wrote:
Sadly, many of the folks who are involved with ARIN are sadly short sighted in this regard. They dismiss both the idea of an address market upon v4 exhaustion and the idea of clear title to address blocks.
I can imagine a similar scenario in the boardrooms of Exxon et al. A young executive suggests that gasoline prices should be raised to $20 per gallon because reserves are dropping. The seasoned executives glance nervously at the unknown Russian oil reserves and the huge Canadian oilsands reserves and wonder what would happen to that plan if huge new supplies suddenly entered the market.
Let's face it, IPv6 is close enough to IPv4 that any attempt to put a price on IPv4 addresses will simply cause a massive migration to free and plentiful IPv6 addresses.
Or a freeing up of hoarded or unused IPv4 addresses. That's one thing spot markets are pretty efficient at doing in times of scarcity. Regards Marshall
--Michael Dillon
Thus spake <Michael.Dillon@btradianz.com>
Let's face it, IPv6 is close enough to IPv4 that any attempt to put a price on IPv4 addresses will simply cause a massive migration to free and plentiful IPv6 addresses.
You assume that there will be a source of free and plentiful IPv6 addresses. AFAIK, none of them are rent-free, and they're not even available unless you have the clue and resources to prented to be an LIR. So, unless there's policy change, most end-user orgs will have no choice but to pay the market rate for IPv4 addresses. Spot markets are good when demand is elastic, but we're faced with a market that has growing inelastic demand that will outstrip fixed supply in a decade. Capitalism doesn't handle that well. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
On 3/6/06 10:25 AM, "Stephen Sprunk" <stephen@sprunk.org> wrote:
Thus spake <Michael.Dillon@btradianz.com>
Let's face it, IPv6 is close enough to IPv4 that any attempt to put a price on IPv4 addresses will simply cause a massive migration to free and plentiful IPv6 addresses.
You assume that there will be a source of free and plentiful IPv6 addresses. AFAIK, none of them are rent-free, and they're not even available unless you have the clue and resources to prented to be an LIR.
So, unless there's policy change, most end-user orgs will have no choice but to pay the market rate for IPv4 addresses. Spot markets are good when demand is elastic, but we're faced with a market that has growing inelastic demand that will outstrip fixed supply in a decade. Capitalism doesn't handle that well.
S
Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
Stephen, There will be a average cost per host to transition from v4 to v6. When the cost of IPv4 addresses exceeds the transition cost, then you have the one thing missing from IPv6 discussions: an ROI. Many organizations wont even look at this without an ROI. Folks who want to see v6 adopted would be well advised to support the creation of a hard ROI through these means. (Just as an aside - capitalism has historically been the best mechanism for resource allocation - scarce resource or plentiful. Command economics have never historically beaten a fair and open market, despite the beliefs of a certain sector of those involved with IP address allocation. ARIN (and/or RIPE, APNIC) should really use a bit of their budget surplus to provide a few grants to economics professors who are experts in commodity market issues. As engineers, we grope in the dark concerning fairly well established scientific principles we are unfamiliar with. Its like reinventing the wheel. :( -- Daniel Golding
Thus spake "Daniel Golding" <dgolding@burtongroup.com>
On 3/6/06 10:25 AM, "Stephen Sprunk" <stephen@sprunk.org> wrote:
So, unless there's policy change, most end-user orgs will have no choice but to pay the market rate for IPv4 addresses. Spot markets are good when demand is elastic, but we're faced with a market that has growing inelastic demand that will outstrip fixed supply in a decade. Capitalism doesn't handle that well.
There will be a average cost per host to transition from v4 to v6. When the cost of IPv4 addresses exceeds the transition cost, then you have the one thing missing from IPv6 discussions: an ROI.
Please quantify the cost of not being able to multihome your mission-critical business. Compare to the cost of obtaining an IPv4 PI block. Both are likely to exceed the possible revenue for small businesses at some point not too far off. IPv6 is not a replacement for IPv4 today; it's less attractive for a number of reasons, and running out of IPv4 addresses will only solve one (maybe two) of the problems.
Many organizations wont even look at this without an ROI. Folks who want to see v6 adopted would be well advised to support the creation of a hard ROI through these means.
That'd be interesting to see, but there's just too many variables we don't (and can't) have numbers for yet. Maybe it'd be a useful exercise to at least identify what needs to be quantified...
ARIN (and/or RIPE, APNIC) should really use a bit of their budget surplus to provide a few grants to economics professors who are experts in commodity market issues. As engineers, we grope in the dark concerning fairly well established scientific principles we are unfamiliar with. Its like reinventing the wheel. :(
That would require the RIRs to admit that IP addresses are marketable commodities, which is something that, to date, they have refused to do. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
On 3/6/06 6:14 PM, "Stephen Sprunk" <stephen@sprunk.org> wrote:
Thus spake "Daniel Golding" <dgolding@burtongroup.com>
On 3/6/06 10:25 AM, "Stephen Sprunk" <stephen@sprunk.org> wrote:
So, unless there's policy change, most end-user orgs will have no choice but to pay the market rate for IPv4 addresses. Spot markets are good when demand is elastic, but we're faced with a market that has growing inelastic demand that will outstrip fixed supply in a decade. Capitalism doesn't handle that well.
There will be a average cost per host to transition from v4 to v6. When the cost of IPv4 addresses exceeds the transition cost, then you have the one thing missing from IPv6 discussions: an ROI.
Please quantify the cost of not being able to multihome your mission-critical business. Compare to the cost of obtaining an IPv4 PI block. Both are likely to exceed the possible revenue for small businesses at some point not too far off.
This is of course, making the assumption (which may be in error) at PI IPv6 space will happen, through 2005-1 or some other policy. Absent PI IPv6 space, IPv6 simply wont happen, shim6 or not. -- Daniel Golding
On Mon, 06 Mar 2006 17:05:52 -0500 Daniel Golding <dgolding@burtongroup.com> wrote:
ARIN (and/or RIPE, APNIC) should really use a bit of their budget surplus to provide a few grants to economics professors who are experts in commodity market issues. As engineers, we grope in the dark concerning fairly well established scientific principles we are unfamiliar with. Its like reinventing the wheel. :(
For a view by some computer scientists, see: Yakov Rekhter, Paul Resnick, and Steven M. Bellovin, "Financial Incentives for Route Aggregation and Efficient Address Utilization in the Internet," in Proceedings of Telecommunications Policy Research Conference, Solomons, MD..Also in Brian Kahin and James H. Keller, eds., Coordinating the Internet, MIT Press, 1997 http://www.cs.columbia.edu/~smb/papers/piara/index.html We even held a BoF at an IETF, long ago. The enthusiasm was, shall we say, underwhelming.
Not to digress too far, but, I guess that depends on your definition of best. I am sure that many peoples of this world would argue that capitalism has been rather catastrophic in terms of resource allocation and resulting effects with regard to oil, for example. Owen
Thus spake <Michael.Dillon@btradianz.com>
Let's face it, IPv6 is close enough to IPv4 that any attempt to put a price on IPv4 addresses will simply cause a massive migration to free and plentiful IPv6 addresses.
You assume that there will be a source of free and plentiful IPv6
addresses. Why not? It is CORE idea of IPcv6 addres space (no matter whart is written abouyt allocation policies). And I can bet, we will see how every business can get PI space when required (in ipv6) for multi home, in less than 3 - 4 years. Except if IPv6 will be replaced by something simpler and more reliable like IPv8 (but it looks very unlikely now).
On 3-mrt-2006, at 17:04, Stephen Sprunk wrote:
Keep in mind that current RIR allocations/assignments are effectively leases (though the RIRs deny that fact) and, like any landlord, they can refuse to renew a lease or increase the rent at any point.
I can only imagine the fun the lawyers are going to have with this: 1. Get address space from Internic, no questions asked 2. ARIN is formed and starts making policies that say address space isn't owned 3. ARIN never enforces these no ownership policies (that I know of) 4. ARIN tries to take away the addresses That's the best advertisement IPv6 could ever hope for: "no lawyers!"
So even if it's free, deploying IPv6 today isn't all that useful. But when you're the last one running IPv4, you'll really want to move over to IPv6, even if it's very expensive.
Ah, but why? As long as IPv4 has similar or better performance characteristics to IPv6, why would anyone _need_ to migrate? Add to that the near certainty that vendors will create NAT devices that will allow an entire v4 enterprise to reach the v6 Internet...
Don't they teach you IPv6 network design in CCIE school? Once you've worked with link local addressing/routing and generating addresses from EUI-64s you never want to go back to the tedious address and subnet management that's necessary in IPv4. So building boxes just so you can stick to IPv4 when the rest of the world is already on IPv6 seems a bit backward to me. Since you can't express the IPv6 address space in the IPv4 address space (the reverse is easy and available today), the translation needs to happen a bit higher in the stack. When I was testing running IPv6-only I installed an Apache 2 proxy in order to reach the IPv4 web from my IPv6-only system. But it worked the other way around too, of course: using the proxy, I could visit sites over IPv6 with IPv4- only systems.
Thus spake "Iljitsch van Beijnum" <iljitsch@muada.com>
On 3-mrt-2006, at 17:04, Stephen Sprunk wrote:
Keep in mind that current RIR allocations/assignments are effectively leases (though the RIRs deny that fact) and, like any landlord, they can refuse to renew a lease or increase the rent at any point.
I can only imagine the fun the lawyers are going to have with this:
1. Get address space from Internic, no questions asked 2. ARIN is formed and starts making policies that say address space isn't owned 3. ARIN never enforces these no ownership policies (that I know of) 4. ARIN tries to take away the addresses
That's the best advertisement IPv6 could ever hope for: "no lawyers!"
Thanks for silently snipping the paragraph that partially answered that. There may be some legal battles over it, but since the orgs have no records of ever purchasing those legacy addresses, it's hard to claim true ownership -- not that one could easily establish owning a number even with a bill of sale. My guess is we'll continue to grandfather them forever, but RIR policy will change to requiring orgs to start paying rent on them in order to receive any new assignments (either v4 or v6). Wait a few years, and we can reclaim most of the space without the lawyers being able to interfere. v6 does have an advantage (to the RIRs) of not having legacy issues, but that's a disadvantage for the orgs getting space. Consider that the vast majority of orgs with multiple legacy swamp allocations haven't traded them in for a rent-free CIDR one; part of that is inertia, but part is the risk that doing so will more likely expose them to rent in the future.
So even if it's free, deploying IPv6 today isn't all that useful. But when you're the last one running IPv4, you'll really want to move over to IPv6, even if it's very expensive.
Ah, but why? As long as IPv4 has similar or better performance characteristics to IPv6, why would anyone _need_ to migrate? Add to that the near certainty that vendors will create NAT devices that will allow an entire v4 enterprise to reach the v6 Internet...
Don't they teach you IPv6 network design in CCIE school?
There weren't CCIE schools back when I got mine, but my understanding is that the ones today still don't teach anything (or at least anything useful) about IPv6.
Once you've worked with link local addressing/routing and generating addresses from EUI-64s you never want to go back to the tedious address and subnet management that's necessary in IPv4.
When you're using RFC1918 space, as nearly all leaf orgs do today, subnet assignment isn't tedious: just give every VLAN a /24 or so and be done with it; similar to assigning /64s. Maintaining DHCP servers sucks, but it's an accepted cost that doesn't amount to much in the budget since they're already paid for (or free with your routers). I agree that IPv6 is better from this perspective, but unless one is building out a greenfield network, the transition cost is higher than the cost of status quo. Just upgrading all those L3 switches to v6-capable models will cost large enterprises tens of millions of dollars (and don't say regular upgrade cycles will fix that, as obsolete equipment just moves out of the core to other places).
So building boxes just so you can stick to IPv4 when the rest of the world is already on IPv6 seems a bit backward to me.
It's not a matter of building boxes: all that needs to happen is for Cisco to release an upgrade for PIX (ditto for other vendors) that is free with a maintenance contract, and every enterprise will be doing it overnight. What's to stop the vendors from doing it? All it takes is one big (or several small) RFP(s) asking for the feature, and it'll be there.
Since you can't express the IPv6 address space in the IPv4 address space (the reverse is easy and available today), the translation needs to happen a bit higher in the stack.
Off-the-cuff solution: translate all incoming v6 addresses to temporary v4 addresses (172.16/12 will do nicely). You'll need to intercept DNS, but most NAT devices do that today anyways for other reasons.
When I was testing running IPv6-only I installed an Apache 2 proxy in order to reach the IPv4 web from my IPv6-only system. But it worked the other way around too, of course: using the proxy, I could visit sites over IPv6 with IPv4-only systems.
Which supports my point: why upgrade when you can proxy / translate / whatever for (almost) free? Especially when you're using 10/8 internally and thus will never directly feel any v4 exhaustion pain? S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
One thing that Geoff hasn't been cynical enough to put forward is the idea that orgs with lots of valuable, monetized address space may very well end up lobbying the IAB and RIRs to erect new cost structures around green-fields IPv6 allocations as well, to make sure that the profit-providing marketplace survives for as long as possible by making the IPv6 migration process as expensive and inconvenient as possible.
I'd suggest that there is no need to do so, as that particular reality is already in place. For customers its still just mail, its still just google, ebay, yahoo, etc. Why should they pay their ISP any more money to have the packets colored red, blue or green? Or use 32 bit address fields in their packets or 128bits? The barriers to IPv6 adoption are already in place in terms of a customer-unfunded transition to get effectively to a place thats not very much different from where they were when they started. It really doesn't make much sense does it?
What will happen when the MCI's of the world discover that the race to $0 for IP transit prices has created a world in which they make more money by selling their IPv4 addresses than they make by selling Internet access? Will we see them coming out as a strong supporter of restrictive RIR policies and IPv6 technologies which don't work as a way of artificially boosting the price of IPv6?
The real question for such holders is when to sell. In a market with rising demand and insufficient supply, the typical seller's strategy is to hang on to further exacerbate demand. At what point such behaviour results in collapse of the market as buyers find the substitution of IPv6 viable is the key question for the seller.
It's going to be a fun ride :-)
indeed. Geoff
On Mar 2, 2006, at 7:49 AM, Michael.Dillon@btradianz.com wrote:
Clearly, it would be extremely unwise for an ISP or an enterprise to rely on shim6 for multihoming. Fortunately they won't have to do this because the BGP multihoming option will be available.
Are you *sure* BGP multihoming will be available? This is my interpretation of the IPv6 /32 allocation policy: To receive an allocation of a /32, you must: "A) Be an LIR". I think you can consider a hosting company an LIR. "B) Not be an end site". A little less cut and dry, but I'll accept that a hosting company doesn't fit the definition of an end site. "C) plan to provide IPv6 connectivity to organizations to which it will assign /48s, by advertising that connectivity through its single aggregated address allocation". This is the one where I don't think a hosting company fits. If all of your hosting is "shared", the servers are your responsibility, and you're not providing connectivity to anyone but yourself. I don't think you qualify at all at this point. If you're selling dedicated servers or colo space, it's a little better, but I still don't think you fit. The average dedicated hosting/colo company now runs many customers servers sharing one subnet. Each customer gets /32's assigned per server, unless you're a huge colo customer, you're not getting space SWIPed to you. When deciding who gets space out of your /32:
Assignments are to be made in accordance with the existing guidelines (RFC3177,RIRs-on-48), which are summarized here as:
- /48 in the general case, except for very large subscribers
- /64 when it is known that one and only one subnet is needed by design
- /128 when it is absolutely known that one and only one device is connecting.
One customer on one dedicated server gets a /128. Even if you stretch plausibility, they only get a /64. I don't see any way you can justify giving colo customers /48s, unless they're deploying huge networks in your datacenter. The final rule for getting a /32 is: "D) be an existing, known ISP in the ARIN region or have a plan for making at least 200 /48 assignments to other organizations within five years." Unless you're providing transit/connectivity to 200 companies/ networks, I can't see how you justify assigning even ONE /48, let alone 200. The other PI assignment policies that have been proposed either require that you have a /19 already in IPv4 (lots of hosting companies don't have anything this size), or have tens/hundreds of thousands of devices. Even if a hosting company does get a /32 or a /44 or whatever, the "you can't deaggregate your assignment at all" policy rules out having multiple independent POPs unless you somehow arrange to get multiple allocations(which isn't possible now).
If all of your hosting is "shared", the servers are your responsibility, and you're not providing connectivity to anyone but yourself. I don't think you qualify at all at this point.
If you're selling dedicated servers or colo space, it's a little better, but I still don't think you fit. The average dedicated hosting/colo company now runs many customers servers sharing one subnet. Each customer gets /32's assigned per server, unless you're a huge colo customer, you're not getting space SWIPed to you.
If your current business model means that your business cannot continue in an IPv6 world, then a competent business manager will change that model. If the IPv6 address policy requires hosting providers to sell the blades in their servers, and to SWIP a /48 per customer end-site (i.e. blade) then they will do so. There is precedent for this when IPv4 addressing policy drove hosting providers to name-based virtual hosting. I'm not terribly worried about situations like this because I know the applicants can adjust their business plans, and future business models, to work with the policy. A while back on PPML I noted that RIPE had allocated IPv6 space to several organizations with no IPv4 allocations. Under the assumption that these were not existing IPv4 ISPs, I had a look into several of these organizations. One of them was the IT function of a retail chain operating in Germany, Poland and Turkey. If they could qualify as an LIR by organizing their business with a separate IT and network services function, then I think the rules are less restrictive than most people think. If you feel you should qualify as an LIR, then apply for your /32. If you get rejected, asked for detailed reasons why. If the reasons are due to misunderstanding or a lack of information, then remedy the situation and reapply. If the reasons have to do with your structure or your plans, then change them and reapply. If you can't do that then I would question whether you have a serious intention to be an IPv6 service provider.
- /128 when it is absolutely known that one and only one device is connecting.
One customer on one dedicated server gets a /128.
You know, the problem with those blade servers is that they generate too much heat and use too much electricity in one place. This increases HVAC costs and increase the risk of power problems. It also creates a massive downside during a power outage if you can't keep the HVAC running. Much better to find a cheaper real-estate location where you can rent rack positions rather than blades. Customers can put whatever they want in their rack positions. Maybe it's a WLAN proxy relay station or their own blade server. You can't know how many devices will be at the rack position therefore you should allocate them a /48. After all, you are a hotel. You rent rooms, you don't police what goes on inside the room.
The other PI assignment policies that have been proposed either require that you have a /19 already in IPv4 (lots of hosting companies don't have anything this size), or have tens/hundreds of thousands of devices.
It has also been suggested that the simple presence of multihoming should be sufficient justification for PI space.
Even if a hosting company does get a /32 or a /44 or whatever, the "you can't deaggregate your assignment at all" policy rules out having multiple independent POPs unless you somehow arrange to get multiple allocations(which isn't possible now).
People have done creative things with tunnels in the past. The widespread existence of MPLS backbones makes that even easier. You will always be able to find one situation that simply will not fit a given policy. Regardless, we still need to have some reasonable policy that creates a level playing field, does not unecessarily restrain trade, and creates possibilities that smart entrepreneurs can exploit to expand the network. --Michael Dillon
On 2-mrt-2006, at 17:05, Michael.Dillon@btradianz.com wrote:
If you feel you should qualify as an LIR,
With RIPE, an LIR is simply an organization that pays the membership fee and thus gets to submit requests for address space and AS numbers. ARIN doesn't seem to use this terminology except in their IPv6 address allocation policy. -- I've written another book! http://www.runningipv6.net/
If you feel you should qualify as an LIR,
With RIPE, an LIR is simply an organization that pays the membership fee and thus gets to submit requests for address space and AS numbers. ARIN doesn't seem to use this terminology except in their IPv6 address allocation policy.
That's what we were talking about. Shim6 and IPv6. The term LIR is used in IPv6 allocation policy in all regions and refers to an entity that assigns /48 blocks to its subscribers. Such an entity can receive a /32 from ARIN or RIPE or APNIC or LACNIC or AFRINIC. --Michael Dillon
On 3-mrt-2006, at 11:30, Michael.Dillon@btradianz.com wrote:
If you feel you should qualify as an LIR,
With RIPE, an LIR is simply an organization that pays the membership fee and thus gets to submit requests for address space and AS numbers. ARIN doesn't seem to use this terminology except in their IPv6 address allocation policy.
That's what we were talking about. Shim6 and IPv6. The term LIR is used in IPv6 allocation policy in all regions and refers to an entity that assigns /48 blocks to its subscribers. Such an entity can receive a /32 from ARIN or RIPE or APNIC or LACNIC or AFRINIC.
No, a LIR has the privilege of paying their RIR a fee. Wether they can get a /32 is another matter.
At 08:16 AM 3/4/2006 +0800, Randy Bush wrote:
On 3-mrt-2006, at 11:30, Michael.Dillon@btradianz.com wrote:
The term LIR is used in IPv6 allocation policy in all regions
no
Hmm...sure looks like it to me http://lacnic.net/en/politicas/ipv6.html 2.4 A Local Internet Registry (LIR) is an IR that primarily assigns address space to the users of the network services that it provides. LIRs are generally ISPs, whose customers are primarily end users and possibly other ISPs. http://www.arin.net/policy/nrpm.html 6.2.4 A Local Internet Registry (LIR) is an IR that primarily assigns address space to the users of the network services that it provides. LIRs are generally ISPs, whose customers are primarily end users and possibly other ISPs. http://www.afrinic.net/docs/policies/afpol-v6200407-000.htm 2.2. Local Internet Registry (LIR) A Local Internet Registry primarily assigns address space to the users of the network services that it provides. LIRs are generally ISPs whose customers are primarily End Users and possibly other ISPs. http://www.ripe.net/ripe/docs/ipv6policy.html 2.4. Local Internet Registry (LIR) A Local Internet Registry is an IR that primarily assigns address space to the users of the network services that it provides. LIRs are generally ISPs whose customers are primarily End Users and possibly other ISPs. http://www.apnic.net/docs/policy/ipv6-address-policy.html 2.4. Local Internet Registry (LIR) A Local Internet Registry (LIR) is an IR that primarily assigns address space to the users of the network services that it provides. LIRs are generally ISPs, whose customers are primarily end users and possibly other ISPs.
randy@psg.com wrote on 04/03/2006 00:16:28:
On 3-mrt-2006, at 11:30, Michael.Dillon@btradianz.com wrote:
The term LIR is used in IPv6 allocation policy in all regions
no
Yes. I checked all 5 RIR sites and they all use the term LIR in their IPv6 policy. This is by design since the original IPv6 policy was jointly written by all RIRs. http://www.afrinic.net/docs/policies/afpol-v6200407-000.htm http://lacnic.net/sp/politicas/ipv6.html http://www.apnic.net/docs/policy/ipv6-address-policy.html http://www.ripe.net/ripe/docs/ipv6policy.html http://www.arin.net/policy/nrpm.html#ipv6 --Michael Dillon
On Fri, Mar 03, 2006 at 10:30:44AM +0000, Michael.Dillon@btradianz.com wrote:
If you feel you should qualify as an LIR,
With RIPE, an LIR is simply an organization that pays the membership fee and thus gets to submit requests for address space and AS numbers. ARIN doesn't seem to use this terminology except in their IPv6 address allocation policy.
That's what we were talking about. Shim6 and IPv6. The term LIR is used in IPv6 allocation policy in all regions and refers to an entity that assigns /48 blocks to its subscribers. Such an entity can receive a /32 from ARIN or RIPE or APNIC or LACNIC or AFRINIC.
--Michael Dillon
hum... so what happens if i chose to delegated /56's to my customers? does that invalidate my LIR status 'cause i'm not toeing the /48 line? presuming of course i have LIR status along w/ the /32 that has ben delegated to me. to borrow a line, "some days your the IANA, some days your the endnode" --bill
The other PI assignment policies that have been proposed either require that you have a /19 already in IPv4 (lots of hosting companies don't have anything this size), or have tens/hundreds of thousands of devices.
It has also been suggested that the simple presence of multihoming should be sufficient justification for PI space.
Current PI policy in the ARIN region is /22 for IPv4.
Even if a hosting company does get a /32 or a /44 or whatever, the "you can't deaggregate your assignment at all" policy rules out having multiple independent POPs unless you somehow arrange to get multiple allocations(which isn't possible now).
People have done creative things with tunnels in the past. The widespread existence of MPLS backbones makes that even easier. You will always be able to find one situation that simply will not fit a given policy. Regardless, we still need to have some reasonable policy that creates a level playing field, does not unecessarily restrain trade, and creates possibilities that smart entrepreneurs can exploit to expand the network.
Another option is to create separate ORGs for each colo and get an allocation for each ORG. Owen -- If it wasn't crypto-signed, it probably didn't come from me.
* Michael.Dillon@btradianz.com (Michael.Dillon@btradianz.com) [Thu 02 Mar 2006, 17:03 CET]:
If your current business model means that your business cannot continue in an IPv6 world, then a competent business manager will change that model. If the IPv6
I assume that you mean that the IPv6 model will be changed, no? Because I hope that it has become clear from this thread that the adoption of IPv6 by a significant amount of players currently reachable over IPv4 is far from certain, and unless IPv6 offers at least feature compatibility it may not even happen. [..]
If you feel you should qualify as an LIR, then apply for your /32. If you get rejected, asked for detailed reasons why. If the reasons are due to misunderstanding or a lack of information, then remedy the situation and reapply. If the reasons have to do with your structure or your plans, then change them and reapply. If you can't do that then I would question whether you have a serious intention to be an IPv6 service provider.
Why should a certain business model be forced upon you? To multihome right now you need to cough up some money for equipment and some clue for configuration. Why would IPv6 require you to change your business model to achieve the same? [unattributed wrote:]
One customer on one dedicated server gets a /128.
That is, of course, insane. IPv6 addresses are far from rare. Why do you a priori rule out applications like https and virtualisation? -- Niels.
On Mar 1, 2006, at 9:07 AM, Joe Abley wrote:
On 1-Mar-2006, at 02:56, Kevin Day wrote:
If you include "Web hosting company" in your definition of ISP, that's not true.
Right. I wasn't; I listed them separately.
It's important to note that even if you are a hosting company who *does* qualify for PI v6 space, you still need shim6-capable servers, if you want to make them optimally available to multi- homed, shim6-capable hosts. The difference PI makes is in the distribution of addresses to servers (the servers only need a single set).
Which isn't a point to be glossed over. PA v.s. PI is a big deal to people for many reasons. (Portability between providers being the biggest.)
I'm just one guy, one ASN, and one content/hosting network. But I can tell you that to switch to using shim6 instead of BGP speaking would be a complete overhaul of how we do things.
You are not alone in fearing change.
It's not so much fearing change, as the bar of effort/difficulty in transitioning from 4 to 6 being really high. If IPv6 is made as painless as possible to people to transition to it NOW, people will. If you can tell every ISP, NSP, hosting company and end site that they can continue doing what they're doing now in 4, but with vastly more address space, you'd have a lot of convertors. Every thing that you make people do differently (even if it is for a greater good, and even if it benefits them directly) is one more reason people will give NOT to do it until they have to. Telling a hosting company "Here, you can get a /32 or /44 of your own which is virtually unlimited space for your needs, continue using BGP and TE as you do now, just deploy IPv6 on top of IPv4 and you're live" is an easy sell. Telling a hosting company "You have to lose the independence of PI space. You need to completely start over with your traffic engineering and do it in a way totally orthogonal to how you have to continue doing it in IPv4 space(adding workload instead of replacing what you're doing in IPv4). You now need to trust your customers/end users to do the right thing with routing. Routing now involves routers, servers and DNS - instead of a handful of devices making routing decisions now ALL of your devices need to make routing decisions. Even if you do get PI space somehow, you can't deaggregate it even if you truly run multiple isolated networks.(Don't say 'Get additional allocations then!' because that still results in the same number of routes added to the global table, while wasting even more space)" is a really hard sell. The only carrot you can offer someone is "You can have lots more space", which I personally don't think is worth even half of those negatives. If significant percentages of networks are going to heavily push back/ delay deployment of IPv6, IPv4 will be exhausted before a critical mass has switched to IPv6, making the whole "how do we protect the long term future of IPv6" rational for these policies a little less important. It reminds me of a story from engineer who worked for AT&T/Bell when touch tone dialing was first being tested in a few markets. AT&T wanted to offer touch tone dialing as a convenience for users, but they also had a desire to standardize the dialing procedure everywhere. In some markets you could make local calls by just dialing the last 4 or 5 digits of their number, others required the full 7. Some required a 1+XXX-XXXX to make calls outside their city, some only required the 1 if it was a toll call. AT&T really wanted to change their network where everyone in every city used the same procedure to make outbound calls. They decided that they'd make the new dialing plan mandatory with touch tone service. The carrot was "Look how much easier it is to dial compared to a rotary phone!", and they got the benefits of forcing a standardized system on everyone everywhere at the same time. The customer gets something that makes their life easier, and the operator of the network gets to make their job easier by standardization and reduced overhead of supporting hundreds of incompatible dialing plans in each exchange that people were used to. They tested it in a few cities, with a few customers(business and residential). A large number of people perceived touch tone dialing negatively, so much so that they asked for their rotary phones back. It had nothing to do with the push-button interface, it was asking people to take a perceived negative along with a positive they weren't sure they needed in the first place. Asking people to make too many changes at once outweighed the convenience of faster dialing. Users found it too confusing to have the new system (touch tone dialing, a.k.a. IPv6) work so much differently than what they were used to (rotary dialing a.k.a. IPv4) that they couldn't see past the change into what the advantage was. In the end, they gave the customers touch tone dialing, then gradually deployed the new dialing plan, using permissive grace periods where both dialing plans worked for as long as possible. I'm worried the same will happen with IPv4/IPv6. The temptation of virtually unlimited addressing is really nice. But I think the negatives of the allocation policy and the direction of how multihoming is going will scare off the willing participants, and we'll be stuck with only getting people to switch when they're forced to due to IPv4 exhaustion. My advice: Get people to switch now. If you have IPv4 PI space, you can get IPv6 PI space. If you have a / 21 now, you get a /48. If you have a /20 now, you get a /47. If you have a /19 now you get a /46. Etc. If you have anything bigger than a /48, you can rely on people accepting deaggregated prefixes of /48 or shorter. Push shim6 for people who don't need fancy traffic engineering, as a tool for small/medium business. Heck, even residential if you want to go that far. Get a critical mass of people using IPv6 as soon as possible. Once it's there, once people are comfortable with it, and once IPv6-acting- as-much-like-IPv4-as-possible has proven itself, let people WILLINGLY deploy shim6 if it truly would be advantageous to them. I'd have no problem with raising the initial/yearly fee per ASN if it made everyone comfortable that they were only being used where it was truly needed. On top of all of that, I'm still not convinced that IPv6-acting-as- much-like-IPv4-as-possible isn't going to have significantly less routes than IPv4, even if everyone moved today. Aren't a large number of the routes being advertised due to people having to go back and ask for more/bigger allocations again and again? If everyone out there right now had to announce only 1 route per POP, and 1 route per multihomed customer with PI space, wouldn't your outbound routes shrink pretty significantly? That would be a huge step in the right direction, and would buy loads of time to allow for a much more gradual transition.
Putting routing decisions in the control of servers we don't operate scares me. I wouldn't rely on 90% of our customers to get this right unless it was completely idiot proof. Even if it was, I don't see how we can trust that users aren't messing with things to "game the system" somehow.
This is the kind of feedback that the shim6 architects need. There is talk at present of whether the protocol needs to be able to accommodate a site-policy middlebox function to enforce site policy in the event that host behaviour needs to be controlled. The scope of that policy mediation function depends strongly on people like you saying "at a high level, this is the kind of decision I am not happy with the hosts making".
While I'm happy to give that feedback anywhere it's needed/welcome, I'm kind of surprised alarm bells didn't go off already about this. :)
We deal with long lived TCP sessions (hours/days). I don't see how routing updates can happen that won't result in a disconnect/ reconnect, which isn't acceptable.
One of the primary objectives of shim6 is to provide session survivability over re-homing events. Since routing protocols are not used to manage re-homing, the speed at which a session can recover from a topological event depends on the operation of the shim6 protocol between client and server.
I'd... be really curious to see how that works, without having to add intelligence to the application layer and stateful firewall layer to handle this. I don't mean to take an "I'll believe it when I see it" stance, but I think a lot of layers(that may exist on other hosts) would have to be changed to support doing that.
We have peering arrangements with about 120 ASNs. How do we mix BGP IPv6 peering and Shim6 for transit?
You advertise all your PA netblocks to all your peers.
Ok, I was a bit too vague there... How do we ensure that peering connections are always used instead of transit connections? Currently for outbound, we can localpref peer-learned routes over everything else. How do all of our servers on our end know that routes learned from a BGP session on our own routers are desirable? For inbound, we can either rely on our peers to do the same, prepend what we send out to transit to make peer routes look even better to our peers, or if we want to force the issue we can send peers more specific routes than we send to transit (operating on the principle of "most specific wins", no matter what else is done). I'm not seeing how BGP routing information can enter into Shim6's decision making, in any scalable fashion, and again.. something that updates near instantly.
So far it looks like Shim6 is going to rely on DNS. The DNS caching issue is a real problem. We need changes to happen faster than DNS caching will allow.
Well, not quite.
If you change a transit provider, then you need to remove a set of AAAA records from the servers you operate, and substitute a new set. The time taken for this change to propagate in the DNS is non- zero, assuming you use reasonable TTLs. This is your point above, I think.
Reasonable TTLs on our end or not, lots and lots of people don't respect TTLs. Seriously, try this sometime. Set the TTL for a very busy site to 5 minutes. Wait 2 weeks, to make SURE everyone is seeing the new TTL. Change the IP in the A record. Watch how long it takes for traffic to stop coming in the old IP. If you see 90% of your traffic moved in less than 4 hours, I'd be surprised. If you saw 99% of your traffic moved in less than a day, I'd be even more surprised. I don't know if this is intentional misbehavior of DNS caches, broken software, broken OSes, or where the issue is. But, I can attest that having to make sudden changes from one provider's PA space to another with no grace period will result in support issues for at least a day of "Why can't I reach your site?".
Renumbering for hosting providers can be a monstrous pain in the neck, especially for hosting providers who rely on third parties (or, horrors, their customers) to maintain the zone files within which services are named.
Yep. We don't control the DNS for quite a number of the sites we're hosting. Making our customer get involved every time we add/remove a transit connection isn't going to be fun. Especially when "big" providers who can qualify for PI space of their own don't have to do this.
Some hosting providers of my acquaintance insist on customer zones being redelegated to the hosting providers' nameservers, so that any renumbering that needs to happen can be coordinated by the hosting provider directly. Hosting providers who don't do this, and who use PA addresses with shim6 to multi-home, are definitely going to face some challenges.
That's possible for some hosting providers, not for others. It's not uncommon for a customer to use one provider for some services(web hosting, for example) and another provider for others(secure/e- commerce, for example). Trying to make two competing providers play together nicely when managing DNS for one domain is a recipe for disaster, even with sub-delegation.
Some of our applications are extremely sensitive to jitter/ latency. We've spent ages tweaking route-maps manually (and through automated continual tweaking) to make sure we avoid any congested links. [...]
The site-policy middleware that I alluded to earlier seems like the analogous place to specify this policy. Such a facility might actually give you more control than you have now -- tweaking BGP attributes to accomplish this kind of thing is often like a game of whack-a-mole; if you were able to control the route taken by traffic in both directions by influencing the locator selection for each and every session, you'd have far greater, and more fine- grained, control over your external traffic than BGP/swamp-abuse gives you currently.
While I don't doubt that there are advantages and disadvantages to each way of doing things, I'd much prefer to be given the choice of selecting the one that works best for us, possibly a mix of both.
We'd still be relying on PA space. No matter how great dhcp6 is, there will be significant renumbering pain when providers are changed. Static ACLs, firewall rules, etc. If you're including customer machines in the renumbering, many simply won't do it.
Agreed, renumbering is a pain. Dhcp6 sounds like a scary thing to use with servers. Customers suck. Change in operational practices will be required.
Lest I sound too much like a foam-at-the-mouth shim6 advocate, I think it would be perfectly fine if, in the final analysis, the conclusion was that shim6 and PA/renumbering was not an option for hosting providers. A reasoned technical argument which came to that conclusion would provide a solid basis for the RIRs to modify their allocation policies such that hosting providers could use PI space instead. As perhaps the recent attempt to change the v6 PI policy indicates, the chances of making changes without such a reasoned argument are slim.
And that's kind of my overall point I've been not-so-successfully trying to make. IPv4 is running out. We need people to switch to IPv6, sooner rather than later. Instead of trying to make the process as painless as possible, and with using tools that are available now, a large swath of what probably would be the FIRST people(content/hosting companies) who would want to move to IPv6 are being told to "hang tight until we figure out this multihoming thing, just don't expect it to work at all the same as how you do it now." The additional bite of "You can't do what you used to do, because if everyone did it the internet would break. Oh, and your bigger competitors can still do things like that, just not you." isn't helping matters. Stopping the routing table from exploding in the future is obviously a goal everyone needs to have. Everyone needs to think about this NOW rather than when it's too late, I agree. But, policies shouldn't be written depending on tools that don't exist yet, which is basically what's happening now. If IPv6 were permitted to be used in the same manner that IPv4 is now (PI space is accessible to just about everyone, everyone with an ASN can run BGP, you're trusted that if you deaggregate your space it's for a good reason, etc), people could actually begin moving things now. Taking an existing IPv4 network and overlaying an IPv6 network over the top of it is relatively easy, we went from planning to full deployment in a week. Even if 100% of the IPv4 networks moved to IPv6 today, we'd still have a smaller table size in 6 than 4. Growth would be slower (ISPs and NSPs wouldn't continually be adding more networks as they grew, initial allocations should be enough for just about everyone). Then when Shim6 is developed, you can rely on the current release of every major OS supporting it, router/middleware/etc vendors supporting it, and all the kinks are worked out, you can let the people who find Shim6 appropriate for their needs to use it. Make one of the requirements to get an ASN a justification for why non-ASN/non- public-routing doesn't work for your organization. Then let each network operator choose the right tools for the job. Without totally upending my network, I need: 1) PI space 2) The ability to deaggregate that PI space where truly needed. (or the ability to request multiple PI blocks, but I don't see how that helps matters) 3) BGP announcements to the world If IPv6 can't give me those, and the only thing it's offering is more space... That's just not worth the serious amount of labor and reengineering of our network. If that's the value proposition, we'd hold out on IPv4 as long as possible... And I'm *FOR* IPv6. :) -- Kevin (wondering how many people are muttering 'kook' right now)
On 1-Mar-2006, at 13:32, Kevin Day wrote:
We have peering arrangements with about 120 ASNs. How do we mix BGP IPv6 peering and Shim6 for transit?
You advertise all your PA netblocks to all your peers.
Ok, I was a bit too vague there...
How do we ensure that peering connections are always used instead of transit connections?
Currently for outbound, we can localpref peer-learned routes over everything else. How do all of our servers on our end know that routes learned from a BGP session on our own routers are desirable?
If a client has a set of locators, some of which are reachable via peering connections and some of which are reachable via transit connections, you want to be able to bias the locator selection such that (perhaps, e.g.) peering locators are always preferred to those which involve transit providers. Although simple performance benefits might cause hosts to make that decision on their own, you don't want to leave the decision up to the hosts, since if they get it wrong it will cost you money. This seems inordinately reasonable. Did I summarise correctly?
[...]
But, policies shouldn't be written depending on tools that don't exist yet, which is basically what's happening now.
Actually, I think policies are conservative because there's a lack of solid, technical argument for loosening them. If (for example) there was consensus between operators and shim6 architects that the requirements of hosting providers are definitively not met by shim6, for technical/architectural reasons, then it'd be far easier to convince RIR memberships and boards that policy modifications should be made to accommodate operators like you.
[...]
Even if 100% of the IPv4 networks moved to IPv6 today, we'd still have a smaller table size in 6 than 4. Growth would be slower (ISPs and NSPs wouldn't continually be adding more networks as they grew, initial allocations should be enough for just about everyone).
The trouble with this is that for every argument that says "PI for all will be fine" there's a corresponding argument that says "PI for all will not scale".
Without totally upending my network, I need:
1) PI space 2) The ability to deaggregate that PI space where truly needed. (or the ability to request multiple PI blocks, but I don't see how that helps matters) 3) BGP announcements to the world
For sure, the simplest and cheapest thing for you would be to obtain PI space and continue as you have been doing with v4. The implications of that are not necessarily simple nor cheap for others, however, if you're one of $BIGNUM people doing it.
-- Kevin (wondering how many people are muttering 'kook' right now)
:-) Joe
ok... i've slept some. let me rephrase my agnst this way... when/if a shim6 proof of concept is built, THEN is the time to start debating the merits of shim6 and setting policies on addressing plans. Find one(or more) of the converted, build the darned thing, run some tests, and then there will be concrete, empirical evidence to back the shim6 proponent assertions. not just thought experiments conducted on paper. --bill (show me) manning
Date: Wed, 1 Mar 2006 23:46:22 +0000 From: bmanning@...
when/if a shim6 proof of concept is built,
Let's look at IPv4 options: 0x83 0x04 0x04 0x?? 0x?? 0x?? 0x?? usually doesn't make it very far. Try % traceroute -n -g ip.of.some.router and.of.the.destination from a few endpoints. This is for a reason, yes? Or maybe people just blindly copy "no ip soure-route" from the Cymru secure IOS template... and maybe Rob was just having fun when he labelled it "noxious". :-) It's not shim6, but it's something of an analog that's here today. Perhaps shim6's "intelligent" decisions would assuage transit's concerns about endpoints selecting the links. I doubt it. (Would anyone turn on LSRR if your downstreams spoke multihop eBGP with other endpoints, then used that information to select the source route?) Something along the lines of "no ip shim6" makes these threads moot. ;-) Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
For those watching and grumbling, I'll move the discussion to a shim6 mailing list, or in private if anyone wants to continue beyond this. Just make sure you cc: me if you move the discussion somewhere else. On Mar 1, 2006, at 12:55 PM, Joe Abley wrote:
On 1-Mar-2006, at 13:32, Kevin Day wrote:
We have peering arrangements with about 120 ASNs. How do we mix BGP IPv6 peering and Shim6 for transit?
You advertise all your PA netblocks to all your peers.
If a client has a set of locators, some of which are reachable via peering connections and some of which are reachable via transit connections, you want to be able to bias the locator selection such that (perhaps, e.g.) peering locators are always preferred to those which involve transit providers.
Although simple performance benefits might cause hosts to make that decision on their own, you don't want to leave the decision up to the hosts, since if they get it wrong it will cost you money.
This seems inordinately reasonable. Did I summarise correctly?
For the most part, yes. The information of "who is a peer" is only received into our network via BGP. There needs to be a near realtime interface between BGP on N routers, being made available to influence shim6 decisions on X hosts. A quick look at our peering sessions now shows about 20,000 peer routes. I'm a bit worried that I need to send 20,000+ pieces of routing information to hundreds/thousands of hosts, if I want them to make decisions based on that.
Without totally upending my network, I need:
1) PI space 2) The ability to deaggregate that PI space where truly needed. (or the ability to request multiple PI blocks, but I don't see how that helps matters) 3) BGP announcements to the world
For sure, the simplest and cheapest thing for you would be to obtain PI space and continue as you have been doing with v4. The implications of that are not necessarily simple nor cheap for others, however, if you're one of $BIGNUM people doing it.
No, I'm just trying to be practical here... Estimates of IPv4 pool exhaustion range from Mid 2008 (Tony Hain's ARIN presentation) to roughly 2012 (Geoff Huston's ARIN presentation). Sooner if a mad dash for space starts happening (or isn't happening already). Does anyone here really believe that there is time for: 1) Getting shim6 from proposal to standard 2) Develop reference implementations to prove that it works, and base other development on 3) Convincing OS/Router/other vendors that shim6 is practical and needed, and the best of all ipv6 multihoming solutions 4) Having every major OS/Router/other vendor plan, develop and release software that supports shim6 5) Allow time for application, firewall and other vendors to release upgrades to support shim6 features that may be necessary 6) Developing any "middleware" applications as discussed earlier, on top of adding basic shim6 support 7) Having ISPs/Hosting companies/Enterprises/End users to upgrade all critical and legacy systems to shim6 compatible software 8) Allowing for ISPs/Hosting companies to convince all their customers to upgrade the servers that are outside of the ISP's control (owned by the customers), or deal with losing reliability/ redundancy 8a) In a way that doesn't encourage customers to just move to a hosting provider that has PI space and doesn't need to deal with shim6. 9) Replacing hardware or software that was designed to be ipv6 compatible, but cannot be upgraded to a shim6 stack. (Vendor out of business, vendor won't produce upgrade, etc) 10) The learning curve for network engineers to learn a whole new way of traffic engineering PLUS the existing resistance/foot-dragging about just implementing IPv6 as it is? Far enough in advance of IPv4 exhaustion that IPv6 is a stable and established platform, with a critical mass of ISPs, NSPs content companies and end users on it? All of the above need to happen before PA holders can realistically switch to IPv6 using Shim6. I can't see Shim6 being ready and deployable until well after we already need to have a large percentage of the world using IPv6. My argument isn't that Shim6 will never work for anyone, or that it couldn't be made to work for us EVENTUALLY. I'm saying the pain level of transitioning to IPv6 is too high for any multihoming PA-holding company to willingly do it until they're forced to. The world won't end if we separate "transition to IPv6 addressing" and "transition to more scalable routing/allocation policies" into to different battles, that each need to be won on their own merits. Win the war of just getting people to use IPv6 addressing alone, then when reasonable alternatives to multihoming are ready and usable, let those who can use them transition on their own. I realize pool exhaustion != death of IPv4, but it's a big step towards it. As an aside, for those thinking I'm arguing out of self-interest... We have a /32, none of this discussion is to save me any work or headache. I'm not sure if that helps or hurts my credibility or perceived intentions though. :) -- Kevin
On 2 mar 2006, at 06.16, Kevin Day wrote:
No, I'm just trying to be practical here... Estimates of IPv4 pool exhaustion range from Mid 2008 (Tony Hain's ARIN presentation) to roughly 2012 (Geoff Huston's ARIN presentation). Sooner if a mad dash for space starts happening (or isn't happening already).
Does anyone here really believe that there is time for:
So what I think we might need (that I wrote in an internet-draft some years ago) is the following things in exactly this order : 0. PI space with an artifically high barrier on entry yet available when needed (read cost+administration=LIR or equiv.). 1. Ducttape ala shim6 2. One of breakthrough in graph-theory or a completely new addressing/ routing paradigm. Most like the latter. That will take us past IPv4 exhaustion+IPv6 initial deployment, through wider uptake through to the 10-15 years from now when we might have an idea of what 2 is. I used to believe that it would take 10 years to deploy a standardised version of a stack change, I must say I changing my mind and I am starting to agree with however said that we just need to wait for the next <insert favourite OS> major security hole+patch. - kurtis -
On Mar 3, 2006, at 5:55 AM, Kurt Erik Lindqvist wrote:
On 2 mar 2006, at 06.16, Kevin Day wrote:
No, I'm just trying to be practical here... Estimates of IPv4 pool exhaustion range from Mid 2008 (Tony Hain's ARIN presentation) to roughly 2012 (Geoff Huston's ARIN presentation). Sooner if a mad dash for space starts happening (or isn't happening already).
Does anyone here really believe that there is time for:
So what I think we might need (that I wrote in an internet-draft some years ago) is the following things in exactly this order :
0. PI space with an artifically high barrier on entry yet available when needed (read cost+administration=LIR or equiv.). 1. Ducttape ala shim6 2. One of breakthrough in graph-theory or a completely new addressing/routing paradigm. Most like the latter.
I will bet anyone reading this $ 20 USD right now that what will actually happen is the development of a spot market in IPv4 address space. Regards Marshall
That will take us past IPv4 exhaustion+IPv6 initial deployment, through wider uptake through to the 10-15 years from now when we might have an idea of what 2 is. I used to believe that it would take 10 years to deploy a standardised version of a stack change, I must say I changing my mind and I am starting to agree with however said that we just need to wait for the next <insert favourite OS> major security hole+patch.
- kurtis -
On Fri, 3 Mar 2006, Marshall Eubanks wrote:
I will bet anyone reading this $ 20 USD right now that what will actually happen is the development of a spot market in IPv4 address space.
That's a sucker bet. What's worse is that unless people start changing their tune soon and make the ownership of IP space official, this will be a black market (like it is now, just much bigger). -- Brandon Ross AIM: BrandonNRoss Director, Network Engineering ICQ: 2269442 Internap Skype: brandonross Yahoo: BrandonNRoss
On 3-mrt-2006, at 21:43, Brandon Ross wrote:
I will bet anyone reading this $ 20 USD right now that what will actually happen is the development of a spot market in IPv4 address space.
That's a sucker bet.
What's worse is that unless people start changing their tune soon and make the ownership of IP space official, this will be a black market (like it is now, just much bigger).
But that will end as soon as interdomain routing is protected by certificates given out by the RIRs.
On Fri, Mar 03, 2006 at 09:50:55PM +0100, Iljitsch van Beijnum wrote:
On 3-mrt-2006, at 21:43, Brandon Ross wrote:
I will bet anyone reading this $ 20 USD right now that what will actually happen is the development of a spot market in IPv4 address space.
That's a sucker bet.
What's worse is that unless people start changing their tune soon and make the ownership of IP space official, this will be a black market (like it is now, just much bigger).
But that will end as soon as interdomain routing is protected by certificates given out by the RIRs.
er, not at all. RIR's issuing certificates and expecting the routing system to kowtow is a stretch. --bill
On Fri, Mar 03, 2006 at 09:50:55PM +0100, Iljitsch van Beijnum wrote:
On 3-mrt-2006, at 21:43, Brandon Ross wrote:
What's worse is that unless people start changing their tune soon and make the ownership of IP space official, this will be a black market (like it is now, just much bigger).
But that will end as soon as interdomain routing is protected by certificates given out by the RIRs.
No, it'll end as soon as those certificates become mandatory. Which will, in my humble estimation, happen at some point near the year 4523. ("Hey, guys, you're all using these CIDR blocks which have real monetary value, how about you all agree to deploy this technology which will make them all valueless again? Huh? Hello? Anyone listening? Where'd everyone go?") - mark -- Mark Newton Email: newton@internode.com.au (W) Network Engineer Email: newton@atdot.dotat.org (H) Internode Systems Pty Ltd Desk: +61-8-82282999 "Network Man" - Anagram of "Mark Newton" Mobile: +61-416-202-223
Thus spake "Mark Newton" <newton@internode.com.au>
On Fri, Mar 03, 2006 at 09:50:55PM +0100, Iljitsch van Beijnum wrote:
On 3-mrt-2006, at 21:43, Brandon Ross wrote:
What's worse is that unless people start changing their tune soon and make the ownership of IP space official, this will be a black market (like it is now, just much bigger).
But that will end as soon as interdomain routing is protected by certificates given out by the RIRs.
No, it'll end as soon as those certificates become mandatory.
Which will, in my humble estimation, happen at some point near the year 4523.
I agree that RIR certs will never become truly mandatory, but it'll be a Good Idea(tm) to have one to prevent hijacking. However, some bright accountant at a big telco is going to figure out it's not RIR certs they want to see -- they'll want to issue their own certs to squeeze revenue from non-customers. "You want to buy transit from our peers instead of us? That's great. But, if you want reliable access to our customers from your PI block, you have to pay $100/mo for a routing slot." Bingo, the swamp problem becomes self-correcting. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
At 07:43 AM 4/03/2006, Brandon Ross wrote:
On Fri, 3 Mar 2006, Marshall Eubanks wrote:
I will bet anyone reading this $ 20 USD right now that what will actually happen is the development of a spot market in IPv4 address space.
That's a sucker bet.
What's worse is that unless people start changing their tune soon and make the ownership of IP space official, this will be a black market (like it is now, just much bigger).
There is a lot to agree with in this statement, but also perhaps more to do that just that - how to support such a market such that uniqueness of current control over address resources is clearly demonstrable is necessary, but so is coherence of transfer of such control between consenting parties. I _think_ you are referring to some form of title registration or similar instrument that could support such a market - yes? cheers, Geoff
On 3 mar 2006, at 21.37, Marshall Eubanks wrote:
I will bet anyone reading this $ 20 USD right now that what will actually happen is the development of a spot market in IPv4 address space.
I won't bet against you, but it will only take you that far. At one point IPv4 addresses will just be black tulips... - kurtis -
On Mar 4, 2006, at 8:03 AM, Kurt Erik Lindqvist wrote:
On 3 mar 2006, at 21.37, Marshall Eubanks wrote:
I will bet anyone reading this $ 20 USD right now that what will actually happen is the development of a spot market in IPv4 address space.
I won't bet against you, but it will only take you that far. At one point IPv4 addresses will just be black tulips...
- kurtis -
Be careful. If a /24 becomes worth briefly a landed estate (roughly what the reference translates too), I suspect that a lot of the readers of this list (but not, alas, me) would retire happy. Regards Marshall
At 07:37 AM 4/03/2006, Marshall Eubanks wrote:
On Mar 3, 2006, at 5:55 AM, Kurt Erik Lindqvist wrote:
On 2 mar 2006, at 06.16, Kevin Day wrote:
No, I'm just trying to be practical here... Estimates of IPv4 pool exhaustion range from Mid 2008 (Tony Hain's ARIN presentation) to roughly 2012 (Geoff Huston's ARIN presentation). Sooner if a mad dash for space starts happening (or isn't happening already).
Does anyone here really believe that there is time for:
So what I think we might need (that I wrote in an internet-draft some years ago) is the following things in exactly this order :
0. PI space with an artifically high barrier on entry yet available when needed (read cost+administration=LIR or equiv.). 1. Ducttape ala shim6 2. One of breakthrough in graph-theory or a completely new addressing/routing paradigm. Most like the latter.
I will bet anyone reading this $ 20 USD right now that what will actually happen is the development of a spot market in IPv4 address space.
That was part of the speculative component of my report at the time, and, no, I won't be lining up to take your bet - I agree with you that such a market is pretty much an inevitably. Of course you could always start a futures market right now! cheers, Geoff .
Kevin Day wrote:
If you include "Web hosting company" in your definition of ISP, that's not true. Unless you're providing connectivity to 200 or more networks, you can't get a /32. If all of your use is internal(fully managed hosting) or aren't selling leased lines or anything, you are not considered an LIR by the current IPv6 policies.
Leased lines are not required. You can assign a /48 to any separate organization you provide connectivity to even if they are colocated. A business model where you don't assign /48's to any customers does seem to preclude being an LIR. Web hosting companies that do assign /48's to some customers would qualify.
Even the proposed ARIN 2006-4 assignment policy for "end sites" doesn't help a lot of small to mid sized hosting companies. For that, to just get a /48, you need to already have a /19 or larger, and be using 80% of that. That's 6553 IPs being utilized. If you're running a managed hosting company (name based vhosts) and deploying 1 IP per web server, you're pretty huge before you've hit 6553 devices. Even assuming 20% of that is wasted, you're still talking about more than 5000 servers. 40 1U servers per rack, you need to have 125 racks of packed to the gills servers before you'd qualify for PI space. That excludes every definition I have of "small-to-medium" in the hosting arena.
The latest revision of 2005-1 is also on the table. It would allow for a /48 assignment for any organization that qualifies for IPv4 space, (even /22). Name based virtual hosting is not required either.
You don't get PI space, and Shim6 is looking like your only alternative for multihoming.
We are only limited by our own imaginations and and by what actually works. This is a hard problem to solve and the solution doesn't have to come from the IETF. - Kevihn
How about some actual technical complaints about shim6? good question. to give such discussion a base, could you point us to the documents which describe how to deploy it in the two most common situation operators see o a large multi-homed enterprise customer There are no documents describing deployment. Probably there should be.
The general approach is presumably well-known (for those for whom it is not, go browse around <http://www.ietf.org/html.charters/shim6- charter.html>, and perhaps in particular <http://www.ietf.org/ internet-drafts/draft-ietf-shim6-proto-03.txt>.
Deployment in an enterprise is a matter of:
(a) deploying hosts with shim6-capable stacks within the enterprise;
(b) arranging for those hosts to receive addresses in each PA assignment made by each transit provider (multiple PA addresses per interface), e.g. using dhcp6;
(c) optionally, perhaps, installing shim6 middleware at some suitable place between host and border in order to impose site policy or modulate locator selection by the hosts.
and this last will handle the normal site border (and these days intra-site, e.g., departmental, borders) issues such as o dns within the enterprise is isolated from that of outside o firewalls, algs, and sometimes nats o security policy in general o load balancing between upstreams o ... i.e, what handles the impedance mismatch between the goal, which is *site* multi-homing, and the tool, which is *host* multihoming? and how does it handle it, how is it managed, ...?
You will note I have glossed over several hundred minor details (and several hundred more not-so-minor ones). The protocols are not yet published; there is no known implementation.
possibly this contributes to the sceptisim with which this is viewed?
o a small to medium multi-homed tier-n isp A small-to-medium, multi-homed, tier-n ISP can get PI space from their RIR, and don't need to worry about shim6 at all. Ditto larger ISPs, up to and including the largest.
as it is not yet clear if small isps can get pi space, and the issue of multi-homing is central to the discussion of this issue, and routing table growth is another vector here, perhaps this needs to be explored a bit more. randy
On 1-Mar-2006, at 18:29, Randy Bush wrote:
You will note I have glossed over several hundred minor details (and several hundred more not-so-minor ones). The protocols are not yet published; there is no known implementation.
possibly this contributes to the sceptisim with which this is viewed?
Quite probably. However, if we're waiting for an implementation before we give our requirements for the protocol that was implemented, we should prepare to be disappointed. ("Yossarian!")
A small-to-medium, multi-homed, tier-n ISP can get PI space from their RIR, and don't need to worry about shim6 at all. Ditto larger ISPs, up to and including the largest.
as it is not yet clear if small isps can get pi space, [...]
Actually, this part is most definitely non-fiction. According to the harmonised IPv6 management policy in effect across all the RIRs, anybody who can demonstrate an even vaguely plausible plan to connect 200 IPv6 customers within two years qualifies for a / 32 PI allocation. I've assisted many small, tier-N ISPs to do just this (including hosting companies, for whom the connected customers were colocated customer servers). Joe
On Wed, 1 Mar 2006, Randy Bush wrote:
How about some actual technical complaints about shim6?
good question. to give such discussion a base, could you point us to the documents which describe how to deploy it in the two most common situation operators see o a large multi-homed enterprise customer o a small to medium multi-homed tier-n isp
never under-estimate the range and productivity of Pekka! http://www.netlab.hut.fi/opetus/s38030/k05/Multihoming.pdf
thanks.
randy
-- Lucy E. Lynch Academic User Services Computing Center University of Oregon llynch @darkwing.uoregon.edu (541) 346-1774
On Wed, 1 Mar 2006, Lucy E. Lynch wrote:
point us to the documents which describe how to deploy it in the two most common situation operators see o a large multi-homed enterprise customer o a small to medium multi-homed tier-n isp
never under-estimate the range and productivity of Pekka!
Ouch. I didn't expect that. In case you're interested in a summary paper on shim6, take a look at a newer version (at publication accepted): http://staff.csc.fi/psavola/shim6.pdf It isn't as good as I'd have hoped, and describes a moving target, but it should give a hopefully short and relatively concise summary. Unfortunately, it _doesn't_ describe how to solve the problems that Randy was referring to... :-) -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
--- Joe Abley <jabley@isc.org> wrote:
How about some actual technical complaints about shim6? The jerking knees become tedious to watch, after a while.
Okay, if I'm an enterprise with 6 ISPs but don't qualify for PI space, I'll need to get PA space from all of them, for Shim6 to work, right? Then each server on my network is going to need to maintain state for 6 different contexts for each of the various external customers who attempt to reach them. Assuming that I have busy servers, that's a whole lot of state. It's cheaper and easier to upgrade or modify N routers than the M servers behind them, given that M is certainly greater than N, and in many cases in multiple orders of magnitude greater. Also, the current drafts don't support middleboxes, which a huge number of enterprises use - in fact the drafts specifically preclude their existence, which renders this a complete non-starter for most of my clients. My single biggest issue here however is the complexity: given that today's architecture can deliver relatively simple and robust multihoming to enterprises, and rerouting DOES work today for persistent sessions (albeit imperfectly), what is the benefit to be gained from doing something this hard? As far as I can tell, the whole reason for these discussions is the insistence on the strict PA-addressing model, with no ability to advertise PA space to other providers. I think that we could spend our time better in coming up with a different approach to addressing hierarchy instead. Besides, /48s are cheap now, but if every enterprise gets multiple /48s from multiple providers, they might become dear more quickly than is desired. -David David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On 1-Mar-2006, at 11:22, David Barak wrote:
Also, the current drafts don't support middleboxes, which a huge number of enterprises use - in fact the drafts specifically preclude their existence, which renders this a complete non-starter for most of my clients.
I have not yet reviewed the lastest shim6 protocol draft, but I've seem discussion around it in which people have talked about middlebox support (in the context of "do we want to leave the door open to middleboxes, or should we insist that this is all done on the host stack?").
My single biggest issue here however is the complexity: given that today's architecture can deliver relatively simple and robust multihoming to enterprises, and rerouting DOES work today for persistent sessions (albeit imperfectly), what is the benefit to be gained from doing something this hard?
The current system is complex too, and it will get more complex as the amount of state in the routing system increases. Contrary to what some might think, reading this thread, inter-domain traffic engineering is only achievable using BGP in fairly coarse terms, and the success or failure of the TE tweaks in terms of the desired outcome is often non-determinstic, depending on it does on the routing policies of others. The current system has the advantage, of course, that its strengths and weaknesses are somewhat well-known.
As far as I can tell, the whole reason for these discussions is the insistence on the strict PA-addressing model, with no ability to advertise PA space to other providers.
The whole reason for the strict PA-addressing model is concern over whether open-slather on PI address space will result in an Internet that will scale. Joe (Failing miserably to keep quiet. Must try harder.)
--- Joe Abley <jabley@isc.org> wrote:
On 1-Mar-2006, at 11:22, David Barak wrote:
As far as I can tell, the whole reason for these discussions is the insistence on the strict PA-addressing model, with no ability to advertise PA space to other providers.
The whole reason for the strict PA-addressing model is concern over whether open-slather on PI address space will result in an Internet that will scale.
Is it easier to scale N routers, or scale 10000*N hosts? If we simply moved to an "everyone with an ASN gets a /32" model, we'd have about 30,000 /32s. It would be a really long time before we had as many routes in the table as we do today, let alone the umpteen-bazillion routes which scare everyone so badly.
Joe
(Failing miserably to keep quiet. Must try harder.)
(don't worry - you have content in these posts. content is always welcome...) David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On Wed, 2006-03-01 at 09:05 -0800, David Barak wrote: [..]
Is it easier to scale N routers, or scale 10000*N hosts? If we simply moved to an "everyone with an ASN gets a /32" model, we'd have about 30,000 /32s. It would be a really long time before we had as many routes in the table as we do today, let alone the umpteen-bazillion routes which scare everyone so badly.
Today indeed, but you are missing the point that IPv6 should last for the couple of next decennia. In IPv4 the starters also got a nice /8 as a bonus and the result: new small entities complaining that the first ones got the cool stuff and they can't have any. You might have noticed the 32-bit ASN talk. This is there for a reason: ASN's will go to 32bit mode. Can you say 4 million routes? :) Simple isn't always good. The KISS principle doesn't always work... The current 30k in-use ASN's (afaik they are even less) will and can explode when that means you can get easy address space. Btw, this is policy talk, you might want to bring that to ARIN PPML or the various other lists. If you want to propose a PI policy, then please make a decent proposal and send the relevant RIR group. That endsites require "PI" is inevitable, but the way those routes end up in the routing tables and the amount of address space each endsite is getting should be relevant to need, not to the fact that you got an ASN. (Which would mean I would qualify for 2x /32's... which is very silly as the couple of /48's I use is way more than enough. Please don't mix up addressing and routing. "PI addressing" as you mention is addressing. SHIM6 will become a routing trick. Greets, Jeroen (who simply would like a policy where endsites that want it could request a /48 or /40 depending on requirements from a dedicated block which one day might be used for identity purposes and not pop up in the bgp tables or whatever we have then anymore....)
Please don't mix up addressing and routing. "PI addressing" as you mention is addressing. SHIM6 will become a routing trick.
I think that is overly pessimistic. I would say that SHIM6 _MAY_ become a routing trick, but, so far, SHIM6 is a still-born piece of overly complicated vaporware of minimal operational value, if any. Personally, I think a better solution is to stop overloading IDR meaning onto IP addresses and use ASNs for IDR and prefixes for intradomain routing only.
Greets, Jeroen
(who simply would like a policy where endsites that want it could request a /48 or /40 depending on requirements from a dedicated block which one day might be used for identity purposes and not pop up in the bgp tables or whatever we have then anymore....)
I would, for one. Policy proposal 2005-1 (I am the author) comes reasonably close to that. It will be discussed at the ARIN policy meeting in Montreal in April. Owen -- If it wasn't crypto-signed, it probably didn't come from me.
Owen DeLong wrote:
Please don't mix up addressing and routing. "PI addressing" as you mention is addressing. SHIM6 will become a routing trick.
I think that is overly pessimistic. I would say that SHIM6 _MAY_ become a routing trick, but, so far, SHIM6 is a still-born piece of overly complicated vaporware of minimal operational value, if any.
Personally, I think a better solution is to stop overloading IDR meaning onto IP addresses and use ASNs for IDR and prefixes for intradomain routing only.
Full ACK! For the IDR we then can use perfect match lookups which scale very well and pretty cheaply to many millions of table entries. BGP scales very well too if you've got a decent cpu in your router. Our OpenBGPD easily does 30 flapping constandly full-feeds with 1 million routes each. Lets get pragmatic and realistic! -- Andre
Does this mean that you support 2005-1, or do you think a new ARIN proposal is needed ? Regards Marshall Eubanks On Mar 2, 2006, at 4:28 AM, Andre Oppermann wrote:
Owen DeLong wrote:
Please don't mix up addressing and routing. "PI addressing" as you mention is addressing. SHIM6 will become a routing trick.
I think that is overly pessimistic. I would say that SHIM6 _MAY_ become a routing trick, but, so far, SHIM6 is a still-born piece of overly complicated vaporware of minimal operational value, if any. Personally, I think a better solution is to stop overloading IDR meaning onto IP addresses and use ASNs for IDR and prefixes for intradomain routing only.
Full ACK! For the IDR we then can use perfect match lookups which scale very well and pretty cheaply to many millions of table entries. BGP scales very well too if you've got a decent cpu in your router. Our OpenBGPD easily does 30 flapping constandly full-feeds with 1 million routes each.
Lets get pragmatic and realistic!
-- Andre
Marshall Eubanks wrote:
Does this mean that you support 2005-1, or do you think a new ARIN proposal is needed ?
What I'm saying is that we should reconsider parts of IPv6' design decisions and fix stuff while we can. Opening the floodgates right now, which 2005-1 will do, will only cement the current IPv4 way of doing things with longest-prefix match. Doing longest-prefix match for high pps rates and high prefix counts in hardware is complex and expensive. Way more so than doing perfect match on 32 bits (giving 4bn routeable slots). To answer your question: I do support the rationale behind 2005-1 to allow for PI address space according to current IPv4 rules but I think it is premature right now to make the decision in this way. Once the first /48 according to it went out we have to support and carry it forever in the DFZ. Right now I'm against 2005-1. We should take a hard look at the current customer requirements and market drivers and look at either adjustments to current policies or even certain changes to IPv6 itself to align them. IMHO we have to find the best cross-section satisfying the following requirements: ) PI space to avoid renumbering when switching ISP's (independence) ) PI space to multi-home with two or more ISP's (performance/redundancy) ) PA space for ISP's to hand out to single-homed customers/consumers ) Efficient and cost-effective implementation of DFZ packet forwarding I'm a strong supporter of the original layered approach where different functionality resides on different levels of the stack and is not or only to least possible extent intermixed. Putting routing decisions into the transport layer (4) as it is done or proposed with SCTP and SHIM6 is Total Evilness(tm) in my book. Topology and such should be of no concern to transport. The network layer (3) must handle that in a transparent and independent fashion. This allows for future changes and improvements without having to change everything everywhere. And to make it clear I'm totally against geo-addressing finer than the size of RIR regions. Why should anyone take me seriously? Well, I'm running a genuine 4-digit AS number for as long as the RIR assigned it to me amost a decade ago. And I'm an operating system developer (FreeBSD) working on the network stack. This way I can claim to see all sides of the dice which helps a lot for the Big Picture(tm). -- Andre
Regards Marshall Eubanks
On Mar 2, 2006, at 4:28 AM, Andre Oppermann wrote:
Owen DeLong wrote:
Please don't mix up addressing and routing. "PI addressing" as you mention is addressing. SHIM6 will become a routing trick.
I think that is overly pessimistic. I would say that SHIM6 _MAY_ become a routing trick, but, so far, SHIM6 is a still-born piece of overly complicated vaporware of minimal operational value, if any. Personally, I think a better solution is to stop overloading IDR meaning onto IP addresses and use ASNs for IDR and prefixes for intradomain routing only.
Full ACK! For the IDR we then can use perfect match lookups which scale very well and pretty cheaply to many millions of table entries. BGP scales very well too if you've got a decent cpu in your router. Our OpenBGPD easily does 30 flapping constandly full-feeds with 1 million routes each.
Lets get pragmatic and realistic!
-- Andre
On 2-mrt-2006, at 21:42, Andre Oppermann wrote:
To answer your question: I do support the rationale behind 2005-1 to allow for PI address space according to current IPv4 rules but I think it is premature right now to make the decision in this way. Once the first /48 according to it went out we have to support and carry it forever in the DFZ. Right now I'm against 2005-1.
This is in and of itself enough to reject 2005-1, and I urge the ARIN constituency to do exactly that. We've had restrictive policies around the world for many years now, and so far we've been able to live with it. The IETF is making good progress with its multihoming in IPv6 efforts: implementable RFCs should be forthcoming within a year. Currently, IPv6 deployment is not such that lack of multihoming is creating big problems. If this situation changes, a policy proposal like this one can presumably be adopted fast enough to avoid significant problems. I've talked long and hard about why it's bad to have nearly unrestricted PI in IPv6 because the routing system can't scale (either at all or at reasonable cost) to accommodate this, but apparently this argument isn't universally convincing among operators. However, within the IETF there is reasonable consensus that there is enough of a risk to warrant efforts to provide multihoming benefits that don't impact routing. Also, having /48s for PI is a bad choice as it procludes easy filtering of accidental deaggregated PA prefixes. ISPs are getting / 32s or larger, and customers often get a /48. Deaggregating a /32 into /48s has the potential to increase the global routing table by 65000 routes. Such an event will almost certainly overload routers that don't filter those prefixes out. Experience in IPv4 shows us that accidental deaggregation is relatively common. The easiest way to avoid problems when this happens is filter out all /48s. Today, there must already be exceptions for root server /48s, but as the number of exceptions grows the filtes will become more fragile and the risk of deaggregation that isn't caught by filters increases. Finallly, allow me to observe that deciding on this issues regionally while the resulting routes must be carried world wide doesn't make sense. We really need a global forum for this.
On Mar 2, 2006, at 2:06 PM, Iljitsch van Beijnum wrote:
The IETF is making good progress with its multihoming in IPv6 efforts: implementable RFCs should be forthcoming within a year.
Which proposals do you consider to be implementable? ---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice Everything has been said. But nobody listens. -- Roger Shattuck
Hello; On Mar 2, 2006, at 5:06 PM, Iljitsch van Beijnum wrote:
On 2-mrt-2006, at 21:42, Andre Oppermann wrote:
To answer your question: I do support the rationale behind 2005-1 to allow for PI address space according to current IPv4 rules but I think it is premature right now to make the decision in this way. Once the first /48 according to it went out we have to support and carry it forever in the DFZ. Right now I'm against 2005-1.
This is in and of itself enough to reject 2005-1, and I urge the ARIN constituency to do exactly that. We've had restrictive policies around the world for many years now, and so far we've been able to live with it. The IETF
2005-1 is fairly close to 2002-3, which has been in place for almost 3 years, and so far we've been able to live with it.
is making good progress with its multihoming in IPv6 efforts: implementable RFCs should be forthcoming within a year. Currently, IPv6 deployment is not such that lack of multihoming is creating big problems. If this situation changes, a policy proposal like this one can presumably be adopted fast enough to avoid significant problems.
I've talked long and hard about why it's bad to have nearly unrestricted PI in IPv6 because the routing system can't scale (either at all or at reasonable cost) to accommodate this, but apparently this argument isn't universally convincing among operators. However, within the IETF there is reasonable consensus that there is enough of a risk to warrant efforts to provide multihoming benefits that don't impact routing.
The IETF is at serious risk of being overtaken by events here, in my humble opinion. I have just returned from China, where there is a serious effort focused on deploying IPv6, and where there are 111 million Internet users, most with broadband, according to government statistics. I do not think that they and the other people waiting on IPv6 are going to wait a decade for this to be sorted out.
Also, having /48s for PI is a bad choice as it procludes easy filtering of accidental deaggregated PA prefixes. ISPs are getting / 32s or larger, and customers often get a /48. Deaggregating a /32 into /48s has the potential to increase the global routing table by 65000 routes. Such an event will almost certainly overload routers that don't filter those prefixes out. Experience in IPv4 shows us that accidental deaggregation is relatively common. The easiest way to avoid problems when this happens is filter out all /48s. Today, there must already be exceptions for root server /48s, but as the number of exceptions grows the filtes will become more fragile and the risk of deaggregation that isn't caught by filters increases.
This to be honest sounds like the sort of thing that router vendors would love to build filters for, much like dampening routing flaps or rate limiting MSDP storms. After all, an ASN going from one address block to 65,000 should be detectable automatically. I see no reason why this will lead to the filtering of all IPv6 /48's.
Finallly, allow me to observe that deciding on this issues regionally while the resulting routes must be carried world wide doesn't make sense. We really need a global forum for this.
I fully agree here. Maybe a meeting should be organized in the fall of 2006 or early 2007 to discuss this, either under the auspices of the NRO (http://www.nro.net) or independently. Regards Marshall
On 3-mrt-2006, at 4:54, Marshall Eubanks wrote:
I've talked long and hard about why it's bad to have nearly unrestricted PI in IPv6 because the routing system can't scale (either at all or at reasonable cost) to accommodate this, but apparently this argument isn't universally convincing among operators. However, within the IETF there is reasonable consensus that there is enough of a risk to warrant efforts to provide multihoming benefits that don't impact routing.
The IETF is at serious risk of being overtaken by events here, in my humble opinion. I have just returned from China, where there is a serious effort focused on deploying IPv6, and where there are 111 million Internet users, most with broadband, according to government statistics. I do not think that they and the other people waiting on IPv6 are going to wait a decade for this to be sorted out.
There are currently 265 AS numbers assigned in China (9 more than what Sweden has) so apparantly multihoming isn't too high on their list of priorities. There haven't been any developments the last few years that warrant such a policy change. Yes, some people are saying that they won't deploy IPv6 without easy multihoming, but I've yet to hear someone say that they WILL deploy IPv6 within a year WITH easy multihoming. For instance, Google has had 2001:4860::/32 since april but I've yet to see any of their services work over IPv6. Yes, there are problems with the current policy. For instance, if you're a large transit ISP and your customers all have their own address space, you're not going to assign address space to them so you don't qualify for IPv6 address space. However, it would be nice to be able to give your routers addresses. Why not create a policy that addresses these issues, rather than try to do a full 180 on what's been in place for years? Last but not least, a /48 isn't going to give large hosting companies and large enterprises what they want/need, which is generally multiple address blocks for multiple locations. We can debate whether deagregating a /32 in IPv6 is going to work well (I say: don't count on it), but nobody in their right mind is going to accept prefixes longer than a /48. So this policy will allow creating a swamp in IPv6 and still not address the needs it is supposed to address.
Deaggregating a /32 into /48s has the potential to increase the global routing table by 65000 routes. Such an event will almost certainly overload routers that don't filter those prefixes out.
This to be honest sounds like the sort of thing that router vendors would love to build filters for, much like dampening routing flaps or rate limiting MSDP storms.
I don't think creating a potential problem just so vendors can have a crack at trying to solve it is a good use of our collective time.
After all, an ASN going from one address block to 65,000 should be detectable automatically.
Not very easily. It means keeping statistics on the number of prefixes per origin AS, which is additional work and additional knobs that can be turned into the wrong direction.
I see no reason why this will lead to the filtering of all IPv6 /48's.
A couple of years ago I discovered that the F root server has an IPv6 address: 2001:500::1035. This was assigned as a /48 by ARIN even though their IPv6 policy (still) says: [wait wait wait until I fall back to IPv4 because www.arin.net is currently unreachable over IPv6] "6.4.3. Minimum Allocation RIRs will apply a minimum size for IPv6 allocations, to facilitate prefix-based filtering. The minimum allocation size for IPv6 address space is /32." The ISP that I used at the time installed prefix length filters accordingly so I couldn't reach the F root server over IPv6. Moral of the story: if you build in a way for people to screw up, they'll do it. After that, they'll start throwing out some babies with the bath water.
Thus spake "Iljitsch van Beijnum" <iljitsch@muada.com>
On 3-mrt-2006, at 4:54, Marshall Eubanks wrote:
I've talked long and hard about why it's bad to have nearly unrestricted PI in IPv6 because the routing system can't scale (either at all or at reasonable cost) to accommodate this, but apparently this argument isn't universally convincing among operators. However, within the IETF there is reasonable consensus that there is enough of a risk to warrant efforts to provide multihoming benefits that don't impact routing.
The IETF is at serious risk of being overtaken by events here, in my humble opinion. I have just returned from China, where there is a serious effort focused on deploying IPv6, and where there are 111 million Internet users, most with broadband, according to government statistics. I do not think that they and the other people waiting on IPv6 are going to wait a decade for this to be sorted out.
There are currently 265 AS numbers assigned in China (9 more than what Sweden has) so apparantly multihoming isn't too high on their list of priorities.
There's a general lack of competition over there; remember, it's a communist country, not a capitalist one. If one _wanted_ to multihome in China, there's a limited business possibility of doing so. That's changing slowly, but it's still the reality today.
There haven't been any developments the last few years that warrant such a policy change. Yes, some people are saying that they won't deploy IPv6 without easy multihoming, but I've yet to hear someone say that they WILL deploy IPv6 within a year WITH easy multihoming. For instance, Google has had 2001:4860::/32 since april but I've yet to see any of their services work over IPv6.
That's particularly telling. Of all companies, I'd have expected Google to go IPv6 long before there was a compelling business reason. It'd be interesting to hear from someone there why they haven't rolled it out yet.
Yes, there are problems with the current policy. For instance, if you're a large transit ISP and your customers all have their own address space, you're not going to assign address space to them so you don't qualify for IPv6 address space. However, it would be nice to be able to give your routers addresses. Why not create a policy that addresses these issues, rather than try to do a full 180 on what's been in place for years?
Current ARIN policy allows assigning a /32 to an LIR if they're merely "a known ISP in the ARIN region". The 200 site requirement is only for new entrants to the ISP world, or for enterprises that want to pretend to be an LIR.
Last but not least, a /48 isn't going to give large hosting companies and large enterprises what they want/need, which is generally multiple address blocks for multiple locations. We can debate whether deagregating a /32 in IPv6 is going to work well (I say: don't count on it), but nobody in their right mind is going to accept prefixes longer than a /48.
So this policy will allow creating a swamp in IPv6 and still not address the needs it is supposed to address.
I'm not sure it'll create a swamp, but at least one (I don't have both in front of me) of the proposals allows for assignment of a shorter prefix if it is justified. One could make a reasonable case that announcing /48s in four locations justifies a /46 (or even /45). Conversely, one could convince their upstream(s) to accept longer prefixes as long as they're tagged no-export. That keeps the routing tables at other ISPs to a minimum, but still gets you the majority of the benefit of deaggregating. Either way, be sure to announce the agregate in all locations as well.
Deaggregating a /32 into /48s has the potential to increase the global routing table by 65000 routes. Such an event will almost certainly overload routers that don't filter those prefixes out.
This to be honest sounds like the sort of thing that router vendors would love to build filters for, much like dampening routing flaps or rate limiting MSDP storms.
I don't think creating a potential problem just so vendors can have a crack at trying to solve it is a good use of our collective time.
This is why we've proposed assigning mainly /48s to end-user orgs: ISPs can easily filter at the /48 boundary. These assignments should be out of a single block, which would make it easy for ISPs to set different limits on the PI block (just like the microallocation block(s)) and, if necessary, drop all routes from it if it actually turns into a swamp.
I see no reason why this will lead to the filtering of all IPv6 /48's.
A couple of years ago I discovered that the F root server has an IPv6 address: 2001:500::1035. This was assigned as a /48 by ARIN even though their IPv6 policy (still) says:
[wait wait wait until I fall back to IPv4 because www.arin.net is currently unreachable over IPv6]
"6.4.3. Minimum Allocation RIRs will apply a minimum size for IPv6 allocations, to facilitate prefix-based filtering.
The minimum allocation size for IPv6 address space is /32."
The ISP that I used at the time installed prefix length filters accordingly so I couldn't reach the F root server over IPv6.
Moral of the story: if you build in a way for people to screw up, they'll do it. After that, they'll start throwing out some babies with the bath water.
There's a different policy for IPv6 microallocations, and your ISP messed up by not noticing it. Not surprising given how little time and attention folks have been spending on IPv6 to date. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
On 3 mar 2006, at 04.54, Marshall Eubanks wrote:
Finallly, allow me to observe that deciding on this issues regionally while the resulting routes must be carried world wide doesn't make sense. We really need a global forum for this.
I fully agree here. Maybe a meeting should be organized in the fall of 2006 or early 2007 to discuss this, either under the auspices of the NRO (http://www.nro.net) or independently.
Problem is that there simply is not one-size fits all when it comes to allocation policies. Besides that, AFAIK the NRO have no policy role at all - by design. Policy is set my open meetings at each of the RIRs. - kurtis -
AO> Date: Thu, 02 Mar 2006 21:42:49 +0100 AO> From: Andre Oppermann AO> Doing longest-prefix match for high pps rates and high prefix counts AO> in hardware is complex and expensive. True, but... AO> Way more so than doing perfect match on 32 bits (giving 4bn AO> routeable slots). ...how many routers' FIBs are 32-bit perfect match right now? Or even a 24+8 radix tree? That said, one can use a hybrid { array + { btree | skip list } } for O(k + log(N)) FIB lookups when hardware doesn't support full exact match. Transition workload from logarithmic to scalar as technology permits. Classful routing is simple. Simplicity is good. However, it's still not quite time to get carried away with huge tables. (Of course, IPv6 is a good chance to "start over" with a defragmented table in which a full table would have _fewer_ routes than IPv4.) Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
On 2 mar 2006, at 21.42, Andre Oppermann wrote:
Putting routing decisions into the transport layer (4) as it is done or proposed with SCTP and SHIM6 is Total Evilness(tm) in my book.
Not that shim6 is a change to transport though, but a change at layer 3... - kurtis -
On Sat, 4 Mar 2006 13:35:02 +0100, "Kurt Erik Lindqvist" <kurtis@kurtis.pp.se> said:
On 2 mar 2006, at 21.42, Andre Oppermann wrote:
Putting routing decisions into the transport layer (4) as it is done or proposed with SCTP and SHIM6 is Total Evilness(tm) in my book.
Not that shim6 is a change to transport though, but a change at layer 3...
Isn't the fact that shim6 doesn't affect the forwarding-plane of routers an argument that is used to its advantage? It seems more like something mingling the transport and session layers if anyone ask me (not that the old iso-model is all that relevant anymore imho). //per -- Per Heldal http://heldal.eml.cc/
On 6 mar 2006, at 11.10, Per Heldal wrote:
On Sat, 4 Mar 2006 13:35:02 +0100, "Kurt Erik Lindqvist" <kurtis@kurtis.pp.se> said:
On 2 mar 2006, at 21.42, Andre Oppermann wrote:
Putting routing decisions into the transport layer (4) as it is done or proposed with SCTP and SHIM6 is Total Evilness(tm) in my book.
Not that shim6 is a change to transport though, but a change at layer 3...
Isn't the fact that shim6 doesn't affect the forwarding-plane of routers an argument that is used to its advantage? It seems more like something mingling the transport and session layers if anyone ask me (not that the old iso-model is all that relevant anymore imho).
Ok, so shim6 doesn't require a change to the transport layer and it doesn't change the forwarding plane. It does create a mapping state at the end-nodes. So claiming it to be either is probably wrong. - kurtis -
On Mon, 6 Mar 2006 11:24:59 +0100, "Kurt Erik Lindqvist" <kurtis@kurtis.pp.se> said: [snip]
Ok, so shim6 doesn't require a change to the transport layer and it doesn't change the forwarding plane. It does create a mapping state at the end-nodes. So claiming it to be either is probably wrong.
I stand corrected. Was commenting from a flawed perspective. The most correct is probably to consider it a sub-layer to the existing L3. //per -- Per Heldal http://heldal.eml.cc/
[Pekka, thanks for the Shim6 Summary paper ;) ] On Wed, 2006-03-01 at 14:58 -0800, Owen DeLong wrote:
Please don't mix up addressing and routing. "PI addressing" as you mention is addressing. SHIM6 will become a routing trick.
I think that is overly pessimistic. I would say that SHIM6 _MAY_ become a routing trick, but, so far, SHIM6 is a still-born piece of overly complicated vaporware of minimal operational value, if any.
Vaporware part is true, upto now, operational value is to be seen.
Personally, I think a better solution is to stop overloading IDR meaning onto IP addresses and use ASNs for IDR and prefixes for intradomain routing only.
Did you notice that 32bit ASN's are coming and that IPv4 addresses are 32bits? :) Which effectively means that we are going to route IPv6 with an IPv4 address space. Or when one would use the 32bit ASN for IPv4: routing a 32bit address space with an 32bit routing ID. The mere difference
Greets, Jeroen
(who simply would like a policy where endsites that want it could request a /48 or /40 depending on requirements from a dedicated block which one day might be used for identity purposes and not pop up in the bgp tables or whatever we have then anymore....)
I would, for one. Policy proposal 2005-1 (I am the author) comes reasonably close to that. It will be discussed at the ARIN policy meeting in Montreal in April.
Yep, 2005-1 fits my idea pretty well. Takes care of the folks needing address space now while being able to use it differently later when it is needed. Though as Joe Abley also mentioned (and I also quite a number of times already ;) anyone with even a vague definition of a plan for 200 customers can get a /32 IPv6 without a problem. Just check the GRH list for companies in your neighbourhood who did get it. Greets, Jeroen
I think that is overly pessimistic. I would say that SHIM6 _MAY_ become a routing trick, but, so far, SHIM6 is a still-born piece of overly complicated vaporware of minimal operational value, if any.
Vaporware part is true, upto now, operational value is to be seen.
Well... I can only go based on the existing drafts since there's no running code to base an opinion on, but, my opinion of the drafts is that it will provide minimal operational value. It's the wrong answer to the wrong question, in my opinion.
Personally, I think a better solution is to stop overloading IDR meaning onto IP addresses and use ASNs for IDR and prefixes for intradomain routing only.
Did you notice that 32bit ASN's are coming and that IPv4 addresses are 32bits? :) Which effectively means that we are going to route IPv6 with an IPv4 address space. Or when one would use the 32bit ASN for IPv4: routing a 32bit address space with an 32bit routing ID. The mere difference
Yes, I am well aware of 32bit ASNs. However, some things to consider: 1. Just because ASNs are 32 bits doesn't mean we'll instantly issue all 4 billion of them. The reality is that we probably only need about 18 bits to express all the ASNs well need for the life of IPv6, but, 32 is the next convenient size and there's really no benefit to going with less than 32. 2. In my current thinking on how to achieve ASN based IDR, we would not need ASNs for every organization that multihomes, only for each organization that provides transit. This would greatly reduce some of the current and future demand for ASNs.
Yep, 2005-1 fits my idea pretty well. Takes care of the folks needing address space now while being able to use it differently later when it is needed.
Though as Joe Abley also mentioned (and I also quite a number of times already ;) anyone with even a vague definition of a plan for 200 customers can get a /32 IPv6 without a problem. Just check the GRH list for companies in your neighbourhood who did get it.
True, but, until recently, I was being told that ARIN insisted that the 200 "customers" had to be non-related third parties. E.g. Chevron couldn't use all their different business units as 200 customers of Chevron Corporate IT. It appears based on some recent allocations that they may have relaxed that stance. Regards, Owen
On Thu, 2006-03-02 at 02:21 -0800, Owen DeLong wrote:
Personally, I think a better solution is to stop overloading IDR meaning onto IP addresses and use ASNs for IDR and prefixes for intradomain routing only.
Did you notice that 32bit ASN's are coming and that IPv4 addresses are 32bits? :) Which effectively means that we are going to route IPv6 with an IPv4 address space. Or when one would use the 32bit ASN for IPv4: routing a 32bit address space with an 32bit routing ID. The mere difference
Yes, I am well aware of 32bit ASNs. However, some things to consider:
1. Just because ASNs are 32 bits doesn't mean we'll instantly issue all 4 billion of them. The reality is that we probably only need about 18 bits to express all the ASNs well need for the life of IPv6, but, 32 is the next convenient size and there's really no benefit to going with less than 32.
True. If we would take the 170k routes that are in BGP at the moment then a 18bits address space is enough to give every route a dedicated ASN. The issue is that there are way more people who might want to multihome than that, just take the number of businesses on this planet, add some future growth and we'll end up using the 24th bit too quite quickly. Which is, according to some people who do routing code, no problem at all. Like shim6, see first then believe.
2. In my current thinking on how to achieve ASN based IDR, we would not need ASNs for every organization that multihomes, only for each organization that provides transit. This would greatly reduce some of the current and future demand for ASNs.
Paper/draft/description/website? :)
Yep, 2005-1 fits my idea pretty well. Takes care of the folks needing address space now while being able to use it differently later when it is needed.
Though as Joe Abley also mentioned (and I also quite a number of times already ;) anyone with even a vague definition of a plan for 200 customers can get a /32 IPv6 without a problem. Just check the GRH list for companies in your neighbourhood who did get it.
True, but, until recently, I was being told that ARIN insisted that the 200 "customers" had to be non-related third parties. E.g. Chevron couldn't use all their different business units as 200 customers of Chevron Corporate IT. It appears based on some recent allocations that they may have relaxed that stance.
It might have been that ARIN was a bit stricter, the other RIR's though have never given any real problems as far as I know. The few ones that I heared of that couldn't get it, either didn't try or didn't want to "lie" about their plans. Greets, Jeroen
--On March 2, 2006 11:31:51 AM +0100 Jeroen Massar <jeroen@unfix.org> wrote:
On Thu, 2006-03-02 at 02:21 -0800, Owen DeLong wrote:
Personally, I think a better solution is to stop overloading IDR meaning onto IP addresses and use ASNs for IDR and prefixes for intradomain routing only.
Did you notice that 32bit ASN's are coming and that IPv4 addresses are 32bits? :) Which effectively means that we are going to route IPv6 with an IPv4 address space. Or when one would use the 32bit ASN for IPv4: routing a 32bit address space with an 32bit routing ID. The mere difference
Yes, I am well aware of 32bit ASNs. However, some things to consider:
1. Just because ASNs are 32 bits doesn't mean we'll instantly issue all 4 billion of them. The reality is that we probably only need about 18 bits to express all the ASNs well need for the life of IPv6, but, 32 is the next convenient size and there's really no benefit to going with less than 32.
True. If we would take the 170k routes that are in BGP at the moment then a 18bits address space is enough to give every route a dedicated ASN. The issue is that there are way more people who might want to multihome than that, just take the number of businesses on this planet, add some future growth and we'll end up using the 24th bit too quite quickly. Which is, according to some people who do routing code, no problem at all. Like shim6, see first then believe.
2. In my current thinking on how to achieve ASN based IDR, we would not need ASNs for every organization that multihomes, only for each organization that provides transit. This would greatly reduce some of the current and future demand for ASNs.
Paper/draft/description/website? :)
Paper: Haven't gotten that far yet. Draft: Haven't gotten that far yet. Description: See below Website: Haven't gotten that far yet. Description: This is still knocking around in my head so far. I've discussed and described it to a few folks, but, there are lots of details to work out yet. So, this will require a fair amount of imagination on your part, and, it will require letting go of a lot of assumptions built on the current dogma and paradigm. This is in many ways a completely different paradigm for interdomain routing. Basically, internet routers would come in three flavors: 1. Intradomain Routers -- Routers which have a default route and limited or no detailed knowledge of topology beyond the local ASN. 2. DFZ Edge Routers -- Routers which participate in the IDR process ("full BGP feeds") which have adjacencies with Intradomain Routers. 3. DFZ Core Routers -- Routers which participate in IDR as in 2 above, but, which do not have any adjacencies with routers from category 1 above. In the long run, routers in category 2 and 3 would only carry prefix information for routes terminating in the local AS. For all exterior routes and peering sessions, only AS PATH data would be exchanged, without any prefix information. (In the interim, BGP would be unchanged and routing table bloat would continue to be an issue, but, the routing process could change on a router-by-router basis without requiring a "flag day" conversion). Routers in category 2 would insert an IPv6 extension header of type 53 with a new subtype (yet to be defined, probably 1) which would contain the Destination ASN for the packet. The lookup of Prefix->ASN mapping would be accomplished by a process similar to DNS (See Route Resolvers below). Routers in category 2 and 3 would forward packets by the following ruleset: Is extension header present? Yes: Is it my local ASN? (A) Yes: -- Prefix route available? Yes: Route packet by IGP No: Perform exterior resolution and rewrite ASN header if possible. Otherwise, drop packet. (see loop prevention below for details) (B) No: -- Forward based on ASPATH data to reach AS No: Resolve ASN -- Local? Yes: -- Continue process from (A) above No: -- Insert Extension header and continue from (B) above. Unresolvable: -- Drop packet, send Unreachable no route Route Resolver: Two new RR types and one new hierarchy would need to be added to DNS. The RR types would be AS and SIG, which would provide AS data similar to MX records and Cryptographic Signature data which could be used to trace the delegation of authority for the prefix back to IANA. The new hierarchy would be something like in-as.inet. and would be used to map IPv6 prefixes to NS/SIG records until the most specific match was found, yielding an AS/SIG record pairs. The signature of NS records would contain the public key of the next level delegated authority so as to make it possible to validate that the records it returns had to be signed by the appropriate private key. Loop Prevention: The reason the ASN redirect is necessary is to support clients who multihome without their own ASN. They would advertise records for each upstream ASN. There would be no global knowledge of which ASN was or was not workable at a given time until DNS changed (probably a manual process). In order to prevent loops in this redirect process, it would be necessary to have a second extension header (probably another subtype of type 53) which would contain a list of destination AS already visited. Each router performing AS Redirect would not consider any AS already in this list and would add the AS which it removed on to the list before forwarding. OK... That's it in a nutshell. Yes, there are lots of details that still aren't worked out, but, I think it is a feasible process for IDR and it would allow virtually unlimited PI addressing with routing table growth tied to the number of transit ASNs instead of to the number of prefixes or multihomed leaf sites.
True, but, until recently, I was being told that ARIN insisted that the 200 "customers" had to be non-related third parties. E.g. Chevron couldn't use all their different business units as 200 customers of Chevron Corporate IT. It appears based on some recent allocations that they may have relaxed that stance.
It might have been that ARIN was a bit stricter, the other RIR's though have never given any real problems as far as I know. The few ones that I heared of that couldn't get it, either didn't try or didn't want to "lie" about their plans.
Nonetheless, this was a showstopper for a number of enterprises I know in North America. Owen -- If it wasn't crypto-signed, it probably didn't come from me.
On Thu, Mar 02, 2006 at 02:21:45AM -0800, Owen DeLong wrote:
Yes, I am well aware of 32bit ASNs. However, some things to consider: 1. Just because ASNs are 32 bits doesn't mean we'll instantly issue all 4 billion of them. The reality is that we probably only need about 18 bits to express all the ASNs well need for the life of IPv6, but, 32 is the next convenient size and there's really no benefit to going with less than 32.
It's probably worth using this juncture to point out that exactly the same principle applies to the bit-width gap between IPv4 and IPv6: the fact that IPv6 gives 128 bits doesn't mean we're going to allocate all of them right away. The number of networks in use is not driven by the size of the address space; it's driven by the number of enterprises who wish to connect to the Internet, the number of sites at which they wish to connect, the number of interfaces they wish to use to carry out their interconnections, and the number of hosts they're bringing along with each connection. Notice that none of that has anything to do with the version number of the protocol which those hosts are speaking. By any way you measure it, Internet growth is a function of end-user demand, not a function of the number of bits in an IP address. We can spend another decade talking about how to manage routing table growth if we really want to, but during that decade the routing table is going to *keep growing anyway*. If we want to prevent it from growing, the only real way to do it is going to be at the demand side -- which is another way of saying that we'd need to address and control all of the clauses I iterated through two paragraphs ago. When you distill *that* message to its essence, you can restate it like this: "We can control the growth of the IP routing table by making it harder for people to connect their networks to the Internet." Because that's what the advocates for IPv6 universal PA space are *really* saying, once you remove all the smoke and mirrors. ... which neatly explains a major reason for why IPv6 hasn't taken off, and why shim6 remains vapourware despite many years of discussion and the presence of a clear, unambiguous demand for a solution to the multihoming problem. What's the way out? Acknowledgement of the fact that the size of the routing table is a function of the size of the Internet. Y'know, one of those "duh!" statements. If we expect the Internet to grow past 32-bit limitations, we're going to have to expect the routing table to grow past the constraints which that 32-bit world has imposed upon it. Solving *that* problem is, IMHO, overwhelmingly preferable to breaking multihoming and handing routing policy decisions over to the viruses and worms which control each host. (note that I'm not pretending that solving the routing table growth problem is -easy-, it's just plainly obvious to me that it'll need to be solved anyway, and the IPv6 PA advocacy that keeps coming up seems to be an exercise in denial...) - mark -- Mark Newton Email: newton@internode.com.au (W) Network Engineer Email: newton@atdot.dotat.org (H) Internode Systems Pty Ltd Desk: +61-8-82282999 "Network Man" - Anagram of "Mark Newton" Mobile: +61-416-202-223
On Thu, 2 Mar 2006, Owen DeLong wrote:
I think that is overly pessimistic. I would say that SHIM6 _MAY_
Yes, I am well aware of 32bit ASNs. However, some things to consider:
1. Just because ASNs are 32 bits doesn't mean we'll instantly issue all 4 billion of them. The reality is that we probably only need about 18 bits to express all the ASNs well need for
Probably 20 bits or 24 bits rather then 18. It maybe worth it to reserve top 8 bits for some experimental future use.
the life of IPv6, but, 32 is the next convenient size and there's really no benefit to going with less than 32.
2. In my current thinking on how to achieve ASN based IDR, we would not need ASNs for every organization that multihomes, only for each organization that provides transit. This would greatly reduce some of the current and future demand for ASNs.
I thought of something of this sort some time ago but never got around to full proposal on it (maybe somebody else did?). I'm trying to go through my notes now - maybe it could prove useful if research or engineering people do indeed want to work on something like that. My thinking was that its a big waste of memory (in the global bgp table) to announce every IPv6 route in full in particular for cases when its sub-allocation and aggregate is already being announced. As an example lets take some ip block like say A100:1000::/32 that is allocated to an ISP 'A' by RIR. Now lets say that this ISP 'A' (AS1) has a customer who received A100:1000:0010::/48 and later same customer got connection from ISP 'B' (AS2) as well and wants to multihome. For normal BGP that would mean that full route of A100:1000:0010::/48 would appear in BGP and increase its size quite a bit. But it maybe possible to do limited bgp multi-homing by having such /48 and similar routes included as attributes of the main route, i.e. A100:1000::/32 route would appear with extended attributes like Subroutes: 0010/16 (2) Where "0010/16" indicates subroute as seen within A100:1000::/32 ip block (i.e. netmask here is added to netmask of A100:1000 route to get full netmask on the internet) and "(2)" is an additional ASN that such subroute can also be found through. Now if properly designed, such subroute attribute can be compacted to be around 64 bits extra each (slightly more if multihoming is through more then one ASN and subblock is smaller size then /16) and 64bits data for each multi-homed customer in BGP appear to me something that we can deal with. Unfortunately with this design, the issue then becomes how some AS10 (the end-site) knows how to get to AS2 (one way maybe to do ASN routes as part of BGP in addition to ipv4 and ipv6 routes). As well as what to do if connection from customer to AS1 or from AS1 to internet drops since its AS1 that announces such subroutes as part of its aggregate and purpose of multi-homing it to let it work without it (this can be done with AS2 also announcing A100:1000::/32 but with special attribute indicating its valid only for subroutes and such route should not be propagated if same ASN also sees primary AS1 direct route). Another possibility of similar design (kind-of already mentioned above) is to allow AS2 to announce A100:1000::/32 on the internet as it would its own route but indicate that it is valid only for specific subroutes and not as an aggregate (in fact in this design AS1 would not even have subroutes in its main route announcement). While this means entire full route on the net, the hope is that if AS1 and AS2 have number of common multi-homed customers, the net would be spared from separate BGP routes for each one and the end-site AS10 would only see something like: BGP routing table for A1000:1000::/32 9 8 7 6 1 Origin IGP 5 4 3 2 Origin IGP, Subroutes-Only Subroutes: 0010/16 0101/16 So from above if router needs to send data to either A1000:1000:0010::/48 or A100:1000:0101::/48 it would choose 2nd path through peer AS5 as having smaller as-path. All these approaches (especially second one) however certain problems when you have to consider route security & authorization (i.e. SIDR/SBGP space) as its necessary to convey that AS2 is to be allowed to announce A100:1000: block for specific subroutes. But I think these issues can be worked out like having AS2 sendscertificate request for subblock to AS1 which it then signs and gives new certificate authorizing announcement with specific subblock attribute to AS2. More general issues with these approaches is obviously that there is no possibility of PI space as customers that need multi-homing would have to get space from one of its ISPs (well, actually it is still possible to do PI - it is just that multiple ISPs would be expected to announce large aggregate of the PI block with bunch of subroutes on equal basis). Anyway, if you have comments or if something like this has already been discussed and tested, I'd like to hear. Otherwise, hopefully knowing about this design may prove useful if shim6 does not work out. -- William Leibzon Elan Networks william@elan.net
On 2-mrt-2006, at 13:44, william(at)elan.net wrote:
2. In my current thinking on how to achieve ASN based IDR, we would not need ASNs for every organization that multihomes, only for each organization that provides transit. This would greatly reduce some of the current and future demand for ASNs.
Yes, we wouldn't want to run out of AS numbers just now we're creating 4.29 billion new ones...
My thinking was that its a big waste of memory (in the global bgp table) to announce every IPv6 route in full in particular for cases when its sub-allocation and aggregate is already being announced.
Yes, it would be cool if the routers or route servers could automatically detect this and clean up the routing table. Unfortunately: A --- B / \ X Y \ / C --- D If X uses 172.16.1.0/24 but A also announces 172.16.0.0/12, then A or B could decide to suppress the /24. However, Y will see the /24 through D and C but not through B and A, so Y will now send all of its traffic to X through C and D.
But it maybe possible to do limited bgp multi-homing by having such /48 and similar routes included as attributes of the main route, i.e. A100:1000::/32 route would appear with extended attributes like Subroutes: 0010/16 (2)
Some years ago, I suggested doing this by adding a bitmap to the aggregate route: a single bit is enough to convey holes in the aggregate, with two or three bits you can also do some traffic engineering. This will get you from a /16 aggregate to individual / 24s with 32 bytes (1 bit per more specific) or a /32 to /48s with 8 kilobytes. Such an approach does depend on relatively tight packing of end-users that share the same ISPs, though.
All these approaches (especially second one) however certain problems when you have to consider route security & authorization (i.e. SIDR/SBGP space)
IDR security doesn't come cheap anyway: be prepared to double or quadruple your router's memory and install crypto hardware.
On Thu, 2 Mar 2006, Iljitsch van Beijnum wrote:
My thinking was that its a big waste of memory (in the global bgp table) to announce every IPv6 route in full in particular for cases when its sub-allocation and aggregate is already being announced.
Yes, it would be cool if the routers or route servers could automatically detect this and clean up the routing table. Unfortunately:
A --- B / \ X Y \ / C --- D
If X uses 172.16.1.0/24 but A also announces 172.16.0.0/12, then A or B could decide to suppress the /24. However, Y will see the /24 through D and C but not through B and A, so Y will now send all of its traffic to X through C and D.
If you read through my design, it would be that Y should assume that aggregate as seen from B is always valid path even if it is not directly indicated by inclusion of special subroute attribute. This maybe both good and bad as far as such design goes.
But it maybe possible to do limited bgp multi-homing by having such /48 and similar routes included as attributes of the main route, i.e. A100:1000::/32 route would appear with extended attributes like Subroutes: 0010/16 (2)
Some years ago, I suggested doing this by adding a bitmap to the aggregate route: a single bit is enough to convey holes in the aggregate, with two or three bits you can also do some traffic engineering. This will get you from a /16 aggregate to individual /24s with 32 bytes (1 bit per more specific) or a /32 to /48s with 8 kilobytes.
Can be done with bitmaps, but unless aggregate is very well filled with sub-allocations, this would be waste of memory too. I think individual subroutes is more reasonable as long as each one can be well compacted (0010/16 is 16-bits for netblock, max 7 bits for netmask).
Such an approach does depend on relatively tight packing of end-users that share the same ISPs, though.
All these approaches (especially second one) however certain problems when you have to consider route security & authorization (i.e. SIDR/SBGP space)
IDR security doesn't come cheap anyway: be prepared to double or quadruple your router's memory and install crypto hardware.
Yes, most likely it will require dedicated box to process the security data and verify ip routes (Note: in usual way dedicated box might be represented as being separate card in the router). -- William Leibzon Elan Networks william@elan.net
On Wed, Mar 01, 2006 at 09:05:17AM -0800, David Barak wrote:
--- Joe Abley <jabley@isc.org> wrote:
On 1-Mar-2006, at 11:22, David Barak wrote:
As far as I can tell, the whole reason for these discussions is the insistence on the strict PA-addressing model, with no ability to advertise PA space to other providers.
The whole reason for the strict PA-addressing model is concern over whether open-slather on PI address space will result in an Internet that will scale.
Is it easier to scale N routers, or scale 10000*N hosts? If we simply moved to an "everyone with an ASN gets a /32" model, we'd have about 30,000 /32s. It would be a really long time before we had as many routes in the table as we do today, let alone the umpteen-bazillion routes which scare everyone so badly.
I think you're missing that some people do odd things with their IPs as well, like have one ASN and 35 different sites where they connect to their upstream Tier69.net all with the same ASN. This means that their 35 offices/sites will each need a /32, not one per the entire asn in the table. And they may use different carriers in different cities. Obviously this doesn't fit the definition that some have of "autonomous system", as these are 35 different discrete networks that share a globally unique identifier of sorts. - jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
I think you're missing that some people do odd things with their IPs as well, like have one ASN and 35 different sites where they connect to their upstream Tier69.net all with the same ASN. This means that their 35 offices/sites will each need a /32, not one per the entire asn in the table.
People who are doing that have not read the definition of the term ASN and there is no reason that the community or public policy should concern itself with supporting such violations of the RFCs. An AS is a collection of prefixes with a consistent and common routing policy. By definition, an AS must be a contiguous collection of prefixes or it is not properly a single AS. Using the same ASN to represent multiple AS is a clear violation.
And they may use different carriers in different cities. Obviously this doesn't fit the definition that some have of "autonomous system", as these are 35 different discrete networks that share a globally unique identifier of sorts.
It doesn't fit the RFC definition of AS. Therefore, there is no reason to support such usage on a continuing basis. You violate the RFC's you takes your chances. Owen -- If it wasn't crypto-signed, it probably didn't come from me.
On Wed, Mar 01, 2006 at 03:01:22PM -0800, Owen DeLong wrote:
I think you're missing that some people do odd things with their IPs as well, like have one ASN and 35 different sites where they connect to their upstream Tier69.net all with the same ASN. This means that their 35 offices/sites will each need a /32, not one per the entire asn in the table.
People who are doing that have not read the definition of the term ASN and there is no reason that the community or public policy should concern itself with supporting such violations of the RFCs. An AS is a collection of prefixes with a consistent and common routing policy. By definition, an AS must be a contiguous collection of prefixes or it is not properly a single AS. Using the same ASN to represent multiple AS is a clear violation.
It doesn't fit the RFC definition of AS. Therefore, there is no reason to support such usage on a continuing basis. You violate the RFC's you takes your chances.
I guess all those root servers that use the same asn but connect to different networks (anycast) should get shut down quickly. This is a part of networking life today in the v4 space, and without any current changes, it will (is) the same in v6 routing as there is nothing different except a few more bits 32 => 128. No new routing protocol, nothing, except this shim6 thing which people don't seem interested in because it means network operators can't do the traffic engineering they need to. - jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
--On March 2, 2006 9:37:12 AM -0500 Jared Mauch <jared@puck.nether.net> wrote:
On Wed, Mar 01, 2006 at 03:01:22PM -0800, Owen DeLong wrote:
I think you're missing that some people do odd things with their IPs as well, like have one ASN and 35 different sites where they connect to their upstream Tier69.net all with the same ASN. This means that their 35 offices/sites will each need a /32, not one per the entire asn in the table.
People who are doing that have not read the definition of the term ASN and there is no reason that the community or public policy should concern itself with supporting such violations of the RFCs. An AS is a collection of prefixes with a consistent and common routing policy. By definition, an AS must be a contiguous collection of prefixes or it is not properly a single AS. Using the same ASN to represent multiple AS is a clear violation.
It doesn't fit the RFC definition of AS. Therefore, there is no reason to support such usage on a continuing basis. You violate the RFC's you takes your chances.
I guess all those root servers that use the same asn but connect to different networks (anycast) should get shut down quickly.
No... In the case of anycast, there is a consistent routing policy for the address. There are services that don't work because of that routing policy, but, that's a decision of the service provider in question. However, they are using the equivalent of one /32 per entire ASN, not one per site. If they are advertising different prefixes from different sites in an inconsistent manner using the same ASN, that is broken. That's not what anycast does.
This is a part of networking life today in the v4 space, and without any current changes, it will (is) the same in v6 routing as there is nothing different except a few more bits 32 => 128.
Anycast is part of networking life today. What you described initially is _NOT_ how anycast works. Owen -- If it wasn't crypto-signed, it probably didn't come from me.
--- Jared Mauch <jared@puck.nether.net> wrote:
I think you're missing that some people do odd things with their IPs as well, like have one ASN and 35 different sites where they connect to their upstream Tier69.net all with the same ASN. This means that their 35 offices/sites will each need a /32, not one per the entire asn in the table.
No, that's an argument for a /32 and a bunch of /48 allocations heard by a single provider, who's getting paid to carry them, but are not advertised to the rest of the Internet.
And they may use different carriers in different cities. Obviously this doesn't fit the definition that some have of "autonomous system", as these are 35 different discrete networks that share a globally unique identifier of sorts.
Well, wait a minute - what would these people do TODAY? Some build tunnel backbones, some use one ASN per city, some do "allowas-in" or other things of that nature. I would venture to say that most medium to large enterprises don't use straight-Internet with no VPN of any kind to support their enterprise backbones anymore, simply for security reasons. My argument still stands - if having an ASN is equated with having a routable netblock, then each of those cases results in the enterprise being able to pass packets, and only the "one ASN per city" approach requires multiple netblocks. -David David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On 1-mrt-2006, at 18:05, David Barak wrote:
Is it easier to scale N routers, or scale 10000*N hosts?
Is it easier for the government to make a 5 year plan or for everyone to spend time and energy finding the best deal for everything? Every router has to search through its FIB tables for every packet it forwards. That's something like 10 FIB lookups for every packet flowing between two hosts. The hosts only have to search through their TCBs for every packet. Number of TCBs in nearly all hosts is smaller than the average FIB size (even if you consider that many routers don't have a full table). 2 x relatively small is a lot less than 10 x relatively large. Or, in other words: on the host you only pay if you actually communicate. In routers, you pay more as there is more routing information, whether the extra information is used or not.
If we simply moved to an "everyone with an ASN gets a /32" model, we'd have about 30,000 /32s. It would be a really long time before we had as many routes in the table as we do today, let alone the umpteen-bazillion routes which scare everyone so badly.
1. We've already walked the edge of the cliff several times (CIDR had to be implemented in a big hurry, later flap dampening and prefix length filtering were needed) 2. We'll have to live with IPv6 a long time 3. Route processing and FIB lookups scale worse than linear 4. If the global routing table meltdown happens, it will be extremely costly in a short time 5. Even if the meltdown doesn't happen a smaller routing table makes everything cheaper and gives us more implementation options (5000 entry TCAM is nice, 500000 entries not so much as it basically uses 100 times as much power) 6. Moore can't go on forever, there are physical limitations But the most important thing we should remember is that currently, routing table growth is artificially limited by relatively strict requirements for getting a /24 or larger. With IPv6 this goes away, and we don't know how many people will want to multihome then.
--- Iljitsch van Beijnum <iljitsch@muada.com> wrote:
But the most important thing we should remember is that currently, routing table growth is artificially limited by relatively strict requirements for getting a /24 or larger. With IPv6 this goes away, and we don't know how many people will want to multihome then.
So why not approach Shim6 as something for basement multihomers rather than enterprises? Honestly, the cost of the second connection is the limiting factor in most decisions not to multihome today, not the difficulty of getting BGP, an ASN, or a /24 from a provider... For your "I have a cablemodem AND a DSL" folks, Shim6 sounds like exactly what they need. However, once you start talking about enterprise-wide policies, etc, Shim6 starts to look like a really heavy hammer. -David David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On Wed, 1 Mar 2006, Iljitsch van Beijnum wrote:
3. Route processing and FIB lookups scale worse than linear
6. Moore can't go on forever, there are physical limitations
The funny part: Those on this list who have cited Moore's Law don't seem to have an understanding that it does not directly apply to custom routing logic (since general-purpose CPUs are no longer fast enough to do the lookups on the high end). In addition, GP CPUs are no longer scaling exponentially, but rather closer to quadratically and approaching linear. In short, Moore's Law is dying, and even if it weren't, it is not a valid argument for "let the swamp in". -- -- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>
Thus spake "Todd Vierling" <tv@duh.org>
On Wed, 1 Mar 2006, Iljitsch van Beijnum wrote:
3. Route processing and FIB lookups scale worse than linear
6. Moore can't go on forever, there are physical limitations
The funny part: Those on this list who have cited Moore's Law don't seem to have an understanding that it does not directly apply to custom routing logic (since general-purpose CPUs are no longer fast enough to do the lookups on the high end). In addition, GP CPUs are no longer scaling exponentially, but rather closer to quadratically and approaching linear.
In short, Moore's Law is dying,
Moore's Law says nothing about performance; it only refers to transistor densities. In fact, current CPUs are still following the predicted curve, but they're turning fewer and fewer of those transistors into actual performance improvements. That's what the move to dual-core is about: finding more productive ways to use the wealth transistors now available. However, I agree that custom logic for routers does not necessarily follow the same curve; the volume is still low enough that vendors can't (or don't) use the best processes available. Heck, even the best available main CPUs are several years behind what's available in the PC market (why ship a 2GHz CPU when you can ship a 500MHz one at ten times the price?).
and even if it weren't, it is not a valid argument for "let the swamp in".
One of the key attributes of the v4 swamp is that most orgs got more than one assignment (aka routing slot), often dozens to hundreds; the proposed policies for a "v6 swamp" do not allow that. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
Thus spake "Iljitsch van Beijnum" <iljitsch@muada.com>
Is it easier to scale N routers, or scale 10000*N hosts? ... 2 x relatively small is a lot less than 10 x relatively large. Or, in other words: on the host you only pay if you actually communicate. In routers, you pay more as there is more routing information, whether
On 1-mrt-2006, at 18:05, David Barak wrote: the extra information is used or not.
OTOH, hosts go a lot longer between upgrades and generally don't have professional admins. It'll be a long, long time (if ever) until shim6 is deployed widely enough for folks to literally bet their company on host-based multihoming.
If we simply moved to an "everyone with an ASN gets a /32" model, we'd have about 30,000 /32s. It would be a really long time before we had as many routes in the table as we do today, let alone the umpteen-bazillion routes which scare everyone so badly.
1. We've already walked the edge of the cliff several times (CIDR had to be implemented in a big hurry, later flap dampening and prefix length filtering were needed)
At least this time we know what the likely problems are, and we can build in safeguards that can be quickly implemented if we get too close to the edge. Not that I agree we'll even get there...
2. We'll have to live with IPv6 a long time
Perhaps. I know the goal was for it to last 100 years, but what technology has ever lasted that long without significant improvements that altered it almost beyond recognition?
3. Route processing and FIB lookups scale worse than linear
With an mtrie+ FIB, routing lookups scale far, far better than linear. What matters is the length of the prefix being matched, not how many there are. TCAMs scale linearly, provided you can build them big enough (and costs certainly aren't linear).
4. If the global routing table meltdown happens, it will be extremely costly in a short time 5. Even if the meltdown doesn't happen a smaller routing table makes everything cheaper and gives us more implementation options (5000 entry TCAM is nice, 500000 entries not so much as it basically uses 100 times as much power)
Agreed.
6. Moore can't go on forever, there are physical limitations
Every time folks claim that, someone makes a breakthrough that continues the curve. Surely we can't count on this forever, but so far money has consistently trumped "physical limitations".
But the most important thing we should remember is that currently, routing table growth is artificially limited by relatively strict requirements for getting a /24 or larger. With IPv6 this goes away, and we don't know how many people will want to multihome then.
The requirements for getting a /24 are pretty darn lax, actually, and the current proposals for PI space being debated within ARIN are significantly more restrictive. The reality today is that v4 routing tables are well within our capabilities and growing slowly. If we were on the verge of another serious problem, like we where when the CIDR fire drill happened, ISPs could easily cut the tables in half simply by filtering prefixes longer than RIR minima. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
OTOH, hosts go a lot longer between upgrades and generally don't have professional admins. It'll be a long, long >time (if ever) until shim6 is deployed widely enough for folks to literally bet
On Mar 3, 2006, at 10:50 AM, Stephen Sprunk wrote: their company on host-based >multihoming. This issue alone means that shim6 isn't viable. Besides the already- mentioned security and complexity issues, enterprise IT departments - i.e., the customers who need multihoming and cannot live without it - are not going to be amused when told that the tens and hundreds of thousands of desktops, laptops, PDAs, and other IP-enabled devices on their networks are now essentially routers, with multiple IP addresses and complex middleware required to simply access 'the Internet' . . . they're starved for resources and talent like everyone else, and the network is -not- their business, simply a means to an end. It's all overhead, to them. Many customers have trouble simply supporting (and patching/ upgrading) basic OS and apps and IPv4. Expecting them to support something like shim6 is as unrealistic as expecting them to re- address at the drop of a hat due to changing business relationships with their SPs (see RFC 4192 for an exposition on the effort required to renumber, and discussion on the concept of network renumbering as a frequent procedure). ---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice Everything has been said. But nobody listens. -- Roger Shattuck
Roland Dobbins wrote:
OTOH, hosts go a lot longer between upgrades and generally don't have professional admins. It'll be a long, long time (if ever) until shim6 is deployed widely enough for folks to literally bet their company on host-based multihoming. This issue alone means that shim6 isn't viable. Besides the already- mentioned security and complexity issues, enterprise IT departments - i.e., the customers who need multihoming and cannot live without it - are not going to be amused when told that the tens and hundreds of
On Mar 3, 2006, at 10:50 AM, Stephen Sprunk wrote: thousands of desktops, laptops, PDAs, and other IP-enabled devices on their networks are now essentially routers, with multiple IP addresses and complex middleware required to simply access 'the Internet' . . .
We've been here before; we shift a lot of data in the content arena, and our web-head loadbalancers, installed only a year ago, don't even support ipv6 in the current software build. -a
On 1-mrt-2006, at 17:22, David Barak wrote:
I think that we could spend our time better in coming up with a different approach to addressing hierarchy instead.
I agree. The address space is one dimensional. This means you can encode a single thing in it in a hierarchical manner "for free". With PA, that's the ISP: for any address, it's very easy to determine which ISP it belongs to and thus route the packet to that ISP. (We're so used to this that we don't even notice anymore.) However, this doesn't work for multihoming because rather than a linear space starting with ISP A and ending with ISP Z we now have a matrix: A-A, A-B, A-C ... Z-X, Z-Y, Z-Z. (Worse with more than two ISPs.) You can't do a longest match first lookup on a multidimensional space, so in routing, every end-user becomes his own ISP and occupies a slot at the top of the hierarchy. The thing is, it's not even hard to aggregate differently: just have router A hold the first quarter of the global routing table (0/2 with v4), router B the second quarter (64/2), router C the second quarter (128/2) and router D the fourth quarter (192/2), for example. There is one snag, though: either you need four routers in each location, or you have to bring the traffic to the place where the router handling that part of the table is located. Now I happen to think that we can massage this such that it's not necessary to add extra routers to speak of or backhaul traffic through places where it doesn't belong so basically all of this is free (no new protocols!), but unfortunately, I'm having a hard time convincing others that this is a workable approach.
--On February 28, 2006 5:15:37 PM -0500 John Payne <john@sackheads.org> wrote:
On Feb 28, 2006, at 2:22 PM, Iljitsch van Beijnum wrote:
Should be doable with a DNS SRV record like mechanism. Don't worry too much about this one.
Where does the assumption that the network operators control the DNS for the end hosts come from?
Thin air I think. Certainly isn't the case with a large number of domains we host.
On Feb 28, 2006, at 1:22 PM, Iljitsch van Beijnum wrote:
On 28-feb-2006, at 17:09, Kevin Day wrote:
4) Being able to do 1-3 in realtime, in one place, without waiting for DNS caching or connections to expire
How fast is real time?
And are we just talking about changing preferences here, or about what happens when there are outages?
5-30 seconds? Including already established connections. "Oh, crap. We're going over our commit on provider C because of a traffic surge on one of our sites. We need to rebalance this before we get dinged for 95th percentile overage." "Packet loss to AS1234 through provider A suddenly skyrocketed. We need to bypass A to that ASN until it's fixed." "1 of the 2 lines in our trunk to provider B went down, we're at half bandwidth. We need to shed some load immediately." We also have incredibly long TCP sessions for some of our services (streaming video/audio). We need to be able to make routing changes while those are active, without relying on a keepalive failing to make the hosts re-evaluate their path decision. If I'm a VOIP provider, I can't wait for someone to hang up a phone call for new routing policy to take effect. A VPN provider could have sessions open for days/weeks. We make extensive use of near-immediate routing changes on both inbound and outbound, relying on the fact that they take effect immediately. No matter where we put the routing information, how are the end nodes that are now making the routing decisions going to see the changes quickly? And how do they see changes for already established connections? Anything done in DNS is just too slow. As an example, take a busy/ popular website. Put a 5 minute TTL on the records weeks in advance. Change the IP and watch how long it takes for 100% of the traffic to stop reaching the old IP. 90% within 1-3 hours, 99% within 24 hours. You'll still get hits to the old IP days later. Too many people blatantly disregard DNS caching, or just get it wrong.
5) Being able to make routing/policy changes without having to rely on the owners/administrators of the machines/sites/domains themselves to do the right thing. (i.e. untrusted/not-maintained- by-us systems/networks on our network)
If you're a multihomed hosting company you would want to do TE for your entire POP, but you wouldn't necessarily be able to change information in the DNS for all the hosts/services that your customers run. Is that what you mean?
Exactly. More detail in my followup message.
6) Anycast?
I don't think shim6 applies to interdomain anycast. (Which is a hack anyway.)
Well, it's a hack that many people are using. If we can't do anycast after we migrate to IPv6, that again raises the bar of transitioning.
7) During what will be a very lengthy dual-stack transitional period, having to do TE in two entirely different ways. BGP +Prepending+Selective-announcements along side Shim6 doesn't really sound like fun to me. We can't treat bits as bits, we have to consider if they're IPv4 bits or IPv6 bits, and engineer them differently, even though they're sharing the same lines and are probably going to have a 1:1 addressing relationship between IPv4 and IPv6 services.
:-)
This is a result of the transition to IPv6, regardless of shim6.
It is, but it's one more thing in the list of "We have to do things differently, and it's questionable if it's better - if not flat out worse" things about moving to IPv6. From a hosting company's standpoint: Pros: 1) Virtually unlimited IP space Cons: 1) Even if you qualified for PI space in IPv4, unless you're huge, you're not getting PI space in IPv6. Want to change providers? You're renumbering all of your customers. 2) If you do need to move, your new provider can't temporarily announce your space from your old provider, which is possible now. 3) No matter how easily configurable IPv6 makes renumbering, you are going to have customers leave rather than deal with readdressing. Some just won't respond/do anything at all no matter how much you harass them that they need to take an action. "Big" hosting companies who do enough connectivity sales to justify PI space get the upper hand. 4) Once you publish AAAA records, every user who has broken their IPv6 stack on their desktop (even if they don't have IPv6 connectivity at all) suddenly can't reach you. 5) The only proposal that looks like it has any traction at all to multihome(shim6) requires trust in customers to administer their boxes to our instructions a lot more closely, and/or requires control over DNS for each site we host. 6) If you do get PI space, the mantra of "Announce only/exactly what you were allocated. No more specifics. No deaggregation." requires a complete redesign of how a lot of us do things. And now adding shim6 to the mix: 7) You can't run BGP or traffic engineer your network the way you're doing with IPv4. You now have two places you have to make routing policy decisions, and they're done in completely different ways. 8) If you're using shim6, public/private peering is probably not possible either. (And yes, there are those who participate in peering arrangements who don't provide transit to others, and wouldn't qualify for PI space) The "migrate to IPv6" pain v.s. benefit ratio for those actually running the content side of the internet is pretty poor at the moment. I don't think you'll be finding many doing it willingly at this stage, or in the foreseeable future. And don't confuse this with laziness or some dislike to IPv6. I went into our transition attempt really wanting to make this work, and eventually dropped it because it would require too many business- model changing transitions to do so.
On top of those, even if shim6 accomplishes the failover and reliability goals, I can't see how shim6 is going to make path decisions as optimal as IPv4/BGP/etc.
Really??? The way I see it, BGP decisions today are mediocre at best. If anything, I would expect things to get better with shim6.
BGP has the benefit of each network in the middle being able to add their say into things. Each transit network can prepend/localpref/med/ etc to produce an end-to-end decision. Shim6 presents both ends with multiple choices, but little in the way of information as to which one to prefer. It's also moving the decision making into LOTS of equipment, instead of the borders. Any fancy ideas we come up with to make better decisions has to be deployed everywhere, and possibly on equipment we don't control. BGP allows information to be added to the routing decision making process that isn't visible from each end. We're making use of that now. -- Kevin
participants (46)
-
Alexei Roudnev -
Andre Oppermann -
Andrew Dul -
Andy Davidson -
bmanning＠vacation.karoshi.com -
Brandon Ross -
Christian Kuhtz -
Christopher L. Morrow -
Daniel Golding -
David Barak -
David Meyer -
Edward B. DREGER -
Eliot Lear -
Geoff Huston -
Gustavo Lozano -
Iljitsch van Beijnum -
Jared Mauch -
Jeroen Massar -
Joe Abley -
John Curran -
John Kristoff -
John Payne -
JORDI PALET MARTINEZ -
Kevin Day -
Kevin Loch -
Kurt Erik Lindqvist -
Lucy E. Lynch -
Mark Newton -
Marshall Eubanks -
Matthew Petach -
Michael Loftis -
Michael.Dillon＠btradianz.com -
Mikael Abrahamsson -
Niels Bakker -
Owen DeLong -
Paul Jakma -
Pekka Savola -
Per Heldal -
Randy Bush -
Roland Dobbins -
Stephen Sprunk -
Steven M. Bellovin -
Todd Vierling -
Tony Hain -
Tony Li -
william(at)elan.net