Hello, Things I understand: IPv6 is the long term solution to IPv4 exhaustion. For IPv6 to work correctly, most of the IPv4 content has to be on IPv6. That's not there yet. IPv6 deployment to end users is not trivial (end user support, CPE support, etc...). Translation techniques are generally evil. IPv6->IPv4 still requires 1 IPv4 IP per end user or else you're doing NAT. IPv4->IPv6 (1-1) doesn't solve our main problem of giving users access to the IPv4 Internet. I expect like most companies we're faced with having to extend the life of IPv4 since our users will continue to want access to the IPv4 content. Doing that by giving them an IPv6 address is not very feasible yet for many reasons. NAT444 seems like the only solution available while we slowly transition over to IPv6 over the next 20 years. Based on the this RFC, NAT444 breaks a lot of applications! http://tools.ietf.org/html/draft-donley-nat444-impacts-01 Has anyone deployed NAT444? Can folks share their experiences? Does it really break this many apps? What other options do we have? Thanks, Serge
On Thu, Sep 1, 2011 at 11:36 AM, Serge Vautour <sergevautour@yahoo.ca> wrote:
Hello,
Things I understand: IPv6 is the long term solution to IPv4 exhaustion. For IPv6 to work correctly, most of the IPv4 content has to be on IPv6. That's not there yet. IPv6 deployment to end users is not trivial (end user support, CPE support, etc...). Translation techniques are generally evil. IPv6->IPv4 still requires 1 IPv4 IP per end user or else you're doing NAT. IPv4->IPv6 (1-1) doesn't solve our main problem of giving users access to the IPv4 Internet.
Correct, all content is not there yet... but World IPv6 Day showed that Google, Facebook, Yahoo, Microsoft and 400+ others are just about ready to go. http://en.wikipedia.org/wiki/World_IPv6_Day IPv6->IPv4 does not require 1 to 1, .... any protocol translation is a form of NATish things, and stateful NAT64 has many desirable properties IF you already do NAT44. Specifically, it is nice that IPv6 flows bypass the NAT .... and as more content becomes IPv6, NAT becomes less and less used. In this way, unlike NAT44 or NAT444, NAT64 has an exit strategy that ends with proper E2E networking with IPv6... the technology and economic incentives push the right way (more IPv6...) Have a look at http://tools.ietf.org/html/rfc6146 There are multiple opensource and big vendor (C, J, B, LB guys...) implementation of NAT64 / DNS64 ... I have trialed it and plan to deploy it, YMMV... It works great for web and email, not so great for gaming and Skype.
I expect like most companies we're faced with having to extend the life of IPv4 since our users will continue to want access to the IPv4 content. Doing that by giving them an IPv6 address is not very feasible yet for many reasons. NAT444 seems like the only solution available while we slowly transition over to IPv6 over the next 20 years. Based on the this RFC, NAT444 breaks a lot of applications!
This is just putting IPv4 on life support without moving needle towards a long term solution. NAT64 = good investment to get IPv6 off the blocks. NAT444 = life support / money pit with forklift exit required.
Has anyone deployed NAT444? Can folks share their experiences? Does it really break this many apps? What other options do we have?
Yes, expect it to be deployed in places where the access gear can only do IPv4 and there is no money or technology available to bring in IPv6. Cameron
Thanks, Serge
On 9/1/11 11:52 AM, Cameron Byrne wrote:
Hello,
Things I understand: IPv6 is the long term solution to IPv4 exhaustion. For IPv6 to work correctly, most of the IPv4 content has to be on IPv6. That's not there yet. IPv6 deployment to end users is not trivial (end user support, CPE support, etc...). Translation techniques are generally evil. IPv6->IPv4 still requires 1 IPv4 IP per end user or else you're doing NAT. IPv4->IPv6 (1-1) doesn't solve our main problem of giving users access to the IPv4 Internet. Correct, all content is not there yet... but World IPv6 Day showed
On Thu, Sep 1, 2011 at 11:36 AM, Serge Vautour<sergevautour@yahoo.ca> wrote: that Google, Facebook, Yahoo, Microsoft and 400+ others are just about ready to go.
http://en.wikipedia.org/wiki/World_IPv6_Day
IPv6->IPv4 does not require 1 to 1, .... any protocol translation is a form of NATish things, and stateful NAT64 has many desirable properties IF you already do NAT44. Specifically, it is nice that IPv6 flows bypass the NAT .... and as more content becomes IPv6, NAT becomes less and less used. In this way, unlike NAT44 or NAT444, NAT64 has an exit strategy that ends with proper E2E networking with IPv6... the technology and economic incentives push the right way (more IPv6...)
Have a look at http://tools.ietf.org/html/rfc6146
There are multiple opensource and big vendor (C, J, B, LB guys...) implementation of NAT64 / DNS64 ... I have trialed it and plan to deploy it, YMMV... It works great for web and email, not so great for gaming and Skype. http://tools.ietf.org/html/rfc6333 http://tools.ietf.org/html/draft-bpw-pcp-nat-pmp-interworking-00 moves CPE NAT to the ISP tunneled over 192.0.0.0/29.
Has anyone deployed NAT444? Can folks share their experiences? Does it really break this many apps? What other options do we have? Yes, expect it to be deployed in places where the access gear can only do IPv4 and there is no money or technology available to bring in IPv6. A false economy when support outweigh CPE cost.
-Doug
NAT444 alone is not enough. You will need to deploy it along with 6rd or DS-lite. Whilst you still have global v4, use it. The best is to deploy dual-stack, but that won't last for too long. Regards, as- On 1 Sep 2011, at 15:36, Serge Vautour wrote:
Hello,
Things I understand: IPv6 is the long term solution to IPv4 exhaustion. For IPv6 to work correctly, most of the IPv4 content has to be on IPv6. That's not there yet. IPv6 deployment to end users is not trivial (end user support, CPE support, etc...). Translation techniques are generally evil. IPv6->IPv4 still requires 1 IPv4 IP per end user or else you're doing NAT. IPv4->IPv6 (1-1) doesn't solve our main problem of giving users access to the IPv4 Internet.
I expect like most companies we're faced with having to extend the life of IPv4 since our users will continue to want access to the IPv4 content. Doing that by giving them an IPv6 address is not very feasible yet for many reasons. NAT444 seems like the only solution available while we slowly transition over to IPv6 over the next 20 years. Based on the this RFC, NAT444 breaks a lot of applications!
http://tools.ietf.org/html/draft-donley-nat444-impacts-01
Has anyone deployed NAT444? Can folks share their experiences? Does it really break this many apps? What other options do we have?
Thanks, Serge
* Arturo Servin
NAT444 alone is not enough.
You will need to deploy it along with 6rd or DS-lite.
In a typical DS-Lite deployment you won't be using NAT444. One of the key advantages of DS-Lite (and A+P, I believe) is that there's only one level of NAT between the end user and the public internet. -- Tore Anderson Redpill Linpro AS - http://www.redpill-linpro.com
In a typical DS-Lite deployment you won't be using NAT444. One of the key advantages of DS-Lite (and A+P, I believe) is that there's only one level of NAT between the end user and the public internet.
yep. and in ds-lite that nat is in the core, so you talk to comcast's lawyers when you need a new alg. with a+p, the nat is under customer control, and they just go to best buy to get the cpe that supports the alg that they want. randy
-----Original Message----- From: Arturo Servin [mailto:arturo.servin@gmail.com] Sent: 07 September 2011 01:37 To: Serge Vautour Cc: nanog@nanog.org Subject: Re: NAT444 or ?
NAT444 alone is not enough.
You will need to deploy it along with 6rd or DS-lite.
Whilst you still have global v4, use it. The best is to deploy dual-stack, but that won't last for too long.
Regards, as-
I'm going to have to deploy NAT444 with dual-stack real soon now. So I am expecting to see some issues. A+P would be nicer perhaps, but none of the CPE I have will support it. I'll try and give people who do NAT in their CPE a public address for as long as I can, but it'll soon run out and then NAT444 will have to be used and some things will just not work very well. -- Leigh Porter ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
I'm going to have to deploy NAT444 with dual-stack real soon now.
you may want to review the presentations from last week's apnic meeting in busan. real mesurements. sufficiently scary that people who were heavily pushing nat444 for the last two years suddenly started to say "it was not me who pushed nat444, it was him!" as if none of us had a memory. randy
-----Original Message----- From: Randy Bush [mailto:randy@psg.com] Sent: 07 September 2011 11:18 To: Leigh Porter Cc: North American Network Operators' Group Subject: Re: NAT444 or ?
I'm going to have to deploy NAT444 with dual-stack real soon now.
you may want to review the presentations from last week's apnic meeting in busan. real mesurements. sufficiently scary that people who were heavily pushing nat444 for the last two years suddenly started to say "it was not me who pushed nat444, it was him!" as if none of us had a memory.
randy
Thankyou, I'm watching it now, but I am under no illusion that it will work well. NAT44 is bad enough. -- Leigh ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
On Wed, Sep 07, 2011 at 12:16:28PM +0200, Randy Bush wrote:
I'm going to have to deploy NAT444 with dual-stack real soon now.
you may want to review the presentations from last week's apnic meeting in busan. real mesurements. sufficiently scary that people who were heavily pushing nat444 for the last two years suddenly started to say "it was not me who pushed nat444, it was him!" as if none of us had a memory.
Hm, I fail to find relevant slides discussing that. Could you please point us to those? I'm looking at http://meetings.apnic.net/32 Best regards, Daniel -- CLUE-RIPE -- Jabber: dr@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0
-----Original Message----- From: Daniel Roesen [mailto:dr@cluenet.de] Sent: 07 September 2011 17:38 To: nanog@nanog.org Subject: Re: NAT444 or ?
On Wed, Sep 07, 2011 at 12:16:28PM +0200, Randy Bush wrote:
I'm going to have to deploy NAT444 with dual-stack real soon now.
you may want to review the presentations from last week's apnic meeting in busan. real mesurements. sufficiently scary that people who were heavily pushing nat444 for the last two years suddenly started to say "it was not me who pushed nat444, it was him!" as if none of us had a memory.
Hm, I fail to find relevant slides discussing that. Could you please point us to those?
I'm looking at http://meetings.apnic.net/32
Best regards, Daniel
There is a lot in the IPv6 plenary sessions: http://meetings.apnic.net/32/program/ipv6 This is what I am looking at right now. Randy makes some good comments in those sessions. I have not found anything yet, but I am only on session 3, pertaining specifically to issues around NAT444. I would be looking for issues such as implementing ALGs on both NAT devices, ALG scaling on LSN boxes and issues surrounding application compatibility. I'm also looking at NAT logging for law enforcement issues. Is there anything planned for the next NANOG around these issues? -- Leigh ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
On 08/09/2011, at 2:41 AM, Leigh Porter wrote:
-----Original Message----- From: Daniel Roesen [mailto:dr@cluenet.de] Sent: 07 September 2011 17:38 To: nanog@nanog.org Subject: Re: NAT444 or ?
On Wed, Sep 07, 2011 at 12:16:28PM +0200, Randy Bush wrote:
I'm going to have to deploy NAT444 with dual-stack real soon now.
you may want to review the presentations from last week's apnic meeting in busan. real mesurements. sufficiently scary that people who were heavily pushing nat444 for the last two years suddenly started to say "it was not me who pushed nat444, it was him!" as if none of us had a memory.
Hm, I fail to find relevant slides discussing that. Could you please point us to those?
I'm looking at http://meetings.apnic.net/32
There is a lot in the IPv6 plenary sessions:
http://meetings.apnic.net/32/program/ipv6
This is what I am looking at right now. Randy makes some good comments in those sessions. I have not found anything yet, but I am only on session 3, pertaining specifically to issues around NAT444.
It may not be what Randy was referring to above, but as part of that program at APNIC32 I reported on the failure rate I am measuring for Teredo. I'm not sure its all in the slides I was using, but what I was trying to say was that STUN is simply terrible at reliably negotiating a NAT. I was then wondering what pixie dust CGNs were going to use that would have any impact on the ~50% connection failure rate I'm observing in Teredo. And if there is no such thing as pixie dust (damn!) I was then wondering if NATs are effectively unuseable if you want anything fancier than 1:1 TCP connections (like multi-user games, for example). After all, a 50% connection failure rate for STUN is hardly encouraging news for a CGN deployer if your basic objective is not to annoy your customers. regards, Geoff
Op 8 sep 2011, om 07:26 heeft Geoff Huston het volgende geschreven:
On 08/09/2011, at 2:41 AM, Leigh Porter wrote:
It may not be what Randy was referring to above, but as part of that program at APNIC32 I reported on the failure rate I am measuring for Teredo. I'm not sure its all in the slides I was using, but what I was trying to say was that STUN is simply terrible at reliably negotiating a NAT. I was then wondering what pixie dust CGNs were going to use that would have any impact on the ~50% connection failure rate I'm observing in Teredo. And if there is no such thing as pixie dust (damn!) I was then wondering if NATs are effectively unuseable if you want anything fancier than 1:1 TCP connections (like multi-user games, for example). After all, a 50% connection failure rate for STUN is hardly encouraging news for a CGN deployer if your basic objective is not to annoy your customers.
The striking thing I picked up is that NTT considers the CGN equipment a big black hole where money goes into. Because it won't solve their problem now or in the future and it becomes effectively a piece of equipment they need to buy and then scrap "soon" after. They acknowledge the need, but they'd rather not buy one. That and they (the isp) get called for anything which doesn't work. Regards, Seth
-----Original Message----- From: Seth Mos [mailto:seth.mos@dds.nl] Sent: 08 September 2011 06:43 To: NANOG Subject: Re: NAT444 or ?
Op 8 sep 2011, om 07:26 heeft Geoff Huston het volgende geschreven:
On 08/09/2011, at 2:41 AM, Leigh Porter wrote:
It may not be what Randy was referring to above, but as part of that
program at APNIC32 I reported on the failure rate I am measuring for Teredo. I'm not sure its all in the slides I was using, but what I was trying to say was that STUN is simply terrible at reliably negotiating a NAT. I was then wondering what pixie dust CGNs were going to use that would have any impact on the ~50% connection failure rate I'm observing in Teredo. And if there is no such thing as pixie dust (damn!) I was then wondering if NATs are effectively unuseable if you want anything fancier than 1:1 TCP connections (like multi-user games, for example). After all, a 50% connection failure rate for STUN is hardly encouraging news for a CGN deployer if your basic objective is not to annoy your customers.
I have a concern about some weird and wonderful VPN solutions that people may be using. I am quite sure that some of them will just not work through NAT444, though I have no evidence of this. People have problems with some VPN solutions with single NAT (especially with no ALGs). NAT444 will just be a mess.
The striking thing I picked up is that NTT considers the CGN equipment a big black hole where money goes into. Because it won't solve their problem now or in the future and it becomes effectively a piece of equipment they need to buy and then scrap "soon" after.
Well if you buy the 'right' solution then you can re-use it elsewhere. Many solutions use multi-purpose processing cards to deliver NAT functionality which can be used for other stuff such as firewalling or some other manor of traffic mangling.
They acknowledge the need, but they'd rather not buy one. That and they (the isp) get called for anything which doesn't work.
Well at least these little problems that pop up keep people in jobs ;-) If everything just worked (tm) there would be nothing to do.. -- Leigh ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
On Thursday, September 08, 2011 04:52:56 PM Leigh Porter wrote:
Well if you buy the 'right' solution then you can re-use it elsewhere. Many solutions use multi-purpose processing cards to deliver NAT functionality which can be used for other stuff such as firewalling or some other manor of traffic mangling.
You'll be hard-pressed to NOT find a vendor willing to sell you the "right" solution for the "easy way out" :-). Mark.
...
The striking thing I picked up is that NTT considers the CGN equipment a big black hole where money goes into. Because it won't solve their problem now or in the future and it becomes effectively a piece of equipment they need to buy and then scrap "soon" after.
It would get scrapped when all servers support dual stack. What year is that predicted to occur?
They acknowledge the need, but they'd rather not buy one. That and they (the isp) get called for anything which doesn't work.
-d
On Thursday, September 08, 2011 01:41:58 PM Seth Mos wrote:
The striking thing I picked up is that NTT considers the CGN equipment a big black hole where money goes into. Because it won't solve their problem now or in the future and it becomes effectively a piece of equipment they need to buy and then scrap "soon" after.
They acknowledge the need, but they'd rather not buy one. That and they (the isp) get called for anything which doesn't work.
In spending millions of hard-earned $$ toward vendors, NTT might not be alone in this realization. However, in acknowledging that it's an endless blackhole where you won't see money necessarily coming back relative to the amount being put in, they may just part of a small handful of folk, sadly. GPRS/3G/EDGE has made many a mobile provider especially notorious. Mark.
On Sep 10, 2011, at 12:46 PM, Mark Tinka wrote:
GPRS/3G/EDGE has made many a mobile provider especially notorious.
All this problematic state should be broken up into smaller instantiations and distributed as close to the access edge (RAN, wireline, etc.) as possible in order to a) reduce the amount of state concentrated in a single device and b) to minimize the impact footprint when aberrant traffic inevitably fills up the state tables and said devices choke. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
On Saturday, September 10, 2011 01:52:12 PM Dobbins, Roland wrote:
All this problematic state should be broken up into smaller instantiations and distributed as close to the access edge (RAN, wireline, etc.) as possible in order to a) reduce the amount of state concentrated in a single device and b) to minimize the impact footprint when aberrant traffic inevitably fills up the state tables and said devices choke.
Certainly a consideration when an ISP considers scaling avenues for LSN's. The issue is that there are simply too many variables, not least of which is what business the ISP is in. The mobile types are a lot more problematic because they tend to centralize IP intelligence, and keep the RAN's pretty simple (although the RAN's are now becoming more intelligent thanks to your garden-variety IP vendors getting into the game). What we've seen also, with some mobile carriers, is that if you ask them to consider distributed IP architectures, they/you quickly realize that IP routing isn't really their core business or skill. For your typical ISP, size notwithstanding, it will invariably come down to how much money and effort they're willing to spend or save with either centralized or distributed architectures. Mind you, they're also battling with other problems re: centralized or distributed solutions, e.g., broadband aggregation, the ratio of access:aggregation intelligence, access topology lay-outs, e.t.c. And somehow, in all this mix, LSN's must work, be they small units thrown around the network, or one or two large monsters sitting somewhere in the core. We've certainly considered both options very thoroughly. Mark.
On Sep 10, 2011, at 1:11 PM, Mark Tinka wrote:
What we've seen also, with some mobile carriers, is that if you ask them to consider distributed IP architectures, they/you quickly realize that IP routing isn't really their core business or skill.
Concur. Many/most have essentially become 'accidental ISPs' through the rise in popularity of iDevices, and some at least are beginning to realize this and to staff and re-engineer accordingly. It takes time, though. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
On Sep 9, 2011 10:54 PM, "Dobbins, Roland" <rdobbins@arbor.net> wrote:
On Sep 10, 2011, at 12:46 PM, Mark Tinka wrote:
GPRS/3G/EDGE has made many a mobile provider especially notorious.
All this problematic state should be broken up into smaller instantiations
and distributed as close to the access edge (RAN, wireline, etc.) as possible in order to a) reduce the amount of state concentrated in a single device and b) to minimize the impact footprint when aberrant traffic inevitably fills up the state tables and said devices choke.
Ip mobility via gtp or mobile ip generally does not work when you nat at the 'edge'. If you don't want your ip address to change every time you change cell sites, the nat has to be centralized. Cb
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
The basis of optimism is sheer terror.
-- Oscar Wilde
-----Original Message----- From: Cameron Byrne [mailto:cb.list6@gmail.com] Ip mobility via gtp or mobile ip generally does not work when you nat at the 'edge'. If you don't want your ip address to change every time you change cell sites, the nat has to be centralized.
Cb
Indeed, networks with some kind of anchor point (even xDSL networks with a LNS that terminates PPP sessions) really lend themselves to a big fat NAT box simply because there is a single point where all the connections appear anyway so why not have a single box doing NAT/DPI/etc as well? I'd agree that, usually, distributed is better but these are not distributed networks, there is a single point (or a few large single points) of contact. -- Leigh ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
On Sep 11, 2011, at 4:02 PM, Leigh Porter wrote:
I'd agree that, usually, distributed is better but these are not distributed networks, there is a single point (or a few large single points) of contact.
The point is that these aggregations of state are quite vulnerable, and therefore they should be as distributed as is practicable. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
On Sep 11, 2011 4:33 AM, "Dobbins, Roland" <rdobbins@arbor.net> wrote:
On Sep 11, 2011, at 4:02 PM, Leigh Porter wrote:
I'd agree that, usually, distributed is better but these are not
distributed networks, there is a single point (or a few large single points) of contact.
The point is that these aggregations of state are quite vulnerable, and
therefore they should be as distributed as is practicable.
I don't disagree with that principle, but other priciples around scale, cost, and oam say that we get one big box called a cgn. And, that is the reality of service provider nat in the real world today. For mobile providers, the cgn generally follows the mobility anchor points. For some national providers that means nfl cities, for others that means one per timezone. Cb
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
The basis of optimism is sheer terror.
-- Oscar Wilde
-----Original Message----- From: Geoff Huston [mailto:gih@apnic.net] Sent: Wednesday, September 07, 2011 10:27 PM To: Leigh Porter Cc: nanog@nanog.org list; Daniel Roesen Subject: Re: NAT444 or ?
On 08/09/2011, at 2:41 AM, Leigh Porter wrote:
-----Original Message----- From: Daniel Roesen [mailto:dr@cluenet.de] Sent: 07 September 2011 17:38 To: nanog@nanog.org Subject: Re: NAT444 or ?
On Wed, Sep 07, 2011 at 12:16:28PM +0200, Randy Bush wrote:
I'm going to have to deploy NAT444 with dual-stack real soon now.
you may want to review the presentations from last week's apnic meeting in busan. real mesurements. sufficiently scary that people who
were
heavily pushing nat444 for the last two years suddenly started to say "it was not me who pushed nat444, it was him!" as if none of us had a memory.
Hm, I fail to find relevant slides discussing that. Could you please point us to those?
I'm looking at http://meetings.apnic.net/32
There is a lot in the IPv6 plenary sessions:
http://meetings.apnic.net/32/program/ipv6
This is what I am looking at right now. Randy makes some good comments in those sessions. I have not found anything yet, but I am only on session 3, pertaining specifically to issues around NAT444.
It may not be what Randy was referring to above, but as part of that program at APNIC32 I reported on the failure rate I am measuring for Teredo. I'm not sure its all in the slides I was using, but what I was trying to say was that STUN is simply terrible at reliably negotiating a NAT.
Teredo does not use STUN; Teredo was implemented before STUN was fully spec'd and does its own thing. Teredo tries to determine if the type of NAT it is behind ("cone", "symmetric", etc.) Determining the type of NAT has been found to be difficult, and nearly impossible (*) and removed from the current RFC for STUN (**). I suspect most of Teredo's failures are related to this procedure not working effectively. A CGN can't improve that. (*) http://tools.ietf.org/html/rfc5780#section-1 (**) http://tools.ietf.org/html/rfc5389#section-2
I was then wondering what pixie dust CGNs were going to use that would have any impact on the ~50% connection failure rate I'm observing in Teredo. And if there is no such thing as pixie dust (damn!) I was then wondering if NATs are effectively unuseable if you want anything fancier than 1:1 TCP connections (like multi-user games, for example). After all, a 50% connection failure rate for STUN is hardly encouraging news for a CGN deployer if your basic objective is not to annoy your customers.
If the CGN has both Endpoint-Independent Filtering (***) behavior and Endpoint-Independent Mapping (****) behavior, the CGN won't make Teredo worse -- Teredo will be as bad as today, caused by the home user's own pretty NAT. With that behavior, it also won't make applications that use STUN or ICE worse, or applications that use STUN-like or ICE-like techniques such as Skype. (***) endpoint-independent filtering: means it doesn't filter incoming packets after a mapping is created. See http://tools.ietf.org/html/rfc4787#section-5 for canonical definition. (****) Endpoint-Independent Mapping: means when the internal host sends a packet with the same source port, to any destination, the same public port is mapped. See http://tools.ietf.org/html/rfc4787#section-4.1 for canonical definition -d
On Wed, Sep 07, 2011 at 12:16:28PM +0200, Randy Bush wrote:
I'm going to have to deploy NAT444 with dual-stack real soon now. you may want to review the presentations from last week's apnic meeting in busan. real mesurements. sufficiently scary that people who were heavily pushing nat444 for the last two years suddenly started to say "it was not me who pushed nat444, it was him!" as if none of us had a memory.
Hm, I fail to find relevant slides discussing that. Could you please point us to those?
I had the same question. I found Miyakawa-san's presentation has some dramatic examples of CGN NAT444 effects using Google Maps: http://meetings.apnic.net/__data/assets/file/0011/38297/Miyakawa-APNIC-KEYNO... However these are with a very high address-sharing ratio (several thousands users per address). Using a sparser density (<= 64 users per address) is likely to show much less dramatic user impacts. /JF
On Wed, Sep 07, 2011 at 01:06:11PM -0400, Jean-Francois.TremblayING@videotron.com wrote:
I had the same question. I found Miyakawa-san's presentation has some dramatic examples of CGN NAT444 effects using Google Maps: http://meetings.apnic.net/__data/assets/file/0011/38297/Miyakawa-APNIC-KEYNO...
Those effects are not specific to NAT444, but apply to any provider-based NAT limiting the amount of parallel sessions available to any one customer. Randy was (as I understood him) referring to the evilness of double-NAT in contrast to single-state NAT (as used with e.g. DS-Lite). Best regards, Daniel -- CLUE-RIPE -- Jabber: dr@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0
Op 7 sep 2011, om 19:06 heeft Jean-Francois.TremblayING@videotron.com het volgende geschreven:
On Wed, Sep 07, 2011 at 12:16:28PM +0200, Randy Bush wrote:
I'm going to have to deploy NAT444 with dual-stack real soon now. you may want to review the presentations from last week's apnic meeting in busan. real mesurements. sufficiently scary that people who were heavily pushing nat444 for the last two years suddenly started to say "it was not me who pushed nat444, it was him!" as if none of us had a memory.
Hm, I fail to find relevant slides discussing that. Could you please point us to those?
I had the same question. I found Miyakawa-san's presentation has some dramatic examples of CGN NAT444 effects using Google Maps: http://meetings.apnic.net/__data/assets/file/0011/38297/Miyakawa-APNIC-KEYNO...
However these are with a very high address-sharing ratio (several thousands users per address). Using a sparser density (<= 64 users per address) is likely to show much less dramatic user impacts.
I think you have the numbers off, he started with 1000 users sharing the same IP, since you can only do 62k sessions or so and with a "normal" timeout on those sessions you ran into issues quickly. The summary is that with anything less then 20 tcp sessions per user simultaneous google maps or earth was problematic. From 15 and downwards almost unsable. He deducted from testing that about 10 users per IP was a more realistic limit without taking out the entire CGN "experience". On a personal note, this isn't even taking into question things like broken virus scanners or other software updates that will happily try to do 5 sessions per second, or a msn client lost trying to do 10 per second. The most the windows IP stack will allow on client versions. The real big issue that will be the downfall of NAT444 is the issue with ACLS and automatic blocklists and the loss of granular access control on that which the ISP has no control of. Which roughly estimates to the internet. Regards, Seth
-----Original Message----- From: Seth Mos [mailto:seth.mos@dds.nl] Sent: 07 September 2011 20:26 To: NANOG Subject: Re: NAT444 or ?
I think you have the numbers off, he started with 1000 users sharing the same IP, since you can only do 62k sessions or so and with a "normal" timeout on those sessions you ran into issues quickly.
The summary is that with anything less then 20 tcp sessions per user simultaneous google maps or earth was problematic. From 15 and downwards almost unsable.
He deducted from testing that about 10 users per IP was a more realistic limit without taking out the entire CGN "experience".
On a personal note, this isn't even taking into question things like broken virus scanners or other software updates that will happily try to do 5 sessions per second, or a msn client lost trying to do 10 per second. The most the windows IP stack will allow on client versions.
The real big issue that will be the downfall of NAT444 is the issue with ACLS and automatic blocklists and the loss of granular access control on that which the ISP has no control of. Which roughly estimates to the internet.
Regards,
Seth
I was thinking of an average of around 100 sessions per user for working out how things scale to start with. It would also be handy to be able to apply sensible limits to new sessions, say limit the number of sessions to a single destination IP address and apply an overall session limit of perhaps 200 sessions per source IP address. ACLs and blocklists are going to be a problem, perhaps, as LSN becomes more and more common, such things will gradually die out. Considering that offices, schools etc regularly have far more than 10 users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted these are all different kinds of users, it is just something that operational experience will have to demonstrate. I would love to avoid NAT444, I do not see a viable way around it at the moment. Unless the Department of Work and Pensions release their /8 that is ;-) -- Leigh ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
On Wed, Sep 7, 2011 at 4:05 PM, Leigh Porter <leigh.porter@ukbroadband.com>wrote:
I was thinking of an average of around 100 sessions per user for working out how things scale to start with. It would also be handy to be able to apply sensible limits to new sessions, say limit the number of sessions to a single destination IP address and apply an overall session limit of perhaps 200 sessions per source IP address.
ACLs and blocklists are going to be a problem, perhaps, as LSN becomes more and more common, such things will gradually die out.
Considering that offices, schools etc regularly have far more than 10 users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted these are all different kinds of users, it is just something that operational experience will have to demonstrate.
I would love to avoid NAT444, I do not see a viable way around it at the moment. Unless the Department of Work and Pensions release their /8 that is ;-)
Perhaps it can be made ever so slightly less ugly if endpoints get an "address" that consists of a 32 bit IP address + (n) upper bits of port number. This might be 4 significant bits to share an IP 16 ways, or 8 significant bits to share it 256 ways, or whatever. In a perfect world, all of the endpoints sharing a single IP would be off a single concentration point of whatever sort (same tower, etc)... The end users can still do their own NAT then, their NAT device just has to restrict the external port range to the one assigned (or have packets with bad source ports dropped when sent). Ok, not really pretty, but maybe a little less ugly than twice NAT, and at least users could have "fixed" addresses (including the upper bits of port number). Of course, the 0000 value for upper port bits can be reserved for "business" grade users so they get the nice ports like 80, etc.
On Wed, 07 Sep 2011 16:13:26 EDT, Dorn Hetzel said:
Perhaps it can be made ever so slightly less ugly if endpoints get an "address" that consists of a 32 bit IP address + (n) upper bits of port number.
This might be 4 significant bits to share an IP 16 ways, or 8 significant bits to share it 256 ways, or whatever.
And you store the 4 or 8 bits in what part of the IPv4 header, exactly?
-----Original Message----- From: Valdis.Kletnieks@vt.edu [mailto:Valdis.Kletnieks@vt.edu] Sent: 07 September 2011 23:14 To: Dorn Hetzel Cc: Leigh Porter; NANOG Subject: Re: NAT444 or ?
On Wed, 07 Sep 2011 16:13:26 EDT, Dorn Hetzel said:
Perhaps it can be made ever so slightly less ugly if endpoints get an "address" that consists of a 32 bit IP address + (n) upper bits of port number.
This might be 4 significant bits to share an IP 16 ways, or 8 significant bits to share it 256 ways, or whatever.
And you store the 4 or 8 bits in what part of the IPv4 header, exactly?
Nobody uses the TOS bits, do they? ;-) -- Leigh ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
On Sep 7, 2011, at 1:05 PM, Leigh Porter wrote:
-----Original Message----- From: Seth Mos [mailto:seth.mos@dds.nl] Sent: 07 September 2011 20:26 To: NANOG Subject: Re: NAT444 or ?
I think you have the numbers off, he started with 1000 users sharing the same IP, since you can only do 62k sessions or so and with a "normal" timeout on those sessions you ran into issues quickly.
The summary is that with anything less then 20 tcp sessions per user simultaneous google maps or earth was problematic. From 15 and downwards almost unsable.
He deducted from testing that about 10 users per IP was a more realistic limit without taking out the entire CGN "experience".
On a personal note, this isn't even taking into question things like broken virus scanners or other software updates that will happily try to do 5 sessions per second, or a msn client lost trying to do 10 per second. The most the windows IP stack will allow on client versions.
The real big issue that will be the downfall of NAT444 is the issue with ACLS and automatic blocklists and the loss of granular access control on that which the ISP has no control of. Which roughly estimates to the internet.
Regards,
Seth
I was thinking of an average of around 100 sessions per user for working out how things scale to start with. It would also be handy to be able to apply sensible limits to new sessions, say limit the number of sessions to a single destination IP address and apply an overall session limit of perhaps 200 sessions per source IP address.
ACLs and blocklists are going to be a problem, perhaps, as LSN becomes more and more common, such things will gradually die out.
I think that such things will kill the NAT444 user experience rather than having the NAT444 user experience problems kill the block lists. The people maintaining said lists are generally trying to protect larger systems from abusers and don't have any strong motivation to preserve the user experience of particular ISPs or particular subsets of users.
Considering that offices, schools etc regularly have far more than 10 users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted these are all different kinds of users, it is just something that operational experience will have to demonstrate.
Yes, but, you are counting individual users whereas at the NAT444 level, what's really being counted is end-customer sites not individual users, so the term "users" is a bit misleading in the context. A given end-customer site may be from 1 to 50 or more individual users.
I would love to avoid NAT444, I do not see a viable way around it at the moment. Unless the Department of Work and Pensions release their /8 that is ;-)
The best mitigation really is to get IPv6 deployed as rapidly and widely as possible. The more stuff can go native IPv6, the less depends on fragile NAT444. Owen
-- Leigh
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
-----Original Message----- From: Owen DeLong [mailto:owen@delong.com] Sent: 08 September 2011 01:22 To: Leigh Porter Cc: Seth Mos; NANOG Subject: Re: NAT444 or ?
Considering that offices, schools etc regularly have far more than 10 users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted these are all different kinds of users, it is just something that operational experience will have to demonstrate.
Yes, but, you are counting individual users whereas at the NAT444 level, what's really being counted is end-customer sites not individual users, so the term "users" is a bit misleading in the context. A given end-customer site may be from 1 to 50 or more individual users.
Indeed, my users are using LTE dongles mostly so I expect they will be single users. At the moment on the WiMAX network I see around 35 sessions from a WiMAX modem on average rising to about 50 at peak times. These are a combination of individual users and "home modems". We had some older modems that had integrated NAT that was broken and locked up the modem at 200 sessions. Then some old base station software died at about 10K sessions. So we monitor these things now..
I would love to avoid NAT444, I do not see a viable way around it at the moment. Unless the Department of Work and Pensions release their /8 that is ;-)
The best mitigation really is to get IPv6 deployed as rapidly and widely as possible. The more stuff can go native IPv6, the less depends on fragile NAT444.
Absolutely. Even things like google maps, if that can be dumped on v6, it'll save a load of sessions from people. The sooner services such as Microsoft Update turn on v6 the better as well. I would also like the CDNs to be able to deliver content in v6 (even if the main page is v4) which again will reduce the traffic that has to traverse any NAT. Soon, I think content providers (and providers of other services on the 'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such. -- Leigh ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
On Sep 8, 2011 1:47 AM, "Leigh Porter" <leigh.porter@ukbroadband.com> wrote:
-----Original Message----- From: Owen DeLong [mailto:owen@delong.com] Sent: 08 September 2011 01:22 To: Leigh Porter Cc: Seth Mos; NANOG Subject: Re: NAT444 or ?
Considering that offices, schools etc regularly have far more than 10 users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted these are all different kinds of users, it is just something that operational experience will have to demonstrate.
Yes, but, you are counting individual users whereas at the NAT444 level, what's really being counted is end-customer sites not individual users, so the term "users" is a bit misleading in the context. A given end-customer site may be from 1 to 50 or more individual users.
Indeed, my users are using LTE dongles mostly so I expect they will be
single users. At the moment on the WiMAX network I see around 35 sessions from a WiMAX modem on average rising to about 50 at peak times. These are a combination of individual users and "home modems".
We had some older modems that had integrated NAT that was broken and
locked up the modem at 200 sessions. Then some old base station software died at about 10K sessions. So we monitor these things now..
I would love to avoid NAT444, I do not see a viable way around it at the moment. Unless the Department of Work and Pensions release their /8 that is ;-)
The best mitigation really is to get IPv6 deployed as rapidly and widely as possible. The more stuff can go native IPv6, the less depends on fragile NAT444.
Absolutely. Even things like google maps, if that can be dumped on v6,
it'll save a load of sessions from people. The sooner services such as Microsoft Update turn on v6 the better as well. I would also like the CDNs to be able to deliver content in v6 (even if the main page is v4) which again will reduce the traffic that has to traverse any NAT.
Soon, I think content providers (and providers of other services on the
'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such.
What do you mean by performance increase? If performance equals latency, v4 will win for a long while still. Cgn does not add measurable latency. Cb
-- Leigh
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
I wonder if the discussion as useful as it is isn't forgetting that the edge of Internet has a stake in getting this right too! This is not just an ISP problem but one where content providers and services that is the users need to get from here to there in good order. So What can users do to encourage ISPs to deploy v6 to them? What can users do to ease the pain in reaching IPv4 only sites once they are on IPv6 tails? Is there not a bit of CPE needed here? What should the CPE do? and not do? should it deprecate NAT/PAT when it receives 1918 allocation from a CGN? and less technically but relevant I think is to ask about cost? who pays? Christian On 8 Sep 2011, at 15:02, Cameron Byrne wrote:
On Sep 8, 2011 1:47 AM, "Leigh Porter" <leigh.porter@ukbroadband.com> wrote:
-----Original Message----- From: Owen DeLong [mailto:owen@delong.com] Sent: 08 September 2011 01:22 To: Leigh Porter Cc: Seth Mos; NANOG Subject: Re: NAT444 or ?
Considering that offices, schools etc regularly have far more than 10 users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted these are all different kinds of users, it is just something that operational experience will have to demonstrate.
Yes, but, you are counting individual users whereas at the NAT444 level, what's really being counted is end-customer sites not individual users, so the term "users" is a bit misleading in the context. A given end-customer site may be from 1 to 50 or more individual users.
Indeed, my users are using LTE dongles mostly so I expect they will be
single users. At the moment on the WiMAX network I see around 35 sessions from a WiMAX modem on average rising to about 50 at peak times. These are a combination of individual users and "home modems".
We had some older modems that had integrated NAT that was broken and
locked up the modem at 200 sessions. Then some old base station software died at about 10K sessions. So we monitor these things now..
I would love to avoid NAT444, I do not see a viable way around it at the moment. Unless the Department of Work and Pensions release their /8 that is ;-)
The best mitigation really is to get IPv6 deployed as rapidly and widely as possible. The more stuff can go native IPv6, the less depends on fragile NAT444.
Absolutely. Even things like google maps, if that can be dumped on v6,
it'll save a load of sessions from people. The sooner services such as Microsoft Update turn on v6 the better as well. I would also like the CDNs to be able to deliver content in v6 (even if the main page is v4) which again will reduce the traffic that has to traverse any NAT.
Soon, I think content providers (and providers of other services on the
'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such.
What do you mean by performance increase? If performance equals latency, v4 will win for a long while still. Cgn does not add measurable latency.
Cb
-- Leigh
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
Can we really push an IPv6 agenda for CDN's when IPv6 routing at high backend levels is still not complete? I certainly don't have the 'clout' to push that, but full routing between Cogent and HE needs to be fixed. Lyle Giese LCR Computer Services, Inc. On 09/08/11 10:04, Christian de Larrinaga wrote:
I wonder if the discussion as useful as it is isn't forgetting that the edge of Internet has a stake in getting this right too! This is not just an ISP problem but one where content providers and services that is the users need to get from here to there in good order.
So
What can users do to encourage ISPs to deploy v6 to them? What can users do to ease the pain in reaching IPv4 only sites once they are on IPv6 tails?
Is there not a bit of CPE needed here? What should the CPE do? and not do? should it deprecate NAT/PAT when it receives 1918 allocation from a CGN? and less technically but relevant I think is to ask about cost? who pays?
Christian
On 8 Sep 2011, at 15:02, Cameron Byrne wrote:
On Sep 8, 2011 1:47 AM, "Leigh Porter"<leigh.porter@ukbroadband.com> wrote:
-----Original Message----- From: Owen DeLong [mailto:owen@delong.com] Sent: 08 September 2011 01:22 To: Leigh Porter Cc: Seth Mos; NANOG Subject: Re: NAT444 or ?
Considering that offices, schools etc regularly have far more than 10 users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted these are all different kinds of users, it is just something that operational experience will have to demonstrate.
Yes, but, you are counting individual users whereas at the NAT444 level, what's really being counted is end-customer sites not individual users, so the term "users" is a bit misleading in the context. A given end-customer site may be from 1 to 50 or more individual users.
Indeed, my users are using LTE dongles mostly so I expect they will be
single users. At the moment on the WiMAX network I see around 35 sessions from a WiMAX modem on average rising to about 50 at peak times. These are a combination of individual users and "home modems".
We had some older modems that had integrated NAT that was broken and
locked up the modem at 200 sessions. Then some old base station software died at about 10K sessions. So we monitor these things now..
I would love to avoid NAT444, I do not see a viable way around it at the moment. Unless the Department of Work and Pensions release their /8 that is ;-)
The best mitigation really is to get IPv6 deployed as rapidly and widely as possible. The more stuff can go native IPv6, the less depends on fragile NAT444.
Absolutely. Even things like google maps, if that can be dumped on v6,
it'll save a load of sessions from people. The sooner services such as Microsoft Update turn on v6 the better as well. I would also like the CDNs to be able to deliver content in v6 (even if the main page is v4) which again will reduce the traffic that has to traverse any NAT.
Soon, I think content providers (and providers of other services on the
'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such.
What do you mean by performance increase? If performance equals latency, v4 will win for a long while still. Cgn does not add measurable latency.
Cb
-- Leigh
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
Can we really push an IPv6 agenda for CDN's when IPv6 routing at high backend levels is still not complete? I certainly don't have the 'clout' to push that, but full routing between Cogent and HE needs to be fixed.
if you are worried about full v4 or v6 or v8-juice routing between cogent and X, for many values of X, then you will never be unworried. randy
On 9/8/11 08:49 , Lyle Giese wrote:
Can we really push an IPv6 agenda for CDN's when IPv6 routing at high backend levels is still not complete? I certainly don't have the 'clout' to push that, but full routing between Cogent and HE needs to be fixed.
It's your job to run your network such that you have connectivity to the destinations your customers want to reach not Cogent's or HE's...
Lyle Giese LCR Computer Services, Inc.
On 09/08/11 10:04, Christian de Larrinaga wrote:
I wonder if the discussion as useful as it is isn't forgetting that the edge of Internet has a stake in getting this right too! This is not just an ISP problem but one where content providers and services that is the users need to get from here to there in good order.
So
What can users do to encourage ISPs to deploy v6 to them? What can users do to ease the pain in reaching IPv4 only sites once they are on IPv6 tails?
Is there not a bit of CPE needed here? What should the CPE do? and not do? should it deprecate NAT/PAT when it receives 1918 allocation from a CGN? and less technically but relevant I think is to ask about cost? who pays?
Christian
On 8 Sep 2011, at 15:02, Cameron Byrne wrote:
On Sep 8, 2011 1:47 AM, "Leigh Porter"<leigh.porter@ukbroadband.com> wrote:
-----Original Message----- From: Owen DeLong [mailto:owen@delong.com] Sent: 08 September 2011 01:22 To: Leigh Porter Cc: Seth Mos; NANOG Subject: Re: NAT444 or ?
Considering that offices, schools etc regularly have far more than 10 users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted these are all different kinds of users, it is just something that operational experience will have to demonstrate.
Yes, but, you are counting individual users whereas at the NAT444 level, what's really being counted is end-customer sites not individual users, so the term "users" is a bit misleading in the context. A given end-customer site may be from 1 to 50 or more individual users.
Indeed, my users are using LTE dongles mostly so I expect they will be
single users. At the moment on the WiMAX network I see around 35 sessions from a WiMAX modem on average rising to about 50 at peak times. These are a combination of individual users and "home modems".
We had some older modems that had integrated NAT that was broken and
locked up the modem at 200 sessions. Then some old base station software died at about 10K sessions. So we monitor these things now..
I would love to avoid NAT444, I do not see a viable way around it at the moment. Unless the Department of Work and Pensions release their /8 that is ;-)
The best mitigation really is to get IPv6 deployed as rapidly and widely as possible. The more stuff can go native IPv6, the less depends on fragile NAT444.
Absolutely. Even things like google maps, if that can be dumped on v6,
it'll save a load of sessions from people. The sooner services such as Microsoft Update turn on v6 the better as well. I would also like the CDNs to be able to deliver content in v6 (even if the main page is v4) which again will reduce the traffic that has to traverse any NAT.
Soon, I think content providers (and providers of other services on the
'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such.
What do you mean by performance increase? If performance equals latency, v4 will win for a long while still. Cgn does not add measurable latency.
Cb
-- Leigh
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
And these 'perceived' routing issues won't be noticed nor are they important to CDN's? I know what my job is, but that may not matter to the CDN's. Reading this thread, I wanted to mention another problem that I feel has an effect on this issue. Lyle On 09/08/11 11:22, Joel jaeggli wrote:
On 9/8/11 08:49 , Lyle Giese wrote:
Can we really push an IPv6 agenda for CDN's when IPv6 routing at high backend levels is still not complete? I certainly don't have the 'clout' to push that, but full routing between Cogent and HE needs to be fixed.
It's your job to run your network such that you have connectivity to the destinations your customers want to reach not Cogent's or HE's...
Lyle Giese LCR Computer Services, Inc.
On 09/08/11 10:04, Christian de Larrinaga wrote:
I wonder if the discussion as useful as it is isn't forgetting that the edge of Internet has a stake in getting this right too! This is not just an ISP problem but one where content providers and services that is the users need to get from here to there in good order.
So
What can users do to encourage ISPs to deploy v6 to them? What can users do to ease the pain in reaching IPv4 only sites once they are on IPv6 tails?
Is there not a bit of CPE needed here? What should the CPE do? and not do? should it deprecate NAT/PAT when it receives 1918 allocation from a CGN? and less technically but relevant I think is to ask about cost? who pays?
Christian
On 8 Sep 2011, at 15:02, Cameron Byrne wrote:
On Sep 8, 2011 1:47 AM, "Leigh Porter"<leigh.porter@ukbroadband.com> wrote:
-----Original Message----- From: Owen DeLong [mailto:owen@delong.com] Sent: 08 September 2011 01:22 To: Leigh Porter Cc: Seth Mos; NANOG Subject: Re: NAT444 or ?
> Considering that offices, schools etc regularly have far more than 10 users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted these are all different kinds of users, it is just something that operational experience will have to demonstrate. > Yes, but, you are counting individual users whereas at the NAT444 level, what's really being counted is end-customer sites not individual users, so the term "users" is a bit misleading in the context. A given end-customer site may be from 1 to 50 or more individual users.
Indeed, my users are using LTE dongles mostly so I expect they will be
single users. At the moment on the WiMAX network I see around 35 sessions from a WiMAX modem on average rising to about 50 at peak times. These are a combination of individual users and "home modems".
We had some older modems that had integrated NAT that was broken and
locked up the modem at 200 sessions. Then some old base station software died at about 10K sessions. So we monitor these things now..
> I would love to avoid NAT444, I do not see a viable way around it at the moment. Unless the Department of Work and Pensions release their /8 that is ;-) >
The best mitigation really is to get IPv6 deployed as rapidly and widely as possible. The more stuff can go native IPv6, the less depends on fragile NAT444.
Absolutely. Even things like google maps, if that can be dumped on v6,
it'll save a load of sessions from people. The sooner services such as Microsoft Update turn on v6 the better as well. I would also like the CDNs to be able to deliver content in v6 (even if the main page is v4) which again will reduce the traffic that has to traverse any NAT.
Soon, I think content providers (and providers of other services on the
'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such.
What do you mean by performance increase? If performance equals latency, v4 will win for a long while still. Cgn does not add measurable latency.
Cb
-- Leigh
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
And these 'perceived' routing issues won't be noticed nor are they important to CDN's? I know what my job is, but that may not matter to the CDN's. Reading this thread, I wanted to mention another problem that I feel has an effect on this issue. Lyle
A very interesting point. In order to save precious CGN resources, it would not be surprising to see some ISPs asking CDNs to provide a private/non-routed behind-CGN leg for local CDN nodes. For this to work, the CGN users would probably have a different set of DNS servers (arguably also with a private/non-routed leg) or some other way to differentiate these CGN clients. Lots of fun in the future debugging that. /JF
On Fri, 09 Sep 2011 11:09:38 EDT, Jean-Francois.TremblayING@videotron.com said:
A very interesting point. In order to save precious CGN resources, it would not be surprising to see some ISPs asking CDNs to provide a private/non-routed behind-CGN leg for local CDN nodes.
For this to work, the CGN users would probably have a different set of DNS servers (arguably also with a private/non-routed leg) or some other way to differentiate these CGN clients. Lots of fun in the future debugging that.
Especially once you have 10 or 15 CDNs doing this, all of which have different rules of engagement. "Akamai requires us to do X, Hulu wants Y, Foobar wants Y and specifically NOT-X..." ;) And then Cogent will get into another peering spat and.... :)
On Friday 09 Sep 2011 16:25:35 Valdis.Kletnieks@vt.edu wrote:
On Fri, 09 Sep 2011 11:09:38 EDT, Jean- Francois.TremblayING@videotron.com said:
A very interesting point. In order to save precious CGN resources, it would not be surprising to see some ISPs asking CDNs to provide a private/non-routed behind-CGN leg for local CDN nodes.
The actual problem here is that everyone assumes it'll be donkey's years before every last web server in the world is on IPv6. If you're a CDN, though, you can solve this problem for your own network right now by deploying IPv6! Akamai says that you need 650 AS to cover 90% of Internet traffic. I propose that effort getting content networks to go dual stack is better used than effort used to work around NAT444. Further, if making your hosting network IPv6 is hard, the answer is surely to give the job to a CDN operator with v6 clue. I actually rather think CDNs are an important way of getting content onto the IPv6 Internet. In my view CDNing (and its sister, application acceleration) is so important to delivering the heavy video and complex web apps that dominate the modern Internet that this should be a killer. Still, breaking the BBC, Hulu, Level(3), Akamai, Limelight, and Google's video services will probably reduce your transit and backhaul bills significantly. Can't say it'll help with customer retention.
For this to work, the CGN users would probably have a different set of DNS servers (arguably also with a private/non-routed leg) or some other way to differentiate these CGN clients. Lots of fun in the future debugging that.
Especially once you have 10 or 15 CDNs doing this, all of which have different rules of engagement. "Akamai requires us to do X, Hulu wants Y, Foobar wants Y and specifically NOT-X..." ;)
And then Cogent will get into another peering spat and.... :)
-- The only thing worse than e-mail disclaimers...is people who send e-mail to lists complaining about them
I can predict the response from the teen dens of the world! What does CGN mean .. Can't Get Nothing! Christian On 9 Sep 2011, at 17:06, Alexander Harrowell wrote:
On Friday 09 Sep 2011 16:25:35 Valdis.Kletnieks@vt.edu wrote:
On Fri, 09 Sep 2011 11:09:38 EDT, Jean- Francois.TremblayING@videotron.com said:
A very interesting point. In order to save precious CGN resources, it would not be surprising to see some ISPs asking CDNs to provide a private/non-routed behind-CGN leg for local CDN nodes.
The actual problem here is that everyone assumes it'll be donkey's years before every last web server in the world is on IPv6.
If you're a CDN, though, you can solve this problem for your own network right now by deploying IPv6! Akamai says that you need 650 AS to cover 90% of Internet traffic. I propose that effort getting content networks to go dual stack is better used than effort used to work around NAT444.
Further, if making your hosting network IPv6 is hard, the answer is surely to give the job to a CDN operator with v6 clue. I actually rather think CDNs are an important way of getting content onto the IPv6 Internet.
In my view CDNing (and its sister, application acceleration) is so important to delivering the heavy video and complex web apps that dominate the modern Internet that this should be a killer.
Still, breaking the BBC, Hulu, Level(3), Akamai, Limelight, and Google's video services will probably reduce your transit and backhaul bills significantly. Can't say it'll help with customer retention.
For this to work, the CGN users would probably have a different set of DNS servers (arguably also with a private/non-routed leg) or some other way to differentiate these CGN clients. Lots of fun in the future debugging that.
Especially once you have 10 or 15 CDNs doing this, all of which have different rules of engagement. "Akamai requires us to do X, Hulu wants Y, Foobar wants Y and specifically NOT-X..." ;)
And then Cogent will get into another peering spat and.... :)
-- The only thing worse than e-mail disclaimers...is people who send e-mail to lists complaining about them
On Sep 9, 2011, at 11:06 PM, Alexander Harrowell wrote:
Further, if making your hosting network IPv6 is hard, the answer is surely to give the job to a CDN operator with v6 clue.
This is a good strategy for payload-type content from unitary sources which lends itself to caching/redistribution; however, Web applications, things which link to/incorporate lots of content, etc. under the control of other organization, and 'active content' are another story, unfortunately. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
-----Original Message----- From: Christian de Larrinaga [mailto:cdel@firsthand.net] Sent: Thursday, September 08, 2011 8:05 AM To: Cameron Byrne Cc: NANOG Subject: what about the users re: NAT444 or ?
I wonder if the discussion as useful as it is isn't forgetting that the edge of Internet has a stake in getting this right too! This is not just an ISP problem but one where content providers and services that is the users need to get from here to there in good order.
So
What can users do to encourage ISPs to deploy v6 to them? What can users do to ease the pain in reaching IPv4 only sites once they are on IPv6 tails?
Is there not a bit of CPE needed here? What should the CPE do? and not do? should it deprecate NAT/PAT when it receives 1918 allocation from a CGN?
Careful with that idea -- people like their in-home network to continue functioning even when their ISP is down or having an outage. Consider a home NAS holding delivering content to the stereo or the television. It is possible to eliminate reliance on the ISP's network and still have the in-home network function, but it's more difficult than just continuing to run NAT44 in the home like today. (Dual Stack-Lite can accomplish this pretty easily, because the IPv4 addresses in the home can be any IPv4 address whatsoever -- which allows the in-home CPE ("B4", in Dual Stack-Lite parlance) to assign any address it wants with its built-in DHCP server.) -d
and less technically but relevant I think is to ask about cost? who pays?
Christian
On 8 Sep 2011, at 15:02, Cameron Byrne wrote:
On Sep 8, 2011 1:47 AM, "Leigh Porter" <leigh.porter@ukbroadband.com> wrote:
-----Original Message----- From: Owen DeLong [mailto:owen@delong.com] Sent: 08 September 2011 01:22 To: Leigh Porter Cc: Seth Mos; NANOG Subject: Re: NAT444 or ?
Considering that offices, schools etc regularly have far more than
users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted
are all different kinds of users, it is just something that operational experience will have to demonstrate.
Yes, but, you are counting individual users whereas at the NAT444 level, what's really being counted is end-customer sites not individual users, so the term "users" is a bit misleading in the context. A given end-customer site may be from 1 to 50 or more individual users.
Indeed, my users are using LTE dongles mostly so I expect they will be single users. At the moment on the WiMAX network I see around 35 sessions from a WiMAX modem on average rising to about 50 at peak times. These are a combination of individual users and "home modems".
We had some older modems that had integrated NAT that was broken and locked up the modem at 200 sessions. Then some old base station software died at about 10K sessions. So we monitor these things now..
I would love to avoid NAT444, I do not see a viable way around it
at
the moment. Unless the Department of Work and Pensions release
that is ;-)
The best mitigation really is to get IPv6 deployed as rapidly and widely as possible. The more stuff can go native IPv6, the less depends on fragile NAT444.
Absolutely. Even things like google maps, if that can be dumped on v6, it'll save a load of sessions from people. The sooner services such as Microsoft Update turn on v6 the better as well. I would also like the CDNs to be able to deliver content in v6 (even if the main page is v4) which again will reduce the traffic that has to traverse any NAT.
Soon, I think content providers (and providers of other services on
10 these their /8 the 'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such.
What do you mean by performance increase? If performance equals latency, v4 will win for a long while still. Cgn does not add measurable latency.
Cb
-- Leigh
This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email
exactly. don't plan to deploy what breaks things for the user edge. there are two issues here 1/ what ISPs do that might break things at the edge 2/ what edge stuff is doing that will break things at the other end edge of a connection It seems a bit odd that ISPs would actively plot to do 1/ whilst they could be making hard cash helping people at the edge avoid 2/ Odd because it adds a 3/ element which is stuff at the edge which will break stuff in the network. Do (some) operators see more money in a 1/2/3/ world? Christian On 8 Sep 2011, at 17:52, Dan Wing wrote:
Is there not a bit of CPE needed here? What should the CPE do? and not do? should it deprecate NAT/PAT when it receives 1918 allocation from a CGN?
Careful with that idea -- people like their in-home network to continue functioning even when their ISP is down or having an outage.
On Sep 8, 2011, at 9:52 AM, Dan Wing wrote:
-----Original Message----- From: Christian de Larrinaga [mailto:cdel@firsthand.net] Sent: Thursday, September 08, 2011 8:05 AM To: Cameron Byrne Cc: NANOG Subject: what about the users re: NAT444 or ?
I wonder if the discussion as useful as it is isn't forgetting that the edge of Internet has a stake in getting this right too! This is not just an ISP problem but one where content providers and services that is the users need to get from here to there in good order.
So
What can users do to encourage ISPs to deploy v6 to them?
Call up and ask for it? Vote with their $$ and their feet?
What can users do to ease the pain in reaching IPv4 only sites once they are on IPv6 tails?
1. Encourage the sites they care about to implement IPv6. 2. Why is being on an IPv6 tail exclusive of being on an IPv4 tail. I would want to be on a dual-stack tail (which is what I currently have).
Is there not a bit of CPE needed here? What should the CPE do? and not do? should it deprecate NAT/PAT when it receives 1918 allocation from a CGN?
Careful with that idea -- people like their in-home network to continue functioning even when their ISP is down or having an outage. Consider a home NAS holding delivering content to the stereo or the television. It is possible to eliminate reliance on the ISP's network and still have the in-home network function, but it's more difficult than just continuing to run NAT44 in the home like today. (Dual Stack-Lite
One can do that with or without NAT. This claim that one cannot keep a network running without a service provider connected if you don't run NAT is a myth of dubious origin.
can accomplish this pretty easily, because the IPv4 addresses in the home can be any IPv4 address whatsoever -- which allows the in-home CPE ("B4", in Dual Stack-Lite parlance) to assign any address it wants with its built-in DHCP server.)
There are other ways to accomplish this as well.
-d
and less technically but relevant I think is to ask about cost? who pays?
In some cases, ISPs will provide new CPE to their end users. In other cases, end-users will be expected to pay to upgrade their own. Owen
Christian
On 8 Sep 2011, at 15:02, Cameron Byrne wrote:
On Sep 8, 2011 1:47 AM, "Leigh Porter" <leigh.porter@ukbroadband.com> wrote:
-----Original Message----- From: Owen DeLong [mailto:owen@delong.com] Sent: 08 September 2011 01:22 To: Leigh Porter Cc: Seth Mos; NANOG Subject: Re: NAT444 or ?
Considering that offices, schools etc regularly have far more than
users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted
are all different kinds of users, it is just something that operational experience will have to demonstrate.
Yes, but, you are counting individual users whereas at the NAT444 level, what's really being counted is end-customer sites not individual users, so the term "users" is a bit misleading in the context. A given end-customer site may be from 1 to 50 or more individual users.
Indeed, my users are using LTE dongles mostly so I expect they will be single users. At the moment on the WiMAX network I see around 35 sessions from a WiMAX modem on average rising to about 50 at peak times. These are a combination of individual users and "home modems".
We had some older modems that had integrated NAT that was broken and locked up the modem at 200 sessions. Then some old base station software died at about 10K sessions. So we monitor these things now..
I would love to avoid NAT444, I do not see a viable way around it
at
the moment. Unless the Department of Work and Pensions release
that is ;-)
The best mitigation really is to get IPv6 deployed as rapidly and widely as possible. The more stuff can go native IPv6, the less depends on fragile NAT444.
Absolutely. Even things like google maps, if that can be dumped on v6, it'll save a load of sessions from people. The sooner services such as Microsoft Update turn on v6 the better as well. I would also like the CDNs to be able to deliver content in v6 (even if the main page is v4) which again will reduce the traffic that has to traverse any NAT.
Soon, I think content providers (and providers of other services on
10 these their /8 the 'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such.
What do you mean by performance increase? If performance equals latency, v4 will win for a long while still. Cgn does not add measurable latency.
Cb
-- Leigh
This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email
One can do that with or without NAT. This claim that one cannot keep a network running without a service provider connected if you don't run NAT is a myth of dubious origin.
If the hosts are running DHCP, and the ISP is running the DHCP server? I guess they will fall back (after a while) to link-local and continue on their merry way.
can accomplish this pretty easily, because the IPv4 addresses in the home can be any IPv4 address whatsoever -- which allows the in-home CPE ("B4", in Dual Stack-Lite parlance) to assign any address it wants with its built-in DHCP server.)
There are other ways to accomplish this as well.
-d
-d
and less technically but relevant I think is to ask about cost? who pays?
In some cases, ISPs will provide new CPE to their end users. In other cases, end-users will be expected to pay to upgrade their own.
Owen
Christian
On 8 Sep 2011, at 15:02, Cameron Byrne wrote:
On Sep 8, 2011 1:47 AM, "Leigh Porter"
wrote:
-----Original Message----- From: Owen DeLong [mailto:owen@delong.com] Sent: 08 September 2011 01:22 To: Leigh Porter Cc: Seth Mos; NANOG Subject: Re: NAT444 or ?
> Considering that offices, schools etc regularly have far more
users per IP, I think this limit is a little low. I've happily had around 300 per public IP address on a large WiFi network, granted
are all different kinds of users, it is just something that operational experience will have to demonstrate. > Yes, but, you are counting individual users whereas at the NAT444 level, what's really being counted is end-customer sites not individual users, so the term "users" is a bit misleading in the context. A given end-customer site may be from 1 to 50 or more individual users.
Indeed, my users are using LTE dongles mostly so I expect they will be single users. At the moment on the WiMAX network I see around 35 sessions from a WiMAX modem on average rising to about 50 at peak times. These are a combination of individual users and "home modems".
We had some older modems that had integrated NAT that was broken and locked up the modem at 200 sessions. Then some old base station software died at about 10K sessions. So we monitor these things now..
> I would love to avoid NAT444, I do not see a viable way around
it at
the moment. Unless the Department of Work and Pensions release
10 these their /8
that is ;-) >
The best mitigation really is to get IPv6 deployed as rapidly and widely as possible. The more stuff can go native IPv6, the less depends on fragile NAT444.
Absolutely. Even things like google maps, if that can be dumped on v6, it'll save a load of sessions from people. The sooner services such as Microsoft Update turn on v6 the better as well. I would also like
<leigh.porter@ukbroadband.com> than the
to be able to deliver content in v6 (even if the main page is v4) which again will reduce the traffic that has to traverse any NAT.
Soon, I think content providers (and providers of other services
on
CDNs the
'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such.
What do you mean by performance increase? If performance equals latency, v4 will win for a long while still. Cgn does not add measurable latency.
Cb
-- Leigh
This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email
On Sep 13, 2011, at 10:18 PM, Dan Wing wrote:
One can do that with or without NAT. This claim that one cannot keep a network running without a service provider connected if you don't run NAT is a myth of dubious origin.
If the hosts are running DHCP, and the ISP is running the DHCP server? I guess they will fall back (after a while) to link-local and continue on their merry way.
That's some pretty big IFs. Even if I were using DHCP to get the prefix from my service provider via DHCP-PD, I'd back-stop that with some form of local DHCP server and deal with the need for manual intervention when the provider renumbered me. In my experience, getting renumbered is a rare enough experience that I don't pay Comcast $60/year for a static address. Owen
can accomplish this pretty easily, because the IPv4 addresses in the home can be any IPv4 address whatsoever -- which allows the in-home CPE ("B4", in Dual Stack-Lite parlance) to assign any address it wants with its built-in DHCP server.)
There are other ways to accomplish this as well.
-d
-d
and less technically but relevant I think is to ask about cost? who pays?
In some cases, ISPs will provide new CPE to their end users. In other cases, end-users will be expected to pay to upgrade their own.
Owen
Christian
On 8 Sep 2011, at 15:02, Cameron Byrne wrote:
On Sep 8, 2011 1:47 AM, "Leigh Porter"
wrote:
> -----Original Message----- > From: Owen DeLong [mailto:owen@delong.com] > Sent: 08 September 2011 01:22 > To: Leigh Porter > Cc: Seth Mos; NANOG > Subject: Re: NAT444 or ? > >> Considering that offices, schools etc regularly have far more
> users per IP, I think this limit is a little low. I've happily had > around 300 per public IP address on a large WiFi network, granted
> are all different kinds of users, it is just something that operational > experience will have to demonstrate. >> > Yes, but, you are counting individual users whereas at the NAT444 > level, what's really being counted is end-customer sites not individual > users, so the term > "users" is a bit misleading in the context. A given end-customer site > may be from 1 to 50 or more individual users.
Indeed, my users are using LTE dongles mostly so I expect they will be single users. At the moment on the WiMAX network I see around 35 sessions from a WiMAX modem on average rising to about 50 at peak times. These are a combination of individual users and "home modems".
We had some older modems that had integrated NAT that was broken and locked up the modem at 200 sessions. Then some old base station software died at about 10K sessions. So we monitor these things now..
> >> I would love to avoid NAT444, I do not see a viable way around it at > the moment. Unless the Department of Work and Pensions release
10 these their /8
> that is ;-) >> > > The best mitigation really is to get IPv6 deployed as rapidly and > widely as possible. The more stuff can go native IPv6, the less depends > on fragile NAT444.
Absolutely. Even things like google maps, if that can be dumped on v6, it'll save a load of sessions from people. The sooner services such as Microsoft Update turn on v6 the better as well. I would also like
<leigh.porter@ukbroadband.com> than the
to be able to deliver content in v6 (even if the main page is v4) which again will reduce the traffic that has to traverse any NAT.
Soon, I think content providers (and providers of other services
on
CDNs the
'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such.
What do you mean by performance increase? If performance equals latency, v4 will win for a long while still. Cgn does not add measurable latency.
Cb
-- Leigh
This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email
On Thursday, September 08, 2011 04:48:16 PM Leigh Porter wrote:
Soon, I think content providers (and providers of other services on the 'net) will roll v6 because of the performance increase as v6 will not have to traverse all this NAT and be subject to session limits, timeouts and such.
I certainly hope so - let's hope ISP's go out and deploy v6 beyond the core, as content providers deploy v6 for their content, rather have a stand-off between both ends of fence on who should move first. Mark.
However these are with a very high address-sharing ratio (several thousands users per address). Using a sparser density (<= 64 users per address) is likely to show much less dramatic user impacts.
I think you have the numbers off, he started with 1000 users sharing the same IP, since you can only do 62k sessions or so
These numbers were not off. From page 19: "...we should assign at least 1000 [..] ports per customer to assure good performance of IPv4 applications" "At least 1000 ports per customers" is roughly the same than "less than 64 users per address" as I stated above. Btw, 90% of subscribers have less than 100 active connections at any time, if I read these tiny graphs correctly: http://www.wand.net.nz/~salcock/pam2009_final.pdf
and with a "normal" timeout on those sessions you ran into issues quickly.
Agreed for UDP, but most of these sessions are TCP, which arguably time out rather rapidly after a FIN and an extra hold time. Normal duration of a TCP session is usually under a few seconds. This study saw an average connection time of 8 seconds for DSL, but it's from 2004. http://www.google.com/#q=A+Comparative+Study+of+TCP/IP+Traffic+Behavior+in+B... /JF
On 9/7/2011 3:24 PM, Seth Mos wrote:
I think you have the numbers off, he started with 1000 users sharing the same IP, since you can only do 62k sessions or so and with a "normal" timeout on those sessions you ran into issues quickly.
Remember that a TCP session is defined not just by the port, but by the combination of source address:source port:destination address:destination port. So that's 62k sessions *per destination* per ip address. In theory, this particular performance problem should only arise when the NAT gear insists on a unique port per session (which is common, but unnecessary) or when a particular destination is inordinately popular; the latter problem could be addressed by increasing the number of addresses that facebook.com and google.com resolve to. I'm not advocating CGN; my point is not that this problem *should* be solved, merely that it *can* be. -Dave
-----Original Message----- From: David Israel [mailto:davei@otd.com] Sent: 07 September 2011 21:23 To: nanog@nanog.org Subject: Re: NAT444 or ?
I think you have the numbers off, he started with 1000 users sharing
On 9/7/2011 3:24 PM, Seth Mos wrote: the same IP, since you can only do 62k sessions or so and with a "normal" timeout on those sessions you ran into issues quickly.
Remember that a TCP session is defined not just by the port, but by the combination of source address:source port:destination address:destination port. So that's 62k sessions *per destination* per ip address. In theory, this particular performance problem should only arise when the NAT gear insists on a unique port per session (which is common, but unnecessary) or when a particular destination is inordinately popular; the latter problem could be addressed by increasing the number of addresses that facebook.com and google.com resolve to.
Good point, but aside from these scaling issues which I expect can be resolved to a point, the more serious issue, I think, is applications that just do not work with double NAT. Now, I have not conducted any serious research into this, but it seems that draft-donley-nat444-impacts does appear to have highlight issues that may have been down to implementation. Other simple tricks such as ensuring that your own internal services such as DNS are available without traversing NAT also help. Certainly some more work can be done in this area, but I fear that the only way a real idea as to how much NAT444 really doe break things will be operational experience. -- Leigh ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
As HTTP seems to be a major factor causing a lot of short lived connections, and several large ISPs have demonstrated that large scale transparent HTTP proxies seem to work just fine, you could also move the IPv4 port 80 traffic from the CGN to a transparent HTTP proxy. As well as any benefits from caching keeping connections local it can also combine 1000 users trying to load facebook in to a handful of persistent connections to the facebook servers. The proxy can of course also have its own global IPv4 address rather than going through the NAT, I have no experience with large scale HTTP proxy deployments but I strongly suspect a single HTTP proxy can handle traffic for a lot more users than low hundreds currently being suggested for NAT444! and can be scaled out separately if required. As an end user this is probably a little worse with HTTP coming from a different IP address to everything else, but not that much worse. As a provider it may be much easier to scale to larger numbers of customers. The proxy can also take IPv4-only users to a dual stacked site over IPv6, as I am under no illusions that even with IPv6 to every customer you will still have customers behind IPv4-only NAT routers they bought themselves for quite a while. With some DNS tricks this might be useful for those users reaching IPv6-only sites, however it would probably be better if they were unable to reach those sites at all to give them an incentive to fix their IPv6. On 7 September 2011 21:37, Leigh Porter <leigh.porter@ukbroadband.com> wrote:
Other simple tricks such as ensuring that your own internal services such as DNS are available without traversing NAT also help.
As obvious as this probably is, i'm sure someone will overlook it! Also other services such as providers with CDN nodes in their network may want to talk to the CDN operator about having those connected to directly from the internal addresses to avoid traversing the NAT, and I'm sure there are other services as well. - Mike
When you need to pile up this amount of trickery to make something work, it's probably high time for letting the thing die :-) Warm regards Carlos On Thu, Sep 8, 2011 at 8:33 AM, Mike Jones <mike@mikejones.in> wrote:
As HTTP seems to be a major factor causing a lot of short lived connections, and several large ISPs have demonstrated that large scale transparent HTTP proxies seem to work just fine, you could also move the IPv4 port 80 traffic from the CGN to a transparent HTTP proxy. As well as any benefits from caching keeping connections local it can also combine 1000 users trying to load facebook in to a handful of persistent connections to the facebook servers. The proxy can of course also have its own global IPv4 address rather than going through the NAT, I have no experience with large scale HTTP proxy deployments but I strongly suspect a single HTTP proxy can handle traffic for a lot more users than low hundreds currently being suggested for NAT444! and can be scaled out separately if required.
As an end user this is probably a little worse with HTTP coming from a different IP address to everything else, but not that much worse. As a provider it may be much easier to scale to larger numbers of customers. The proxy can also take IPv4-only users to a dual stacked site over IPv6, as I am under no illusions that even with IPv6 to every customer you will still have customers behind IPv4-only NAT routers they bought themselves for quite a while. With some DNS tricks this might be useful for those users reaching IPv6-only sites, however it would probably be better if they were unable to reach those sites at all to give them an incentive to fix their IPv6.
On 7 September 2011 21:37, Leigh Porter <leigh.porter@ukbroadband.com> wrote:
Other simple tricks such as ensuring that your own internal services such as DNS are available without traversing NAT also help.
As obvious as this probably is, i'm sure someone will overlook it! Also other services such as providers with CDN nodes in their network may want to talk to the CDN operator about having those connected to directly from the internal addresses to avoid traversing the NAT, and I'm sure there are other services as well.
- Mike
-- -- ========================= Carlos M. Martinez-Cagnazzo http://www.labs.lacnic.net =========================
-----Original Message----- From: Carlos Martinez-Cagnazzo [mailto:carlosm3011@gmail.com] Sent: 09 September 2011 05:10 To: Mike Jones Cc: nanog@nanog.org Subject: Re: NAT444 or ?
When you need to pile up this amount of trickery to make something work, it's probably high time for letting the thing die :-)
Warm regards
Carlos
You could say the same thing about NAT44 from the very start! IPv4 just needs to die sooner rather than later. For now though, there is a good many years of trickery left ;-) -- Leigh Porter ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
-----Original Message----- From: Leigh Porter [mailto:leigh.porter@ukbroadband.com] Sent: Wednesday, September 07, 2011 1:38 PM To: David Israel; nanog@nanog.org Subject: RE: NAT444 or ?
-----Original Message----- From: David Israel [mailto:davei@otd.com] Sent: 07 September 2011 21:23 To: nanog@nanog.org Subject: Re: NAT444 or ?
I think you have the numbers off, he started with 1000 users sharing
On 9/7/2011 3:24 PM, Seth Mos wrote: the same IP, since you can only do 62k sessions or so and with a "normal" timeout on those sessions you ran into issues quickly.
Remember that a TCP session is defined not just by the port, but by the combination of source address:source port:destination address:destination port. So that's 62k sessions *per destination* per ip address. In theory, this particular performance problem should only arise when the NAT gear insists on a unique port per session (which is common, but unnecessary) or when a particular destination is inordinately popular; the latter problem could be addressed by increasing the number of addresses that facebook.com and google.com resolve to.
Good point, but aside from these scaling issues which I expect can be resolved to a point, the more serious issue, I think, is applications that just do not work with double NAT. Now, I have not conducted any serious research into this, but it seems that draft-donley-nat444- impacts does appear to have highlight issues that may have been down to implementation.
Draft-donley-nat444-impacts conflates bandwidth constraints with CGN with in-home NAT. Until those are separated and then analyzed carefully, it is harmful to draw conclusions such as "NAT444 bad; NAT44 good".
Other simple tricks such as ensuring that your own internal services such as DNS are available without traversing NAT also help.
Yep. But some users want to use other DNS servers for performance (e.g., Google's or OpenDNS servers, especially considering they could point the user at a 'better' (closer) CDN based on Client IP), to avoid ISP DNS hijacking, or for content control (e.g., "parental control" of DNS hostnames). That traffic will, necessarily, traverse the CGN. To avoid users burning through their UDP port allocation for those DNS queries it is useful for the CGN to have short timeouts for port 53.
Certainly some more work can be done in this area, but I fear that the only way a real idea as to how much NAT444 really doe break things will be operational experience.
Yep. (Same as everything else.) -d
-- Leigh
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
Good point, but aside from these scaling issues which I expect can be resolved to a point, the more serious issue, I think, is applications that just do not work with double NAT. Now, I have not conducted any serious research into this, but it seems that draft-donley-nat444- impacts does appear to have highlight issues that may have been down to implementation.
Draft-donley-nat444-impacts conflates bandwidth constraints with CGN with in-home NAT. Until those are separated and then analyzed carefully, it is harmful to draw conclusions such as "NAT444 bad; NAT44 good".
Continuing to make this claim does not make it any more true. Draft-donley took networks and measured their real-world functionality without NAT444, then, added NAT444 and repeated the same tests. Regardless of the underlying issue(s), the addition of NAT444 to the mix resulted in the forms of service degradation enumerated in the draft. Further, I would not ever say "NAT444 bad; NAT44 good". I would say, rather, "NAT44 bad, NAT444 worse". I think that's a pretty safe and non- harmful thing to say.
Other simple tricks such as ensuring that your own internal services such as DNS are available without traversing NAT also help.
Yep. But some users want to use other DNS servers for performance (e.g., Google's or OpenDNS servers, especially considering they could point the user at a 'better' (closer) CDN based on Client IP), to avoid ISP DNS hijacking, or for content control (e.g., "parental control" of DNS hostnames). That traffic will, necessarily, traverse the CGN. To avoid users burning through their UDP port allocation for those DNS queries it is useful for the CGN to have short timeouts for port 53.
If the user chooses to use a DNS server on the other side of a NAT, then, they are choosing to inflict whatever damage upon themselves. I'm not saying that short UDP/53 timeouts are a bad idea, but, I am saying that the more stuff you funnel through an LSN at the carrier, the more stuff you will see break. This would lead me to want to avoid funneling anything through said NAT which I could avoid. Then again, I run my own authoritative and recursive nameservers in my home and don't use any NAT at all, so, perhaps my perspective is different from others.
Certainly some more work can be done in this area, but I fear that the only way a real idea as to how much NAT444 really doe break things will be operational experience.
Yep. (Same as everything else.)
I'm sure that will happen soon enough. I, for one, am not looking forward to the experience. Owen
-----Original Message----- From: Owen DeLong [mailto:owen@delong.com] Sent: Tuesday, September 13, 2011 9:43 PM To: Dan Wing Cc: 'Leigh Porter'; 'David Israel'; nanog@nanog.org Subject: Re: NAT444 or ?
Good point, but aside from these scaling issues which I expect can
be
resolved to a point, the more serious issue, I think, is applications that just do not work with double NAT. Now, I have not conducted any serious research into this, but it seems that draft-donley-nat444- impacts does appear to have highlight issues that may have been down to implementation.
Draft-donley-nat444-impacts conflates bandwidth constraints with CGN with in-home NAT. Until those are separated and then analyzed carefully, it is harmful to draw conclusions such as "NAT444 bad; NAT44 good".
Continuing to make this claim does not make it any more true.
Draft-donley took networks and measured their real-world functionality without NAT444, then, added NAT444 and repeated the same tests. Regardless of the underlying issue(s), the addition of NAT444 to the mix resulted in the forms of service degradation enumerated in the draft.
I disagree it reached that conclusion. That may have been its intent.
Further, I would not ever say "NAT444 bad; NAT44 good". I would say, rather, "NAT44 bad, NAT444 worse". I think that's a pretty safe and non-harmful thing to say.
Yes, your statement is completely accurate. I agree that IPv4 address sharing causes additional problems (which encompasses all forms of IPv4 address sharing), and CGN causes additional problems.
Other simple tricks such as ensuring that your own internal services such as DNS are available without traversing NAT also help.
Yep. But some users want to use other DNS servers for performance (e.g., Google's or OpenDNS servers, especially considering they could point the user at a 'better' (closer) CDN based on Client IP), to avoid ISP DNS hijacking, or for content control (e.g., "parental control" of DNS hostnames). That traffic will, necessarily, traverse the CGN. To avoid users burning through their UDP port allocation for those DNS queries it is useful for the CGN to have short timeouts for port 53.
If the user chooses to use a DNS server on the other side of a NAT, then, they are choosing to inflict whatever damage upon themselves. I'm not saying that short UDP/53 timeouts are a bad idea, but, I am saying that the more stuff you funnel through an LSN at the carrier, the more stuff you will see break. This would lead me to want to avoid funneling anything through said NAT which I could avoid. Then again, I run my own authoritative and recursive nameservers in my home and don't use any NAT at all, so, perhaps my perspective is different from others.
Yeah, you are probably of about 1000 or maybe 3000 people in the world that do that. Seems to be a minority.
Certainly some more work can be done in this area, but I fear that the only way a real idea as to how much NAT444 really doe break things will be operational experience.
Yep. (Same as everything else.)
I'm sure that will happen soon enough. I, for one, am not looking forward to the experience.
Neither am I. But if major content providers cannot provide AAAA on their properties, and if ISPs and CPE vendors do not make IPv6 available and working, and if web browsers don't adopt faster fallback to IPv4 when IPv6 is borked .... We will all be behind NATs. -d
David Israel wrote, on 09/07/2011 04:21 PM:
In theory, this particular performance problem should only arise when the NAT gear insists on a unique port per session (which is common, but unnecessary)
What you're describing is known as "endpoint-independent mapping" behaviour. It is good for not breaking applications, not so good for scalability. RFC 4787 section 4.1 makes it a MUST. Simon -- DTN made easy, lean, and smart --> http://postellation.viagenie.ca NAT64/DNS64 open-source --> http://ecdysis.viagenie.ca STUN/TURN server --> http://numb.viagenie.ca
-----Original Message----- From: Simon Perreault [mailto:simon.perreault@viagenie.ca] Sent: Wednesday, September 07, 2011 2:29 PM To: nanog@nanog.org Subject: Re: NAT444 or ?
David Israel wrote, on 09/07/2011 04:21 PM:
In theory, this particular performance problem should only arise when the NAT gear insists on a unique port per session (which is common, but unnecessary)
What you're describing is known as "endpoint-independent mapping" behaviour. It is good for not breaking applications, not so good for scalability. RFC 4787 section 4.1 makes it a MUST.
There are two dimensions of that scalability, of course: Endpoint-independent mapping means better scaling of the NAT itself, because it stores less state (slightly less memory for each active mapping and slightly less per-packet processing). This savings is exchanged for worse IPv4 utilization -- which I agree is not so good for scalability. -d
-----Original Message----- From: Jean-Francois.TremblayING@videotron.com [mailto:Jean- Francois.TremblayING@videotron.com] Sent: Wednesday, September 07, 2011 10:06 AM To: dr@cluenet.de Cc: nanog@nanog.org Subject: Re: NAT444 or ?
On Wed, Sep 07, 2011 at 12:16:28PM +0200, Randy Bush wrote:
I'm going to have to deploy NAT444 with dual-stack real soon now. you may want to review the presentations from last week's apnic meeting in busan. real mesurements. sufficiently scary that people who were heavily pushing nat444 for the last two years suddenly started to say "it was not me who pushed nat444, it was him!" as if none of us had a memory.
Hm, I fail to find relevant slides discussing that. Could you please point us to those?
I had the same question. I found Miyakawa-san's presentation has some dramatic examples of CGN NAT444 effects using Google Maps: http://meetings.apnic.net/__data/assets/file/0011/38297/Miyakawa-APNIC- KEYNOTE-IPv6-2011-8.pptx.pdf
However these are with a very high address-sharing ratio (several thousands users per address). Using a sparser density (<= 64 users per address) is likely to show much less dramatic user impacts.
Try it at home. With aggressive usage of Microsoft's Terraserver, mapquest, or google maps, I'm able to burn through 120 or so TCP connections. Move the map around, zoom in/out, enable/disable traffic, switch between satellite and map and overlay, repeat those steps 2-3 times. Don't be slow and don't wait for everything to paint. Or crash your browser and when it restarts watch how many connections it makes to re-open all your tabs. I understand iTunes opens lots of connections, but I haven't looked at that. To experiment with limited ports at home, load 3rd party firmware onto your NAT -- most of them allow controlling the number of mappings (and by default, have higher limits than stock firmware). Or consume a bunch of your mappings with a script (such as the brain-dead Perl script below) and then start your testing. Some NATs and some servers will kill the TCP sessions after inactivity (which is why I describe the script as brain-dead). -d ---- #!/usr/bin/perl -w use IO::Socket; $max = shift(@ARGV); my $count = 0; my $host = shift(@ARGV) || "www.example.com"; my @remote; print "connecting to $host\n"; while ($count < $max) { printf ("connecting...(%d)\n", $count+1); $remote[$count] = IO::Socket::INET->new( Proto => "tcp", PeerAddr => $host, PeerPort => "80") or warn "got an error"; $count++; } print "press Return to exit\n"; my $junk = <STDIN>; $count = 0; while ($count < $max) { close $remote[$count]; $count++; } exit;
-----Original Message----- From: Randy Bush [mailto:randy@psg.com] Sent: Wednesday, September 07, 2011 3:16 AM To: Leigh Porter Cc: North American Network Operators' Group Subject: Re: NAT444 or ?
I'm going to have to deploy NAT444 with dual-stack real soon now.
you may want to review the presentations from last week's apnic meeting in busan. real mesurements. sufficiently scary that people who were heavily pushing nat444 for the last two years suddenly started to say "it was not me who pushed nat444, it was him!" as if none of us had a memory.
Many of the problems are due to IPv4 address sharing, which will be problems for A+P, CGN, HTTP proxies, and other address sharing technologies. RFC6269 discusses most (or all) of those problems. There are workarounds to those problems, but most are not pretty. The solution is IPv6. -d
On Friday, September 09, 2011 01:44:08 AM Dan Wing wrote:
Many of the problems are due to IPv4 address sharing, which will be problems for A+P, CGN, HTTP proxies, and other address sharing technologies. RFC6269 discusses most (or all) of those problems. There are workarounds to those problems, but most are not pretty. The solution is IPv6.
I do expect some of these workarounds to be vendor and/or platform specific, as more units are deployed and the industry simply can't keep up with the various topologies and customer elasticities ISP's have to maintain. We're already seeing evidence of this as we discuss NAT64 options with vendors, particularly in the area of scale and customer experience perceptions. Mark.
participants (25)
-
Alexander Harrowell -
Arturo Servin -
Cameron Byrne -
Carlos Martinez-Cagnazzo -
Christian de Larrinaga -
Dan Wing -
Daniel Roesen -
David Israel -
Dobbins, Roland -
Dorn Hetzel -
Douglas Otis -
Geoff Huston -
Jean-Francois.TremblayING＠videotron.com -
Joel jaeggli -
Leigh Porter -
Lyle Giese -
Mark Tinka -
Mike Jones -
Owen DeLong -
Randy Bush -
Serge Vautour -
Seth Mos -
Simon Perreault -
Tore Anderson -
Valdis.Kletnieks＠vt.edu