Ah yes, public-but-not-external IPv4 addresses ... I wish a stern note saying don't do that was feasible ... /TJ ------Original Message------ From: Nathan Ward To: nanog list Subject: Re: IPv6 routing /48s Sent: Nov 19, 2008 15:02 On 20/11/2008, at 4:06 AM, Florian Weimer wrote:
* Michael Sinatra:
And it just reinforces the fear that people have against putting AAAA records in DNS for their publicly-accessible resources, especially www.
Won't current Windows clients work just fine in this case?
I have no idea what a fix should look like for some of the non-Windows systems I care about, unfortunately.
No, unfortunately broken 6to4 auto-configuration (ie, in Vista, XPSP2, when on a non-RFC1918 IP address) breaks, and you get 90s timeouts before falling back to IPv4/A. Works fine most of the time, however when that non-RFC1918 address is behind NAT, or some sort of packet filter, then it doesn't work so well, and the client does not have a way to detect that reliably. -- Nathan Ward Sent from my Verizon Wireless BlackBerry
On 20/11/2008, at 10:11 AM, trejrco@gmail.com wrote:
Ah yes, public-but-not-external IPv4 addresses ... I wish a stern note saying don't do that was feasible ...
What people do with their addresses is their business. The problem here is XPSP2/Vista assuming that non-RFC1918 = unfiltered/ unNATed for the purposes of 6to4. Well, deeper problem is that they're using 6to4 on an end host I suppose - it's supposed to be used on routers. I was going to write up a qualification mechanism for it so it could detect if 6to4 was OK or not, but code is already out there on umpteen million PCs that aren't going to do their patches. I still plan to.. hopefully I'll get around to it when I feel a bit less jaded :-) -- Nathan Ward
Nathan Ward wrote:
The problem here is XPSP2/Vista assuming that non-RFC1918 = unfiltered/unNATed for the purposes of 6to4. Well, deeper problem is that they're using 6to4 on an end host I suppose - it's supposed to be used on routers.
While I don't doubt that the 6to4 is broken in such circumstances, how many IPv6 content providers are using 6to4 addressing and not 2001:: addressing? 6to4 by default on xp and vista, in my experience, is only used if a) talking to another 6to4 address or b) there is no IPv4 address available. 6to4 never seemed like a viable method for content providing, though its use at the eyeball layer is somewhat iffy given that it's primary use is for other 6to4 addresses. If prefix policies are altered to use it for 2001:: addressing, problems start arising quickly. A good example is that traceroutes through my he.net tunnel using 6to4 source addresses do not get replies through he.net's network, presumably due to their routers not being 6to4 aware and having no route to respond. Responses pick up again after picking up a network such as NTT that is 6to4 aware. My 2001:: addressing works just fine the entire route. I'm sure there's quite a few networks that aren't 6to4 aware, hindering 6to4 connectivity to non-6to4 addresses. Jack
On 11/19/08 14:05, Jack Bates wrote:
Nathan Ward wrote:
The problem here is XPSP2/Vista assuming that non-RFC1918 = unfiltered/unNATed for the purposes of 6to4. Well, deeper problem is that they're using 6to4 on an end host I suppose - it's supposed to be used on routers.
While I don't doubt that the 6to4 is broken in such circumstances, how many IPv6 content providers are using 6to4 addressing and not 2001:: addressing?
[other references to 2001:: addressing snipped] I hope I am not being toooo picky here, and I realize this is not part of your main point... If your reference to 2001:: addressing simply means "non-tunneled, globally routable IPv6 addressing," then I suppose it is okay. But please note that there is now a lot of native (non-tunneled), globally routable IPv6 addressing that is outside of 2001::/16. ARIN, for example, is allocating blocks out of 2607::/16 and there are quite a large number of prefixes elsewhere in the designated globally-routable 2000::/3 that are *not* 6to4 addresses. The reason I bring this up is that I have already seen certain applications, such as one for registering AAAA records for DNS servers in a certain TLD, that don't allow anything other than 2001::/16. (Fortunately that application was fixed quickly when those responsible were notified.) Just making sure others aren't careening toward making the same mistake. michael
Yes, always worth reminding people: 2000::/3 is the currently active "Global Unicast" pool ... and that doesn't mean native IPv6 (2002::/16 = 6to4, 2001::/32 = Teredo ... ISATAP may be in play, and not discretely indicated in prefix side of address ... and non-auto tunnels cold be mentioned here as well, using global-scope prefixes ) Back to the topic at hand, and in the subject line, global routing of arbitrary /48s is far from guaranteed. /32 is the default cutoff (routing for /35s probably pretty reliable as well) Up to /48s for Critical Infra / Micro-Allocations and PI-designated blocks ... anything else AFAIK is quite possibly troublesome today. /TJ
-----Original Message----- From: Michael Sinatra [mailto:michael@rancid.berkeley.edu] Sent: Wednesday, November 19, 2008 4:16 PM To: Jack Bates Cc: nanog list Subject: Re: IPv6 routing /48s
On 11/19/08 14:05, Jack Bates wrote:
Nathan Ward wrote:
The problem here is XPSP2/Vista assuming that non-RFC1918 = unfiltered/unNATed for the purposes of 6to4. Well, deeper problem is that they're using 6to4 on an end host I suppose - it's supposed to be used on routers.
While I don't doubt that the 6to4 is broken in such circumstances, how many IPv6 content providers are using 6to4 addressing and not 2001:: addressing?
[other references to 2001:: addressing snipped]
I hope I am not being toooo picky here, and I realize this is not part of your main point...
If your reference to 2001:: addressing simply means "non-tunneled, globally routable IPv6 addressing," then I suppose it is okay. But please note that there is now a lot of native (non-tunneled), globally routable IPv6 addressing that is outside of 2001::/16. ARIN, for example, is allocating blocks out of 2607::/16 and there are quite a large number of prefixes elsewhere in the designated globally-routable 2000::/3 that are *not* 6to4 addresses.
The reason I bring this up is that I have already seen certain applications, such as one for registering AAAA records for DNS servers in a certain TLD, that don't allow anything other than 2001::/16. (Fortunately that application was fixed quickly when those responsible were notified.) Just making sure others aren't careening toward making the same mistake.
michael
Michael Sinatra wrote:
If your reference to 2001:: addressing simply means "non-tunneled, globally routable IPv6 addressing," then I suppose it is okay. But please note that there is now a lot of native (non-tunneled), globally routable IPv6 addressing that is outside of 2001::/16. ARIN, for example, is allocating blocks out of 2607::/16 and there are quite a large number of prefixes elsewhere in the designated globally-routable 2000::/3 that are *not* 6to4 addresses.
heh, these days, lots of it is still tunneled, though through more conventional means. But yes, I should have been more clear. Just too used to seeing 2001::/16 and too lazy to figure out the proper terminology (The original topic is something I've been heavily testing lately while I figure out how closely I can get to customer edges and how they will react).
The reason I bring this up is that I have already seen certain applications, such as one for registering AAAA records for DNS servers in a certain TLD, that don't allow anything other than 2001::/16. (Fortunately that application was fixed quickly when those responsible were notified.) Just making sure others aren't careening toward making the same mistake.
Agreed, and thanks for correcting my post. Would hate for others to take my offhanded comments on addressing and use them in production apps. Jack
Just for the record, I like my host being the degenerate case of "6to4 site + site router all in one". This makes my life much easier, as I frequently need IPv6 connectivity and frequently have a public IP(v4) address (EVDO, FTW). Having said that - what applies to me may well not be the common case. (Just wanted to state this in case MS is listening and was thinking about removing the functionality. I think the right approach is to detect the failure (s) when they occur, not to remove the functionality) /TJ
-----Original Message----- From: Jack Bates [mailto:jbates@brightok.net] Sent: Wednesday, November 19, 2008 4:05 PM To: Nathan Ward Cc: nanog list Subject: Re: IPv6 routing /48s
Nathan Ward wrote:
The problem here is XPSP2/Vista assuming that non-RFC1918 = unfiltered/unNATed for the purposes of 6to4. Well, deeper problem is that they're using 6to4 on an end host I suppose - it's supposed to be used on routers.
While I don't doubt that the 6to4 is broken in such circumstances, how many IPv6 content providers are using 6to4 addressing and not 2001:: addressing? 6to4 by default on xp and vista, in my experience, is only used if a) talking to another 6to4 address or b) there is no IPv4 address available.
6to4 never seemed like a viable method for content providing, though its use at the eyeball layer is somewhat iffy given that it's primary use is for other 6to4 addresses. If prefix policies are altered to use it for 2001:: addressing, problems start arising quickly.
A good example is that traceroutes through my he.net tunnel using 6to4 source addresses do not get replies through he.net's network, presumably due to their routers not being 6to4 aware and having no route to respond. Responses pick up again after picking up a network such as NTT that is 6to4 aware. My 2001:: addressing works just fine the entire route.
I'm sure there's quite a few networks that aren't 6to4 aware, hindering 6to4 connectivity to non-6to4 addresses.
Jack
On Wed, Nov 19, 2008 at 5:05 PM, Jack Bates <jbates@brightok.net> wrote:
Nathan Ward wrote:
The problem here is XPSP2/Vista assuming that non-RFC1918 = unfiltered/unNATed for the purposes of 6to4. Well, deeper problem is that they're using 6to4 on an end host I suppose - it's supposed to be used on routers.
While I don't doubt that the 6to4 is broken in such circumstances, how many IPv6 content providers are using 6to4 addressing and not 2001:: addressing?
6to4 v6 addrs are just regular v6 addrs as far as the network is concerned. if you put a 6to4 addr on your server you are saying that you don't have native v6 transport to that host(s) and that you are reachable via the 6to4 tunnel your host presumably has configured. Content providers MIGHT put 6to4 addrs on their servers so that they appear to be 'closer' to their clients, or so they might better control the pathing between client/server... but that's really no different than a non-6to4 addr as far as the applications and network are concerned.
6to4 never seemed like a viable method for content providing, though its use
it doesn't seem clear that 6to4 buys the content provider much on their side of the pipe, sure.
at the eyeball layer is somewhat iffy given that it's primary use is for other 6to4 addresses. If prefix policies are altered to use it for 2001:: addressing, problems start arising quickly.
6to4 is just an ip, 128bits long, but an ip... no differentiation is made in the network for 6to4 vs 'normal v6'... unless someone's putting up acls, or blackholing 6to4's /16, of course.
A good example is that traceroutes through my he.net tunnel using 6to4 source addresses do not get replies through he.net's network, presumably due to their routers not being 6to4 aware and having no route to respond.
can you explain this a little more? is it possible your v6 packets hit something like 6pe inside HE and exit to NTT without hitting a
Responses pick up again after picking up a network such as NTT that is 6to4 aware. My 2001:: addressing works just fine the entire route.
'6to4 aware' doesn't compute... -chris
Christopher Morrow wrote:
6to4 v6 addrs are just regular v6 addrs as far as the network is concerned. if you put a 6to4 addr on your server you are saying that you don't have native v6 transport to that host(s) and that you are reachable via the 6to4 tunnel your host presumably has configured.
Sure it's just another address, except the anycast portion of it for dealing with tunnels. It's also usually set to a different label and priority in windows prefix policies (and at least some linux setups). I was referring to the matter of if a windows box will even choose to use 6to4.
6to4 is just an ip, 128bits long, but an ip... no differentiation is made in the network for 6to4 vs 'normal v6'... unless someone's putting up acls, or blackholing 6to4's /16, of course.
Windows and several other end systems use prefix policies to determine if they use IPv6 or IPv4 and even when using IPv6, if they should use the 6to4 tunnel or not.
can you explain this a little more? is it possible your v6 packets hit something like 6pe inside HE and exit to NTT without hitting a
If a router does not a) know how to encapsulate 6to4 and send it over ipv4 to the destination or b) know how to reach a 6to4 anycast address where the packet can be encapsulated into IPv4, the packet is going to get dropped. Of course, you could be right. he.net could be purposefully not sending icmp replies back to 6to4 addresses for other reasons while replying to my non-6to4 addresses. I hesitate to say filter, as it does push the 6to4 sourced packets on to other networks. Jack
On Wed, Nov 19, 2008 at 6:03 PM, Jack Bates <jbates@brightok.net> wrote:
Christopher Morrow wrote:
6to4 v6 addrs are just regular v6 addrs as far as the network is concerned. if you put a 6to4 addr on your server you are saying that you don't have native v6 transport to that host(s) and that you are reachable via the 6to4 tunnel your host presumably has configured.
Sure it's just another address, except the anycast portion of it for dealing with tunnels. It's also usually set to a different label and priority in windows prefix policies (and at least some linux setups). I was referring to the matter of if a windows box will even choose to use 6to4.
sure... address selection on new connections, standard ipv6 foo for end-hosts. routers in the middle don't care, they route to the /16 route for 2002::/16, or they route to 192.88.99.0/24 ... anycast'd on their respective sides of the version#.
6to4 is just an ip, 128bits long, but an ip... no differentiation is made in the network for 6to4 vs 'normal v6'... unless someone's putting up acls, or blackholing 6to4's /16, of course.
Windows and several other end systems use prefix policies to determine if they use IPv6 or IPv4 and even when using IPv6, if they should use the 6to4 tunnel or not.
right, normal address selection rules.
can you explain this a little more? is it possible your v6 packets hit something like 6pe inside HE and exit to NTT without hitting a
If a router does not a) know how to encapsulate 6to4 and send it over ipv4
routers in the middle shouldn't/won't do this.. once it's encap'd it rides (on v4) to the 192.88.99.1 device that does the 4->6 majics, then on to the v6 destination where it rides back to the 'local' 2002::/16 endpoint where 6->4 encap majics happen then over the v4 to you again.
to the destination or b) know how to reach a 6to4 anycast address where the packet can be encapsulated into IPv4, the packet is going to get dropped. Of course, you could be right. he.net could be purposefully not sending icmp replies back to 6to4 addresses for other reasons while replying to my non-6to4 addresses. I hesitate to say filter, as it does push the 6to4 sourced packets on to other networks.
hopefully mike's message gets it worked out :) -chris
Christopher Morrow wrote:
Jack Bates wrote:
A good example is that traceroutes through my he.net tunnel using 6to4 source addresses do not get replies through he.net's network, presumably due to their routers not being 6to4 aware and having no route to respond.
can you explain this a little more? is it possible your v6 packets hit something like 6pe inside HE and exit to NTT without hitting a
(Chris thank you for automatically going into customer service mode :) A bunch of background first, then some questions to help diagnose this specific case. We don't filter 6to4 in any way. We don't run 6PE. We don't operate any 6to4 gateways. We've been considering it carefully, and haven't taken the plunge. There is sort of a "routing the whole Internet for free" aspect that will occur as v6 takes off and it's not clear how one manages that (i.e. If you do it in the beginning until people depend on it and traffic grows to 100 Gbps and you no longer can afford to do it for free, do you stop? What about all the IPv4 traffic traveling directly between 6to4 gateways on IPv4? abuse, security, man in the middle by definition, etc). This means any 6to4 gateway action is happening on somebody's 6to4 gateway not operated by us. There are people that are using 6to4 on our network that works just fine. You can reach several 6to4 gateways on both IPv4 and IPv6 via our network. However, what is likely happening is a random 6to4 gateway operator may have seen fit to rate limit or filter ICMP. To properly diagnose 6to4 problems you potentially need as many as 4 traceroutes, IPv6 traceroutes from the source and destination endpoints and a IPv4 traceroutes to the 6to4 gateway addresses from the source and destination endpoint. There a few other tips I'm forgetting at the moment, however if you send us email (to ipv6@he.net) we will make sure to research it thoroughly. Because 6to4 gateways are *anycast* the gateways you use in any part of the world in any part of a specific network may be different. This makes debugging it "interesting".
Responses pick up again after picking up a network such as NTT that is 6to4 aware. My 2001:: addressing works just fine the entire route.
'6to4 aware' doesn't compute...
Jack, it seems you are saying traffic passes end to end just fine, you just don't get ICMP responses from some of the hops in the middle. Is this correct? If you would like, please send email to ipv6@he.net with the detail regarding what you are seeing with all of the endpoint information and the traceroutes and we will work from our side to see where the "interesting" 6to4 gateway is that is affecting your traceroute. We will probably also need you to have access to the destination side as well. Mike. -- +---------------- H U R R I C A N E - E L E C T R I C ----------------+ | Mike Leber Wholesale IPv4 and IPv6 Transit 510 580 4100 | | Hurricane Electric AS6939 | | mleber@he.net Internet Backbone & Colocation http://he.net | +---------------------------------------------------------------------+
Mike Leber wrote:
We don't operate any 6to4 gateways.
This I suspected, and actually took as evidence based on the results from traceroutes.
However, what is likely happening is a random 6to4 gateway operator may have seen fit to rate limit or filter ICMP.
This may very well be true. I have nothing but love for he.net. However, the anycast nature of 6to4 does have it's issues. This was just a passing example that I noticed. Packets go through the network, but your network couldn't send ICMPv6 back. Actually not a concern for me, but I doubt it's the only 6to4 issue seen across the network.
To properly diagnose 6to4 problems you potentially need as many as 4 traceroutes, IPv6 traceroutes from the source and destination endpoints and a IPv4 traceroutes to the 6to4 gateway addresses from the source and destination endpoint. There a few other tips I'm forgetting at the moment, however if you send us email (to ipv6@he.net) we will make sure to research it thoroughly.
Will do. Not that I care, but might be something you'll want to check into anyways.
Because 6to4 gateways are *anycast* the gateways you use in any part of the world in any part of a specific network may be different.
This makes debugging it "interesting".
Definitely, and another reason I am heavily against 6to4 except in cases where it's absolutely necessary.
Jack, it seems you are saying traffic passes end to end just fine, you just don't get ICMP responses from some of the hops in the middle. Is this correct?
Correct, traceroute and ping find a void on the 2 routers I pass before I hit NTT's network in the test case I was doing. I haven't tested this in 1/2 a week, though.
If you would like, please send email to ipv6@he.net with the detail regarding what you are seeing with all of the endpoint information and the traceroutes and we will work from our side to see where the "interesting" 6to4 gateway is that is affecting your traceroute. We will probably also need you to have access to the destination side as well.
Will do. Be abit. The "interesting" part is primarily what it was mentioned. Though in another response I agreed that anyone using IPv6 from an end network should consider have 6to4 relays so as not to depend on someone else. In some cases, though, it's just not practical. FYI: Outside of testing, my link to he.net was to take what little 6to4 traffic I had on the network to non-6to4 addresses and give it a better chance. My nearest IPv4 anycast 6to4 was beyond horrid (major isolation). Heaviest traffic load appears to be p2p to teredo destinations. Jack
On 20/11/2008, at 11:05 AM, Jack Bates wrote:
Nathan Ward wrote:
The problem here is XPSP2/Vista assuming that non-RFC1918 = unfiltered/unNATed for the purposes of 6to4. Well, deeper problem is that they're using 6to4 on an end host I suppose - it's supposed to be used on routers.
While I don't doubt that the 6to4 is broken in such circumstances, how many IPv6 content providers are using 6to4 addressing and not 2001:: addressing? 6to4 by default on xp and vista, in my experience, is only used if a) talking to another 6to4 address or b) there is no IPv4 address available.
IPv6 will be used in preference to IPv4 if it is available. If 6to4 is the only IPv6 connectivity then it will be used. See RFC3484. Preference of IPv4 is 10, 6to4 is 30. Things are a bit different with Teredo.. In Vista, if Teredo is the only IPv6 connectivity available, WSAConnectByName will not send queries for AAAA - obviously Teredo can still be used if the application does its own name resolution and opens sockets by INET6 address, not name. This Teredo behaviour is not documented in an RFC anywhere I don't believe, it is Windows specific.
6to4 never seemed like a viable method for content providing, though its use at the eyeball layer is somewhat iffy given that it's primary use is for other 6to4 addresses. If prefix policies are altered to use it for 2001:: addressing, problems start arising quickly.
Right, so, because of how RFC3484 works the above is largely invalid. Because of RFC3484, putting 6to4 addresses on content is actually quite a good idea. It means that hosts with 6to4 connectivity only will not have to go via an RFC3068 (192.88.99.1) relay - they will go direct over an IPv4 best path between their 6to4 router and yours. This works best if RFC3484 is widely deployed - it is, however it does not exist on OS X. Because of this, if you do this you may find OS X clients visiting your content on a 6to4 address when they have native connectivity, or vice versa. So.. stuff to play with, and please go encourage Apple to do RFC3484 if you are a big customer of theirs.
A good example is that traceroutes through my he.net tunnel using 6to4 source addresses do not get replies through he.net's network, presumably due to their routers not being 6to4 aware and having no route to respond. Responses pick up again after picking up a network such as NTT that is 6to4 aware. My 2001:: addressing works just fine the entire route.
No, he.net does not have a 6to4 relay. If they did it would be used by a very large number of networks as he.net is fairly well peered. Because of that, the traffic would be quite high, and any relay deployment there would have to be managed quite carefully.
I'm sure there's quite a few networks that aren't 6to4 aware, hindering 6to4 connectivity to non-6to4 addresses.
6to4 does not require networks to be 6to4 aware - the Internet has many public 6to4 relays available. Note though that most end user IPv6 hosts have a 6to4 relay within 3 IPv6 hops. This is easy to calculate - throw up an AAAA pointing only to a 2002::/16 address, and look at the HLIM of the IPv6 header (not the IPv4 header). You will see that it is generally between 124 and 128, or 60 and 64. (Some hosts start at 128, some 64. Few start at much else it seems.) I did some work looking at IPv6 adoption using Bittorrent DHT to talk to large numbers of peers without downloading any content, this was one of the interesting points that fell out of that work. Content providers wanting to put AAAA records on things should run 6to4 relays until IPv4-only networks are a thing of the past. If they don't, they'll have to rely on third party relays that might not work. -- Nathan Ward
On Thu, 20 Nov 2008, Nathan Ward wrote:
IPv6 will be used in preference to IPv4 if it is available. If 6to4 is the only IPv6 connectivity then it will be used. See RFC3484. Preference of IPv4 is 10, 6to4 is 30.
However, in destination address selection "prefer matching scope" (comes up when v4 address is rfc1918) and "prefer matching label" is checked before preference. -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
participants (8)
-
Christopher Morrow -
Jack Bates -
Michael Sinatra -
Mike Leber -
Nathan Ward -
Pekka Savola -
TJ -
trejrco＠gmail.com