2008.02.20 NANOG 42 IPv4 PTR queries for unallocated space
Notes from day 4 of NANOG 42, we're on the home stretch now, so there's only a bit more of my spewage to wade through before we can resume our normal discussion of the imminent death of the internet as we know it. :) Again, apologies for typos, misspellings of names, etc. Matt 2008.02.20 opening remarks and analysis of PTR queries for IPv4 reserved addresses Trying to measure use of unallocated IPv4 address space Starting with Leo Vigoda, working at IANA now. leo.vigoda@icann.org At last NANOG, did lightning talk about how people were using unallocated space, but didn't show lots of data. Now, has a way to hopefully measure IPv4 addresses that people are trying to use. Not all the addresses, but at least one view of it. Problem? All unallocated unicast space will eventually be allocated; no more free /8's in the IANA pool at some point, as v6 won't get rolled out in time. Some people are using the same addres space that's currently in the 'unallocated' pool. Tried to measure it by looking at DNS queries at l.root for addresses in the unallocated blocks. root servers provide in-addr.arpa delegations, if a /8 isn't allocated, there's no delegation, so the root servers see the queries. Can't measure well-maintained networks with split-horizon DNS, or with good egress filters. Doesn't know how to measure truly private internal use; if you have ideas, let him know. Results slide; long blue line at the far left, lots and lots of queries, then shallows off to the right side; some old rubbish names in in-addr people are querying for. Left side is the fun stuff. 222/8, allocated to APNIC; not sure why it gets a lot more queries than the other /8's. Could be mail server lookups, since there's a lot of mail that comes out of 222/8; could be one rogue server. If you assume it's bogus, and chuck it out, data 'looks' better. 10/8 is near the top, as expected. 0/8 RFC3330 2/8 is unallocated; in the top 15 range. Odd there's unallocated space so close to the top of the queries range. If you take out all the unallocated stuff, there's still a lot of entries; looking at top 10 list: 2/8, 1/8, 23/8, 5/8, 100/8 is there at #5, which is odd. the next /8's they *were* going to give ARIN were 100/101, they decided to NOT give it out to figure out why there's so many hits on it in this round. The top 10 list is the 30 day period ending sunday; he also compared to statistics he used at the esNOG meeting in madrid a few months ago. Not very static; people leave top 10, new come on. 28dec-26jan, vs 19jan to 17feb 2/8 and 1/8 stay in #1 and #2 other /8s shift; 5/8 rises, 23/8 rises, 100/8 drops, 27/8 drops. It's not static, there's movement, people are shifting and shuffling blocks. Not measuring the query source address and AS # that could be gathered to inform people they're trying to talk to unallocated space. Only gathering from l.root, would be good to get data from other root servers. Leo's not a proper researcher, so would be good to get skillset from a real researcher. Would like to get data from nodes all over the world to get more global information; can see if there's more of a local component, or if this is a global phenomenon. This would be a very big data analysis challenge, combining l, k, and other root server operators, and get a more complete view of the data. Use it to either warn people, or help get things fixed if at all possible. Q: Louis Lee, Equinix, want to find out if they've looked to see if the top10 prefixes are in the global routing table when the queries come in? A: No, that hasn't been looked at, but a proper researcher would probably have considered and tied that data in as well. Q: Leo Bicknell, ISC: did they look at the source of queries? Are they all coming from a common resolver? A: no, not yet; would be the sort of thing that a proper researcher would be able to dig into. There are privacy considerations with looking at sources of queries, etc; needs to be thought through very carefully to make sure no per Q: Keith Mitchell, Interesting work, thanks Leo; there's lots of information for gathering, sharing, and protecting privacy in OIRC; if he would like to work with OIRC, they can give pointers on how to handle that data with necessary Q: further instrumenting to see if there's noise in the allocated space to see if this may be just part of overall background noise? A: this is the raw, basic data Q: Duane Wessels, measurement factory; DNS queries behave differently when a result exists vs doesn't exist; may be worthwhile to temporarily make them exist on a set of servers, and collect the data from there, and see if the resolution process follows the delegation. Q: Troy Jessup, Utah edu network; could 1, 2, 100, etc. network queries be due to malware that starts at one end of a block and just starts scanning indiscriminantly. 100 seems to be a common starting point for malware, for example. Haven't looked into it in that much detail--is it just 1.1.1.1/32, or 1.1.1.1/16, how much coverage within the block is there? is it just hot spots? These are really good ideas for the Real Researcher(tm) to delve into, once one can be tracked down. If people want to send suggestions, email him at leo.vigoda@icann.org
Doesn't know how to measure truly private internal use; if you have ideas, let him know.
Make official requests to companies who operate networks along with assurances of confidentiality. Include the NDA that you are willing to sign along with your request. Try to find a legal contact for each company and submit the requests to them as well as to IP addressing folks and engineering/ops folks. Note that "companies who operate networks" does not include "network operators". A second approach that is more likely to get results, but less likely to answer the whys and wherefores, is to go to "network operators" who are in the VPN or private lines business and ask them for stats on what address ranges (and how many instances) their customers use on private IP networks. Probably need the same NDA and approach to legal contacts. I know of a nonISP that has three separate non-overlapping and non-connected private IP networks which use 1/8 addresses. I know of a company that uses 1/8 through 8/8 on private IP networks. I know of a company that chose to use 126/8 addresses because it seemed that 126/8 would be the last IP block for RIRs to allocate. Then big cable companies started running out of 10/8 space... I've even seen an ISP using a /24 carved out of 196/8 in their internal network management systems. I wasn't able to find out if that started life as a typo of 192.168.?.? and I can't remember the exact /24 range. I've received SPAM reports for a former customer who connected in 1994, received a /24, disconnected in 1996 but kept using that /24 internally behind NATs. One day a bright salesperson decided to SPAM some advertising. Since the SPAM content matched this former customer's business, I chased it up and discovered the reason why our address range was in the mail headers. Is it possible to measure this? Maybe sniffing ISP nameserver traffic to identify matching A and PTR queries where the PTR name is clearly located in a different address range?
2/8, 1/8, 23/8, 5/8, 100/8 is there at #5, which is odd.
Odd? It's a round number which probably means that more than one person has picked it when they needed to make up an IP address.
next /8's they *were* going to give ARIN were 100/101, they decided to NOT give it out to figure out why there's so many hits on it in this round.
It would be good to find out why people have these kinds of DNS leaks. Hopefully it will turn out to be configuration errors which can be addressed through best practices. Even when the space becomes allocated, people will continue to use these blocks and this can be expected to become even more common when IPv4 nears exhaustion. --Michael Dillon
On 20 feb 2008, at 20:27, <michael.dillon@bt.com> <michael.dillon@bt.com> wrote:
2/8, 1/8, 23/8, 5/8, 100/8 is there at #5, which is odd.
Odd? It's a round number which probably means that more than one person has picked it when they needed to make up an IP address.
It would be interesting to know how much of this space is really used for something more or less permanent, and how much is just random noise. For instance, I do a training course where people need to configure routers, and I use addresses out of 96.0.0.0/8 for that, because it has to be clear that we're talking about real addresses and not RFC 1918 stuff. Although this doesn't interact with the real internet, often, people end up having real addresses and also 96.0.0.0/8 addresses on their laptops so they probably generate some DNS queries for the 96 range. Would it be useful for IANA to publish the order in which they're going to allocate /8s? That way, it's easier for people to plan getting out of the way of real deployment in time.
On 21 Feb 2008, at 09:51, Iljitsch van Beijnum wrote:
For instance, I do a training course where people need to configure routers, and I use addresses out of 96.0.0.0/8 for that, because it has to be clear that we're talking about real addresses and not RFC 1918 stuff.
Do you think you could consider some of the rfc3330 ranges like TEST- NET - 192.0.2.0/24 - or if you need more than one network, 191.255.0.0/16 ?
Would it be useful for IANA to publish the order in which they're going to allocate /8s? That way, it's easier for people to plan getting out of the way of real deployment in time.
Well it's a pretty safe bet that most of today's unused /8s will be allocated within the next couple of years ! Best wishes, Andy
I know of at least one large telecom provider which is using 100/8. In my opinion, this should not be a reason to delay the use of these addresses for a legitimate purpose. Rewarding address squatting simply isn't a good thing. Owen On Feb 21, 2008, at 1:51 AM, Iljitsch van Beijnum wrote:
On 20 feb 2008, at 20:27, <michael.dillon@bt.com> <michael.dillon@bt.com
wrote:
2/8, 1/8, 23/8, 5/8, 100/8 is there at #5, which is odd.
Odd? It's a round number which probably means that more than one person has picked it when they needed to make up an IP address.
It would be interesting to know how much of this space is really used for something more or less permanent, and how much is just random noise. For instance, I do a training course where people need to configure routers, and I use addresses out of 96.0.0.0/8 for that, because it has to be clear that we're talking about real addresses and not RFC 1918 stuff. Although this doesn't interact with the real internet, often, people end up having real addresses and also 96.0.0.0/8 addresses on their laptops so they probably generate some DNS queries for the 96 range.
Would it be useful for IANA to publish the order in which they're going to allocate /8s? That way, it's easier for people to plan getting out of the way of real deployment in time.
On 21/02/2008 07:22, "Owen DeLong" <owen@delong.com> wrote:
I know of at least one large telecom provider which is using 100/8. In my opinion, this should not be a reason to delay the use of these addresses for a legitimate purpose. Rewarding address squatting simply isn't a good thing.
No one is attempting to reward address squatting. The main reasons for this work are to try and quantify the scale and distribution of the problem. I hope that with some more data and a fuller analysis we will find that there isn't anything major to worry about. But if the problem is significant, I'd like to be able to pre-warn people so that they can take prepare themselves for it. That might mean doing simple things like tweaking technical support and fault finding procedures, possibly something else. Regards, Leo
On 21/02/2008 01:51, "Iljitsch van Beijnum" <iljitsch@muada.com> wrote: [...]
Would it be useful for IANA to publish the order in which they're going to allocate /8s? That way, it's easier for people to plan getting out of the way of real deployment in time.
We have been discussing something along these lines. The exact method for selecting the order in which /8s should be allocated hasn't been decided yet, though. Regards, Leo
Just FYI: Leo's email address is <leo.vegoda@icann.org> Regards, -drc On Feb 20, 2008, at 10:34 AM, Matthew Petach wrote:
Notes from day 4 of NANOG 42, we're on the home stretch now, so there's only a bit more of my spewage to wade through before we can resume our normal discussion of the imminent death of the internet as we know it. :)
Again, apologies for typos, misspellings of names, etc.
Matt
2008.02.20 opening remarks and analysis of PTR queries for IPv4 reserved addresses
Trying to measure use of unallocated IPv4 address space
Starting with Leo Vigoda, working at IANA now. leo.vigoda@icann.org
At last NANOG, did lightning talk about how people were using unallocated space, but didn't show lots of data.
Now, has a way to hopefully measure IPv4 addresses that people are trying to use.
Not all the addresses, but at least one view of it.
Problem? All unallocated unicast space will eventually be allocated; no more free /8's in the IANA pool at some point, as v6 won't get rolled out in time.
Some people are using the same addres space that's currently in the 'unallocated' pool.
Tried to measure it by looking at DNS queries at l.root for addresses in the unallocated blocks. root servers provide in-addr.arpa delegations, if a /8 isn't allocated, there's no delegation, so the root servers see the queries.
Can't measure well-maintained networks with split-horizon DNS, or with good egress filters.
Doesn't know how to measure truly private internal use; if you have ideas, let him know.
Results slide; long blue line at the far left, lots and lots of queries, then shallows off to the right side; some old rubbish names in in-addr people are querying for.
Left side is the fun stuff.
222/8, allocated to APNIC; not sure why it gets a lot more queries than the other /8's. Could be mail server lookups, since there's a lot of mail that comes out of 222/8; could be one rogue server. If you assume it's bogus, and chuck it out, data 'looks' better. 10/8 is near the top, as expected. 0/8 RFC3330 2/8 is unallocated; in the top 15 range. Odd there's unallocated space so close to the top of the queries range. If you take out all the unallocated stuff, there's still a lot of entries; looking at top 10 list: 2/8, 1/8, 23/8, 5/8, 100/8 is there at #5, which is odd. the next /8's they *were* going to give ARIN were 100/101, they decided to NOT give it out to figure out why there's so many hits on it in this round.
The top 10 list is the 30 day period ending sunday; he also compared to statistics he used at the esNOG meeting in madrid a few months ago.
Not very static; people leave top 10, new come on.
28dec-26jan, vs 19jan to 17feb
2/8 and 1/8 stay in #1 and #2 other /8s shift; 5/8 rises, 23/8 rises, 100/8 drops, 27/8 drops.
It's not static, there's movement, people are shifting and shuffling blocks.
Not measuring the query source address and AS # that could be gathered to inform people they're trying to talk to unallocated space.
Only gathering from l.root, would be good to get data from other root servers.
Leo's not a proper researcher, so would be good to get skillset from a real researcher.
Would like to get data from nodes all over the world to get more global information; can see if there's more of a local component, or if this is a global phenomenon. This would be a very big data analysis challenge, combining l, k, and other root server operators, and get a more complete view of the data.
Use it to either warn people, or help get things fixed if at all possible.
Q: Louis Lee, Equinix, want to find out if they've looked to see if the top10 prefixes are in the global routing table when the queries come in? A: No, that hasn't been looked at, but a proper researcher would probably have considered and tied that data in as well.
Q: Leo Bicknell, ISC: did they look at the source of queries? Are they all coming from a common resolver? A: no, not yet; would be the sort of thing that a proper researcher would be able to dig into. There are privacy considerations with looking at sources of queries, etc; needs to be thought through very carefully to make sure no per
Q: Keith Mitchell, Interesting work, thanks Leo; there's lots of information for gathering, sharing, and protecting privacy in OIRC; if he would like to work with OIRC, they can give pointers on how to handle that data with necessary
Q: further instrumenting to see if there's noise in the allocated space to see if this may be just part of overall background noise? A: this is the raw, basic data
Q: Duane Wessels, measurement factory; DNS queries behave differently when a result exists vs doesn't exist; may be worthwhile to temporarily make them exist on a set of servers, and collect the data from there, and see if the resolution process follows the delegation.
Q: Troy Jessup, Utah edu network; could 1, 2, 100, etc. network queries be due to malware that starts at one end of a block and just starts scanning indiscriminantly. 100 seems to be a common starting point for malware, for example. Haven't looked into it in that much detail--is it just 1.1.1.1/32, or 1.1.1.1/16, how much coverage within the block is there? is it just hot spots? These are really good ideas for the Real Researcher(tm) to delve into, once one can be tracked down.
If people want to send suggestions, email him at
leo.vigoda@icann.org
participants (7)
-
Andy Davidson -
David Conrad -
Iljitsch van Beijnum -
Leo Vegoda -
Matthew Petach -
michael.dillon＠bt.com -
Owen DeLong