Good morning and happy holidays all. I'm in the process of creating an automated filtering application and would like to know if anyone can point me to the right place. I'd like to be able to query a site/db/etc., and pull out specific netblocks to create fw rules. Since IP space is always changing, it would be helpful if my queries can be tailored to something like: wget site | Parse IP space | grep Company | create rule Or: wget site | Parse IP space | grep {EDU_IP_SPACE,MIL_SPACE,GOV_SPACE} | create rule Follow? Right now I am using potaroo with something like : wget -qO - http://bgp.potaroo.net/ipv4-stats/allocated-{apnic.html,ripe.html, etc} But this just gives me entire blocks, not who is behind them. Is there any site I could use to query specifics? E.g., for a gov client: wget -qO - this.site.org | grep "\.gov" | parse_with_awk '{print "fw_rule"}' Thanks in advance and Happy New Year to everyone. -- =+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ J. Oquendo SGFA, SGFE, C|EH, CNDA, CHFI, OSCP, CPT "It takes 20 years to build a reputation and five minutes to ruin it. If you think about that, you'll do things differently." - Warren Buffett 227C 5D35 7DCB 0893 95AA 4771 1DCE 1FD1 5CCD 6B5E http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x5CCD6B5E
You may want to look at Capirca (http://code.google.com/p/capirca/) for creating policy files from which to generate your firewall rulesets. I am not aware of a simple categorization of netblocks. My first thought is that an agreement with every RIR for bulk whois data and writing code to parse / categorize would be quite difficult and may not get you a reasonable result after all that work - maybe there is something commercially available. -Ryan On Wed, Dec 29, 2010 at 9:01 AM, J. Oquendo <sil@infiltrated.net> wrote:
Good morning and happy holidays all. I'm in the process of creating an automated filtering application and would like to know if anyone can point me to the right place. I'd like to be able to query a site/db/etc., and pull out specific netblocks to create fw rules. Since IP space is always changing, it would be helpful if my queries can be tailored to something like:
wget site | Parse IP space | grep Company | create rule
Or:
wget site | Parse IP space | grep {EDU_IP_SPACE,MIL_SPACE,GOV_SPACE} | create rule
Follow?
Right now I am using potaroo with something like :
wget -qO - http://bgp.potaroo.net/ipv4-stats/allocated-{apnic.html,ripe.html, etc}
But this just gives me entire blocks, not who is behind them. Is there any site I could use to query specifics? E.g., for a gov client: wget -qO - this.site.org | grep "\.gov" | parse_with_awk '{print "fw_rule"}'
Thanks in advance and Happy New Year to everyone.
--
=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ J. Oquendo SGFA, SGFE, C|EH, CNDA, CHFI, OSCP, CPT
"It takes 20 years to build a reputation and five minutes to ruin it. If you think about that, you'll do things differently." - Warren Buffett
227C 5D35 7DCB 0893 95AA 4771 1DCE 1FD1 5CCD 6B5E http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x5CCD6B5E
On Wed, Dec 29, 2010 at 6:01 AM, J. Oquendo <sil@infiltrated.net> wrote:
Good morning and happy holidays all. I'm in the process of creating an automated filtering application and would like to know if anyone can point me to the right place. I'd like to be able to query a site/db/etc., and pull out specific netblocks to create fw rules.
[...]
But this just gives me entire blocks, not who is behind them. Is there any site I could use to query specifics? E.g., for a gov client: wget -qO - this.site.org | grep "\.gov" | parse_with_awk '{print "fw_rule"}'
Given the current IPv4 climiate, this sounds like a terrible idea. The landscape has changed dramatically from what it once was. Large volumes of mobile carriers use NAT, many IPv6 to IPv4 gateways are out there routing traffic, and we'll soon see a time in which entire countries are transiting over small chunks of IPv4 space. Never mind the fact that applications on services like Google App Engine have a different IP nearly every time they connect because of outbound proxy pools. I think you're going to have a very difficult time resolving an IP to the appropriate owner. Coarse calculation of who might be in charge of a block is possible but fine-grained discovery and classification of an owner is a difficult task. That being said, the tools that I'm using on a daily basis to figure out who actually owns an IP block (or is sending traffic over it) are: - Senderbase (Cisco) - cymru whois (whois.cymru.com - good for fast bgp lookups and geo) - http://multirbl.valli.org/dnsbl-lookup (multi-rbl lookup , good for finding abusers and other issues) - SmartViper (Website ownership) http://www.markosweb.com/ -John
On Wed, Dec 29, 2010 at 2:01 PM, John Adams <jna@retina.net> wrote:
On Wed, Dec 29, 2010 at 6:01 AM, J. Oquendo <sil@infiltrated.net> wrote:
Good morning and happy holidays all. I'm in the process of creating an automated filtering application and would like to know if anyone can point me to the right place. I'd like to be able to query a site/db/etc., and pull out specific netblocks to create fw rules.
[...]
But this just gives me entire blocks, not who is behind them. Is there any site I could use to query specifics? E.g., for a gov client: wget -qO - this.site.org | grep "\.gov" | parse_with_awk '{print "fw_rule"}'
given an ASN you can query their announcements from RouteViews DNS no? (or rsync that and do the lookup locally in whatever form you feel is helpful) That probably has some whois data easily tied to it as well...
Given the current IPv4 climiate, this sounds like a terrible idea. The landscape has changed dramatically from what it once was. Large
if you are updating filters 'quickly' it shouldn't matter, right? you'll catch things (presuming whois is updated and/or BGP is and you can tie things back through asn/netblock relationships, oh... RPKI...) pretty quickly as they move.
volumes of mobile carriers use NAT, many IPv6 to IPv4 gateways are out there routing traffic, and we'll soon see a time in which entire countries are transiting over small chunks of IPv4 space. Never mind
I don't recall the OP saying 'ipv4' only?
the fact that applications on services like Google App Engine have a different IP nearly every time they connect because of outbound proxy pools.
it's probably not 'every time they connect' there's probably some sensible reasoning behind the decision process.. like your query that triggers it comes into "METRO-X" and thus outbound queries come from a netblock for NAT things inside "METRO-X", my query goes to "METRO-Y" so ... diff netblock. Inside a set of queries (10-100?) you'll see a repeated set of ips, I suspect. -chris
participants (4)
-
Christopher Morrow -
J. Oquendo -
John Adams -
Ryan Shea