I'm looking for a good utility that I can run locally to scrape RADB, ARIN, other RIRs to add to my or customer prefix lists... Anybody have a good tool for this? I currently end up visiting https://www.dan.me.uk/filtergen and creating one off that. Thoughts? -- Kate QuadraNet, Inc 530 W 6th Street #901 Los Angeles, CA 90014 kate@quadranet.com<mailto:kate@quadranet.com>
Crap! Sorry about that, I sent the post as an HTML message. :( -- Kate QuadraNet, Inc 530 W 6th Street #901 Los Angeles, CA 90014 kate@quadranet.com From: Kate Gerry Sent: Wednesday, September 21, 2011 5:39 AM To: nanog@nanog.org Cc: Kate Gerry Subject: RADB/RIR Scraper I'm looking for a good utility that I can run locally to scrape RADB, ARIN, other RIRs to add to my or customer prefix lists... Anybody have a good tool for this? I currently end up visiting https://www.dan.me.uk/filtergen and creating one off that. Thoughts? -- Kate QuadraNet, Inc 530 W 6th Street #901 Los Angeles, CA 90014 kate@quadranet.com
here is mine: #!/bin/sh echo "-i origin $1" |nc whois.radb.net 43 On Wed, 21 Sep 2011 05:40:14 -0700 Kate Gerry <kate@quadranet.com> wrote:
Crap! Sorry about that, I sent the post as an HTML message. :(
-- Kate QuadraNet, Inc 530 W 6th Street #901 Los Angeles, CA 90014 kate@quadranet.com
From: Kate Gerry Sent: Wednesday, September 21, 2011 5:39 AM To: nanog@nanog.org Cc: Kate Gerry Subject: RADB/RIR Scraper
I'm looking for a good utility that I can run locally to scrape RADB, ARIN, other RIRs to add to my or customer prefix lists... Anybody have a good tool for this? I currently end up visiting https://www.dan.me.uk/filtergen and creating one off that.
Thoughts?
-- Kate QuadraNet, Inc 530 W 6th Street #901 Los Angeles, CA 90014 kate@quadranet.com
On 21/09/2011 13:39, Kate Gerry wrote:
I'm looking for a good utility that I can run locally to scrape RADB, ARIN, other RIRs to add to my or customer prefix lists... Anybody have a good tool for this? I currently end up visiting https://www.dan.me.uk/filtergen and creating one off that.
there is irrtoolset. If you can get beyond the unfriendly user interface, it will do all sorts of interesting things, including generating prefix lists in both cisco and juniper format. http://irrtoolset.isc.org/ Lots of people don't like irrtoolset, mostly for very good reasons. However for either very simple or very complicated stuff, it can be quite useful. Nick
<http://www.nanog.org/meetings/nanog36/presentations/steenbergen.pdf> has some pointers to tools Richard wrote (and presented a few times now) at nanog meetings. (to save you reading the pdf... which is a good read: <http://irrpt.sourceforge.net/> On Wed, Sep 21, 2011 at 9:01 AM, Nick Hilliard <nick@foobar.org> wrote:
On 21/09/2011 13:39, Kate Gerry wrote:
I'm looking for a good utility that I can run locally to scrape RADB, ARIN, other RIRs to add to my or customer prefix lists... Anybody have a good tool for this? I currently end up visiting https://www.dan.me.uk/filtergen and creating one off that.
there is irrtoolset. If you can get beyond the unfriendly user interface, it will do all sorts of interesting things, including generating prefix lists in both cisco and juniper format.
Lots of people don't like irrtoolset, mostly for very good reasons. However for either very simple or very complicated stuff, it can be quite useful.
Nick
On 21/09/2011 14:56, Christopher Morrow wrote:
<http://www.nanog.org/meetings/nanog36/presentations/steenbergen.pdf>
has some pointers to tools Richard wrote (and presented a few times now) at nanog meetings. (to save you reading the pdf... which is a good read: <http://irrpt.sourceforge.net/>
There's also Marco D'Itri's rpsltool: http://www.linux.it/~md/software/ Of the two, I prefer rpsltool (sorry ras!) - its templating facilities are better. It's also written in perl which tends to be better for text processing. PHP is a pain for text processing. Although both of these these tools are very good, neither of them implements anything near a complete RPSL parser, which is why I made my earlier command about the usefulness of irrtoolset for certain types of user. If you just want basic stuff, irrtoolset will probably do what you want out of the box. If you want moderately complicated stuff, you may run into irritating limitations in irrtoolset's rtconfig command. However, if you want to do really complicated stuff, then it's the only code-base out there which implements an almost complete RPSL implementation. And there is a small number of organisations out there who use irrtoolset to its fullest capabilities. Nick
On Wed, Sep 21, 2011 at 05:39:26AM -0700, Kate Gerry wrote:
I'm looking for a good utility that I can run locally to scrape RADB, ARIN, other RIRs to add to my or customer prefix lists... Anybody have a good tool for this? I currently end up visiting https://www.dan.me.uk/filtergen and creating one off that.
Thoughts?
<http://www.lexa.ru/snar/bgpq/index_en.html> or you could have alook at | whois -h filtergen.level3.com -- help if you get blocked by whois rate limiting. Greetings, Hannes
On Wed, Sep 21, 2011 at 04:09:40PM +0200, Hannes Frederic Sowa wrote:
The url above links to the predecessor project. The code that I actually use is available here: <http://snar.spb.ru/prog/bgpq3/> Greetings, Hannes
On Wed, Sep 21, 2011 at 05:39:26AM -0700, Kate Gerry wrote:
I'm looking for a good utility that I can run locally to scrape RADB, ARIN, other RIRs to add to my or customer prefix lists... Anybody have a good tool for this? I currently end up visiting https://www.dan.me.uk/filtergen and creating one off that.
whois -h filtergen.level3.net -- Brandon Ewing (nicotine@warningg.com)
I've yet to use this myself but it looks like what you are looking for. bgpq3 http://snar.spb.ru/prog/bgpq3/bgpq3-0.1.7.html Sent from my iPhone On Sep 21, 2011, at 8:39, Kate Gerry <kate@quadranet.com> wrote:
I'm looking for a good utility that I can run locally to scrape RADB, ARIN, other RIRs to add to my or customer prefix lists... Anybody have a good tool for this? I currently end up visiting https://www.dan.me.uk/filtergen and creating one off that.
Thoughts?
-- Kate QuadraNet, Inc 530 W 6th Street #901 Los Angeles, CA 90014 kate@quadranet.com<mailto:kate@quadranet.com>
A little bit of topic, but is there a way to get the prefix list and AS number using the description in RADB/others? For example, for "Commonwealth Bank of Australia" I want to get the following route: 203.202.158.0/24 descr: Commonwealth Bank of Australia origin: AS7474 mnt-by: MAINT-AS7474 changed: noc@optus.net.au 20080918 source: RADB Thanks.
I've always found it helpful to use the "inverse query by" feature, where you can query for any object that has x "mnt-by" or "origin" and it will list any objects with that mnt-by or origin you query for. RADB has this built directly into the Advanced Object Query form on the website. -Kyle On Wed, Oct 5, 2011 at 10:10 AM, Mikhail Strizhov < strizhov@netsec.colostate.edu> wrote:
A little bit of topic, but is there a way to get the prefix list and AS number using the description in RADB/others? For example, for "Commonwealth Bank of Australia" I want to get the following
route: 203.202.158.0/24 descr: Commonwealth Bank of Australia origin: AS7474 mnt-by: MAINT-AS7474 changed: noc@optus.net.au 20080918 source: RADB
Thanks.
participants (10)
-
Brandon Ewing -
Christopher Morrow -
Gregory Edigarov -
Hannes Frederic Sowa -
Kate Gerry -
Kyle Duren -
Mikhail Strizhov -
Nick Hilliard -
Rafael Rodriguez -
Randy Bush