Determine difference between 2 BGP feeds
Hi, We receive a BGP feed from different providers on two different routers. While one seems to be a reasonable amount of feeds after reviewing the CIDR report, the other is anywhere from 3K to 10K more routes. Is there a utility that I can use that will pull the routes off each router (Foundry preferred), and then compare them as best it can to see why there is such a difference? I can understand a handful of routes over what CIDR says, but a minimum of 3K more? Thanks, Tuc/TBOH
On Tue, 18 Apr 2006, Scott Tuc Ellentuch at T-B-O-H wrote:
Is there a utility that I can use that will pull the routes off each router (Foundry preferred), and then compare them as best it can to see why there is such a difference? I can understand a handful of routes over what CIDR says, but a minimum of 3K more?
Is one of them as4323? ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Tue, 18 Apr 2006, Scott Tuc Ellentuch at T-B-O-H wrote:
Is there a utility that I can use that will pull the routes off each router (Foundry preferred), and then compare them as best it can to see why there is such a difference? I can understand a handful of routes over what CIDR says, but a minimum of 3K more?
Is one of them as4323?
Actually, no. I wasn't wanting to "name names" to protect the innocent... BUT.... ROUTER1: Neighbor Address AS# State Time Rt:Accepted Filtered Sent ToSend 64.200.58.69 7911 ESTAB 4d21h57m 182287 0 4 0 ROUTER2: Neighbor Address AS# State Time Rt:Accepted Filtered Sent ToSend 69.28.152.229 22822 ESTAB 18d16h51m 186379 0 4 0 Tuc/TBOH
On Tue, Apr 18, 2006 at 04:28:40PM -0400, Scott Tuc Ellentuch at T-B-O-H wrote:
On Tue, 18 Apr 2006, Scott Tuc Ellentuch at T-B-O-H wrote:
Is there a utility that I can use that will pull the routes off each router (Foundry preferred), and then compare them as best it can to see why there is such a difference? I can understand a handful of routes over what CIDR says, but a minimum of 3K more?
Is one of them as4323?
Actually, no. I wasn't wanting to "name names" to protect the innocent... BUT....
ROUTER1: Neighbor Address AS# State Time Rt:Accepted Filtered Sent ToSend 64.200.58.69 7911 ESTAB 4d21h57m 182287 0 4 0
ROUTER2: Neighbor Address AS# State Time Rt:Accepted Filtered Sent ToSend 69.28.152.229 22822 ESTAB 18d16h51m 186379 0 4 0
This is actually fairly common. There are a lot of folks out there who announce more specifics to one network but not another, or who apply no export or limited export community tags in various places. Also, every network has a different filter policy of what they will and won't accept. FWIW my "exported to bgp speaking customers" count at this moment is 182525. I wouldn't get concerned about it unless the network with more prefixes is doing something absurdly stupid like sending you internal /30s and such (which, well, a lot of people do :P). It could also be something like peers agreeing to traffic engineer by sending each other more specifics w/meds, though if they were smart they would be doing that with no-export so as to not make your TE job more difficult. If you really want to compare the differences, try something like: telnet yourrouter | tee outputfile term length 0 sh ip bgp nei x.x.x.x received-routes quit Followed by 30 secs with awk(1), cut(1), diff(1), etc. For floundry, something dirt simple like "grep / | awk '{ print $2 }'" should do the trick. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Hi, Thanks for all the replies! I've consolidated them here hoping to save some noise....
From: Bill Nash <billn@odyssey.billn.net>
Were I faced with this reporting equirement on an on-going basis, I'd suggest establishing a read-only BGP peer with both devices and comparing directly. I've got a perl BGP peering daemon that feeds and maintains a mirror of the BGP routing table into SQL, applying updates and withdrawals as they come in. Setting up something similar, and adding some additional metrics to keep entries unique by peer source would facilitate your end goal with simple SQL grouping mechanics.
This is an idea, thank you. I was hoping for something that would be a bit more "smarter" than BGP . What I was looking for would be something that could say : Router A has route 216.231.96.0/24, 216.231.97.0/24, (etc) while Router B has 216.231.96.0/19 Router B has the following /30's : A.B.C.D, E.F.G.H, I.J.K.L Router A has 216.231.96.0/24, 216.231.97.0/24, but Router B has a route of 216.231.96.0/19 but none of the other /24's.
From: Richard A Steenbergen <ras@e-gerbil.net>
This is actually fairly common. There are a lot of folks out there who announce more specifics to one network but not another, or who apply no export or limited export community tags in various places. Also, every network has a different filter policy of what they will and won't accept.
I understood that this happened, but didn't think it could account for 3K to 10K routes. Guess it can. :)
FWIW my "exported to bgp speaking customers" count at this moment is 182525.
Thats in line with the CIDR report, and I wouldn't mind.
I wouldn't get concerned about it unless the network with more prefixes is doing something absurdly stupid like sending you internal /30s and such (which, well, a lot of people do :P). It could also be something like peers agreeing to traffic engineer by sending each other more specifics w/meds, though if they were smart they would be doing that with no-export so as to not make your TE job more difficult.
Thats what I'm hoping to find out. :)
If you really want to compare the differences, try something like:
telnet yourrouter | tee outputfile term length 0 sh ip bgp nei x.x.x.x received-routes quit
Followed by 30 secs with awk(1), cut(1), diff(1), etc. For floundry, something dirt simple like "grep / | awk '{ print $2 }'" should do the trick.
(See above what I was looking for the output, but again, something to start with, thanks!)
From: md@Linux.IT (Marco d'Itri)
On Apr 18, Scott Tuc Ellentuch at T-B-O-H <ml@t-b-o-h.net> wrote:
Is there a utility that I can use that will pull the routes off each router (Foundry preferred), and then compare them as best it can to see why there is such a difference? I have one, but it's cisco-specific:
http://www.bofh.it/~md/software/cisco-tools-0.2.tgz (the dumppeers script)
himinbjorg# fetch http://www.bofh.it/~md/software/cisco-tools-0.2.tgz fetch: http://www.bofh.it/~md/software/cisco-tools-0.2.tgz: Not Found
Then you can easily find the missing routes with commands like:
awk '{print $1}' < ../routes/1.2.3.4 | sort > ROUTER1 awk '{print $1}' < ../routes/1.2.3.5 | sort > ROUTER2 comm -23 ROUTER1 ROUTER2 > MISSING2
No worries, I'll take a look at it and then see if I can "Foundryize" it. :) Its not such a case of "missing" but maybe more aggregated differently, etc. But again, all leads will be taken!
From: John Kristoff <jtk@ultradns.net>
On Tue, 18 Apr 2006 16:13:12 -0400 (EDT) Scott "Tuc" Ellentuch at T-B-O-H <ml@t-b-o-h.net> wrote:
Is there a utility that I can use that will pull the routes off each router (Foundry preferred), and then compare them as best it can to see why there is such a difference?
I don't know anything about foundry, but if you can simply display the routing table from a terminal, you can go the hacky unix cli tool way. For example, use 'script' to log your terminal session to a file, then presuming you can show the route table and each route includes a 'via upstream-address-line' line for each route (completely untested and I'm sure someone could come up with something much simpler and better):
grep 'via upstream?' script > upstream? perl -ne 'print "$1\n" if /(\d{1,3}(?:\.\d{1,3}){3}\/\d{1,3})/' upstream? | sort > upstream?.sored comm -23 upstream1.txt upstream2.txt comm -13 upstream1.txt upstream2.txt
Thanks!
From: Warren Kumari <warren@kumari.net>
On Apr 18, 2006, at 1:19 PM, Mike Walter wrote:
Sounds to me like one of your providers is not feeding you the full internet routing table. Have you checked with them to see if they are providing you that?
Sounds to me like a: you are only looking at best routes or b: one of the providers is sending you more specific customer routes (that they summarize before sending to non-customers).
Personally I would just slurp one set of routes into an array in perl and then delete them if they appear in the other set. Any left over in either set are unique....
It wouldn't take aggregate differences into account.
From: "Majdi S. Abbas" <msa@latt.net>
We receive a BGP feed from different providers on two different routers. While one seems to be a reasonable amount of feeds after reviewing the CIDR report, the other is anywhere from 3K to 10K more routes.
Thanks, Tuc/TBOH -snip-
I refer both of you to the following message that I posted a few years ago, rather than restate it all:
http://www.merit.edu/mail.archives/nanog/2001-02/msg00347.html
Hope this helps.
--msa
No, I agree, I don't think I'm MISSING, just want to know what the differences are to see why there is such a disparity. Maybe I need to get the provider to filter or change communities, etc. ---------------------------------------------------------------------- Thanks everyone! Tuc/TBOH
On Tue, 18 Apr 2006 16:13:12 -0400 (EDT) Scott "Tuc" Ellentuch at T-B-O-H <ml@t-b-o-h.net> wrote:
Is there a utility that I can use that will pull the routes off each router (Foundry preferred), and then compare them as best it can to see why there is such a difference?
I don't know anything about foundry, but if you can simply display the routing table from a terminal, you can go the hacky unix cli tool way. For example, use 'script' to log your terminal session to a file, then presuming you can show the route table and each route includes a 'via upstream-address-line' line for each route (completely untested and I'm sure someone could come up with something much simpler and better): grep 'via upstream?' script > upstream? perl -ne 'print "$1\n" if /(\d{1,3}(?:\.\d{1,3}){3}\/\d{1,3})/' upstream? | sort > upstream?.sored comm -23 upstream1.txt upstream2.txt comm -13 upstream1.txt upstream2.txt John
On Apr 18, Scott Tuc Ellentuch at T-B-O-H <ml@t-b-o-h.net> wrote:
Is there a utility that I can use that will pull the routes off each router (Foundry preferred), and then compare them as best it can to see why there is such a difference? I have one, but it's cisco-specific:
http://www.bofh.it/~md/software/cisco-tools-0.2.tgz (the dumppeers script) Then you can easily find the missing routes with commands like: awk '{print $1}' < ../routes/1.2.3.4 | sort > ROUTER1 awk '{print $1}' < ../routes/1.2.3.5 | sort > ROUTER2 comm -23 ROUTER1 ROUTER2 > MISSING2 -- ciao, Marco
Were I faced with this reporting equirement on an on-going basis, I'd suggest establishing a read-only BGP peer with both devices and comparing directly. I've got a perl BGP peering daemon that feeds and maintains a mirror of the BGP routing table into SQL, applying updates and withdrawals as they come in. Setting up something similar, and adding some additional metrics to keep entries unique by peer source would facilitate your end goal with simple SQL grouping mechanics. - billn On Tue, 18 Apr 2006, Marco d'Itri wrote:
On Apr 18, Scott Tuc Ellentuch at T-B-O-H <ml@t-b-o-h.net> wrote:
Is there a utility that I can use that will pull the routes off each router (Foundry preferred), and then compare them as best it can to see why there is such a difference? I have one, but it's cisco-specific:
http://www.bofh.it/~md/software/cisco-tools-0.2.tgz (the dumppeers script)
Then you can easily find the missing routes with commands like:
awk '{print $1}' < ../routes/1.2.3.4 | sort > ROUTER1 awk '{print $1}' < ../routes/1.2.3.5 | sort > ROUTER2 comm -23 ROUTER1 ROUTER2 > MISSING2
-- ciao, Marco
Much of what Bill described below is already present using Nick Feamster's bgptools release: http://nms.lcs.mit.edu/software/bgp/ bgptools/ Start with zebra / quagga / etc., which do a great job of dumping tables and updates. Then use bgptools to take the MRT-formatted dumps that Zebra spits out and turn them into text, etc. With the '-q' option, can insert the BGP updates or table snapshot directly into a SQL database. then the libbgpdump.a library gives you lots of cool things on top of that. You'd have to do a little work to get the analysis tool you want, but it's pretty easy. Use the 'buildtree' starting program to build the prefix tree from each provider and then compare those two trees (see which prefixes are present/not present, see if any parts of the IP space are unreachable in in one and unreachable in the other, etc.) It starts as Bill suggested - a read-only BGP peer from the devices, which takes about 3 seconds to set up. -Dave On Apr 18, 2006, at 5:01 PM, Bill Nash wrote:
Were I faced with this reporting equirement on an on-going basis, I'd suggest establishing a read-only BGP peer with both devices and comparing directly. I've got a perl BGP peering daemon that feeds and maintains a mirror of the BGP routing table into SQL, applying updates and withdrawals as they come in. Setting up something similar, and adding some additional metrics to keep entries unique by peer source would facilitate your end goal with simple SQL grouping mechanics.
- billn
On Tue, 18 Apr 2006, Marco d'Itri wrote:
On Apr 18, Scott Tuc Ellentuch at T-B-O-H <ml@t-b-o-h.net> wrote:
Is there a utility that I can use that will pull the routes off each router (Foundry preferred), and then compare them as best it can to see why there is such a difference? I have one, but it's cisco-specific:
http://www.bofh.it/~md/software/cisco-tools-0.2.tgz (the dumppeers script)
Then you can easily find the missing routes with commands like:
awk '{print $1}' < ../routes/1.2.3.4 | sort > ROUTER1 awk '{print $1}' < ../routes/1.2.3.5 | sort > ROUTER2 comm -23 ROUTER1 ROUTER2 > MISSING2
-- ciao, Marco
On Tue, 18 Apr 2006, David Andersen wrote:
Much of what Bill described below is already present using Nick Feamster's bgptools release: http://nms.lcs.mit.edu/software/bgp/bgptools/
Start with zebra / quagga / etc., which do a great job of dumping tables and updates.
Then use bgptools to take the MRT-formatted dumps that Zebra spits out and turn them into text, etc. With the '-q' option, can insert the BGP updates or table snapshot directly into a SQL database.
My peer actually comes from a Zebra box, so I'm not talking directly to any production devices, in the event that I want to bounce my db feed up and down (debugging, featuritis treatments, etc) Z/Q + bgptools is a great suggestion for doing complex reporting/comparison on the routing tables, though. I've got a need for a more real-time view, so my setup fits me a little better than your suggestion, but potato/potatoe. =)
then the libbgpdump.a library gives you lots of cool things on top of that. You'd have to do a little work to get the analysis tool you want, but it's pretty easy. Use the 'buildtree' starting program to build the prefix tree from each provider and then compare those two trees (see which prefixes are present/not present, see if any parts of the IP space are unreachable in in one and unreachable in the other, etc.)
This is pretty interesting, I'll have to tinker with it, especially since I know one of my providers doesn't give me a full routing table.
It starts as Bill suggested - a read-only BGP peer from the devices, which takes about 3 seconds to set up.
And for folks to whom this is new stuff: don't be an idiot, put Zebra/Quagga up as a peer/buffer for attaching analysis tools to your network. *Never* attach development grade tools to a production device, most especially when you're dealing with a routing table. Not that I've ever taken down a live router in this manner[1], I'm just saying.. ;) - billn [1] All smirking current/past coworkers are kindly invited to stfu. =)
participants (7)
-
Bill Nash -
David Andersen -
John Kristoff -
Jon Lewis -
md＠Linux.IT -
Richard A Steenbergen -
Scott Tuc Ellentuch at T-B-O-H