Internet routing table "completeness" monitoring?
Has anyone put in place a method to identify if one their BGP peers suddenly withdraws X% of their prefixes? e.g I should expect ~420k prefixes in a "complete"[1] routing table from a transit peer today. If suddenly I'm only getting 390k prefixes I'd guess a major network was depeered or similiar. If so how are people doing this? SNMP MIB, screen scrape? [1] Varying levels of completeless apply.
On (2012-10-03 00:43 -0400), ML wrote:
Has anyone put in place a method to identify if one their BGP peers suddenly withdraws X% of their prefixes?
I've had monitoring for this for many years, over SNMP. Right now my limits are a) prefix count went or came from 0 or b) relative difference is minimum 1.5x and absolute difference is minimum of 1000 Output what I get as emails: rtr1: AS702 2001:600:202::15 ge-1-0-4.BR2.LND18.ALTER.NET 0 => 34 rtr2: AS2119 148.122.8.213 ti3001b300-ge3-1-0.ti.telenor.net 688 => 0 (1/3) rtr2: AS2119 2001:4600:10::4d ti3001b300-ge3-1-0.ti.telenor.net 13 => 0 (2/3) rtr3: AS3491 80.81.192.50 br02.frf02.pccwbtn.net 37548 => 4710 And there are about 10-20 emails per day, even when looking only rather 'coarse' changes. But to be honest, I almost never peek at the folder where I get these, I'm probably moving the output on IRC channel, as I've found it superior way to keep track of alarms compared to emails for my workflow. -- ++ytti
----- Original Message -----
From: "ML" <ml@kenweb.org>
Has anyone put in place a method to identify if one their BGP peers suddenly withdraws X% of their prefixes?
e.g I should expect ~420k prefixes in a "complete"[1] routing table from a transit peer today. If suddenly I'm only getting 390k prefixes I'd guess a major network was depeered or similiar.
If so how are people doing this? SNMP MIB, screen scrape?
Well, if I had to do it, I think I'd just point munin at the router, yes, using SNMP, and put the prefix count graph up on the Big Wall, as a filled curve. That thing jumps around, someone will likely notice. This assumes a staffed NOC, of course, but it still gives you something to look at historically if you note a problem in an unstaffed situation as well. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
I have cacti graph the amount of prefixes announced and withdrawn from a BGP peer on each BGP router. -----Original Message----- From: ML [mailto:ml@kenweb.org] Sent: Tuesday, October 02, 2012 11:43 PM To: North American Networking and Offtopic Gripes List Subject: Internet routing table "completeness" monitoring? Has anyone put in place a method to identify if one their BGP peers suddenly withdraws X% of their prefixes? e.g I should expect ~420k prefixes in a "complete"[1] routing table from a transit peer today. If suddenly I'm only getting 390k prefixes I'd guess a major network was depeered or similiar. If so how are people doing this? SNMP MIB, screen scrape? [1] Varying levels of completeless apply.
I agree, and just use the Threshold plugin so when it drops below or goes above a certain # to notify you. http://docs.cacti.net/plugin:thold -----Original Message----- From: Joseph Jackson [mailto:jjackson@aninetworks.net] Sent: Wednesday, October 03, 2012 9:51 AM To: ml@kenweb.org; North American Networking and Offtopic Gripes List Subject: RE: Internet routing table "completeness" monitoring? I have cacti graph the amount of prefixes announced and withdrawn from a BGP peer on each BGP router.
On Wed, Oct 3, 2012 at 9:55 AM, Eric Tykwinski <eric-list@truenet.com> wrote:
I agree, and just use the Threshold plugin so when it drops below or goes above a certain # to notify you. http://docs.cacti.net/plugin:thold
is a threshold helpful here? (well, it's helpful to a point at least) what if your neighbour starts deaggragating (or sending you their internal deaggragates) in place of 50k real routes? no alarm, no 'change' from a numbers perspective, but certainly a traffic shift and reach-ability change :( Isn't a speed-of-change threshold also interesting here?
On Wed, 3 Oct 2012, Christopher Morrow wrote:
is a threshold helpful here? (well, it's helpful to a point at least) what if your neighbour starts deaggragating (or sending you their internal deaggragates) in place of 50k real routes? no alarm, no 'change' from a numbers perspective, but certainly a traffic shift and reach-ability change :(
As long as you have some control over the number of polling intervals between the detection of a noteworthy change and sending an alarm. Otherwise, there is a real danger of your NOC having to investigate a lot of noisy alerts. If that persists for too long, the NOC will grow tired of responding to these alerts, and send them all to the bit bucket, or implement their own polling thresholds that meet their needs more effectively. If a network you have no business relationship with and several AS hops away from you goes away, how much effort do you want to expend investigating that? That probably depends on your customers. If you see a few hundred routes disappear and determine them to be for an ISP on the other side of the planet, that's one thing. If your view of something like Google or Facebook suddenly disappears, that could be another thing entirely ;)
Isn't a speed-of-change threshold also interesting here?
+1 on that :) jms
On Wed, 3 Oct 2012, Joseph Jackson wrote:
I have cacti graph the amount of prefixes announced and withdrawn from a BGP peer on each BGP router.
+1 Note that not all router OSs support fetching data like that via SNMP. We use a custom built thing internally that does this two, which we then tack on an alert threshold for. So if a downstream peer sends us less than that, we get an alert. Handy for those times when they call and ask us what we did to their network. :-) Prior to that, we had a script which whould login, munge the 'show ip bgp summary' table output, figure out the deltas and graph or report as needed on a particularly troublesome peer.
-----Original Message----- From: ML [mailto:ml@kenweb.org] Sent: Tuesday, October 02, 2012 11:43 PM To: North American Networking and Offtopic Gripes List Subject: Internet routing table "completeness" monitoring?
Has anyone put in place a method to identify if one their BGP peers suddenly withdraws X% of their prefixes?
e.g I should expect ~420k prefixes in a "complete"[1] routing table from a transit peer today. If suddenly I'm only getting 390k prefixes I'd guess a major network was depeered or similiar.
If so how are people doing this? SNMP MIB, screen scrape?
[1] Varying levels of completeless apply.
wfms
On Wed, 3 Oct 2012, Joseph Jackson wrote:
I have cacti graph the amount of prefixes announced and withdrawn from a BGP peer on each BGP router.
+1
Note that not all router OSs support fetching data like that via SNMP.
We use a custom built thing internally that does this two, which we
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/3/2012 10:16 AM, William F. Maton Sotomayor wrote: then tack on an alert threshold for. So if a downstream peer sends us less than that, we get an alert. Handy for those times when they call and ask us what we did to their network. :-)
Prior to that, we had a script which whould login, munge the 'show ip
bgp summary' table output, figure out the deltas and graph or report as needed on a particularly troublesome peer.
-----Original Message----- From: ML [mailto:ml@kenweb.org] Sent: Tuesday, October 02, 2012 11:43 PM To: North American Networking and Offtopic Gripes List Subject: Internet routing table "completeness" monitoring?
Has anyone put in place a method to identify if one their BGP peers
suddenly withdraws X% of their prefixes?
e.g I should expect ~420k prefixes in a "complete"[1] routing table
from a transit peer today. If suddenly I'm only getting 390k prefixes I'd guess a major network was depeered or similiar.
If so how are people doing this? SNMP MIB, screen scrape?
[1] Varying levels of completeless apply.
wfms
So, there is something called the BGP Monitoring Protocol (BMP): http://tools.ietf.org/html/draft-ietf-grow-bmp-06 http://www.nanog.org/meetings/nanog45/abstracts.php?pt=MTM2NiZuYW5vZzQ1&nm=nanog45 Looks like it is supported in JunOS. Has anyone used it? If so, what monitoring software are you using? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iQEcBAEBAgAGBQJQbFsYAAoJEBxhAh+LWUKi/WwIAKxVmcCfI6xng0lAL5UxHSGK v+fp5Q1d40hCu1aWWeeV/eYvyOEZSwkY/jSK0EaVrdmv0y4GQttQqSfGk+1RQOK/ jZAD/r0Fd9K9vMLJ5Te+bkjYuOp23G2bDn8QWuls4HBc1nHxkPc6arzUTgTzEFAZ +Ljk3MQSHiZ+nb6fkX+Yb7e3UJZjpkuWaNeqGcKz9FR9PJT5UZI8Yf2iwMHtVl/l R9XfAIG/xDd64oKEDbt8JmtQkqS64ZnCFe+B/yEuTKC6Pl0DUm4orH7322oxjxOW yDloqmhO5pCmVETFHwn0ocTpDKqG2zE4zA4bHL+w+xePSS61DpUf51V3eg8JgZo= =Ofld -----END PGP SIGNATURE-----
participants (9)
-
Andrew Gallo
-
Christopher Morrow
-
Eric Tykwinski
-
Jay Ashworth
-
Joseph Jackson
-
Justin M. Streiner
-
ML
-
Saku Ytti
-
William F. Maton Sotomayor