BGP noob needs monitoring advice
Earlier this year I got a /24 of PA space, set up our shiny new router, got BGP working with both my upstreams, and heaved a sigh of relief: "I'll never have to think about THAT again!" (Okay, quit laughing; I SAID I was a noob!) Now, I discover that one of my upstreams quit announcing our route in November (fortunately the provider who assigned us the /24, so we're still covered in their /18) and the other upstream apparently started filtering our announcements last week. I'm working with both of them to get that fixed, but it's made it clear to me that I need to be monitoring this. My question for the group is, how? I can and do monitor my own router, and I can see that I'm receiving full routes from both ISPs. I am capable of manually accessing route servers and looking glass servers to check if they're receiving routes to me, but I'd like something more automated. Free is nice, $$ is not a problem, $$$$ might become a problem. Thanks in advance for any suggestions. -- Dave Pooser Manager of Information Services Alford Media http://www.alfordmedia.com
Hey: Manually speaking, you can always telnet to route-views.routeviews.org which is a restricted Cisco interface. Log in with username "rviews" and don't enable. From the prompt you can do all the "show ip bgp" commands you need to see whether or not your /24 is being announced via your upstream providers. As an example 'sho ip bgp x.x.x.x' where x.x.x.x is your /24. You should see the announcement originating from your AS over multiple providers that includes both of yours. If not, you know you have a problem. Mike -- Michael K. Smith - CISSP, GSEC, GISP Chief Technical Officer - Adhost Internet LLC mksmith@adhost.com w: +1 (206) 404-9500 f: +1 (206) 404-9050 PGP: B49A DDF5 8611 27F3 08B9 84BB E61E 38C0 (Key ID: 0x9A96777D)
-----Original Message----- From: Dave Pooser [mailto:dave.nanog@alfordmedia.com] Sent: Tuesday, December 20, 2011 10:53 AM To: nanog@nanog.org Subject: BGP noob needs monitoring advice
Earlier this year I got a /24 of PA space, set up our shiny new router, got BGP working with both my upstreams, and heaved a sigh of relief: "I'll never have to think about THAT again!" (Okay, quit laughing; I SAID I was a noob!)
Now, I discover that one of my upstreams quit announcing our route in November (fortunately the provider who assigned us the /24, so we're still covered in their /18) and the other upstream apparently started filtering our announcements last week. I'm working with both of them to get that fixed, but it's made it clear to me that I need to be monitoring this.
My question for the group is, how? I can and do monitor my own router, and I can see that I'm receiving full routes from both ISPs. I am capable of manually accessing route servers and looking glass servers to check if they're receiving routes to me, but I'd like something more automated. Free is nice, $$ is not a problem, $$$$ might become a problem.
Thanks in advance for any suggestions. -- Dave Pooser Manager of Information Services Alford Media http://www.alfordmedia.com
At 13:52 20/12/2011 -0500, Dave Pooser wrote: Use one of the following services: http://cyclops.cs.ucla.edu/ http://bgpmon.net/ You'll get an email whenever a routing change takes place in regards to the prefix you are monitoring. -Hank
Earlier this year I got a /24 of PA space, set up our shiny new router, got BGP working with both my upstreams, and heaved a sigh of relief: "I'll never have to think about THAT again!" (Okay, quit laughing; I SAID I was a noob!)
Now, I discover that one of my upstreams quit announcing our route in November (fortunately the provider who assigned us the /24, so we're still covered in their /18) and the other upstream apparently started filtering our announcements last week. I'm working with both of them to get that fixed, but it's made it clear to me that I need to be monitoring this.
My question for the group is, how? I can and do monitor my own router, and I can see that I'm receiving full routes from both ISPs. I am capable of manually accessing route servers and looking glass servers to check if they're receiving routes to me, but I'd like something more automated. Free is nice, $$ is not a problem, $$$$ might become a problem.
Thanks in advance for any suggestions. -- Dave Pooser Manager of Information Services Alford Media http://www.alfordmedia.com
Is http://cyclops.cs.ucla.edu/ still working? I don't seem to received emails from them anymore when we stop announcing to one of our upstream providers. On the other hand http://bgpmon.net/ does send me emails when an announcement disappears from an upstream, although it's usually a day later. On 12/20/2011 02:03 PM, Hank Nussbacher wrote:
At 13:52 20/12/2011 -0500, Dave Pooser wrote:
Use one of the following services: http://cyclops.cs.ucla.edu/ http://bgpmon.net/ You'll get an email whenever a routing change takes place in regards to the prefix you are monitoring.
-Hank
Earlier this year I got a /24 of PA space, set up our shiny new router, got BGP working with both my upstreams, and heaved a sigh of relief: "I'll never have to think about THAT again!" (Okay, quit laughing; I SAID I was a noob!)
Now, I discover that one of my upstreams quit announcing our route in November (fortunately the provider who assigned us the /24, so we're still covered in their /18) and the other upstream apparently started filtering our announcements last week. I'm working with both of them to get that fixed, but it's made it clear to me that I need to be monitoring this.
My question for the group is, how? I can and do monitor my own router, and I can see that I'm receiving full routes from both ISPs. I am capable of manually accessing route servers and looking glass servers to check if they're receiving routes to me, but I'd like something more automated. Free is nice, $$ is not a problem, $$$$ might become a problem.
Thanks in advance for any suggestions. -- Dave Pooser Manager of Information Services Alford Media http://www.alfordmedia.com
Hi, .-- My secret spy satellite informs me that at 11-12-20 11:16 AM Bret Clark wrote:
Is http://cyclops.cs.ucla.edu/ still working? I don't seem to received emails from them anymore when we stop announcing to one of our upstream providers. On the other hand http://bgpmon.net/ does send me emails when an announcement disappears from an upstream, although it's usually a day later.
Just to clarify this: For all alert types below BGPmon.net sends out an alert within minutes: 1) prefix withdrawal (prefix disappeared) 2) new upstream 3) new prefix 4) origin AS changes 5) ASpath regex failure 6) policy violation 7) RPKI validation failure There's one other feature, the routing-report feature, that runs only once a day. It's similar as the cidr report, but specific to your AS. I like to refer to it as a rancid for your BGP announcements. It's basically a diff between how your routes were visible today and yesterday. This specific feature will also notify the user if you lost / gained one or more upstreams per prefix. Also see http://bgpmon.net/blog/?p=257 for more information about that specific feature. Cheers, Andree
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12/20/2011 2:43 PM, Andree Toonk wrote:
Hi,
.-- My secret spy satellite informs me that at 11-12-20 11:16 AM Bret Clark wrote:
Is http://cyclops.cs.ucla.edu/ still working? I don't seem to received emails from them anymore when we stop announcing to one of our upstream providers. On the other hand http://bgpmon.net/ does send me emails when an announcement disappears from an upstream, although it's usually a day later.
Just to clarify this: For all alert types below BGPmon.net sends out an alert within minutes: 1) prefix withdrawal (prefix disappeared) 2) new upstream 3) new prefix 4) origin AS changes 5) ASpath regex failure 6) policy violation 7) RPKI validation failure
There's one other feature, the routing-report feature, that runs only once a day. It's similar as the cidr report, but specific to your AS. I like to refer to it as a rancid for your BGP announcements.
It's basically a diff between how your routes were visible today and yesterday. This specific feature will also notify the user if you lost / gained one or more upstreams per prefix. Also see http://bgpmon.net/blog/?p=257 for more information about that specific feature.
Unless I'm misunderstanding something, I'm concerned regarding the IPv4 bogon list on http://bgpmon.net/showbogons.php?inet=4 . It clearly includes several /8's that should not be there. The data seems to be stale as if some job is no longer pulling the updated data. It states it's being pulled from http://www.cymru.com/Documents/bogon-bn-nonagg.txt , but that clearly does not contain 100/8, 5/8, 181/8, 49/8 and a few others... and hasn't for quite some time. - -Vinny -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) iEYEARECAAYFAk7x3G8ACgkQUyX7ywEAl3rMjACg87ma/guBPU8mmhy/jfxz6Dzx s6wAnRGXMTU2P1tRE+Azm+eSMKW1YENL =RyFn -----END PGP SIGNATURE-----
Hi Vinny, .-- My secret spy satellite informs me that at 11-12-21 5:17 AM Vinny Abello wrote:
Unless I'm misunderstanding something, I'm concerned regarding the IPv4 bogon list on http://bgpmon.net/showbogons.php?inet=4 . It clearly includes several /8's that should not be there. The data seems to be stale as if some job is no longer pulling the updated data. It states it's being pulled from http://www.cymru.com/Documents/bogon-bn-nonagg.txt , but that clearly does not contain 100/8, 5/8, 181/8, 49/8 and a few others... and hasn't for quite some time.
The http://bgpmon.net/showbogons.php?inet=4 page show a list of bogons that were announced at a certain point in time, so the page show historical announcements as well (check the date). For example the last 100/8 bogons were detected on 2010-10-29, at that time it was still considered a bogon. The list is not stale, there's just very few IPv4 bogons left :) We do still see RFC1918 announcements: http://www.bgpmon.net/showbogons.php?inet=4&global=yes&private=yes And of course IPv6 bogons, Last month for example: 18c::/16 http://www.bgpmon.net/showbogons.php?global=yes&private=yes&inet=6 Cheers, Andree
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12/21/2011 12:31 PM, Andree Toonk wrote:
Hi Vinny,
.-- My secret spy satellite informs me that at 11-12-21 5:17 AM Vinny Abello wrote:
Unless I'm misunderstanding something, I'm concerned regarding the IPv4 bogon list on http://bgpmon.net/showbogons.php?inet=4 . It clearly includes several /8's that should not be there. The data seems to be stale as if some job is no longer pulling the updated data. It states it's being pulled from http://www.cymru.com/Documents/bogon-bn-nonagg.txt , but that clearly does not contain 100/8, 5/8, 181/8, 49/8 and a few others... and hasn't for quite some time.
The http://bgpmon.net/showbogons.php?inet=4 page show a list of bogons that were announced at a certain point in time, so the page show historical announcements as well (check the date).
For example the last 100/8 bogons were detected on 2010-10-29, at that time it was still considered a bogon.
The list is not stale, there's just very few IPv4 bogons left :) We do still see RFC1918 announcements: http://www.bgpmon.net/showbogons.php?inet=4&global=yes&private=yes
And of course IPv6 bogons, Last month for example: 18c::/16 http://www.bgpmon.net/showbogons.php?global=yes&private=yes&inet=6
Ahh, that would be the part where I'm misunderstanding something, the date! LOL... Makes sense now. I thought this was a recent snapshot all from 2011. I see of course that I was wrong. Thanks for clearing that up for me. My excuse is lack of coffee when I wrote that. ;) - -Vinny -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) iEYEARECAAYFAk7yW/EACgkQUyX7ywEAl3opowCgsFRAy3osAmML5T6LncNLNzWh UBsAniwvOTKrOT7uwIYWBDie29MDq47r =H3PD -----END PGP SIGNATURE-----
Depending on the nature of your redundant connections, your traffic engineering/bgp settings, and the visibility of the routing through the lost provider to the internet route servers mentioned, you may/may not be able to easily monitor this. Some failures are harder to find than others. Suggestions: 1) On the provider that stopped accepting your prefix, your inbound traffic would have dropped to 0. Monitor for this if this isn't by design already. 2) Use the bgpmon suggested by Dave below to see events which are visible to the route server they use. On Tue, Dec 20, 2011 at 1:03 PM, Hank Nussbacher <hank@efes.iucc.ac.il>wrote:
At 13:52 20/12/2011 -0500, Dave Pooser wrote:
Use one of the following services: http://cyclops.cs.ucla.edu/ http://bgpmon.net/ You'll get an email whenever a routing change takes place in regards to the prefix you are monitoring.
-Hank
Earlier this year I got a /24 of PA space, set up our shiny new router,
got BGP working with both my upstreams, and heaved a sigh of relief: "I'll never have to think about THAT again!" (Okay, quit laughing; I SAID I was a noob!)
Now, I discover that one of my upstreams quit announcing our route in November (fortunately the provider who assigned us the /24, so we're still covered in their /18) and the other upstream apparently started filtering our announcements last week. I'm working with both of them to get that fixed, but it's made it clear to me that I need to be monitoring this.
My question for the group is, how? I can and do monitor my own router, and I can see that I'm receiving full routes from both ISPs. I am capable of manually accessing route servers and looking glass servers to check if they're receiving routes to me, but I'd like something more automated. Free is nice, $$ is not a problem, $$$$ might become a problem.
Thanks in advance for any suggestions. -- Dave Pooser Manager of Information Services Alford Media http://www.alfordmedia.com
On 12/20/2011 1:52 PM, Dave Pooser wrote:
My question for the group is, how? I can and do monitor my own router, and I can see that I'm receiving full routes from both ISPs. I am capable of
you might want to start with a good monitoring software like Argus - http://argus.tcp4me.com/ Group "Upstream Connections" { Group "T3 to whomever" { Service Ping { hostname: far-side.example.net } Service UDP/SNMP { eqvalue: 6 label: BGP uname: BGP oid: .1.3.6.1.2.1.15.3.1.2.x.x.x.x hostname: your-router.example.net } } Group "T3 to whomever2" { Service Ping { hostname: far-other-side.example.net } Service UDP/SNMP { eqvalue: 6 label: BGP uname: BGP oid: .1.3.6.1.2.1.15.3.1.2.x.x.x.x hostname: your-router.example.net } } } something like that will alert you when BGP is anything other than happy. your oid may vary. use snmpwalk to help. then you could also add: Service Prog { frequency: 1800 command: chkbgp.pl -a <ASN> -n <network> -r <route_server> nexepect: evil } *http://jeremy.kister.net/code/perl/chkbgp.pl -- Jeremy Kister http://jeremy.kister.net./
participants (9)
-
Andree Toonk -
Bret Clark -
Dave Pooser -
Hank Nussbacher -
Jeremy Kister -
Michael K. Smith - Adhost -
PC -
Richard Laager -
Vinny Abello