validating reachability via an ISP
Hi all, I have an enterprise network and do not provide transit. In one of our datacenters we have our own prefixes and rely on two ISPs as BGP neighbors to provide global reachability for our prefixes. One is a large regional provider and the other is a large global provider. Recently we took our link to the global provider offline to perform maintenance on our router. Nearly immediately we were hit with alerts that our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our route had been withdrawn. We were not unreachable from every AS, but we certainly were from some of the largest. The root cause is that the our prefix is not being adequately re-distributed globally by the regional ISP. This is unexpected and we are working through this with them now. My question is, how can I monitor global reachability for a prefix via this or any specific provider I use over time? Are there various route-servers I can programmatically query for my prefix and get results that include AS paths? Then I could verify that an "acceptable" number of paths exist that include the AS of the all the ISPs I rely upon. And what would an "acceptable" number of alternate paths be? thanks in advance, -andy
On 3/29/2018 2:22 AM, Andy Litzinger wrote:
Hi all, I have an enterprise network and do not provide transit. In one of our datacenters we have our own prefixes and rely on two ISPs as BGP neighbors to provide global reachability for our prefixes. One is a large regional provider and the other is a large global provider.
Recently we took our link to the global provider offline to perform maintenance on our router. Nearly immediately we were hit with alerts that our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our route had been withdrawn. We were not unreachable from every AS, but we certainly were from some of the largest.
The root cause is that the our prefix is not being adequately re-distributed globally by the regional ISP. This is unexpected and we are working through this with them now.
My question is, how can I monitor global reachability for a prefix via this or any specific provider I use over time? Are there various route-servers I can programmatically query for my prefix and get results that include AS paths? Then I could verify that an "acceptable" number of paths exist that include the AS of the all the ISPs I rely upon. And what would an "acceptable" number of alternate paths be?
If your global provider supports, you could send your announcements with a BGP community per RFC1998 telling them to not-prefer-so-much that advertisement, "use it as a backup". that would shift a lot of incoming traffic to the other link (regional provider). You'll still have the global provider link. this is a smaller change towards taking global provider offline, keeping some fallback. Frank
If your prefix is larger than /24 you can test with a more specific prefix such as a /24. Announce the test prefix on just one transit provider. Then check with BGP services such as the looking glass service provided by the NLNOG RING network. There will be no interruption in the traffic as it will follow the less specific prefix everywhere the test prefix was not picked up. If you only have a /24 you will need to test in a service window. Regards Baldur Den tor. 29. mar. 2018 01.24 skrev Andy Litzinger < andy.litzinger.lists@gmail.com>:
Hi all, I have an enterprise network and do not provide transit. In one of our datacenters we have our own prefixes and rely on two ISPs as BGP neighbors to provide global reachability for our prefixes. One is a large regional provider and the other is a large global provider.
Recently we took our link to the global provider offline to perform maintenance on our router. Nearly immediately we were hit with alerts that our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our route had been withdrawn. We were not unreachable from every AS, but we certainly were from some of the largest.
The root cause is that the our prefix is not being adequately re-distributed globally by the regional ISP. This is unexpected and we are working through this with them now.
My question is, how can I monitor global reachability for a prefix via this or any specific provider I use over time? Are there various route-servers I can programmatically query for my prefix and get results that include AS paths? Then I could verify that an "acceptable" number of paths exist that include the AS of the all the ISPs I rely upon. And what would an "acceptable" number of alternate paths be?
thanks in advance, -andy
On Wed, Mar 28, 2018 at 7:22 PM, Andy Litzinger <andy.litzinger.lists@gmail.com> wrote:
Hi all, I have an enterprise network and do not provide transit. In one of our datacenters we have our own prefixes and rely on two ISPs as BGP neighbors to provide global reachability for our prefixes. One is a large regional provider and the other is a large global provider.
Recently we took our link to the global provider offline to perform maintenance on our router. Nearly immediately we were hit with alerts that our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our route had been withdrawn. We were not unreachable from every AS, but we certainly were from some of the largest.
The root cause is that the our prefix is not being adequately re-distributed globally by the regional ISP. This is unexpected and we are working through this with them now.
My question is, how can I monitor global reachability for a prefix via this or any specific provider I use over time? Are there various route-servers I can programmatically query for my prefix and get results that include AS paths? Then I could verify that an "acceptable" number of paths exist that include the AS of the all the ISPs I rely upon. And what would an "acceptable" number of alternate paths be?
I did something similar a few years ago, by querying routeviews and validating AS paths using python and pexpect. You could adapt it for your use pretty easily. The code's here: https://github.com/awentzell/check-as-path
Hi Andrew, thanks for sharing your repo, it looks very relevant and I should be able to get it to do what I want with a slight tweak to the regex's. Hopefully the route-views collectors or some other ones I dig up will be able to adequately show at least 3 AS path hops through my regional provider to get to my AS/prefix. I really want to see AS paths where there are at least 2 different AS hops before my regional providers AS. This will help me feel good that the global partners of my regional ISP are re-advertising my prefix to their peers. Otherwise if the AS path only includes the first and second hop upstream I can only infer that the 2nd hop AS is accepting routes from the 1st hop AS (my regional ISP), but not that the 2nd hop AS is actually re-advertising the route to anyone else. thanks! -andy On Thu, Mar 29, 2018 at 7:51 AM, Andrew Wentzell <awentzell@gmail.com> wrote:
Hi all, I have an enterprise network and do not provide transit. In one of our datacenters we have our own prefixes and rely on two ISPs as BGP neighbors to provide global reachability for our prefixes. One is a large regional provider and the other is a large global provider.
Recently we took our link to the global provider offline to perform maintenance on our router. Nearly immediately we were hit with alerts
our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our route had been withdrawn. We were not unreachable from every AS, but we certainly were from some of the largest.
The root cause is that the our prefix is not being adequately re-distributed globally by the regional ISP. This is unexpected and we are working through this with them now.
My question is, how can I monitor global reachability for a prefix via
or any specific provider I use over time? Are there various route-servers I can programmatically query for my prefix and get results that include AS paths? Then I could verify that an "acceptable" number of paths exist
On Wed, Mar 28, 2018 at 7:22 PM, Andy Litzinger <andy.litzinger.lists@gmail.com> wrote: that this that
include the AS of the all the ISPs I rely upon. And what would an "acceptable" number of alternate paths be?
I did something similar a few years ago, by querying routeviews and validating AS paths using python and pexpect. You could adapt it for your use pretty easily. The code's here:
Hi Andy, You can use Qrator.Radar API: https://api.radar.qrator.net/. The get-all-paths method will return the set of active paths for selected prefix. 2018-03-29 2:22 GMT+03:00 Andy Litzinger <andy.litzinger.lists@gmail.com>:
Hi all, I have an enterprise network and do not provide transit. In one of our datacenters we have our own prefixes and rely on two ISPs as BGP neighbors to provide global reachability for our prefixes. One is a large regional provider and the other is a large global provider.
Recently we took our link to the global provider offline to perform maintenance on our router. Nearly immediately we were hit with alerts that our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our route had been withdrawn. We were not unreachable from every AS, but we certainly were from some of the largest.
The root cause is that the our prefix is not being adequately re-distributed globally by the regional ISP. This is unexpected and we are working through this with them now.
My question is, how can I monitor global reachability for a prefix via this or any specific provider I use over time? Are there various route-servers I can programmatically query for my prefix and get results that include AS paths? Then I could verify that an "acceptable" number of paths exist that include the AS of the all the ISPs I rely upon. And what would an "acceptable" number of alternate paths be?
thanks in advance, -andy
-- | Alexander Azimov | HLL l QRATOR | tel.: +7 499 241 81 92 | mob.: +7 915 360 08 86 | skype: mitradir | mailto: aa@qrator.net | visit: www.qrator.net
Hi, If you are fine with a commercial tool, Packet Design Route Explorer product supports peering analysis: BGP-RIB-Visualization-1.png <https://www.packetdesign.com/wp-content/uploads/2018/02/BGP-RIB-Visualization-1.png> is one among its features. https://www.packetdesign.com/solutions/peering-analysis/ Disclaimer: I work for Packet Design Thanks, Don - Don Thomas Jacob LinkedIn <https://www.linkedin.com/in/donthomasjacob/> | Twitter <https://twitter.com/DonThomasJacob> On Thu, Mar 29, 2018 at 8:39 PM, Alexander Azimov <aa@qrator.net> wrote:
Hi Andy,
You can use Qrator.Radar API: https://api.radar.qrator.net/. The get-all-paths method will return the set of active paths for selected prefix.
2018-03-29 2:22 GMT+03:00 Andy Litzinger <andy.litzinger.lists@gmail.com>:
Hi all, I have an enterprise network and do not provide transit. In one of our datacenters we have our own prefixes and rely on two ISPs as BGP neighbors to provide global reachability for our prefixes. One is a large regional provider and the other is a large global provider.
Recently we took our link to the global provider offline to perform maintenance on our router. Nearly immediately we were hit with alerts that our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our route had been withdrawn. We were not unreachable from every AS, but we certainly were from some of the largest.
The root cause is that the our prefix is not being adequately re-distributed globally by the regional ISP. This is unexpected and we are working through this with them now.
My question is, how can I monitor global reachability for a prefix via this or any specific provider I use over time? Are there various route-servers I can programmatically query for my prefix and get results that include AS paths? Then I could verify that an "acceptable" number of paths exist that include the AS of the all the ISPs I rely upon. And what would an "acceptable" number of alternate paths be?
thanks in advance, -andy
-- | Alexander Azimov | HLL l QRATOR | tel.: +7 499 241 81 92 | mob.: +7 915 360 08 86 | skype: mitradir | mailto: aa@qrator.net | visit: www.qrator.net
On 29/03/2018, 00:22, Andy Litzinger <andy.litzinger.lists@gmail.com> wrote:
The root cause is that the our prefix is not being adequately re-distributed globally by the regional ISP. This is unexpected and we are working through this with them now.
Hi, Andy — Are you failing to advertise it, or are they filtering it on ingress, or are they failing to send it to their other peers? One configuration mishap which is starting to come along more and more partial or poor reachability caused by route objects which are not correctly published in the IRRDB. It is going to be essential to make sure that you have properly recorded IRR route objects in, for instance, RADB. More BGP speakers properly filter their peers using information that is published there. Avoid future reachability problems by checking this today! Yours, A friendly route-server operator with strict filtering -a -- Andy Davidson Asteroid International BV https://www.asteroidhq.com @asteroidhq @andyd -------------------------------------------------- Local interconnection. Where you need it.
+1 for Route Explorer On Thu, Apr 5, 2018 at 2:49 AM, Andy Davidson <andy@nosignal.org> wrote:
On 29/03/2018, 00:22, Andy Litzinger <andy.litzinger.lists@gmail.com> wrote:
The root cause is that the our prefix is not being adequately re-distributed globally by the regional ISP. This is unexpected and we
are
working through this with them now.
Hi, Andy —
Are you failing to advertise it, or are they filtering it on ingress, or are they failing to send it to their other peers?
One configuration mishap which is starting to come along more and more partial or poor reachability caused by route objects which are not correctly published in the IRRDB. It is going to be essential to make sure that you have properly recorded IRR route objects in, for instance, RADB. More BGP speakers properly filter their peers using information that is published there. Avoid future reachability problems by checking this today!
Yours, A friendly route-server operator with strict filtering
-a
-- Andy Davidson Asteroid International BV https://www.asteroidhq.com @asteroidhq @andyd -------------------------------------------------- Local interconnection. Where you need it.
Hi Andy, The root cause was they regional ISP was failing to advertise my prefix due to a mistake in their export policy. While I'm glad we were able to figure out the issue I'm generally more interested in figuring out a way that I can programmatically monitor that my ISPs are providing me with good reachability, even when their route isn't necessarily the best/installed route to me. I do have route objects defined in an IRR for this prefix. Perhaps you wouldn't mind explaining to me how route-server or ISP operators generally validate route-objects before accepting routes? Especially in the case where I'm not peering with your route server but my ISP is. Do you query the IRR DB to recurse from the ISP AS to my AS and validate route objects there? thanks! -andy On Thu, Apr 5, 2018 at 12:49 AM, Andy Davidson <andy@nosignal.org> wrote:
On 29/03/2018, 00:22, Andy Litzinger <andy.litzinger.lists@gmail.com> wrote:
The root cause is that the our prefix is not being adequately re-distributed globally by the regional ISP. This is unexpected and we
are
working through this with them now.
Hi, Andy —
Are you failing to advertise it, or are they filtering it on ingress, or are they failing to send it to their other peers?
One configuration mishap which is starting to come along more and more partial or poor reachability caused by route objects which are not correctly published in the IRRDB. It is going to be essential to make sure that you have properly recorded IRR route objects in, for instance, RADB. More BGP speakers properly filter their peers using information that is published there. Avoid future reachability problems by checking this today!
Yours, A friendly route-server operator with strict filtering
-a
-- Andy Davidson Asteroid International BV https://www.asteroidhq.com @asteroidhq @andyd -------------------------------------------------- Local interconnection. Where you need it.
participants (8)
-
Alexander Azimov
-
Andrew Wentzell
-
Andy Davidson
-
Andy Litzinger
-
Baldur Norddahl
-
Ben Bartsch
-
Don Thomas Jacob
-
Frank Habicht