BGP Optimizers (Was: Validating possible BGP MITM attack)
Dear all, disclaimer: [ The following is targetted at the context where a BGP optimizer generates BGP announcement that are ordinarily not seen in the Default-Free Zone. The OP indicated they announce a /23, and were unpleasantly surprised to see two unauthorized announcements for /24 more-specifics pop up in their alerting system. No permission was granted to create and announce these more-specifics. The AS_PATH for those /24 announcements was entirely fabricated. Original thread https://mailman.nanog.org/pipermail/nanog/2017-August/092124.html ] On Thu, Aug 31, 2017 at 11:13:18AM -0700, Andy Litzinger wrote:
Presuming it was a route optimizer and the issue was ongoing, what would be the suggested course of action?
I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked. It is extremely irresponsible behavior to use software that generates _fake_ BGP more-specifics for the purpose of traffic engineering. You simply cannot expect that those more-specifics will never escape into the global DFZ! Relying on NO_EXPORT is not sufficient: we regularly see software bugs related to NO_EXPORT, and community-squashing configuration mistakes happen all the time. Consider the following: if you leak your own internal more-specifics, at least you are the legitimate destination. (You may suffer from suboptimal routing, but it isn't guaranteed downtime.) However if you generate fake more-specifics for prefixes belonging to OTHER organisations, you essentially are complicit in BGP hijacking. If those fake more-specifics accidentally leak into the DFZ, you are bringing down the actual owner of such prefixes, and depriving people from access to the Internet. Example case: https://mailman.nanog.org/pipermail/nanog/2013-January/054846.html
reach out to those 2 AS owners and see if they could stop it?
Yes, absolutely! And if everyone of the affected parties of this localized hijack leak (or should we say 'victims') reaches out to the wrongdoers, they contribute peer pressure to rectify the situation. Just make sure you assign blame to the correct party. :)
Or is it something I just have to live with as a traffic engineering solution they are using and mark the alerts as false positives?
No, this is not something we should just accept. The Default-Free Zone is a shared resource. Anyone using "BGP optimizers" is not only risking their own online presence, but also everyone else's. Using a BGP optimizer is essentially trading a degree of risk to society for the purpose of saving a few bucks or milliseconds. It is basically saying: "The optimizer helps me, but may hurt others, and I am fine with that". An analogy: We don't want transporters of radioactive material to cut corners by using non-existant roads to bring the spent nuclear fuels from A to B faster, nor do we want them to rely on non-robust shielding that works "most of the time". We share the BGP DFZ environment, 'BGP optimizers' are downright pollution contributors. Kind regards, Job ps. If you want to do BGP optimization: don't have the BGP optimizer generate fake BGP announcements, but focus only on manipulating non-transitive attributes (like LOCAL_PREF) on real announcements you actually received from others. Or Perhaps BGP optimizers shouldn't even talk BGP but rather push freshly generated TE-optimized routing-policy to your edge boxes. Or perhaps the 'BGP optimizer' vendors should come to IETF to figure out a safe way (maybe a new AFI/SAFI to manipulate real announcements)... there are ways to do this, without risk to others! p.s. providing a publicly available BGP looking glasses will contribute to proving your innocence in cases like these. Since in many cases the AS_PATH is a complete fabrication, we need to manually check every AS in the AS_PATH to see whether the AS carries the fake more-specific. A public looking glass speeds up this fault-finding process. If you don't want to host a webinterface yourself, please consider sending a BGP feed to the Route Views Project or RIPE RIS, or for something queryable in a real-time fashion the NLNOG RING Looking Glass http://lg.ring.nlnog.net/
I would like to use a BGP optimizer, but I'm too poor. :-\ That said, I'm also an eyeball network, so modifications of my own advertisements are what affects the desired traffic, not so much the outbound routes. I know the BGP optimization industry is weighted towards content networks. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Job Snijders" <job@ntt.net> To: nanog@nanog.org Sent: Thursday, August 31, 2017 3:06:49 PM Subject: BGP Optimizers (Was: Validating possible BGP MITM attack) Dear all, disclaimer: [ The following is targetted at the context where a BGP optimizer generates BGP announcement that are ordinarily not seen in the Default-Free Zone. The OP indicated they announce a /23, and were unpleasantly surprised to see two unauthorized announcements for /24 more-specifics pop up in their alerting system. No permission was granted to create and announce these more-specifics. The AS_PATH for those /24 announcements was entirely fabricated. Original thread https://mailman.nanog.org/pipermail/nanog/2017-August/092124.html ] On Thu, Aug 31, 2017 at 11:13:18AM -0700, Andy Litzinger wrote:
Presuming it was a route optimizer and the issue was ongoing, what would be the suggested course of action?
I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked. It is extremely irresponsible behavior to use software that generates _fake_ BGP more-specifics for the purpose of traffic engineering. You simply cannot expect that those more-specifics will never escape into the global DFZ! Relying on NO_EXPORT is not sufficient: we regularly see software bugs related to NO_EXPORT, and community-squashing configuration mistakes happen all the time. Consider the following: if you leak your own internal more-specifics, at least you are the legitimate destination. (You may suffer from suboptimal routing, but it isn't guaranteed downtime.) However if you generate fake more-specifics for prefixes belonging to OTHER organisations, you essentially are complicit in BGP hijacking. If those fake more-specifics accidentally leak into the DFZ, you are bringing down the actual owner of such prefixes, and depriving people from access to the Internet. Example case: https://mailman.nanog.org/pipermail/nanog/2013-January/054846.html
reach out to those 2 AS owners and see if they could stop it?
Yes, absolutely! And if everyone of the affected parties of this localized hijack leak (or should we say 'victims') reaches out to the wrongdoers, they contribute peer pressure to rectify the situation. Just make sure you assign blame to the correct party. :)
Or is it something I just have to live with as a traffic engineering solution they are using and mark the alerts as false positives?
No, this is not something we should just accept. The Default-Free Zone is a shared resource. Anyone using "BGP optimizers" is not only risking their own online presence, but also everyone else's. Using a BGP optimizer is essentially trading a degree of risk to society for the purpose of saving a few bucks or milliseconds. It is basically saying: "The optimizer helps me, but may hurt others, and I am fine with that". An analogy: We don't want transporters of radioactive material to cut corners by using non-existant roads to bring the spent nuclear fuels from A to B faster, nor do we want them to rely on non-robust shielding that works "most of the time". We share the BGP DFZ environment, 'BGP optimizers' are downright pollution contributors. Kind regards, Job ps. If you want to do BGP optimization: don't have the BGP optimizer generate fake BGP announcements, but focus only on manipulating non-transitive attributes (like LOCAL_PREF) on real announcements you actually received from others. Or Perhaps BGP optimizers shouldn't even talk BGP but rather push freshly generated TE-optimized routing-policy to your edge boxes. Or perhaps the 'BGP optimizer' vendors should come to IETF to figure out a safe way (maybe a new AFI/SAFI to manipulate real announcements)... there are ways to do this, without risk to others! p.s. providing a publicly available BGP looking glasses will contribute to proving your innocence in cases like these. Since in many cases the AS_PATH is a complete fabrication, we need to manually check every AS in the AS_PATH to see whether the AS carries the fake more-specific. A public looking glass speeds up this fault-finding process. If you don't want to host a webinterface yourself, please consider sending a BGP feed to the Route Views Project or RIPE RIS, or for something queryable in a real-time fashion the NLNOG RING Looking Glass http://lg.ring.nlnog.net/
Actually, I do remember that one of them would optimize inbound routes, but only billed on outbound usage (as it was content-focused). My in is over 8x my out, so hrm... maybe I'm on to something. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Thursday, August 31, 2017 8:55:46 PM Subject: Re: BGP Optimizers (Was: Validating possible BGP MITM attack) I would like to use a BGP optimizer, but I'm too poor. :-\ That said, I'm also an eyeball network, so modifications of my own advertisements are what affects the desired traffic, not so much the outbound routes. I know the BGP optimization industry is weighted towards content networks. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Job Snijders" <job@ntt.net> To: nanog@nanog.org Sent: Thursday, August 31, 2017 3:06:49 PM Subject: BGP Optimizers (Was: Validating possible BGP MITM attack) Dear all, disclaimer: [ The following is targetted at the context where a BGP optimizer generates BGP announcement that are ordinarily not seen in the Default-Free Zone. The OP indicated they announce a /23, and were unpleasantly surprised to see two unauthorized announcements for /24 more-specifics pop up in their alerting system. No permission was granted to create and announce these more-specifics. The AS_PATH for those /24 announcements was entirely fabricated. Original thread https://mailman.nanog.org/pipermail/nanog/2017-August/092124.html ] On Thu, Aug 31, 2017 at 11:13:18AM -0700, Andy Litzinger wrote:
Presuming it was a route optimizer and the issue was ongoing, what would be the suggested course of action?
I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked. It is extremely irresponsible behavior to use software that generates _fake_ BGP more-specifics for the purpose of traffic engineering. You simply cannot expect that those more-specifics will never escape into the global DFZ! Relying on NO_EXPORT is not sufficient: we regularly see software bugs related to NO_EXPORT, and community-squashing configuration mistakes happen all the time. Consider the following: if you leak your own internal more-specifics, at least you are the legitimate destination. (You may suffer from suboptimal routing, but it isn't guaranteed downtime.) However if you generate fake more-specifics for prefixes belonging to OTHER organisations, you essentially are complicit in BGP hijacking. If those fake more-specifics accidentally leak into the DFZ, you are bringing down the actual owner of such prefixes, and depriving people from access to the Internet. Example case: https://mailman.nanog.org/pipermail/nanog/2013-January/054846.html
reach out to those 2 AS owners and see if they could stop it?
Yes, absolutely! And if everyone of the affected parties of this localized hijack leak (or should we say 'victims') reaches out to the wrongdoers, they contribute peer pressure to rectify the situation. Just make sure you assign blame to the correct party. :)
Or is it something I just have to live with as a traffic engineering solution they are using and mark the alerts as false positives?
No, this is not something we should just accept. The Default-Free Zone is a shared resource. Anyone using "BGP optimizers" is not only risking their own online presence, but also everyone else's. Using a BGP optimizer is essentially trading a degree of risk to society for the purpose of saving a few bucks or milliseconds. It is basically saying: "The optimizer helps me, but may hurt others, and I am fine with that". An analogy: We don't want transporters of radioactive material to cut corners by using non-existant roads to bring the spent nuclear fuels from A to B faster, nor do we want them to rely on non-robust shielding that works "most of the time". We share the BGP DFZ environment, 'BGP optimizers' are downright pollution contributors. Kind regards, Job ps. If you want to do BGP optimization: don't have the BGP optimizer generate fake BGP announcements, but focus only on manipulating non-transitive attributes (like LOCAL_PREF) on real announcements you actually received from others. Or Perhaps BGP optimizers shouldn't even talk BGP but rather push freshly generated TE-optimized routing-policy to your edge boxes. Or perhaps the 'BGP optimizer' vendors should come to IETF to figure out a safe way (maybe a new AFI/SAFI to manipulate real announcements)... there are ways to do this, without risk to others! p.s. providing a publicly available BGP looking glasses will contribute to proving your innocence in cases like these. Since in many cases the AS_PATH is a complete fabrication, we need to manually check every AS in the AS_PATH to see whether the AS carries the fake more-specific. A public looking glass speeds up this fault-finding process. If you don't want to host a webinterface yourself, please consider sending a BGP feed to the Route Views Project or RIPE RIS, or for something queryable in a real-time fashion the NLNOG RING Looking Glass http://lg.ring.nlnog.net/
Sorry for now taking up 1/4 of this thread.... My words in the last message don't match what I was thinking, but I think you all get the point. I'm sick, maybe I should be in bed instead of on NANOG. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Thursday, August 31, 2017 9:02:07 PM Subject: Re: BGP Optimizers (Was: Validating possible BGP MITM attack) Actually, I do remember that one of them would optimize inbound routes, but only billed on outbound usage (as it was content-focused). My in is over 8x my out, so hrm... maybe I'm on to something. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Thursday, August 31, 2017 8:55:46 PM Subject: Re: BGP Optimizers (Was: Validating possible BGP MITM attack) I would like to use a BGP optimizer, but I'm too poor. :-\ That said, I'm also an eyeball network, so modifications of my own advertisements are what affects the desired traffic, not so much the outbound routes. I know the BGP optimization industry is weighted towards content networks. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Job Snijders" <job@ntt.net> To: nanog@nanog.org Sent: Thursday, August 31, 2017 3:06:49 PM Subject: BGP Optimizers (Was: Validating possible BGP MITM attack) Dear all, disclaimer: [ The following is targetted at the context where a BGP optimizer generates BGP announcement that are ordinarily not seen in the Default-Free Zone. The OP indicated they announce a /23, and were unpleasantly surprised to see two unauthorized announcements for /24 more-specifics pop up in their alerting system. No permission was granted to create and announce these more-specifics. The AS_PATH for those /24 announcements was entirely fabricated. Original thread https://mailman.nanog.org/pipermail/nanog/2017-August/092124.html ] On Thu, Aug 31, 2017 at 11:13:18AM -0700, Andy Litzinger wrote:
Presuming it was a route optimizer and the issue was ongoing, what would be the suggested course of action?
I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked. It is extremely irresponsible behavior to use software that generates _fake_ BGP more-specifics for the purpose of traffic engineering. You simply cannot expect that those more-specifics will never escape into the global DFZ! Relying on NO_EXPORT is not sufficient: we regularly see software bugs related to NO_EXPORT, and community-squashing configuration mistakes happen all the time. Consider the following: if you leak your own internal more-specifics, at least you are the legitimate destination. (You may suffer from suboptimal routing, but it isn't guaranteed downtime.) However if you generate fake more-specifics for prefixes belonging to OTHER organisations, you essentially are complicit in BGP hijacking. If those fake more-specifics accidentally leak into the DFZ, you are bringing down the actual owner of such prefixes, and depriving people from access to the Internet. Example case: https://mailman.nanog.org/pipermail/nanog/2013-January/054846.html
reach out to those 2 AS owners and see if they could stop it?
Yes, absolutely! And if everyone of the affected parties of this localized hijack leak (or should we say 'victims') reaches out to the wrongdoers, they contribute peer pressure to rectify the situation. Just make sure you assign blame to the correct party. :)
Or is it something I just have to live with as a traffic engineering solution they are using and mark the alerts as false positives?
No, this is not something we should just accept. The Default-Free Zone is a shared resource. Anyone using "BGP optimizers" is not only risking their own online presence, but also everyone else's. Using a BGP optimizer is essentially trading a degree of risk to society for the purpose of saving a few bucks or milliseconds. It is basically saying: "The optimizer helps me, but may hurt others, and I am fine with that". An analogy: We don't want transporters of radioactive material to cut corners by using non-existant roads to bring the spent nuclear fuels from A to B faster, nor do we want them to rely on non-robust shielding that works "most of the time". We share the BGP DFZ environment, 'BGP optimizers' are downright pollution contributors. Kind regards, Job ps. If you want to do BGP optimization: don't have the BGP optimizer generate fake BGP announcements, but focus only on manipulating non-transitive attributes (like LOCAL_PREF) on real announcements you actually received from others. Or Perhaps BGP optimizers shouldn't even talk BGP but rather push freshly generated TE-optimized routing-policy to your edge boxes. Or perhaps the 'BGP optimizer' vendors should come to IETF to figure out a safe way (maybe a new AFI/SAFI to manipulate real announcements)... there are ways to do this, without risk to others! p.s. providing a publicly available BGP looking glasses will contribute to proving your innocence in cases like these. Since in many cases the AS_PATH is a complete fabrication, we need to manually check every AS in the AS_PATH to see whether the AS carries the fake more-specific. A public looking glass speeds up this fault-finding process. If you don't want to host a webinterface yourself, please consider sending a BGP feed to the Route Views Project or RIPE RIS, or for something queryable in a real-time fashion the NLNOG RING Looking Glass http://lg.ring.nlnog.net/
Nanog's quite a nice list to spend your sick day on. ;) On 31/08/2017 19:04, Mike Hammett wrote:
Sorry for now taking up 1/4 of this thread....
My words in the last message don't match what I was thinking, but I think you all get the point. I'm sick, maybe I should be in bed instead of on NANOG.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
----- Original Message -----
From: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Thursday, August 31, 2017 9:02:07 PM Subject: Re: BGP Optimizers (Was: Validating possible BGP MITM attack)
Actually, I do remember that one of them would optimize inbound routes, but only billed on outbound usage (as it was content-focused). My in is over 8x my out, so hrm... maybe I'm on to something.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
----- Original Message -----
From: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Thursday, August 31, 2017 8:55:46 PM Subject: Re: BGP Optimizers (Was: Validating possible BGP MITM attack)
I would like to use a BGP optimizer, but I'm too poor. :-\
That said, I'm also an eyeball network, so modifications of my own advertisements are what affects the desired traffic, not so much the outbound routes. I know the BGP optimization industry is weighted towards content networks.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
----- Original Message -----
From: "Job Snijders" <job@ntt.net> To: nanog@nanog.org Sent: Thursday, August 31, 2017 3:06:49 PM Subject: BGP Optimizers (Was: Validating possible BGP MITM attack)
Dear all,
disclaimer:
[ The following is targetted at the context where a BGP optimizer generates BGP announcement that are ordinarily not seen in the Default-Free Zone. The OP indicated they announce a /23, and were unpleasantly surprised to see two unauthorized announcements for /24 more-specifics pop up in their alerting system. No permission was granted to create and announce these more-specifics. The AS_PATH for those /24 announcements was entirely fabricated. Original thread https://mailman.nanog.org/pipermail/nanog/2017-August/092124.html ]
On Thu, Aug 31, 2017 at 11:13:18AM -0700, Andy Litzinger wrote:
Presuming it was a route optimizer and the issue was ongoing, what would be the suggested course of action? I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.
It is extremely irresponsible behavior to use software that generates _fake_ BGP more-specifics for the purpose of traffic engineering. You simply cannot expect that those more-specifics will never escape into the global DFZ! Relying on NO_EXPORT is not sufficient: we regularly see software bugs related to NO_EXPORT, and community-squashing configuration mistakes happen all the time.
Consider the following: if you leak your own internal more-specifics, at least you are the legitimate destination. (You may suffer from suboptimal routing, but it isn't guaranteed downtime.) However if you generate fake more-specifics for prefixes belonging to OTHER organisations, you essentially are complicit in BGP hijacking. If those fake more-specifics accidentally leak into the DFZ, you are bringing down the actual owner of such prefixes, and depriving people from access to the Internet. Example case: https://mailman.nanog.org/pipermail/nanog/2013-January/054846.html
reach out to those 2 AS owners and see if they could stop it? Yes, absolutely! And if everyone of the affected parties of this localized hijack leak (or should we say 'victims') reaches out to the wrongdoers, they contribute peer pressure to rectify the situation. Just make sure you assign blame to the correct party. :)
Or is it something I just have to live with as a traffic engineering solution they are using and mark the alerts as false positives? No, this is not something we should just accept. The Default-Free Zone is a shared resource. Anyone using "BGP optimizers" is not only risking their own online presence, but also everyone else's. Using a BGP optimizer is essentially trading a degree of risk to society for the purpose of saving a few bucks or milliseconds. It is basically saying: "The optimizer helps me, but may hurt others, and I am fine with that".
An analogy: We don't want transporters of radioactive material to cut corners by using non-existant roads to bring the spent nuclear fuels from A to B faster, nor do we want them to rely on non-robust shielding that works "most of the time".
We share the BGP DFZ environment, 'BGP optimizers' are downright pollution contributors.
Kind regards,
Job
ps. If you want to do BGP optimization: don't have the BGP optimizer generate fake BGP announcements, but focus only on manipulating non-transitive attributes (like LOCAL_PREF) on real announcements you actually received from others. Or Perhaps BGP optimizers shouldn't even talk BGP but rather push freshly generated TE-optimized routing-policy to your edge boxes. Or perhaps the 'BGP optimizer' vendors should come to IETF to figure out a safe way (maybe a new AFI/SAFI to manipulate real announcements)... there are ways to do this, without risk to others!
p.s. providing a publicly available BGP looking glasses will contribute to proving your innocence in cases like these. Since in many cases the AS_PATH is a complete fabrication, we need to manually check every AS in the AS_PATH to see whether the AS carries the fake more-specific. A public looking glass speeds up this fault-finding process. If you don't want to host a webinterface yourself, please consider sending a BGP feed to the Route Views Project or RIPE RIS, or for something queryable in a real-time fashion the NLNOG RING Looking Glass http://lg.ring.nlnog.net/
Job Snijders <job@ntt.net> writes:
Using a BGP optimizer is essentially trading a degree of risk to society for the purpose of saving a few bucks or milliseconds. It is basically saying: "The optimizer helps me, but may hurt others, and I am fine with that".
People drive SUVs. Bjørn
On 31/08/17 22:06, Job Snijders wrote:> I strongly recommend to turn off those BGP optimizers, glue the ports
shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.
Yes.
p.s. providing a publicly available BGP looking glasses will contribute to proving your innocence in cases like these. Since in many cases the AS_PATH is a complete fabrication, we need to manually check every AS in the AS_PATH to see whether the AS carries the fake more-specific. A public looking glass speeds up this fault-finding process. If you don't want to host a webinterface yourself, please consider sending a BGP feed to the Route Views Project or RIPE RIS, or for something queryable in a real-time fashion the NLNOG RING Looking Glass http://lg.ring.nlnog.net/
As a RIPE RIS operator, we regularly get people complaining 'oh but we are not advertising that prefix, your system must be broken'. Usually it is one of these BGP-optimizer more-specifics leaking out. Cheers, Colin
We regularly see poorly configured "optimizers" or networks hijacking our prefixes (originating /25's, /24 of /23's etc). Thankfully, most of the time filters are in place to stop them leaking badly, but I agree, these are toxic. -Tom On Fri, Sep 1, 2017 at 6:06 AM, Job Snijders <job@ntt.net> wrote:
Dear all,
disclaimer:
[ The following is targetted at the context where a BGP optimizer generates BGP announcement that are ordinarily not seen in the Default-Free Zone. The OP indicated they announce a /23, and were unpleasantly surprised to see two unauthorized announcements for /24 more-specifics pop up in their alerting system. No permission was granted to create and announce these more-specifics. The AS_PATH for those /24 announcements was entirely fabricated. Original thread https://mailman.nanog.org/pipermail/nanog/2017-August/092124.html ]
On Thu, Aug 31, 2017 at 11:13:18AM -0700, Andy Litzinger wrote:
Presuming it was a route optimizer and the issue was ongoing, what would be the suggested course of action?
I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.
It is extremely irresponsible behavior to use software that generates _fake_ BGP more-specifics for the purpose of traffic engineering. You simply cannot expect that those more-specifics will never escape into the global DFZ! Relying on NO_EXPORT is not sufficient: we regularly see software bugs related to NO_EXPORT, and community-squashing configuration mistakes happen all the time.
Consider the following: if you leak your own internal more-specifics, at least you are the legitimate destination. (You may suffer from suboptimal routing, but it isn't guaranteed downtime.) However if you generate fake more-specifics for prefixes belonging to OTHER organisations, you essentially are complicit in BGP hijacking. If those fake more-specifics accidentally leak into the DFZ, you are bringing down the actual owner of such prefixes, and depriving people from access to the Internet. Example case: https://mailman.nanog.org/pipermail/nanog/2013-January/054846.html
reach out to those 2 AS owners and see if they could stop it?
Yes, absolutely! And if everyone of the affected parties of this localized hijack leak (or should we say 'victims') reaches out to the wrongdoers, they contribute peer pressure to rectify the situation. Just make sure you assign blame to the correct party. :)
Or is it something I just have to live with as a traffic engineering solution they are using and mark the alerts as false positives?
No, this is not something we should just accept. The Default-Free Zone is a shared resource. Anyone using "BGP optimizers" is not only risking their own online presence, but also everyone else's. Using a BGP optimizer is essentially trading a degree of risk to society for the purpose of saving a few bucks or milliseconds. It is basically saying: "The optimizer helps me, but may hurt others, and I am fine with that".
An analogy: We don't want transporters of radioactive material to cut corners by using non-existant roads to bring the spent nuclear fuels from A to B faster, nor do we want them to rely on non-robust shielding that works "most of the time".
We share the BGP DFZ environment, 'BGP optimizers' are downright pollution contributors.
Kind regards,
Job
ps. If you want to do BGP optimization: don't have the BGP optimizer generate fake BGP announcements, but focus only on manipulating non-transitive attributes (like LOCAL_PREF) on real announcements you actually received from others. Or Perhaps BGP optimizers shouldn't even talk BGP but rather push freshly generated TE-optimized routing-policy to your edge boxes. Or perhaps the 'BGP optimizer' vendors should come to IETF to figure out a safe way (maybe a new AFI/SAFI to manipulate real announcements)... there are ways to do this, without risk to others!
p.s. providing a publicly available BGP looking glasses will contribute to proving your innocence in cases like these. Since in many cases the AS_PATH is a complete fabrication, we need to manually check every AS in the AS_PATH to see whether the AS carries the fake more-specific. A public looking glass speeds up this fault-finding process. If you don't want to host a webinterface yourself, please consider sending a BGP feed to the Route Views Project or RIPE RIS, or for something queryable in a real-time fashion the NLNOG RING Looking Glass http://lg.ring.nlnog.net/
participants (6)
-
Bjørn Mork -
Colin Petrie -
Job Snijders -
Large Hadron Collider -
Mike Hammett -
Tom Paseka