I dont see that as the solution. Someone else will offend again. However, I also don't see trusting major backbones as our filters (for many other reasons). Our software should be handling what's effectively a buffer overflow or equivalent (beware long paths that are actually shellcode). Quagga among others seems to be subject to this bug, pre 0.99.23 or so (.99.24+ seems ok). So upgrading is a solution. There was also some chatter on the quagga mailing list on how it's more pleasant to stab your eyeballs out rather than constructing extremely long regexp's that might work as a filter. https://lists.quagga.net/pipermail/quagga-users/2017-September/thread.html /kc On Sat, Sep 30, 2017 at 05:30:03PM +0200, Niels Raijer said:
My message to NANOG about this from 12:31 UTC today is still in the moderation queue. I had opened a support case with Cogent before writing my message to NANOG and Cogent has let me know approximately 40 minutes ago that they have contacted their customer.
Niels
On 30 Sep 2017, at 17:09, sthaug@nethelp.no wrote:
If you're on cogent, since 22:30 UTC yesterday or so this has been happening (or happened).
Still happening here. I count 562 prepends (563 * 262197) in the advertisement we receive from Cogent. I see no good reason why we should accept that many prepends.
Steinar Haug, Nethelp consulting, sthaug@nethelp.no
-- Ken Chase - math@sizone.org Guelph Canada
On Sat, Sep 30, 2017 at 12:47 PM, Ken Chase <math@sizone.org> wrote:
I dont see that as the solution. Someone else will offend again.
However, I also don't see trusting major backbones as our filters (for many other reasons). Our software should be handling what's effectively a buffer overflow or equivalent (beware long paths that are actually shellcode).
Quagga among others seems to be subject to this bug, pre 0.99.23 or so (.99.24+ seems ok). So upgrading is a solution.
ii quagga 0.99.22.4-3ubu i386 BGP/OSPF/RIP routing daemon interestingly enough that isn't crashlooping nor is it bouncing bgp sessions: 192.168.100.100 4 MYASN 1642717 8864 0 0 0 2d23h32m 672475 and it's happily showing me the route even... There was also some chatter on the quagga mailing list on how it's more
pleasant to stab your eyeballs out rather than constructing extremely long regexp's that might work as a filter.
https://lists.quagga.net/pipermail/quagga-users/2017-September/thread.html
/kc
My message to NANOG about this from 12:31 UTC today is still in the moderation queue. I had opened a support case with Cogent before writing my message to NANOG and Cogent has let me know approximately 40 minutes ago
On Sat, Sep 30, 2017 at 05:30:03PM +0200, Niels Raijer said: that they have contacted their customer.
Niels
On 30 Sep 2017, at 17:09, sthaug@nethelp.no wrote:
If you're on cogent, since 22:30 UTC yesterday or so this has been
happening
(or happened).
Still happening here. I count 562 prepends (563 * 262197) in the advertisement we receive from Cogent. I see no good reason why we should accept that many prepends.
Steinar Haug, Nethelp consulting, sthaug@nethelp.no
-- Ken Chase - math@sizone.org Guelph Canada
I don't quite understand the exact situation that causes the issue - our cogent facing router (quagga .99.22 debian) was receiving the route but that session stayed up - it was it while sending or the other igp router (also quagga .99.22) receiving (I think the latter) that was crashing their session. Not quite sure why the cogent session didn't crash as well (or first, before propagating the bad route). At any rate, we should likely take this discussion to the quagga-users-l /kc On Sat, Sep 30, 2017 at 09:28:28PM -0400, Christopher Morrow said:
ii quagga 0.99.22.4-3ubu i386 BGP/OSPF/RIP routing daemon
interestingly enough that isn't crashlooping nor is it bouncing bgp sessions: 192.168.100.100 4 MYASN 1642717 8864 0 0 0 2d23h32m 672475
and it's happily showing me the route even...
-- Ken Chase - math@sizone.org Guelph Canada
On Sun, Oct 1, 2017 at 1:05 AM, Ken Chase <math@sizone.org> wrote:
I don't quite understand the exact situation that causes the issue - our cogent facing router (quagga .99.22 debian) was receiving the route but that session stayed up - it was it while sending or the other igp router (also quagga .99.22) receiving (I think the latter) that was crashing their session. Not quite sure why the cogent session didn't crash as well (or first, before propagating the bad route).
Hi Ken, Technically the route is not bad, just really inconsiderate. The bug happens when quagga sends the the long-AS path route to a peer. As I understand it, when the announcement is larger than one segment, Quagga double-counts the some of the bytes when computing the number of bytes in the AS path. It receives the announcement just fine, but then it corrupts what it sends to the neighbor who then chokes. Bug and patch here: https://lists.quagga.net/pipermail/quagga-dev/2017-September/033284.html Regards, Bill Herrin -- William Herrin ................ herrin@dirtside.com bill@herrin.us Dirtside Systems ......... Web: <http://www.dirtside.com/>
On 01/10/2017 04:28, Christopher Morrow wrote:
On Sat, Sep 30, 2017 at 12:47 PM, Ken Chase <math@sizone.org> wrote:
I dont see that as the solution. Someone else will offend again.
However, I also don't see trusting major backbones as our filters (for many other reasons). Our software should be handling what's effectively a buffer overflow or equivalent (beware long paths that are actually shellcode).
Quagga among others seems to be subject to this bug, pre 0.99.23 or so (.99.24+ seems ok). So upgrading is a solution.
ii quagga 0.99.22.4-3ubu i386 BGP/OSPF/RIP routing daemon
interestingly enough that isn't crashlooping nor is it bouncing bgp sessions:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=2009-1572 Quagga 0.99.11 and earlier affected. Fixed in 2009. -Hank
192.168.100.100 4 MYASN 1642717 8864 0 0 0 2d23h32m 672475
and it's happily showing me the route even...
There was also some chatter on the quagga mailing list on how it's more
pleasant to stab your eyeballs out rather than constructing extremely long regexp's that might work as a filter.
https://lists.quagga.net/pipermail/quagga-users/2017-September/thread.html
/kc
My message to NANOG about this from 12:31 UTC today is still in the moderation queue. I had opened a support case with Cogent before writing my message to NANOG and Cogent has let me know approximately 40 minutes ago
On Sat, Sep 30, 2017 at 05:30:03PM +0200, Niels Raijer said: that they have contacted their customer.
Niels
On 30 Sep 2017, at 17:09, sthaug@nethelp.no wrote:
If you're on cogent, since 22:30 UTC yesterday or so this has been
happening
(or happened).
Still happening here. I count 562 prepends (563 * 262197) in the advertisement we receive from Cogent. I see no good reason why we should accept that many prepends.
Steinar Haug, Nethelp consulting, sthaug@nethelp.no
-- Ken Chase - math@sizone.org Guelph Canada
On Sun, 1 Oct 2017, Hank Nussbacher wrote:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=2009-1572 Quagga 0.99.11 and earlier affected. Fixed in 2009.
It was fixed in other OSes as well after this, I presume: http://blog.ipspace.net/2009/02/root-cause-analysis-oversized-as-paths.html -- Mikael Abrahamsson email: swmike@swm.pp.se
looks to me as if 262206 is trying a silly tactic to down-pref inbound from cogent. as cogent probably prefers customers to peers, it may not be working as 262206 expected, so they keep pounding with the same hammer hoping for a miracle. cogent accepts it as they are being paid to do so; normal practice. perhaps our ire should be directed at 262206, not cogent? has anyone tried to contact them? randy
Den 2. okt. 2017 00.44 skrev "Randy Bush" <randy@psg.com>: looks to me as if 262206 is trying a silly tactic to down-pref inbound from cogent. as cogent probably prefers customers to peers, it may not be working as 262206 expected, so they keep pounding with the same hammer hoping for a miracle. cogent accepts it as they are being paid to do so; normal practice. perhaps our ire should be directed at 262206, not cogent? has anyone tried to contact them? randy It is amazing how well the DFZ works given half the participants (*) have no clue how. If that is what they want, all they need is to split that /23 into two /24 and only announce that on their other transit. (*) I should probably include myself in that half. Regards Baldur
participants (7)
-
Baldur Norddahl -
Christopher Morrow -
Hank Nussbacher -
Ken Chase -
Mikael Abrahamsson -
Randy Bush -
William Herrin