So back in the.. hell I don't know like... early 2010s there was a push for 'route optimization' from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily 'optimize' [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses... which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~
On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
It's not even that. GPU's are very good at parallelized vector computations. They are very very good at THAT, but ONLY that. This is no different conceptually than router ASICs. They are designed to do ONE thing very well, BGP bestpath selection is a completely different computational process. On Thu, Dec 5, 2024 at 12:06 PM Jason Bothe via NANOG <nanog@nanog.org> wrote:
WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages.
J~
On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
I don't know that you need to spread BGP best path analysis onto a GPU, but conducting the testing that those boxes do to the entire Internet instead of just top X destinations would be quite parallel. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Tom Beecher" <beecher@beecher.cc> To: "Jason Bothe" <jbothe@me.com> Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 11:11:47 AM Subject: Re: Route optimization using GPUs? It's not even that. GPU's are very good at parallelized vector computations. They are very very good at THAT, but ONLY that. This is no different conceptually than router ASICs. They are designed to do ONE thing very well, BGP bestpath selection is a completely different computational process. On Thu, Dec 5, 2024 at 12:06 PM Jason Bothe via NANOG < nanog@nanog.org > wrote: WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ <blockquote> On Dec 5, 2024, at 8:13 AM, Drew Weaver < drew.weaver@thenap.com > wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew </blockquote>
Sure, alright but given what you just said doesn’t it seem odd that there is still a static BGP tiebreaker in 2024? From: Tom Beecher <beecher@beecher.cc> Sent: Thursday, December 5, 2024 12:12 PM To: Jason Bothe <jbothe@me.com> Cc: Drew Weaver <drew.weaver@thenap.com>; nanog@nanog.org Subject: Re: Route optimization using GPUs? It's not even that. GPU's are very good at parallelized vector computations. They are very very good at THAT, but ONLY that. This is no different conceptually than router ASICs. They are designed to do ONE thing very well, BGP bestpath selection is a completely different computational process. On Thu, Dec 5, 2024 at 12:06 PM Jason Bothe via NANOG <nanog@nanog.org<mailto:nanog@nanog.org>> wrote: WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
Sure, alright but given what you just said doesn’t it seem odd that there is still a static BGP tiebreaker in 2024?
No. Also not sure what your point is here. On Wed, Dec 11, 2024 at 8:55 AM Drew Weaver <drew.weaver@thenap.com> wrote:
Sure, alright but given what you just said doesn’t it seem odd that there is still a static BGP tiebreaker in 2024?
*From:* Tom Beecher <beecher@beecher.cc> *Sent:* Thursday, December 5, 2024 12:12 PM *To:* Jason Bothe <jbothe@me.com> *Cc:* Drew Weaver <drew.weaver@thenap.com>; nanog@nanog.org *Subject:* Re: Route optimization using GPUs?
It's not even that.
GPU's are very good at parallelized vector computations. They are very very good at THAT, but ONLY that. This is no different conceptually than router ASICs. They are designed to do ONE thing very well,
BGP bestpath selection is a completely different computational process.
On Thu, Dec 5, 2024 at 12:06 PM Jason Bothe via NANOG <nanog@nanog.org> wrote:
WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages.
J~
On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Jason Bothe via NANOG" <nanog@nanog.org> To: "Drew Weaver" <drew.weaver@thenap.com> Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
“I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.” -Job Snijders https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html From: NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org> on behalf of Mike Hammett <nanog@ics-il.net> Date: Thursday, December 5, 2024 at 12:24 PM To: Jason Bothe <jbothe@me.com> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.com/v3/__http:/www.ics-il.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ$> Midwest-IX http://www.midwest-ix.com<https://urldefense.com/v3/__http:/www.midwest-ix.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw$> ________________________________ From: "Jason Bothe via NANOG" <nanog@nanog.org> To: "Drew Weaver" <drew.weaver@thenap.com> Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
Eh, different people have different opinions. I think most of the hatred towards them is unwarranted, ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Rich Compton" <RICH_COMPTON@comcast.com> To: "Mike Hammett" <nanog@ics-il.net>, "Jason Bothe" <jbothe@me.com> Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 2:10:47 PM Subject: Re: Route optimization using GPUs? “I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.” -Job Snijders https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html From: NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org> on behalf of Mike Hammett <nanog@ics-il.net> Date: Thursday, December 5, 2024 at 12:24 PM To: Jason Bothe <jbothe@me.com> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com From: "Jason Bothe via NANOG" <nanog@nanog.org> To: "Drew Weaver" <drew.weaver@thenap.com> Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
* nanog@ics-il.net (Mike Hammett) [Thu 05 Dec 2024, 21:18 CET]:
Eh, different people have different opinions.
I think most of the hatred towards them is unwarranted,
You speak like a person whose prefixes were never hijacked and blackholed by one of those things. https://honestnetworker.net/2016/08/15/when-you-overhear-im-considering-depl... -- Niels.
Not by the box, but by the operator of the box. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Niels Bakker" <niels=nanog@bakker.net> To: nanog@nanog.org Sent: Thursday, December 5, 2024 2:27:23 PM Subject: Re: Route optimization using GPUs? * nanog@ics-il.net (Mike Hammett) [Thu 05 Dec 2024, 21:18 CET]:
Eh, different people have different opinions.
I think most of the hatred towards them is unwarranted,
You speak like a person whose prefixes were never hijacked and blackholed by one of those things. https://honestnetworker.net/2016/08/15/when-you-overhear-im-considering-depl... -- Niels.
* nanog@ics-il.net (Mike Hammett) [Thu 05 Dec 2024, 21:32 CET]:
Not by the box, but by the operator of the box.
We've been over this numerous times around the time when that Honest Networker page in my email was created. Noction ship their boxes with insanely dangerous defaults that are bound to lead to catastrophic route leaks. "Unsafe at any speed" applies to the whole category but especially to that implementation. I don't know if they've improved or lost business since. Their major accomplishment as a company may turn out to be, unintended though it may be, hastening the global acceptance and rollout of RPKI OV. -- Niels.
Yeah, that is kind of why I said "Whatever Noction is doing" in the original post. I'm not really calling to resurrect Noction. I'm saying that BGP path selection could be better and it's kind of silly that it's 2024 and things are still being handled by a static tie breaker. Thanks, -Drew -----Original Message----- From: NANOG <nanog-bounces+drew.weaver=thenap.com@nanog.org> On Behalf Of Niels Bakker Sent: Thursday, December 5, 2024 3:40 PM To: nanog@nanog.org Subject: Re: Route optimization using GPUs? * nanog@ics-il.net (Mike Hammett) [Thu 05 Dec 2024, 21:32 CET]:
Not by the box, but by the operator of the box.
We've been over this numerous times around the time when that Honest Networker page in my email was created. Noction ship their boxes with insanely dangerous defaults that are bound to lead to catastrophic route leaks. "Unsafe at any speed" applies to the whole category but especially to that implementation. I don't know if they've improved or lost business since. Their major accomplishment as a company may turn out to be, unintended though it may be, hastening the global acceptance and rollout of RPKI OV. -- Niels.
I think most of the hatred towards them is unwarranted,
This is essentially saying "I've never had a problem , so I don't think it's a big deal." On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett <nanog@ics-il.net> wrote:
Eh, different people have different opinions.
I think most of the hatred towards them is unwarranted,
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
------------------------------ *From: *"Rich Compton" <RICH_COMPTON@comcast.com> *To: *"Mike Hammett" <nanog@ics-il.net>, "Jason Bothe" <jbothe@me.com> *Cc: *nanog@nanog.org *Sent: *Thursday, December 5, 2024 2:10:47 PM *Subject: *Re: Route optimization using GPUs?
“I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.”
-Job Snijders
https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html
*From: *NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org> on behalf of Mike Hammett <nanog@ics-il.net> *Date: *Thursday, December 5, 2024 at 12:24 PM *To: *Jason Bothe <jbothe@me.com> *Cc: *nanog@nanog.org <nanog@nanog.org> *Subject: *Re: Route optimization using GPUs?
IIRC, the widespread outages are the result of exporting things that shouldn't be exported.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com <https://urldefense.com/v3/__http:/www.ics-il.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ$>
Midwest-IX http://www.midwest-ix.com <https://urldefense.com/v3/__http:/www.midwest-ix.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw$>
------------------------------
*From: *"Jason Bothe via NANOG" <nanog@nanog.org> *To: *"Drew Weaver" <drew.weaver@thenap.com> *Cc: *nanog@nanog.org *Sent: *Thursday, December 5, 2024 11:03:39 AM *Subject: *Re: Route optimization using GPUs?
WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages.
J~
On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
*shrugs* Incorrectly assigning the blame doesn't really help anyone. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Tom Beecher" <beecher@beecher.cc> To: "Mike Hammett" <nanog@ics-il.net> Cc: "Rich Compton" <RICH_COMPTON@comcast.com>, nanog@nanog.org Sent: Thursday, December 5, 2024 2:39:52 PM Subject: Re: Route optimization using GPUs? I think most of the hatred towards them is unwarranted, This is essentially saying "I've never had a problem , so I don't think it's a big deal." On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett < nanog@ics-il.net > wrote: <blockquote> Eh, different people have different opinions. I think most of the hatred towards them is unwarranted, ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com From: "Rich Compton" < RICH_COMPTON@comcast.com > To: "Mike Hammett" < nanog@ics-il.net >, "Jason Bothe" < jbothe@me.com > Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 2:10:47 PM Subject: Re: Route optimization using GPUs? “I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.” -Job Snijders https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html From: NANOG <nanog-bounces+rich_compton= comcast.com@nanog.org > on behalf of Mike Hammett < nanog@ics-il.net > Date: Thursday, December 5, 2024 at 12:24 PM To: Jason Bothe < jbothe@me.com > Cc: nanog@nanog.org < nanog@nanog.org > Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com From: "Jason Bothe via NANOG" < nanog@nanog.org > To: "Drew Weaver" < drew.weaver@thenap.com > Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ <blockquote> On Dec 5, 2024, at 8:13 AM, Drew Weaver < drew.weaver@thenap.com > wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew </blockquote> </blockquote>
On Thu, Dec 05, 2024 at 3:41 PM, Mike Hammett <nanog@ics-il.net> wrote:
*shrugs* Incorrectly assigning the blame doesn't really help anyone.
Sure, but the fact remains that there is blame to be assigned. It doesn't really matter to the affected network if the fault lies with the box itself, or the operator of the box, or the person who makes the morning coffee for the person who operates the box — the fact still remains that this call of devices have caused significant disruption for a whole bunch of external networks. "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… "Route optimizers don't hijack routes, operators with route optimizers hijack routes" falls into the same category… W
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
------------------------------ *From: *"Tom Beecher" <beecher@beecher.cc> *To: *"Mike Hammett" <nanog@ics-il.net> *Cc: *"Rich Compton" <RICH_COMPTON@comcast.com>, nanog@nanog.org *Sent: *Thursday, December 5, 2024 2:39:52 PM *Subject: *Re: Route optimization using GPUs?
I think most of the hatred towards them is unwarranted,
This is essentially saying "I've never had a problem , so I don't think it's a big deal."
On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett <nanog@ics-il.net> wrote:
Eh, different people have different opinions.
I think most of the hatred towards them is unwarranted,
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
------------------------------ *From: *"Rich Compton" <RICH_COMPTON@comcast.com> *To: *"Mike Hammett" <nanog@ics-il.net>, "Jason Bothe" <jbothe@me.com> *Cc: *nanog@nanog.org *Sent: *Thursday, December 5, 2024 2:10:47 PM *Subject: *Re: Route optimization using GPUs?
“I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.”
-Job Snijders
https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html
*From: *NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org> on behalf of Mike Hammett <nanog@ics-il.net> *Date: *Thursday, December 5, 2024 at 12:24 PM *To: *Jason Bothe <jbothe@me.com> *Cc: *nanog@nanog.org <nanog@nanog.org> *Subject: *Re: Route optimization using GPUs?
IIRC, the widespread outages are the result of exporting things that shouldn't be exported.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com <https://urldefense.com/v3/__http:/www.ics-il.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ$>
Midwest-IX http://www.midwest-ix.com <https://urldefense.com/v3/__http:/www.midwest-ix.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw$>
------------------------------
*From: *"Jason Bothe via NANOG" <nanog@nanog.org> *To: *"Drew Weaver" <drew.weaver@thenap.com> *Cc: *nanog@nanog.org *Sent: *Thursday, December 5, 2024 11:03:39 AM *Subject: *Re: Route optimization using GPUs?
WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages.
J~
On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
Warren, * "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… While that is also a factually true statement, you are also painting broad strokes over those who are responsible with those weapons. Employment requirements, hunting season, target practice at a range, skeet shooting, are just a few reasons to have them. Let's not dismiss those who follow the law, get qualified on a regular basis or have adequate training when/where/why/how to properly use them. "One rotten apple spoils the whole bunch", does not work here. This same thing also applies to operators of route optimizers. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits. For the incidents those route optimizers have caused, the vendors, their customers, and upstream ISPs are still in business. Ryan Hamel ________________________________ From: NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org> on behalf of Warren Kumari <warren@kumari.net> Sent: Thursday, December 5, 2024 1:17 PM To: Mike Hammett <nanog@ics-il.net> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. On Thu, Dec 05, 2024 at 3:41 PM, Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: *shrugs* Incorrectly assigning the blame doesn't really help anyone. Sure, but the fact remains that there is blame to be assigned. It doesn't really matter to the affected network if the fault lies with the box itself, or the operator of the box, or the person who makes the morning coffee for the person who operates the box — the fact still remains that this call of devices have caused significant disruption for a whole bunch of external networks. "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… "Route optimizers don't hijack routes, operators with route optimizers hijack routes" falls into the same category… W ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<http://www.ics-il.com/> Midwest-IX http://www.midwest-ix.com<http://www.midwest-ix.com/> ________________________________ From: "Tom Beecher" <beecher@beecher.cc<mailto:beecher@beecher.cc>> To: "Mike Hammett" <nanog@ics-il.net<mailto:nanog@ics-il.net>> Cc: "Rich Compton" <RICH_COMPTON@comcast.com<mailto:RICH_COMPTON@comcast.com>>, nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 2:39:52 PM Subject: Re: Route optimization using GPUs? I think most of the hatred towards them is unwarranted, This is essentially saying "I've never had a problem , so I don't think it's a big deal." On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: Eh, different people have different opinions. I think most of the hatred towards them is unwarranted, ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<http://www.ics-il.com/> Midwest-IX http://www.midwest-ix.com<http://www.midwest-ix.com/> ________________________________ From: "Rich Compton" <RICH_COMPTON@comcast.com<mailto:RICH_COMPTON@comcast.com>> To: "Mike Hammett" <nanog@ics-il.net<mailto:nanog@ics-il.net>>, "Jason Bothe" <jbothe@me.com<mailto:jbothe@me.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 2:10:47 PM Subject: Re: Route optimization using GPUs? “I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.” -Job Snijders https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html From: NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org<mailto:comcast.com@nanog.org>> on behalf of Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> Date: Thursday, December 5, 2024 at 12:24 PM To: Jason Bothe <jbothe@me.com<mailto:jbothe@me.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.com/v3/__http:/www.ics-il.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ$> Midwest-IX http://www.midwest-ix.com<https://urldefense.com/v3/__http:/www.midwest-ix.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw$> ________________________________ From: "Jason Bothe via NANOG" <nanog@nanog.org<mailto:nanog@nanog.org>> To: "Drew Weaver" <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
True, I didn't even think of all of the upstreams of those networks being responsible for accepting bad routes. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Ryan Hamel" <ryan@rkhtech.org> To: "Warren Kumari" <warren@kumari.net>, "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 4:34:44 PM Subject: Re: Route optimization using GPUs? Warren, * "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… While that is also a factually true statement, you are also painting broad strokes over those who are responsible with those weapons. Employment requirements, hunting season, target practice at a range, skeet shooting, are just a few reasons to have them. Let's not dismiss those who follow the law, get qualified on a regular basis or have adequate training when/where/why/how to properly use them. "One rotten apple spoils the whole bunch", does not work here. This same thing also applies to operators of route optimizers. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits. For the incidents those route optimizers have caused, the vendors, their customers, and upstream ISPs are still in business. Ryan Hamel From: NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org> on behalf of Warren Kumari <warren@kumari.net> Sent: Thursday, December 5, 2024 1:17 PM To: Mike Hammett <nanog@ics-il.net> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. On Thu, Dec 05, 2024 at 3:41 PM, Mike Hammett < nanog@ics-il.net > wrote: *shrugs* Incorrectly assigning the blame doesn't really help anyone. Sure, but the fact remains that there is blame to be assigned. It doesn't really matter to the affected network if the fault lies with the box itself, or the operator of the box, or the person who makes the morning coffee for the person who operates the box — the fact still remains that this call of devices have caused significant disruption for a whole bunch of external networks. "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… "Route optimizers don't hijack routes, operators with route optimizers hijack routes" falls into the same category… W <blockquote> ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com From: "Tom Beecher" < beecher@beecher.cc > To: "Mike Hammett" < nanog@ics-il.net > Cc: "Rich Compton" < RICH_COMPTON@comcast.com >, nanog@nanog.org Sent: Thursday, December 5, 2024 2:39:52 PM Subject: Re: Route optimization using GPUs? <blockquote> I think most of the hatred towards them is unwarranted, </blockquote> This is essentially saying "I've never had a problem , so I don't think it's a big deal." On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett < nanog@ics-il.net > wrote: <blockquote> Eh, different people have different opinions. I think most of the hatred towards them is unwarranted, ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com From: "Rich Compton" < RICH_COMPTON@comcast.com > To: "Mike Hammett" < nanog@ics-il.net >, "Jason Bothe" < jbothe@me.com > Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 2:10:47 PM Subject: Re: Route optimization using GPUs? “I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.” -Job Snijders https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html From: NANOG <nanog-bounces+rich_compton= comcast.com@nanog.org > on behalf of Mike Hammett < nanog@ics-il.net > Date: Thursday, December 5 , 2024 at 12:24 PM To: Jason Bothe < jbothe@me.com > Cc: nanog@nanog.org < nanog@nanog.org > Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com From: "Jason Bothe via NANOG" < nanog@nanog.org > To: "Drew Weaver" < drew.weaver@thenap.com > Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ <blockquote> On Dec 5, 2024, at 8:13 AM, Drew Weaver < drew.weaver@thenap.com > wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew </blockquote> </blockquote> </blockquote>
They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits
Nobody is asserting that operators are still not responsible for doing this. Stop it. What I think some of you guys fail to understand is that you can have perfectly appropriate policy protections in place that the prefixes generated by the optimizer can still bypass, because these devices don't follow standard BGP behaviors. This has happened before. On Thu, Dec 5, 2024 at 5:37 PM Ryan Hamel <ryan@rkhtech.org> wrote:
Warren,
- "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot…
While that is also a factually true statement, you are also painting broad strokes over those who are responsible with those weapons. Employment requirements, hunting season, target practice at a range, skeet shooting, are just a few reasons to have them. Let's not dismiss those who follow the law, get qualified on a regular basis or have adequate training when/where/why/how to properly use them.
"One rotten apple spoils the whole bunch", does not work here.
This same thing also applies to operators of route optimizers. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits. For the incidents those route optimizers have caused, the vendors, their customers, and upstream ISPs are still in business.
Ryan Hamel
------------------------------ *From:* NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org> on behalf of Warren Kumari <warren@kumari.net> *Sent:* Thursday, December 5, 2024 1:17 PM *To:* Mike Hammett <nanog@ics-il.net> *Cc:* nanog@nanog.org <nanog@nanog.org> *Subject:* Re: Route optimization using GPUs?
Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments.
On Thu, Dec 05, 2024 at 3:41 PM, Mike Hammett <nanog@ics-il.net> wrote:
*shrugs* Incorrectly assigning the blame doesn't really help anyone.
Sure, but the fact remains that there is blame to be assigned.
It doesn't really matter to the affected network if the fault lies with the box itself, or the operator of the box, or the person who makes the morning coffee for the person who operates the box — the fact still remains that this call of devices have caused significant disruption for a whole bunch of external networks.
"Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… "Route optimizers don't hijack routes, operators with route optimizers hijack routes" falls into the same category…
W
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
------------------------------ *From: *"Tom Beecher" <beecher@beecher.cc> *To: *"Mike Hammett" <nanog@ics-il.net> *Cc: *"Rich Compton" <RICH_COMPTON@comcast.com>, nanog@nanog.org *Sent: *Thursday, December 5, 2024 2:39:52 PM *Subject: *Re: Route optimization using GPUs?
I think most of the hatred towards them is unwarranted,
This is essentially saying "I've never had a problem , so I don't think it's a big deal."
On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett <nanog@ics-il.net> wrote:
Eh, different people have different opinions.
I think most of the hatred towards them is unwarranted,
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
------------------------------ *From: *"Rich Compton" <RICH_COMPTON@comcast.com> *To: *"Mike Hammett" <nanog@ics-il.net>, "Jason Bothe" <jbothe@me.com> *Cc: *nanog@nanog.org *Sent: *Thursday, December 5, 2024 2:10:47 PM *Subject: *Re: Route optimization using GPUs?
“I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.”
-Job Snijders
https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html
*From: *NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org> on behalf of Mike Hammett <nanog@ics-il.net> *Date: *Thursday, December 5, 2024 at 12:24 PM *To: *Jason Bothe <jbothe@me.com> *Cc: *nanog@nanog.org <nanog@nanog.org> *Subject: *Re: Route optimization using GPUs?
IIRC, the widespread outages are the result of exporting things that shouldn't be exported.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com <https://urldefense.com/v3/__http:/www.ics-il.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ$>
Midwest-IX http://www.midwest-ix.com <https://urldefense.com/v3/__http:/www.midwest-ix.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw$>
------------------------------
*From: *"Jason Bothe via NANOG" <nanog@nanog.org> *To: *"Drew Weaver" <drew.weaver@thenap.com> *Cc: *nanog@nanog.org *Sent: *Thursday, December 5, 2024 11:03:39 AM *Subject: *Re: Route optimization using GPUs?
WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages.
J~
On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
Tom, What does "these devices don't follow standard BGP behaviors" have to do with adding a NO_EXPORT or specific community on the import policy when a route is accepted, and being belt & suspenders with matching those communities to drop those routes on export to carriers/IX/PNI sessions? Ryan Hamel ________________________________ From: Tom Beecher <beecher@beecher.cc> Sent: Thursday, December 5, 2024 3:05 PM To: Ryan Hamel <ryan@rkhtech.org> Cc: Warren Kumari <warren@kumari.net>; Mike Hammett <nanog@ics-il.net>; nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits Nobody is asserting that operators are still not responsible for doing this. Stop it. What I think some of you guys fail to understand is that you can have perfectly appropriate policy protections in place that the prefixes generated by the optimizer can still bypass, because these devices don't follow standard BGP behaviors. This has happened before. On Thu, Dec 5, 2024 at 5:37 PM Ryan Hamel <ryan@rkhtech.org<mailto:ryan@rkhtech.org>> wrote: Warren, * "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… While that is also a factually true statement, you are also painting broad strokes over those who are responsible with those weapons. Employment requirements, hunting season, target practice at a range, skeet shooting, are just a few reasons to have them. Let's not dismiss those who follow the law, get qualified on a regular basis or have adequate training when/where/why/how to properly use them. "One rotten apple spoils the whole bunch", does not work here. This same thing also applies to operators of route optimizers. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits. For the incidents those route optimizers have caused, the vendors, their customers, and upstream ISPs are still in business. Ryan Hamel ________________________________ From: NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org<mailto:rkhtech.org@nanog.org>> on behalf of Warren Kumari <warren@kumari.net<mailto:warren@kumari.net>> Sent: Thursday, December 5, 2024 1:17 PM To: Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. On Thu, Dec 05, 2024 at 3:41 PM, Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: *shrugs* Incorrectly assigning the blame doesn't really help anyone. Sure, but the fact remains that there is blame to be assigned. It doesn't really matter to the affected network if the fault lies with the box itself, or the operator of the box, or the person who makes the morning coffee for the person who operates the box — the fact still remains that this call of devices have caused significant disruption for a whole bunch of external networks. "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… "Route optimizers don't hijack routes, operators with route optimizers hijack routes" falls into the same category… W ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<http://www.ics-il.com/> Midwest-IX http://www.midwest-ix.com<http://www.midwest-ix.com/> ________________________________ From: "Tom Beecher" <beecher@beecher.cc<mailto:beecher@beecher.cc>> To: "Mike Hammett" <nanog@ics-il.net<mailto:nanog@ics-il.net>> Cc: "Rich Compton" <RICH_COMPTON@comcast.com<mailto:RICH_COMPTON@comcast.com>>, nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 2:39:52 PM Subject: Re: Route optimization using GPUs? I think most of the hatred towards them is unwarranted, This is essentially saying "I've never had a problem , so I don't think it's a big deal." On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: Eh, different people have different opinions. I think most of the hatred towards them is unwarranted, ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<http://www.ics-il.com/> Midwest-IX http://www.midwest-ix.com<http://www.midwest-ix.com/> ________________________________ From: "Rich Compton" <RICH_COMPTON@comcast.com<mailto:RICH_COMPTON@comcast.com>> To: "Mike Hammett" <nanog@ics-il.net<mailto:nanog@ics-il.net>>, "Jason Bothe" <jbothe@me.com<mailto:jbothe@me.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 2:10:47 PM Subject: Re: Route optimization using GPUs? “I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.” -Job Snijders https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html From: NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org<mailto:comcast.com@nanog.org>> on behalf of Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> Date: Thursday, December 5, 2024 at 12:24 PM To: Jason Bothe <jbothe@me.com<mailto:jbothe@me.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.com/v3/__http:/www.ics-il.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ$> Midwest-IX http://www.midwest-ix.com<https://urldefense.com/v3/__http:/www.midwest-ix.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw$> ________________________________ From: "Jason Bothe via NANOG" <nanog@nanog.org<mailto:nanog@nanog.org>> To: "Drew Weaver" <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
Real life example of an optimizer issue I helped a friend sort out many years ago. Prefix 1 : 10.0.0.0/16, Communities A B C Prefix 2 : 10.0.0.0/20, Communities B D E Optimizer created /23s inside the /20. What communities do you think it applied to the new routes? - A B C - B D E - Random assortment of 3-6 values If you guessed random assortment, you're a winner! The optimizer got confused with 2 source prefixes covering what it was trying to generate, and every time it re-optimized it did something different with the communities. This is what I am talking about, that you can do everything "right" with respect to your policies and protections, and the optimizer can do something completely off book and cause an issue. On Thu, Dec 5, 2024 at 6:45 PM Ryan Hamel <ryan@rkhtech.org> wrote:
Tom,
What does "these devices don't follow standard BGP behaviors" have to do with adding a NO_EXPORT or specific community on the import policy when a route is accepted, and being belt & suspenders with matching those communities to drop those routes on export to carriers/IX/PNI sessions?
Ryan Hamel
------------------------------ *From:* Tom Beecher <beecher@beecher.cc> *Sent:* Thursday, December 5, 2024 3:05 PM *To:* Ryan Hamel <ryan@rkhtech.org> *Cc:* Warren Kumari <warren@kumari.net>; Mike Hammett <nanog@ics-il.net>; nanog@nanog.org <nanog@nanog.org> *Subject:* Re: Route optimization using GPUs?
Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments.
They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits
Nobody is asserting that operators are still not responsible for doing this. Stop it.
What I think some of you guys fail to understand is that you can have perfectly appropriate policy protections in place that the prefixes generated by the optimizer can still bypass, because these devices don't follow standard BGP behaviors. This has happened before.
On Thu, Dec 5, 2024 at 5:37 PM Ryan Hamel <ryan@rkhtech.org> wrote:
Warren,
- "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot…
While that is also a factually true statement, you are also painting broad strokes over those who are responsible with those weapons. Employment requirements, hunting season, target practice at a range, skeet shooting, are just a few reasons to have them. Let's not dismiss those who follow the law, get qualified on a regular basis or have adequate training when/where/why/how to properly use them.
"One rotten apple spoils the whole bunch", does not work here.
This same thing also applies to operators of route optimizers. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits. For the incidents those route optimizers have caused, the vendors, their customers, and upstream ISPs are still in business.
Ryan Hamel
------------------------------ *From:* NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org> on behalf of Warren Kumari <warren@kumari.net> *Sent:* Thursday, December 5, 2024 1:17 PM *To:* Mike Hammett <nanog@ics-il.net> *Cc:* nanog@nanog.org <nanog@nanog.org> *Subject:* Re: Route optimization using GPUs?
Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments.
On Thu, Dec 05, 2024 at 3:41 PM, Mike Hammett <nanog@ics-il.net> wrote:
*shrugs* Incorrectly assigning the blame doesn't really help anyone.
Sure, but the fact remains that there is blame to be assigned.
It doesn't really matter to the affected network if the fault lies with the box itself, or the operator of the box, or the person who makes the morning coffee for the person who operates the box — the fact still remains that this call of devices have caused significant disruption for a whole bunch of external networks.
"Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… "Route optimizers don't hijack routes, operators with route optimizers hijack routes" falls into the same category…
W
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
------------------------------ *From: *"Tom Beecher" <beecher@beecher.cc> *To: *"Mike Hammett" <nanog@ics-il.net> *Cc: *"Rich Compton" <RICH_COMPTON@comcast.com>, nanog@nanog.org *Sent: *Thursday, December 5, 2024 2:39:52 PM *Subject: *Re: Route optimization using GPUs?
I think most of the hatred towards them is unwarranted,
This is essentially saying "I've never had a problem , so I don't think it's a big deal."
On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett <nanog@ics-il.net> wrote:
Eh, different people have different opinions.
I think most of the hatred towards them is unwarranted,
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
------------------------------ *From: *"Rich Compton" <RICH_COMPTON@comcast.com> *To: *"Mike Hammett" <nanog@ics-il.net>, "Jason Bothe" <jbothe@me.com> *Cc: *nanog@nanog.org *Sent: *Thursday, December 5, 2024 2:10:47 PM *Subject: *Re: Route optimization using GPUs?
“I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.”
-Job Snijders
https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html
*From: *NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org> on behalf of Mike Hammett <nanog@ics-il.net> *Date: *Thursday, December 5, 2024 at 12:24 PM *To: *Jason Bothe <jbothe@me.com> *Cc: *nanog@nanog.org <nanog@nanog.org> *Subject: *Re: Route optimization using GPUs?
IIRC, the widespread outages are the result of exporting things that shouldn't be exported.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com <https://urldefense.com/v3/__http:/www.ics-il.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ$>
Midwest-IX http://www.midwest-ix.com <https://urldefense.com/v3/__http:/www.midwest-ix.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw$>
------------------------------
*From: *"Jason Bothe via NANOG" <nanog@nanog.org> *To: *"Drew Weaver" <drew.weaver@thenap.com> *Cc: *nanog@nanog.org *Sent: *Thursday, December 5, 2024 11:03:39 AM *Subject: *Re: Route optimization using GPUs?
WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages.
J~
On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
All the solution I was talking about does is select the “best” path egress from a multihomed ASN. It just inserts a smaller prefix into the local routing table. If people outside of your ASN are seeing that then you’ve already lost the game. From: NANOG <nanog-bounces+drew.weaver=thenap.com@nanog.org> On Behalf Of Tom Beecher Sent: Thursday, December 5, 2024 8:25 PM To: Ryan Hamel <ryan@rkhtech.org> Cc: nanog@nanog.org Subject: Re: Route optimization using GPUs? Real life example of an optimizer issue I helped a friend sort out many years ago. Prefix 1 : 10.0.0.0/16<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.0_16&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=RStntx3huOGWR0MfVGqNm4ctlvkpfHbDOrHMoN4BzsU&e=>, Communities A B C Prefix 2 : 10.0.0.0/20<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.0_20&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=hOSdLi-hYquxEayWsZIgvjIPV0glySY5_NWlk0Q0jJ0&e=>, Communities B D E Optimizer created /23s inside the /20. What communities do you think it applied to the new routes? - A B C - B D E - Random assortment of 3-6 values If you guessed random assortment, you're a winner! The optimizer got confused with 2 source prefixes covering what it was trying to generate, and every time it re-optimized it did something different with the communities. This is what I am talking about, that you can do everything "right" with respect to your policies and protections, and the optimizer can do something completely off book and cause an issue. On Thu, Dec 5, 2024 at 6:45 PM Ryan Hamel <ryan@rkhtech.org<mailto:ryan@rkhtech.org>> wrote: Tom, What does "these devices don't follow standard BGP behaviors" have to do with adding a NO_EXPORT or specific community on the import policy when a route is accepted, and being belt & suspenders with matching those communities to drop those routes on export to carriers/IX/PNI sessions? Ryan Hamel ________________________________ From: Tom Beecher <beecher@beecher.cc<mailto:beecher@beecher.cc>> Sent: Thursday, December 5, 2024 3:05 PM To: Ryan Hamel <ryan@rkhtech.org<mailto:ryan@rkhtech.org>> Cc: Warren Kumari <warren@kumari.net<mailto:warren@kumari.net>>; Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>>; nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits Nobody is asserting that operators are still not responsible for doing this. Stop it. What I think some of you guys fail to understand is that you can have perfectly appropriate policy protections in place that the prefixes generated by the optimizer can still bypass, because these devices don't follow standard BGP behaviors. This has happened before. On Thu, Dec 5, 2024 at 5:37 PM Ryan Hamel <ryan@rkhtech.org<mailto:ryan@rkhtech.org>> wrote: Warren, * "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… While that is also a factually true statement, you are also painting broad strokes over those who are responsible with those weapons. Employment requirements, hunting season, target practice at a range, skeet shooting, are just a few reasons to have them. Let's not dismiss those who follow the law, get qualified on a regular basis or have adequate training when/where/why/how to properly use them. "One rotten apple spoils the whole bunch", does not work here. This same thing also applies to operators of route optimizers. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits. For the incidents those route optimizers have caused, the vendors, their customers, and upstream ISPs are still in business. Ryan Hamel ________________________________ From: NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org<mailto:rkhtech.org@nanog.org>> on behalf of Warren Kumari <warren@kumari.net<mailto:warren@kumari.net>> Sent: Thursday, December 5, 2024 1:17 PM To: Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. On Thu, Dec 05, 2024 at 3:41 PM, Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: *shrugs* Incorrectly assigning the blame doesn't really help anyone. Sure, but the fact remains that there is blame to be assigned. It doesn't really matter to the affected network if the fault lies with the box itself, or the operator of the box, or the person who makes the morning coffee for the person who operates the box — the fact still remains that this call of devices have caused significant disruption for a whole bunch of external networks. "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… "Route optimizers don't hijack routes, operators with route optimizers hijack routes" falls into the same category… W ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ics-2Dil.com_&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=kZw8AOOpxu2KBSVPWZMH9dBV1WmXZ3NnkPRfOhtWhww&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.midwest-2Dix.com_&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=ZYL7eHsxMHnJOsOQBuq9jw9bXGxRISlqKICkN-sVVyw&e=> ________________________________ From: "Tom Beecher" <beecher@beecher.cc<mailto:beecher@beecher.cc>> To: "Mike Hammett" <nanog@ics-il.net<mailto:nanog@ics-il.net>> Cc: "Rich Compton" <RICH_COMPTON@comcast.com<mailto:RICH_COMPTON@comcast.com>>, nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 2:39:52 PM Subject: Re: Route optimization using GPUs? I think most of the hatred towards them is unwarranted, This is essentially saying "I've never had a problem , so I don't think it's a big deal." On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: Eh, different people have different opinions. I think most of the hatred towards them is unwarranted, ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ics-2Dil.com_&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=kZw8AOOpxu2KBSVPWZMH9dBV1WmXZ3NnkPRfOhtWhww&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.midwest-2Dix.com_&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=ZYL7eHsxMHnJOsOQBuq9jw9bXGxRISlqKICkN-sVVyw&e=> ________________________________ From: "Rich Compton" <RICH_COMPTON@comcast.com<mailto:RICH_COMPTON@comcast.com>> To: "Mike Hammett" <nanog@ics-il.net<mailto:nanog@ics-il.net>>, "Jason Bothe" <jbothe@me.com<mailto:jbothe@me.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 2:10:47 PM Subject: Re: Route optimization using GPUs? “I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.” -Job Snijders https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__mailman.nanog.org_pipermail_nanog_2017-2DAugust_092131.html&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=zviuljLAPm-GL2Wop7Jqa4Y6JzBL2vdR520fbfopdeI&e=> From: NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org<mailto:comcast.com@nanog.org>> on behalf of Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> Date: Thursday, December 5, 2024 at 12:24 PM To: Jason Bothe <jbothe@me.com<mailto:jbothe@me.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense.com_v3_-5F-5Fhttp-3A_www.ics-2Dil.com-5F-5F-3B-21-21CQl3mcHX2A-21GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-2DFJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ-24&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=nKeCefBjXxEbc2TrByVa3_-77wENFQR7xGBVYL-kps4&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense.com_v3_-5F-5Fhttp-3A_www.midwest-2Dix.com-5F-5F-3B-21-21CQl3mcHX2A-21GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-2DFJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw-24&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=hmpG3hyUYYU58U5AtL1EY2EH_OhpLV05Qvc288aV6zc&e=> ________________________________ From: "Jason Bothe via NANOG" <nanog@nanog.org<mailto:nanog@nanog.org>> To: "Drew Weaver" <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
Ryan Hamel wrote on 05/12/2024 23:45:
What does "these devices don't follow standard BGP behaviors" have to do with adding a NO_EXPORT or specific community on the import policy when a route is accepted, and being belt & suspenders with matching those communities to drop those routes on export to carriers/IX/PNI sessions?
Ryan, BGP ensures loop-free interdomain path computation by inspecting the AS path of each NLRI. If a routing optimiser rewrites all the AS paths for all the NLRIs it receives, then it's just pooped all over the primary component of BGP that's designed to ensure that interdomain BGP actually works in the way that it's supposed to do in the first place, which also acts as an intrinsic safety guard against dfz hijacking. Removing an intrinsic safety guard like this is an inherently risky thing to do. When you elevate the inherent risk of a system, you necessarily elevate either the likelihood of failure or the consequences of a failure, or both. As an industry, we should be well beyond the point of having to tell people that this is a poor idea, in the same way that we don't need to tell people that bypassing electrical fuse boxes is a poor idea, or removing railings on stair-cases, or not wearing motorbike helmets, or anything else designed to mitigate the unfortunate consequences of entirely predictable accidents. Nick
On Fri, Dec 6, 2024 at 8:34 AM Nick Hilliard <nick@foobar.org> wrote:
BGP ensures loop-free interdomain path computation by inspecting the AS path of each NLRI. If a routing optimiser rewrites all the AS paths for all the NLRIs it receives, then it's just pooped all over the primary component of BGP that's designed to ensure that interdomain BGP actually works in the way that it's supposed to do in the first place, which also acts as an intrinsic safety guard against dfz hijacking.
As an industry, we should be well beyond the point of having to tell people that this is a poor idea,
Hi Nick, Have you ever filtered routes from the BGP table and replaced them with a default route? Perhaps the TCAM was too full and you weren't ready to upgrade yet? There's nothing inherently wrong with filtering BGP routes and replacing them in local routes of your own selection. Nor is there anything wrong with using a complicated and detailed local selection process. The error lies in allowing those local routes to accidentally escape your AS. Since people being people, they make mistakes, I thought a little standards work in the area might head off some of those escapes. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
William, Exactly! An example below is where operators/orgs do not have the funds for a full table router deployment and gather top talkers from sFlow, which says what routes are to be installed in TCAM, instead of hitting a default route. https://blog.sflow.com/2015/07/sdn-router-using-merchant-silicon-top.html https://blog.sflow.com/2015/10/active-route-manager.html https://blog.sflow.com/2016/07/internet-router-using-merchant-silicon.html Maybe instead of sFlow, it's FIB compression for switches that can only handle 512K IPv4 routes, or routers that are showing their age with 1M IPv4 route capacity. Ryan Hamel ________________________________ From: NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org> on behalf of William Herrin <bill@herrin.us> Sent: Friday, December 6, 2024 9:01 AM To: Nick Hilliard <nick@foobar.org> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. On Fri, Dec 6, 2024 at 8:34 AM Nick Hilliard <nick@foobar.org> wrote:
BGP ensures loop-free interdomain path computation by inspecting the AS path of each NLRI. If a routing optimiser rewrites all the AS paths for all the NLRIs it receives, then it's just pooped all over the primary component of BGP that's designed to ensure that interdomain BGP actually works in the way that it's supposed to do in the first place, which also acts as an intrinsic safety guard against dfz hijacking.
As an industry, we should be well beyond the point of having to tell people that this is a poor idea,
Hi Nick, Have you ever filtered routes from the BGP table and replaced them with a default route? Perhaps the TCAM was too full and you weren't ready to upgrade yet? There's nothing inherently wrong with filtering BGP routes and replacing them in local routes of your own selection. Nor is there anything wrong with using a complicated and detailed local selection process. The error lies in allowing those local routes to accidentally escape your AS. Since people being people, they make mistakes, I thought a little standards work in the area might head off some of those escapes. Regards, Bill Herrin -- William Herrin bill@herrin.us https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbill.herrin.us%2F&data=05%7C02%7Cryan%40rkhtech.org%7C3cda590fc71545a0ec5808dd1617f0d6%7C81c24bb4f9ec4739ba4d25c42594d996%7C0%7C0%7C638691014279933365%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Ti5JOFv4rHSeqA9KCG%2BXcwfp%2BqUQ1ujsiu6pd8TZ7bc%3D&reserved=0<https://bill.herrin.us/>
* ryan@rkhtech.org (Ryan Hamel) [Fri 06 Dec 2024, 18:46 CET]:
William,
Exactly! An example below is where operators/orgs do not have the funds for a full table router deployment and gather top talkers from sFlow, which says what routes are to be installed in TCAM, instead of hitting a default route.
You realise that this is not just what the problematic "route optimizers" do, right? And also not the problematic bit? -- Niels.
The ones I've used have simply injected more specific routes into my own local routing table to determine what path traffic takes to reach a specific destination. That's pretty much all it does. The same way I use communities to do this based upon geography, or 500 other things [avoiding 1299 getting to certain networks]. The difference is now it's all manual. I wasn't advocating for allowing route optimizers to inject routes that escape your own ASN. I'm not sure how that became the topic. Thanks, -Drew -----Original Message----- From: NANOG <nanog-bounces+drew.weaver=thenap.com@nanog.org> On Behalf Of Niels Bakker Sent: Friday, December 6, 2024 1:03 PM To: nanog@nanog.org Subject: Re: Route optimization using GPUs? * ryan@rkhtech.org (Ryan Hamel) [Fri 06 Dec 2024, 18:46 CET]:
William,
Exactly! An example below is where operators/orgs do not have the funds for a full table router deployment and gather top talkers from sFlow, which says what routes are to be installed in TCAM, instead of hitting a default route.
You realise that this is not just what the problematic "route optimizers" do, right? And also not the problematic bit? -- Niels.
Nick, I understand there are rules and unspoken guidelines/rules for the DFZ, but when it comes to each individual AS, that org/operator can run their AS internally however they please, and maybe they have considered the risks you have mentioned. That said, I can argue that upstreams not filtering their customers properly removes a safety guard, upstreams not implementing RPKI removes a safety guard, not properly prepending communities on synthetic routes to drop them on export again removes a safety guard. I can go on... * As an industry, we should be well beyond the point of having to tell people that this is a poor idea, in the same way that we don't need to tell people that bypassing electrical fuse boxes is a poor idea, or removing railings on stair-cases, or not wearing motorbike helmets, or anything else designed to mitigate the unfortunate consequences of entirely predictable accidents. Where this statement falls short is, those are all regulated by building codes, laws, etc. No laws exist dictating how BGP, routing protocols in general, and topologies must be implemented, nor what safety guidelines must be adhered to. Ryan Hamel ________________________________ From: Nick Hilliard <nick@foobar.org> Sent: Friday, December 6, 2024 8:34 AM To: Ryan Hamel <ryan@rkhtech.org> Cc: Tom Beecher <beecher@beecher.cc>; nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. Ryan Hamel wrote on 05/12/2024 23:45:
What does "these devices don't follow standard BGP behaviors" have to do with adding a NO_EXPORT or specific community on the import policy when a route is accepted, and being belt & suspenders with matching those communities to drop those routes on export to carriers/IX/PNI sessions?
Ryan, BGP ensures loop-free interdomain path computation by inspecting the AS path of each NLRI. If a routing optimiser rewrites all the AS paths for all the NLRIs it receives, then it's just pooped all over the primary component of BGP that's designed to ensure that interdomain BGP actually works in the way that it's supposed to do in the first place, which also acts as an intrinsic safety guard against dfz hijacking. Removing an intrinsic safety guard like this is an inherently risky thing to do. When you elevate the inherent risk of a system, you necessarily elevate either the likelihood of failure or the consequences of a failure, or both. As an industry, we should be well beyond the point of having to tell people that this is a poor idea, in the same way that we don't need to tell people that bypassing electrical fuse boxes is a poor idea, or removing railings on stair-cases, or not wearing motorbike helmets, or anything else designed to mitigate the unfortunate consequences of entirely predictable accidents. Nick
not properly prepending communities on synthetic routes
Let's not normalize 'synthetic route' as a term. It's not a thing that exists. On Fri, Dec 6, 2024 at 12:32 PM Ryan Hamel <ryan@rkhtech.org> wrote:
Nick,
I understand there are rules and unspoken guidelines/rules for the DFZ, but when it comes to each individual AS, that org/operator can run their AS internally however they please, and maybe they have considered the risks you have mentioned.
That said, I can argue that upstreams not filtering their customers properly removes a safety guard, upstreams not implementing RPKI removes a safety guard, not properly prepending communities on synthetic routes to drop them on export again removes a safety guard. I can go on...
- As an industry, we should be well beyond the point of having to tell people that this is a poor idea, in the same way that we don't need to tell people that bypassing electrical fuse boxes is a poor idea, or removing railings on stair-cases, or not wearing motorbike helmets, or anything else designed to mitigate the unfortunate consequences of entirely predictable accidents.
Where this statement falls short is, those are all regulated by building codes, laws, etc. No laws exist dictating how BGP, routing protocols in general, and topologies must be implemented, nor what safety guidelines must be adhered to.
Ryan Hamel
------------------------------ *From:* Nick Hilliard <nick@foobar.org> *Sent:* Friday, December 6, 2024 8:34 AM *To:* Ryan Hamel <ryan@rkhtech.org> *Cc:* Tom Beecher <beecher@beecher.cc>; nanog@nanog.org <nanog@nanog.org> *Subject:* Re: Route optimization using GPUs?
Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments.
Ryan Hamel wrote on 05/12/2024 23:45:
What does "these devices don't follow standard BGP behaviors" have to do with adding a NO_EXPORT or specific community on the import policy when a route is accepted, and being belt & suspenders with matching those communities to drop those routes on export to carriers/IX/PNI sessions?
Ryan,
BGP ensures loop-free interdomain path computation by inspecting the AS path of each NLRI. If a routing optimiser rewrites all the AS paths for all the NLRIs it receives, then it's just pooped all over the primary component of BGP that's designed to ensure that interdomain BGP actually works in the way that it's supposed to do in the first place, which also acts as an intrinsic safety guard against dfz hijacking.
Removing an intrinsic safety guard like this is an inherently risky thing to do. When you elevate the inherent risk of a system, you necessarily elevate either the likelihood of failure or the consequences of a failure, or both.
As an industry, we should be well beyond the point of having to tell people that this is a poor idea, in the same way that we don't need to tell people that bypassing electrical fuse boxes is a poor idea, or removing railings on stair-cases, or not wearing motorbike helmets, or anything else designed to mitigate the unfortunate consequences of entirely predictable accidents.
Nick
Tom, The automotive industry has normalized "synthetic". It's motor oil that is artificially created, vs pulled out of the ground and refined. It's a perfect analogy for routes that were created by third-party software, vs organically created/redistributed from the proper AS. Ryan Hamel ________________________________ From: Tom Beecher <beecher@beecher.cc> Sent: Friday, December 6, 2024 10:10 AM To: Ryan Hamel <ryan@rkhtech.org> Cc: Nick Hilliard <nick@foobar.org>; nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. not properly prepending communities on synthetic routes Let's not normalize 'synthetic route' as a term. It's not a thing that exists. On Fri, Dec 6, 2024 at 12:32 PM Ryan Hamel <ryan@rkhtech.org<mailto:ryan@rkhtech.org>> wrote: Nick, I understand there are rules and unspoken guidelines/rules for the DFZ, but when it comes to each individual AS, that org/operator can run their AS internally however they please, and maybe they have considered the risks you have mentioned. That said, I can argue that upstreams not filtering their customers properly removes a safety guard, upstreams not implementing RPKI removes a safety guard, not properly prepending communities on synthetic routes to drop them on export again removes a safety guard. I can go on... * As an industry, we should be well beyond the point of having to tell people that this is a poor idea, in the same way that we don't need to tell people that bypassing electrical fuse boxes is a poor idea, or removing railings on stair-cases, or not wearing motorbike helmets, or anything else designed to mitigate the unfortunate consequences of entirely predictable accidents. Where this statement falls short is, those are all regulated by building codes, laws, etc. No laws exist dictating how BGP, routing protocols in general, and topologies must be implemented, nor what safety guidelines must be adhered to. Ryan Hamel ________________________________ From: Nick Hilliard <nick@foobar.org<mailto:nick@foobar.org>> Sent: Friday, December 6, 2024 8:34 AM To: Ryan Hamel <ryan@rkhtech.org<mailto:ryan@rkhtech.org>> Cc: Tom Beecher <beecher@beecher.cc<mailto:beecher@beecher.cc>>; nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. Ryan Hamel wrote on 05/12/2024 23:45:
What does "these devices don't follow standard BGP behaviors" have to do with adding a NO_EXPORT or specific community on the import policy when a route is accepted, and being belt & suspenders with matching those communities to drop those routes on export to carriers/IX/PNI sessions?
Ryan, BGP ensures loop-free interdomain path computation by inspecting the AS path of each NLRI. If a routing optimiser rewrites all the AS paths for all the NLRIs it receives, then it's just pooped all over the primary component of BGP that's designed to ensure that interdomain BGP actually works in the way that it's supposed to do in the first place, which also acts as an intrinsic safety guard against dfz hijacking. Removing an intrinsic safety guard like this is an inherently risky thing to do. When you elevate the inherent risk of a system, you necessarily elevate either the likelihood of failure or the consequences of a failure, or both. As an industry, we should be well beyond the point of having to tell people that this is a poor idea, in the same way that we don't need to tell people that bypassing electrical fuse boxes is a poor idea, or removing railings on stair-cases, or not wearing motorbike helmets, or anything else designed to mitigate the unfortunate consequences of entirely predictable accidents. Nick
Ryan- Unfortunately it doesn't appear that you have a solid understanding of core BGP fundamentals. I suggest starting with a read of RFC4271. Have a great weekend and holiday season. On Fri, Dec 6, 2024 at 1:19 PM Ryan Hamel <ryan@rkhtech.org> wrote:
Tom,
The automotive industry has normalized "synthetic". It's motor oil that is artificially created, vs pulled out of the ground and refined. It's a perfect analogy for routes that were created by third-party software, vs organically created/redistributed from the proper AS.
Ryan Hamel
------------------------------ *From:* Tom Beecher <beecher@beecher.cc> *Sent:* Friday, December 6, 2024 10:10 AM *To:* Ryan Hamel <ryan@rkhtech.org> *Cc:* Nick Hilliard <nick@foobar.org>; nanog@nanog.org <nanog@nanog.org> *Subject:* Re: Route optimization using GPUs?
Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments.
not properly prepending communities on synthetic routes
Let's not normalize 'synthetic route' as a term. It's not a thing that exists.
On Fri, Dec 6, 2024 at 12:32 PM Ryan Hamel <ryan@rkhtech.org> wrote:
Nick,
I understand there are rules and unspoken guidelines/rules for the DFZ, but when it comes to each individual AS, that org/operator can run their AS internally however they please, and maybe they have considered the risks you have mentioned.
That said, I can argue that upstreams not filtering their customers properly removes a safety guard, upstreams not implementing RPKI removes a safety guard, not properly prepending communities on synthetic routes to drop them on export again removes a safety guard. I can go on...
- As an industry, we should be well beyond the point of having to tell people that this is a poor idea, in the same way that we don't need to tell people that bypassing electrical fuse boxes is a poor idea, or removing railings on stair-cases, or not wearing motorbike helmets, or anything else designed to mitigate the unfortunate consequences of entirely predictable accidents.
Where this statement falls short is, those are all regulated by building codes, laws, etc. No laws exist dictating how BGP, routing protocols in general, and topologies must be implemented, nor what safety guidelines must be adhered to.
Ryan Hamel
------------------------------ *From:* Nick Hilliard <nick@foobar.org> *Sent:* Friday, December 6, 2024 8:34 AM *To:* Ryan Hamel <ryan@rkhtech.org> *Cc:* Tom Beecher <beecher@beecher.cc>; nanog@nanog.org <nanog@nanog.org> *Subject:* Re: Route optimization using GPUs?
Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments.
Ryan Hamel wrote on 05/12/2024 23:45:
What does "these devices don't follow standard BGP behaviors" have to do with adding a NO_EXPORT or specific community on the import policy when a route is accepted, and being belt & suspenders with matching those communities to drop those routes on export to carriers/IX/PNI sessions?
Ryan,
BGP ensures loop-free interdomain path computation by inspecting the AS path of each NLRI. If a routing optimiser rewrites all the AS paths for all the NLRIs it receives, then it's just pooped all over the primary component of BGP that's designed to ensure that interdomain BGP actually works in the way that it's supposed to do in the first place, which also acts as an intrinsic safety guard against dfz hijacking.
Removing an intrinsic safety guard like this is an inherently risky thing to do. When you elevate the inherent risk of a system, you necessarily elevate either the likelihood of failure or the consequences of a failure, or both.
As an industry, we should be well beyond the point of having to tell people that this is a poor idea, in the same way that we don't need to tell people that bypassing electrical fuse boxes is a poor idea, or removing railings on stair-cases, or not wearing motorbike helmets, or anything else designed to mitigate the unfortunate consequences of entirely predictable accidents.
Nick
Ryan Hamel wrote on 06/12/2024 17:32:
That said, I can argue that upstreams not filtering their customers properly removes a safety guard, upstreams not implementing RPKI removes a safety guard, not properly prepending communities on synthetic routes to drop them on export again removes a safety guard. I can go on...
There's a fundamental difference. Not filtering customers properly fails to implement a safety guard that should have been implemented. Not implementing RPKI fails to implement an additional safety guard. Not properly prepending communities fails to implement an additional safety guard. Rewriting the AS path removes a core descriptive component of NLRIs inherent in the BGP protocol which is critical to implementing other safety guards. Including - as an example of only of the harmful effects of this practice - the ability for the upstream to automatically drop all routes which you just reflected back to it, having just rewritten the AS path to remove their ASN and rewrite the NHIP, because bgp loop-free routing requires this by default in the protocol. When you drop core safety components, accidents are more likely to happen.
Where this statement falls short is, those are all regulated by building codes, laws, etc. No laws exist dictating how BGP, routing protocols in general, and topologies must be implemented, nor what safety guidelines must be adhered to.
The normal progression of many technologies ends in regulation. We already have regulation which covers bgp inter-domain routing security in the EU, and I'd be surprised if it wasn't going to happen in other jurisdictions in due course. In the US, warning shots have already been fired by the white house:
https://www.whitehouse.gov/wp-content/uploads/2024/09/Roadmap-to-Enhancing-I...
This style of document should be taken as notification that interdomain routing security is fresh on the table of regulatory bodies in the US. Nick
On Fri, Dec 6, 2024 at 11:03 AM Nick Hilliard <nick@foobar.org> wrote:
Including - as an example of only of the harmful effects of this practice
Hi Nick, There is consequence to resisting standards work in an area of need just because you're broadly opposed to the technology being used that way in any capacity. One example is IPv6. Another is CGNAT. If you'd rather not follow those examples, stop talking about why route optimizers mustn't be done and move the conversation to what it would take to _safely_ do it. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
Nick, I appreciate the explanation and example, and agree with that as a very strong recommendation. Reading Noction's IRP Lite documentation (https://www.noction.com/wp-content/uploads/2016/09/irp-lite-documentation.pd...) - page 214, with bgpd.as_pathset to "5 4 2 3" by default (table below), it makes a genuine effort to use the same AS-path when possible. 0 - Allow empty AS-PATH 2 - Use non-empty reconstructed AS-PATH (Announce AS-path reconstructed from traceroute) 3 - Reconstruct AS path with provider ASN and prefix origin ASN 4 - Use AS-Path from BMP 5 - Use AS-Path from BGP Alternative paths (RFC 7911) That means (at least for Noction) the operator has to go out of their way to disable safety, so those that claim it has bad defaults, may want to RTFM. Now, I have never had a need to change that value, nor have I advised others to do the same. I agree having an empty AS path is asking for trouble when it gets leaked. Ryan Hamel ________________________________ From: Nick Hilliard <nick@foobar.org> Sent: Friday, December 6, 2024 11:03 AM To: Ryan Hamel <ryan@rkhtech.org> Cc: Tom Beecher <beecher@beecher.cc>; nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. Ryan Hamel wrote on 06/12/2024 17:32:
That said, I can argue that upstreams not filtering their customers properly removes a safety guard, upstreams not implementing RPKI removes a safety guard, not properly prepending communities on synthetic routes to drop them on export again removes a safety guard. I can go on...
There's a fundamental difference. Not filtering customers properly fails to implement a safety guard that should have been implemented. Not implementing RPKI fails to implement an additional safety guard. Not properly prepending communities fails to implement an additional safety guard. Rewriting the AS path removes a core descriptive component of NLRIs inherent in the BGP protocol which is critical to implementing other safety guards. Including - as an example of only of the harmful effects of this practice - the ability for the upstream to automatically drop all routes which you just reflected back to it, having just rewritten the AS path to remove their ASN and rewrite the NHIP, because bgp loop-free routing requires this by default in the protocol. When you drop core safety components, accidents are more likely to happen.
Where this statement falls short is, those are all regulated by building codes, laws, etc. No laws exist dictating how BGP, routing protocols in general, and topologies must be implemented, nor what safety guidelines must be adhered to.
The normal progression of many technologies ends in regulation. We already have regulation which covers bgp inter-domain routing security in the EU, and I'd be surprised if it wasn't going to happen in other jurisdictions in due course. In the US, warning shots have already been fired by the white house:
This style of document should be taken as notification that interdomain routing security is fresh on the table of regulatory bodies in the US. Nick
On Fri, Dec 06, 2024 at 10:55:30PM +0000, Ryan Hamel wrote:
That means (at least for Noction) the operator has to go out of their way to disable safety, so those that claim it has bad defaults, may want to RTFM.
While I appreciate various business drivers and motivations exist to deploy software solutions to modify & optimize routing on the fly, I think I disagree with you on this one point. Operators *literally* have to go out of their way to configure Noction to be safe to use. It is not safe to use out of the box. Page 29: https://www.noction.com/wp-content/uploads/2016/09/irp-lite-documentation.pd... """ improvements should be stopped from propagating across routing domains. A route map is used to address this. [snip] Refer your router capabilities in order to produce the correct route map. The route map MUST be integrated into existing route maps. It is not sufficient to simply append them. """ (red: Noction calls the synthetic unauthorized more-specific hijack route announcements "improvements")
From Noction's other documentation at https://www.noction.com/blog/route-optimizers
""" In order to further reduce the likelihood of these problems occurring in the future, we will be adding a feature within Noction IRP to give an option to tag all the more specific prefixes that it generates with the BGP NO_EXPORT community. -->>> This will not be enabled by default <<<--- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ """ Noction made their software UNSAFE BY DEFAULT. In my opinion this is a very poor product design choice, and the very reason we keep coming back to this specific topic. Other routing optimizers product never make the news, guess what they all have in common? They set NO_EXPORT by default! :-) Efforts to define new extensions to the BGP protocol to make this type of product safer in use (creating a new AFI/SAFI or something else) via IETF is interesting, but it appears Noction is not even doing the bare minimum within the existing standards. Kind regards, Job
Anyway I just wanted to clarify that during my use of route optimization all the devices did was inject a more specific route to a prefix that my network would then use to reach that prefix. Those more specific routes weren't ever advertised to external BGP peers and if they were they shouldn't have been accepted. These days it's a little scarier to me to come in and see a customer ticket indicating that traffic is going from Ohio to Amsterdam and then back to New Jersey before going to Seattle to get to South Korea (real example from a couple of weeks ago) than it would be if something automated just... picked another route. YMMV I guess. -----Original Message----- From: NANOG <nanog-bounces+drew.weaver=thenap.com@nanog.org> On Behalf Of Job Snijders via NANOG Sent: Saturday, December 7, 2024 5:20 AM To: Ryan Hamel <ryan@rkhtech.org> Cc: nanog@nanog.org Subject: Re: Route optimization using GPUs? On Fri, Dec 06, 2024 at 10:55:30PM +0000, Ryan Hamel wrote:
That means (at least for Noction) the operator has to go out of their way to disable safety, so those that claim it has bad defaults, may want to RTFM.
While I appreciate various business drivers and motivations exist to deploy software solutions to modify & optimize routing on the fly, I think I disagree with you on this one point. Operators *literally* have to go out of their way to configure Noction to be safe to use. It is not safe to use out of the box. Page 29: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.noction.com_wp-2Dcontent_uploads_2016_09_irp-2Dlite-2Ddocumentation.pdf&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=-_Rlv_p1lHlsVx5Sa67XIaQJYNw4IADo1JitKZvA8ZI83kk4oZWCXuAmg6M54dd9&s=Ef5Ju3LsdLECff_nlI46a3cLejTooG_OyMBOu2GFcoU&e= """ improvements should be stopped from propagating across routing domains. A route map is used to address this. [snip] Refer your router capabilities in order to produce the correct route map. The route map MUST be integrated into existing route maps. It is not sufficient to simply append them. """ (red: Noction calls the synthetic unauthorized more-specific hijack route announcements "improvements")
From Noction's other documentation at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.noction.com_blog_route-2Doptimizers&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=-_Rlv_p1lHlsVx5Sa67XIaQJYNw4IADo1JitKZvA8ZI83kk4oZWCXuAmg6M54dd9&s=6O4R2ds5EJDg9U9ZcgqJ_tQ5rxAayySPswGNC_-TDPY&e=
""" In order to further reduce the likelihood of these problems occurring in the future, we will be adding a feature within Noction IRP to give an option to tag all the more specific prefixes that it generates with the BGP NO_EXPORT community. -->>> This will not be enabled by default <<<--- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ """ Noction made their software UNSAFE BY DEFAULT. In my opinion this is a very poor product design choice, and the very reason we keep coming back to this specific topic. Other routing optimizers product never make the news, guess what they all have in common? They set NO_EXPORT by default! :-) Efforts to define new extensions to the BGP protocol to make this type of product safer in use (creating a new AFI/SAFI or something else) via IETF is interesting, but it appears Noction is not even doing the bare minimum within the existing standards. Kind regards, Job
On Thu, Dec 5, 2024 at 2:34 PM Ryan Hamel <ryan@rkhtech.org> wrote:
This same thing also applies to operators of route optimizers. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits
Hi Ryan, Is there room or use for standards work here? Just spitballing, but something along the lines of: A synthetic BGP route is a BGP route produced by an AS other than the original origin of the IP addresses contained. Examples include default routes, black-hole routes and routes produced by a route optimizer in a middle network. The producer of a synthetic BGP route MUST mark the route with community XXX. Routers MUST NOT remove the synthetic community XXX from a route. Synthetic routes learned from EBGP sessions MUST be rejected by default. Routers MAY accept synthetic routes learned from EBGP sessions if explicitly configured to do so. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
Bill, While that sounds plausible on paper, keep in mind it will also include IX route servers and MPLS route reflectors. Noction can append communities to its announcements, meaning one can very easily set NO_EXPORT (65535:65281) and/or a community applicable to the AS. The latter going back to what I said about matching a community on export policies to carriers/IX/PNIs, to drop them if the network vendor has a bug in NO_EXPORT, like https://quickview.cloudapps.cisco.com/quickview/bug/CSCuv94859. Ryan Hamel ________________________________ From: William Herrin <bill@herrin.us> Sent: Thursday, December 5, 2024 3:58 PM To: Ryan Hamel <ryan@rkhtech.org> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. On Thu, Dec 5, 2024 at 2:34 PM Ryan Hamel <ryan@rkhtech.org> wrote:
This same thing also applies to operators of route optimizers. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits
Hi Ryan, Is there room or use for standards work here? Just spitballing, but something along the lines of: A synthetic BGP route is a BGP route produced by an AS other than the original origin of the IP addresses contained. Examples include default routes, black-hole routes and routes produced by a route optimizer in a middle network. The producer of a synthetic BGP route MUST mark the route with community XXX. Routers MUST NOT remove the synthetic community XXX from a route. Synthetic routes learned from EBGP sessions MUST be rejected by default. Routers MAY accept synthetic routes learned from EBGP sessions if explicitly configured to do so. Regards, Bill Herrin -- William Herrin bill@herrin.us https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbill.herrin.us%2F&data=05%7C02%7Cryan%40rkhtech.org%7C1ecddd454c53487dd9f008dd1588b6c7%7C81c24bb4f9ec4739ba4d25c42594d996%7C0%7C0%7C638690399122795868%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=oIMMlb2brNWyqSZCspO3CoRsBJfuBmzYcjGUki62xO0%3D&reserved=0<https://bill.herrin.us/>
Yes, some level of shared responsibility has been present on the Internet throughout it’s entire existence. From: NANOG <nanog-bounces+drew.weaver=thenap.com@nanog.org> On Behalf Of Ryan Hamel Sent: Thursday, December 5, 2024 5:35 PM To: Warren Kumari <warren@kumari.net>; Mike Hammett <nanog@ics-il.net> Cc: nanog@nanog.org Subject: Re: Route optimization using GPUs? Warren, * "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… While that is also a factually true statement, you are also painting broad strokes over those who are responsible with those weapons. Employment requirements, hunting season, target practice at a range, skeet shooting, are just a few reasons to have them. Let's not dismiss those who follow the law, get qualified on a regular basis or have adequate training when/where/why/how to properly use them. "One rotten apple spoils the whole bunch", does not work here. This same thing also applies to operators of route optimizers. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits. For the incidents those route optimizers have caused, the vendors, their customers, and upstream ISPs are still in business. Ryan Hamel ________________________________ From: NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org<mailto:nanog-bounces+ryan=rkhtech.org@nanog.org>> on behalf of Warren Kumari <warren@kumari.net<mailto:warren@kumari.net>> Sent: Thursday, December 5, 2024 1:17 PM To: Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. On Thu, Dec 05, 2024 at 3:41 PM, Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: *shrugs* Incorrectly assigning the blame doesn't really help anyone. Sure, but the fact remains that there is blame to be assigned. It doesn't really matter to the affected network if the fault lies with the box itself, or the operator of the box, or the person who makes the morning coffee for the person who operates the box — the fact still remains that this call of devices have caused significant disruption for a whole bunch of external networks. "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… "Route optimizers don't hijack routes, operators with route optimizers hijack routes" falls into the same category… W ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ics-2Dil.com_&d=DwMGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=c5RdX9_eT_rOXXWRn7RdtBigxa-CjsodrYCuf72L3Yar7mThseCqJylnqh84PptD&s=6bg4IieJuZ72py5bKR3cQmMXWcCltpcbVvzPOgkJL5o&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.midwest-2Dix.com_&d=DwMGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=c5RdX9_eT_rOXXWRn7RdtBigxa-CjsodrYCuf72L3Yar7mThseCqJylnqh84PptD&s=UVZ_a2TJ9ofctrjzoB6x-YCPg0XaejrCYRAbXCfSQHA&e=> ________________________________ From: "Tom Beecher" <beecher@beecher.cc<mailto:beecher@beecher.cc>> To: "Mike Hammett" <nanog@ics-il.net<mailto:nanog@ics-il.net>> Cc: "Rich Compton" <RICH_COMPTON@comcast.com<mailto:RICH_COMPTON@comcast.com>>, nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 2:39:52 PM Subject: Re: Route optimization using GPUs? I think most of the hatred towards them is unwarranted, This is essentially saying "I've never had a problem , so I don't think it's a big deal." On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: Eh, different people have different opinions. I think most of the hatred towards them is unwarranted, ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ics-2Dil.com_&d=DwMGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=c5RdX9_eT_rOXXWRn7RdtBigxa-CjsodrYCuf72L3Yar7mThseCqJylnqh84PptD&s=6bg4IieJuZ72py5bKR3cQmMXWcCltpcbVvzPOgkJL5o&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.midwest-2Dix.com_&d=DwMGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=c5RdX9_eT_rOXXWRn7RdtBigxa-CjsodrYCuf72L3Yar7mThseCqJylnqh84PptD&s=UVZ_a2TJ9ofctrjzoB6x-YCPg0XaejrCYRAbXCfSQHA&e=> ________________________________ From: "Rich Compton" <RICH_COMPTON@comcast.com<mailto:RICH_COMPTON@comcast.com>> To: "Mike Hammett" <nanog@ics-il.net<mailto:nanog@ics-il.net>>, "Jason Bothe" <jbothe@me.com<mailto:jbothe@me.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 2:10:47 PM Subject: Re: Route optimization using GPUs? “I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.” -Job Snijders https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__mailman.nanog.org_pipermail_nanog_2017-2DAugust_092131.html&d=DwMGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=c5RdX9_eT_rOXXWRn7RdtBigxa-CjsodrYCuf72L3Yar7mThseCqJylnqh84PptD&s=GRUGP-OK_Z6n0OvayOyKxFpztPLGsaqUnG8vLCZ_yOw&e=> From: NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org<mailto:comcast.com@nanog.org>> on behalf of Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> Date: Thursday, December 5, 2024 at 12:24 PM To: Jason Bothe <jbothe@me.com<mailto:jbothe@me.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense.com_v3_-5F-5Fhttp-3A_www.ics-2Dil.com-5F-5F-3B-21-21CQl3mcHX2A-21GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-2DFJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ-24&d=DwMGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=c5RdX9_eT_rOXXWRn7RdtBigxa-CjsodrYCuf72L3Yar7mThseCqJylnqh84PptD&s=l5Il25Wbz54SEP2QNrAa5vCPk8R7gUHLUhTGIdzXgDw&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense.com_v3_-5F-5Fhttp-3A_www.midwest-2Dix.com-5F-5F-3B-21-21CQl3mcHX2A-21GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-2DFJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw-24&d=DwMGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=c5RdX9_eT_rOXXWRn7RdtBigxa-CjsodrYCuf72L3Yar7mThseCqJylnqh84PptD&s=WGi1YV2jO4Dn2ooKQCNF9oMS8f9AApIXc4tU9sud-p8&e=> ________________________________ From: "Jason Bothe via NANOG" <nanog@nanog.org<mailto:nanog@nanog.org>> To: "Drew Weaver" <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
To be fair it’s a technology and how it’s implemented does make a difference. Wouldn’t you agree? From: NANOG <nanog-bounces+drew.weaver=thenap.com@nanog.org> On Behalf Of Tom Beecher Sent: Thursday, December 5, 2024 3:40 PM To: Mike Hammett <nanog@ics-il.net> Cc: nanog@nanog.org Subject: Re: Route optimization using GPUs? I think most of the hatred towards them is unwarranted, This is essentially saying "I've never had a problem , so I don't think it's a big deal." On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: Eh, different people have different opinions. I think most of the hatred towards them is unwarranted, ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ics-2Dil.com&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=05VYBsDnfxCJ7Vxr7z0MIQ9mIlNuADBqxVE2HDtJGzq5iJ-9eBv_7Jlhc5rqpFTg&s=NLianZlO4ViwCYophTrfZE50uZKYa8yssLj8lsq3pEM&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.midwest-2Dix.com&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=05VYBsDnfxCJ7Vxr7z0MIQ9mIlNuADBqxVE2HDtJGzq5iJ-9eBv_7Jlhc5rqpFTg&s=HpRDI05R4hN7UhPogwyTylvAWP-tGCZNBJAiM-a-8Os&e=> ________________________________ From: "Rich Compton" <RICH_COMPTON@comcast.com<mailto:RICH_COMPTON@comcast.com>> To: "Mike Hammett" <nanog@ics-il.net<mailto:nanog@ics-il.net>>, "Jason Bothe" <jbothe@me.com<mailto:jbothe@me.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 2:10:47 PM Subject: Re: Route optimization using GPUs? “I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.” -Job Snijders https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__mailman.nanog.org_pipermail_nanog_2017-2DAugust_092131.html&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=05VYBsDnfxCJ7Vxr7z0MIQ9mIlNuADBqxVE2HDtJGzq5iJ-9eBv_7Jlhc5rqpFTg&s=JlxUbpyZ5QY4KO6fooZ_v2dzXdWoJwMFmTQjl87JQ6M&e=> From: NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org<mailto:comcast.com@nanog.org>> on behalf of Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> Date: Thursday, December 5, 2024 at 12:24 PM To: Jason Bothe <jbothe@me.com<mailto:jbothe@me.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense.com_v3_-5F-5Fhttp-3A_www.ics-2Dil.com-5F-5F-3B-21-21CQl3mcHX2A-21GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-2DFJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ-24&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=05VYBsDnfxCJ7Vxr7z0MIQ9mIlNuADBqxVE2HDtJGzq5iJ-9eBv_7Jlhc5rqpFTg&s=HwztAxdPZtfgn_s2jg1HMBFPZjPmDJ9psIPNuNJK-i4&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense.com_v3_-5F-5Fhttp-3A_www.midwest-2Dix.com-5F-5F-3B-21-21CQl3mcHX2A-21GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-2DFJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw-24&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=05VYBsDnfxCJ7Vxr7z0MIQ9mIlNuADBqxVE2HDtJGzq5iJ-9eBv_7Jlhc5rqpFTg&s=-IO7Injcjd7v8SWcsxC6eLHhq73be7HAPidnwY5y_Xo&e=> ________________________________ From: "Jason Bothe via NANOG" <nanog@nanog.org<mailto:nanog@nanog.org>> To: "Drew Weaver" <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
Yes, yes, yes and yes J~
On Dec 5, 2024, at 12:10 PM, Compton, Rich <RICH_COMPTON@comcast.com> wrote:
“I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.”
-Job Snijders
https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html
From: NANOG <nanog-bounces+rich_compton=comcast.com@nanog.org> on behalf of Mike Hammett <nanog@ics-il.net> Date: Thursday, December 5, 2024 at 12:24 PM To: Jason Bothe <jbothe@me.com> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: Route optimization using GPUs?
IIRC, the widespread outages are the result of exporting things that shouldn't be exported.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com <https://urldefense.com/v3/__http:/www.ics-il.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ$>
Midwest-IX http://www.midwest-ix.com <https://urldefense.com/v3/__http:/www.midwest-ix.com__;!!CQl3mcHX2A!GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-FJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw$>
From: "Jason Bothe via NANOG" <nanog@nanog.org> To: "Drew Weaver" <drew.weaver@thenap.com> Cc: nanog@nanog.org Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs?
WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages.
J~
On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com> wrote:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
Yes, it really doesn’t have anything to do with one ASN’s particular outbound routing policy. It only matters if that routing policy gets outside of that one ASN. From: Mike Hammett <nanog@ics-il.net> Sent: Thursday, December 5, 2024 2:19 PM To: Jason Bothe <jbothe@me.com> Cc: nanog@nanog.org; Drew Weaver <drew.weaver@thenap.com> Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ________________________________ From: "Jason Bothe via NANOG" <nanog@nanog.org<mailto:nanog@nanog.org>> To: "Drew Weaver" <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
+nanog Greetings, Drew We don't use GPUs, but we have worked on a similar project focused on traffic optimization. I would say that the ping issue is one of the top pain points, even more significant than route recalculation. The more routes you try to monitor and optimize, the more significant the problem will become. Regarding optimization itself, I don't think you would want to optimize the entire routing table at the same time. From a calculation perspective, I don't believe it's a major problem at this point. Please correct me if I'm wrong. Regards, Andrey чт, 5 дек. 2024 г. в 18:32, Drew Weaver <drew.weaver@thenap.com>:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
-- *Andrey Slastenov* Network Expert CCIE #19983 Email:a.slastenov@gmail.com
I'd imagine the GPUs would be used to help the control plane only, to bulk-process several prefix updates? *Pedro Martins Prado* pedro.prado@gmail.com / +353 83 036 1875 (FaceTime & WhatsApp) On Thu, 5 Dec 2024 at 17:58, <sronan@ronan-online.com> wrote:
This is part of why the old systems that were mentioned only optimized the top n routes based on traffic flow (or other configurable metric), because the cost vs. benefit falls off greatly.
Shane
On Dec 5, 2024, at 12:52 PM, Andrey Slastenov <a.slastenov@gmail.com> wrote:
+nanog
Greetings, Drew
We don't use GPUs, but we have worked on a similar project focused on traffic optimization. I would say that the ping issue is one of the top pain points, even more significant than route recalculation. The more routes you try to monitor and optimize, the more significant the problem will become.
Regarding optimization itself, I don't think you would want to optimize the entire routing table at the same time. From a calculation perspective, I don't believe it's a major problem at this point. Please correct me if I'm wrong.
Regards,
Andrey
чт, 5 дек. 2024 г. в 18:32, Drew Weaver <drew.weaver@thenap.com>:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
--
*Andrey Slastenov* Network Expert CCIE #19983 Email:a.slastenov@gmail.com
Actually, some kind of content addressable memory looks more promising for increasing update throughput than GPUs. Rubens On Thu, Dec 5, 2024 at 1:13 PM Drew Weaver <drew.weaver@thenap.com> wrote:
So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing.
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table.
We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have.
Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer.
-Drew
On Thu, Dec 5, 2024 at 8:13 AM Drew Weaver <drew.weaver@thenap.com> wrote:
The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process.
Hi Drew, It was and remains a data problem, not a compute problem. You have to have places to probe who don't object to receiving a usefully high quantity of probes, and the capacity to issue and receive those probes. Processing the results is, by comparison, not a whole lot more consumptive than the normal best-path selection algorithm. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
participants (15)
-
Andrey Slastenov -
Compton, Rich -
Drew Weaver -
Jason Bothe -
Job Snijders -
Mike Hammett -
Nick Hilliard -
Niels Bakker -
Pedro Prado -
Rubens Kuhl -
Ryan Hamel -
sronan＠ronan-online.com -
Tom Beecher -
Warren Kumari -
William Herrin