Can an IXP sell IP transit?
Can an IXP sell traffic? This is a rhetorical question. I know that it can... In fact, it is obvious that it can. It is quite common to see several companies buying and selling traffic through IXPs. But whenever I have been involved with more serious companies, it was common for this type of traffic to be over a Bilateral VLAN between the Downstream and Upstream, and the ASs involved were from the operations themselves (different from the ASN used by Route-Servers). But I have seen a reasonably large scenario in which the IXP operator, maintaining the MLPA LAN with the pair of Route-Servers, adds another participant with the SAME ASN as the route-servers, and through this participant starts to sell traffic. This seemed very strange to me! And that is why I came to ask if this is correct or not. I would appreciate any guidance on the subject. In fact, there were other aggravating factors that worried me: - The IXP activation information itself (VLAN, IPv4/IPv6, Route-Servers, etc.) was indistinguishable from the information in the transit BGP session. And the extra Billing information for anything sent by the transit was not explicit. - The routes reported exchanged by this transit had the ASN transparency function in the AS-Path. Thanks in advance! -- Douglas Fernando Fischer Engº de Controle e Automação
Fundamentally, wouldnt that require the said IXP to be able to send full internet feed (v4 + v6) beyond the peering LAN routes? In some jurisdictions, the regulators require Transit Providers to have some sort of ISP license to sell such capacity. Noah On Mon, 4 Nov 2024, 19:46 Douglas Fischer, <fischerdouglas@gmail.com> wrote:
Can an IXP sell traffic? This is a rhetorical question. I know that it can... In fact, it is obvious that it can.
It is quite common to see several companies buying and selling traffic through IXPs. But whenever I have been involved with more serious companies, it was common for this type of traffic to be over a Bilateral VLAN between the Downstream and Upstream, and the ASs involved were from the operations themselves (different from the ASN used by Route-Servers).
But I have seen a reasonably large scenario in which the IXP operator, maintaining the MLPA LAN with the pair of Route-Servers, adds another participant with the SAME ASN as the route-servers, and through this participant starts to sell traffic.
This seemed very strange to me! And that is why I came to ask if this is correct or not. I would appreciate any guidance on the subject.
In fact, there were other aggravating factors that worried me: - The IXP activation information itself (VLAN, IPv4/IPv6, Route-Servers, etc.) was indistinguishable from the information in the transit BGP session. And the extra Billing information for anything sent by the transit was not explicit. - The routes reported exchanged by this transit had the ASN transparency function in the AS-Path.
Thanks in advance! -- Douglas Fernando Fischer Engº de Controle e Automação
Hello, On 4/11/24 2:14 PM, Noah wrote:
Fundamentally, wouldnt that require the said IXP to be able to send full internet feed (v4 + v6) beyond the peering LAN routes?
I don't think so, I'm sure there are so many upstream providers selling IP transit without having nor sending the full internet feed.
In some jurisdictions, the regulators require Transit Providers to have some sort of ISP license to sell such capacity.
Totally right..
Noah
On Mon, 4 Nov 2024, 19:46 Douglas Fischer, <fischerdouglas@gmail.com> wrote:
Can an IXP sell traffic? This is a rhetorical question. I know that it can... In fact, it is obvious that it can.
It is quite common to see several companies buying and selling traffic through IXPs. But whenever I have been involved with more serious companies, it was common for this type of traffic to be over a Bilateral VLAN between the Downstream and Upstream, and the ASs involved were from the operations themselves (different from the ASN used by Route-Servers).
But I have seen a reasonably large scenario in which the IXP operator, maintaining the MLPA LAN with the pair of Route-Servers, adds another participant with the SAME ASN as the route-servers, and through this participant starts to sell traffic.
This seemed very strange to me! And that is why I came to ask if this is correct or not. I would appreciate any guidance on the subject.
In fact, there were other aggravating factors that worried me: - The IXP activation information itself (VLAN, IPv4/IPv6, Route-Servers, etc.) was indistinguishable from the information in the transit BGP session. And the extra Billing information for anything sent by the transit was not explicit. - The routes reported exchanged by this transit had the ASN transparency function in the AS-Path.
Thanks in advance! -- Douglas Fernando Fischer Engº de Controle e Automação
On Mon, Nov 4, 2024 at 8:44 AM Douglas Fischer <fischerdouglas@gmail.com> wrote:
But I have seen a reasonably large scenario in which the IXP operator, maintaining the MLPA LAN with the pair of Route-Servers, adds another participant with the SAME ASN as the route-servers, and through this participant starts to sell traffic.
Of course they can sell transit. The reason they don't is that it has the potential to create a conflict of interest. When your customer is also a competitor and your customer suffers an outage that's your fault... Well, you see where this is going. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
There is one well-known IXP that sells transit as well. You'd be silly to buy both from the same provider. CH Get Outlook for Android<https://aka.ms/AAb9ysg> ________________________________ From: NANOG <nanog-bounces+chris=thesysadmin.au@nanog.org> on behalf of William Herrin <bill@herrin.us> Sent: Tuesday, November 5, 2024 11:57:13 am To: Douglas Fischer <fischerdouglas@gmail.com> Cc: NANOG <nanog@nanog.org> Subject: Re: Can an IXP sell IP transit? On Mon, Nov 4, 2024 at 8:44 AM Douglas Fischer <fischerdouglas@gmail.com> wrote:
But I have seen a reasonably large scenario in which the IXP operator, maintaining the MLPA LAN with the pair of Route-Servers, adds another participant with the SAME ASN as the route-servers, and through this participant starts to sell traffic.
Of course they can sell transit. The reason they don't is that it has the potential to create a conflict of interest. When your customer is also a competitor and your customer suffers an outage that's your fault... Well, you see where this is going. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
On 11/5/24 02:56, William Herrin wrote:
Of course they can sell transit. The reason they don't is that it has the potential to create a conflict of interest. When your customer is also a competitor and your customer suffers an outage that's your fault... Well, you see where this is going.
Over the coming years, I expect exchange points to do some strangely interesting things, as the towel of revenue continues to get tighter and tighter to squeeze. Especially so if a few of the large content providers continue to pull back from route servers and such. Mark.
Over the coming years, I expect exchange points to do some strangely interesting things, as the towel of revenue continues to get tighter and tighter to squeeze.
Especially so if a few of the large content providers continue to pull back from route servers and such.
Content providers aren't leaving IXP's completely. They're still there, still paying monthly for ports and XCs. Still doing bilateral peering over the IX. There's no revenue hit to an IXP for a CDN to de-peer off the route servers. On Mon, Nov 4, 2024 at 10:20 PM Mark Tinka <mark@tinka.africa> wrote:
On 11/5/24 02:56, William Herrin wrote:
Of course they can sell transit. The reason they don't is that it has the potential to create a conflict of interest. When your customer is also a competitor and your customer suffers an outage that's your fault... Well, you see where this is going.
Over the coming years, I expect exchange points to do some strangely interesting things, as the towel of revenue continues to get tighter and tighter to squeeze.
Especially so if a few of the large content providers continue to pull back from route servers and such.
Mark.
On 11/5/24 18:56, Tom Beecher wrote:
Content providers aren't leaving IXP's completely. They're still there, still paying monthly for ports and XCs. Still doing bilateral peering over the IX. There's no revenue hit to an IXP for a CDN to de-peer off the route servers.
Not sure where I said that content folk were leaving the exchange point... Mark.
On 5 Nov 2024, at 16:56, Tom Beecher wrote:
Especially so if a few of the large content providers continue to pull back from route servers and such. Content providers aren't leaving IXP's completely. They're still there, still paying monthly for ports and XCs. Still doing bilateral peering over the IX. There's no revenue hit to an IXP for a CDN to de-peer off the route servers.
Hi Tom, I don’t really think your last statement is true. UK, and London in particular, is quite a dynamic market. At LONAP we see plenty of networks connect and see an immediate “quick win” of traffic by connection to our route-servers, where adoption among the membership is something like 85-90%. If an operator decides to replace those RS sessions with a (often intractable) portal to request bilateral sessions - or worse, email - that immediate traffic benefit is lost. That can affect the value the IXP provides to its members. Will
On 11/7/24 19:22, Will Hargrave wrote:
Hi Tom,
I don’t really think your last statement is true.
UK, and London in particular, is quite a dynamic market. At LONAP we see plenty of networks connect and see an immediate “quick win” of traffic by connection to our route-servers, where adoption among the membership is something like 85-90%.
If an operator decides to replace those RS sessions with a (often intractable) portal to request bilateral sessions - or worse, email - that immediate traffic benefit is lost. That can affect the value the IXP provides to its members.
Correct. The paradigm shift happening in the content/cloud space is that the "quick win value" eyeball networks enjoy from peering with the route server to reach them does not appear to continue to be worth the admin. effort for said content/cloud folk. Not sure how or if adding a portal layer makes that easier for them to manage. But of course, there is the real issue that continuously upgrading clusters that serve exchange points across most of the exchange points around the world is a task that might have been underestimated, after all. Mark.
* mark@tinka.africa (Mark Tinka) [Thu 07 Nov 2024, 19:25 CET]:
The paradigm shift happening in the content/cloud space is that the "quick win value" eyeball networks enjoy from peering with the route server to reach them does not appear to continue to be worth the admin. effort for said content/cloud folk.
You almost can't have lower effort peering than via an IXP's route server. Most of them these days are well run with IRR-based filtering and RPKI OV. -- Niels.
You almost can't have lower effort peering than via an IXP's route server. Most of them these days are well run with IRR-based filtering and RPKI OV.
i used to resist. my instinct is that the data plane and the control plane should be congruent or you can have hard to debug issues[0]. but, as i have gotten older and lazier, and as you say, route servers have gotten quite reliable, i have come over to the route server side. randy [0] - https://datatracker.ietf.org/doc/draft-ietf-idr-rs-bfd/
On 11/7/24 21:42, Randy Bush wrote:
i used to resist. my instinct is that the data plane and the control plane should be congruent or you can have hard to debug issues[0]. but, as i have gotten older and lazier, and as you say, route servers have gotten quite reliable, i have come over to the route server side.
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service. Mark.
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Plenty of eyeball networks will announce prefixes differently via a bilateral session vs a route server session vs DFZ, then come yelling because traffic isn't going the way they expected it to. There can be times that the administrative overhead of dealing with those folks far outweighs any financial or performance benefits. Route servers are generally useful, but can be a royal pain in the ass too, depending on how they're used. On Thu, Nov 7, 2024 at 3:35 PM Mark Tinka <mark@tinka.africa> wrote:
On 11/7/24 21:42, Randy Bush wrote:
i used to resist. my instinct is that the data plane and the control plane should be congruent or you can have hard to debug issues[0]. but, as i have gotten older and lazier, and as you say, route servers have gotten quite reliable, i have come over to the route server side.
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Mark.
Assuming the code is correct and that you will be getting the email,l you should get any email any time. stbernadettewv.org Domain awaiting transfer initiation Mike On Thu, Nov 7, 2024 at 4:46 PM Tom Beecher <beecher@beecher.cc> wrote:
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Plenty of eyeball networks will announce prefixes differently via a bilateral session vs a route server session vs DFZ, then come yelling because traffic isn't going the way they expected it to. There can be times that the administrative overhead of dealing with those folks far outweighs any financial or performance benefits.
Route servers are generally useful, but can be a royal pain in the ass too, depending on how they're used.
On Thu, Nov 7, 2024 at 3:35 PM Mark Tinka <mark@tinka.africa> wrote:
On 11/7/24 21:42, Randy Bush wrote:
i used to resist. my instinct is that the data plane and the control plane should be congruent or you can have hard to debug issues[0]. but, as i have gotten older and lazier, and as you say, route servers have gotten quite reliable, i have come over to the route server side.
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Mark.
The ... AUTH code did NOT work stbernadettewv.org Canceled - Invalid EPP/authorization key - Please contact current registrar to obtain correct key On Thu, Nov 7, 2024 at 4:51 PM Mike Tindor <mtindor@gmail.com> wrote:
Assuming the code is correct and that you will be getting the email,l you should get any email any time.
stbernadettewv.org Domain awaiting transfer initiation Mike
On Thu, Nov 7, 2024 at 4:46 PM Tom Beecher <beecher@beecher.cc> wrote:
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Plenty of eyeball networks will announce prefixes differently via a bilateral session vs a route server session vs DFZ, then come yelling because traffic isn't going the way they expected it to. There can be times that the administrative overhead of dealing with those folks far outweighs any financial or performance benefits.
Route servers are generally useful, but can be a royal pain in the ass too, depending on how they're used.
On Thu, Nov 7, 2024 at 3:35 PM Mark Tinka <mark@tinka.africa> wrote:
On 11/7/24 21:42, Randy Bush wrote:
i used to resist. my instinct is that the data plane and the control plane should be congruent or you can have hard to debug issues[0]. but, as i have gotten older and lazier, and as you say, route servers have gotten quite reliable, i have come over to the route server side.
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Mark.
Mike, Please verify you are emailing the correct person. I have no idea how this thread relates to a domain name transfer but I highly recommend verifying the recipient of your emails to make sure they go to the right place. Regards, Peter On Thu, Nov 7, 2024 at 4:57 PM Mike Tindor <mtindor@gmail.com> wrote:
The ... AUTH code did NOT work
stbernadettewv.org Canceled - Invalid EPP/authorization key - Please contact current registrar to obtain correct key
On Thu, Nov 7, 2024 at 4:51 PM Mike Tindor <mtindor@gmail.com> wrote:
Assuming the code is correct and that you will be getting the email,l you should get any email any time.
stbernadettewv.org Domain awaiting transfer initiation Mike
On Thu, Nov 7, 2024 at 4:46 PM Tom Beecher <beecher@beecher.cc> wrote:
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Plenty of eyeball networks will announce prefixes differently via a bilateral session vs a route server session vs DFZ, then come yelling because traffic isn't going the way they expected it to. There can be times that the administrative overhead of dealing with those folks far outweighs any financial or performance benefits.
Route servers are generally useful, but can be a royal pain in the ass too, depending on how they're used.
On Thu, Nov 7, 2024 at 3:35 PM Mark Tinka <mark@tinka.africa> wrote:
On 11/7/24 21:42, Randy Bush wrote:
i used to resist. my instinct is that the data plane and the control plane should be congruent or you can have hard to debug issues[0]. but, as i have gotten older and lazier, and as you say, route servers have gotten quite reliable, i have come over to the route server side.
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Mark.
I'm aware that I sent something via email inadvertently to the NANOG list. Sorry about that. If I could remove it I would. Sorry about that Mike On Thu, Nov 7, 2024 at 4:58 PM Peter Potvin < peter.potvin@accuristechnologies.ca> wrote:
Mike,
Please verify you are emailing the correct person. I have no idea how this thread relates to a domain name transfer but I highly recommend verifying the recipient of your emails to make sure they go to the right place.
Regards, Peter
On Thu, Nov 7, 2024 at 4:57 PM Mike Tindor <mtindor@gmail.com> wrote:
The ... AUTH code did NOT work
stbernadettewv.org Canceled - Invalid EPP/authorization key - Please contact current registrar to obtain correct key
On Thu, Nov 7, 2024 at 4:51 PM Mike Tindor <mtindor@gmail.com> wrote:
Assuming the code is correct and that you will be getting the email,l you should get any email any time.
stbernadettewv.org Domain awaiting transfer initiation Mike
On Thu, Nov 7, 2024 at 4:46 PM Tom Beecher <beecher@beecher.cc> wrote:
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Plenty of eyeball networks will announce prefixes differently via a bilateral session vs a route server session vs DFZ, then come yelling because traffic isn't going the way they expected it to. There can be times that the administrative overhead of dealing with those folks far outweighs any financial or performance benefits.
Route servers are generally useful, but can be a royal pain in the ass too, depending on how they're used.
On Thu, Nov 7, 2024 at 3:35 PM Mark Tinka <mark@tinka.africa> wrote:
On 11/7/24 21:42, Randy Bush wrote:
i used to resist. my instinct is that the data plane and the control plane should be congruent or you can have hard to debug issues[0]. but, as i have gotten older and lazier, and as you say, route servers have gotten quite reliable, i have come over to the route server side.
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Mark.
Sorry all for emaling to Nanog inadvertently. I sent a copuole of nonrelevant posts On Thu, Nov 7, 2024 at 4:46 PM Tom Beecher <beecher@beecher.cc> wrote:
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Plenty of eyeball networks will announce prefixes differently via a bilateral session vs a route server session vs DFZ, then come yelling because traffic isn't going the way they expected it to. There can be times that the administrative overhead of dealing with those folks far outweighs any financial or performance benefits.
Route servers are generally useful, but can be a royal pain in the ass too, depending on how they're used.
On Thu, Nov 7, 2024 at 3:35 PM Mark Tinka <mark@tinka.africa> wrote:
On 11/7/24 21:42, Randy Bush wrote:
i used to resist. my instinct is that the data plane and the control plane should be congruent or you can have hard to debug issues[0]. but, as i have gotten older and lazier, and as you say, route servers have gotten quite reliable, i have come over to the route server side.
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
Mark.
On 11/7/24 23:43, Tom Beecher wrote:
Plenty of eyeball networks will announce prefixes differently via a bilateral session vs a route server session vs DFZ, then come yelling because traffic isn't going the way they expected it to. There can be times that the administrative overhead of dealing with those folks far outweighs any financial or performance benefits.
Route servers are generally useful, but can be a royal pain in the ass too, depending on how they're used.
Agreed. I meant outside of policy control (considering most route servers offer a number of knobs that their neighbors can use to influence remote-end forwarding), it would seem that getting off the route servers gives the content/cloud folk an opportunity to screen peering and peering request, and by extension, a trickle-through effect for cluster maintenance, rather than a gush. Mark.
Mark Tinka wrote on 07/11/2024 20:33:
I don't think there is any satisfactory argument that can be made for wanting to avoid route server routing. For the content/cloud folk, I think avoiding it provides a mechanism via which they can screen for the utility of having to keep an exchange point node upgraded and optimized for service.
gonna disagree on this one. Route servers are aimed at leaf networks, networks with simple routing policies, and arguably also networks with complex routing requirements, but where the RS provides reachability to ASNs which they're not picking up elsewhere. They can cause a good deal heartache for networks which have complex policy requirements and in situations where there's already dense interconnection between the ASNs visible on the RS. Several of the larger content networks don't want to use RS's, and they have good technical reasons for not wanting to do so. The flip side of this is that unless you're prepared to implement the sort of routing security configuration that route servers typically implement these days, or to trust that the other party has done this, then bilateral peering sessions will open you up to wide range of routing security problems. I'd be fairly hesitant to implement bilateral peering sessions as a general rule, except with networks that are large enough that they've made the effort to implement good quality filtering at their ixp presence. Nick
On 11/8/24 21:00, Nick Hilliard wrote:
gonna disagree on this one. Route servers are aimed at leaf networks, networks with simple routing policies, and arguably also networks with complex routing requirements, but where the RS provides reachability to ASNs which they're not picking up elsewhere. They can cause a good deal heartache for networks which have complex policy requirements and in situations where there's already dense interconnection between the ASNs visible on the RS. Several of the larger content networks don't want to use RS's, and they have good technical reasons for not wanting to do so.
The original intention for route servers is to simplify turn-up for the majority of networks, most of whom will be eyeball-focused. They did offer cloud & content folk utility as they quickly ramped up their CDN deployments globally over the past decade-and-a-bit. While their challenges are as well as documented as their benefits, moving away from them can "uncomplicate" the routing topology of large operations like those of cloud & content. The unintended side effect is that it can lower traffic load on IXP-specific caches (that traffic would move over to transit until the remote peer can re-establish peering via a bi-lateral session), which bodes well for cost and resource control.
The flip side of this is that unless you're prepared to implement the sort of routing security configuration that route servers typically implement these days, or to trust that the other party has done this, then bilateral peering sessions will open you up to wide range of routing security problems. I'd be fairly hesitant to implement bilateral peering sessions as a general rule, except with networks that are large enough that they've made the effort to implement good quality filtering at their ixp presence.
I'm not sure that bi-lateral sessions are necessarily any less secure than route server sessions, but mandating that peering happens over bi-lateral sessions means that the cloud & content folk will screen requests to peer for that very reason :-). Mark.
* mark@tinka.africa (Mark Tinka) [Sat 16 Nov 2024, 02:53 CET]:
The original intention for route servers is to simplify turn-up for the majority of networks, most of whom will be eyeball-focused. They did offer cloud & content folk utility as they quickly ramped up their CDN deployments globally over the past decade-and-a-bit.
One of the original intentions was that. The other, major, driver was control plane scalability. For a while IXPs grew way too large in regard to the number of available peers for certain routing engines from certain vendors to keep up in case there ever was a switch outage. A router server could radically decrease the number of EBGP peers and thus overhead. -- Niels.
Bi-Lateral benefit is for when there is an issue with the route servers. Example, Equinix IBX Chicago changed filtering policies shortly after ARIN introduced the new IRR database. If your BGP peered customers had not created new entries in the new IRR database their routes were dropped. This dumped a lot of IX traffic for us other than the few bi-laterals we had established. We called all our BGP peered customers that day and they quickly were able to update their entries in the new IRR database. This wasn't a catastrophic event, but you do need to know that things could go wrong with an IX's route servers. Thank you, Kevin McCormick -----Original Message----- From: NANOG <nanog-bounces+kmccormick=mdtc.net@nanog.org> On Behalf Of Mark Tinka Sent: Friday, November 15, 2024 7:52 PM To: Nick Hilliard <nick@foobar.org> Cc: North American Network Operators' Group <nanog@nanog.org> Subject: Re: Can an IXP sell IP transit? CAUTION: This email originated from outside your organization. Exercise caution when opening attachments or clicking links, especially from unknown senders. On 11/8/24 21:00, Nick Hilliard wrote:
gonna disagree on this one. Route servers are aimed at leaf networks, networks with simple routing policies, and arguably also networks with complex routing requirements, but where the RS provides reachability to ASNs which they're not picking up elsewhere. They can cause a good deal heartache for networks which have complex policy requirements and in situations where there's already dense interconnection between the ASNs visible on the RS. Several of the larger content networks don't want to use RS's, and they have good technical reasons for not wanting to do so.
The original intention for route servers is to simplify turn-up for the majority of networks, most of whom will be eyeball-focused. They did offer cloud & content folk utility as they quickly ramped up their CDN deployments globally over the past decade-and-a-bit. While their challenges are as well as documented as their benefits, moving away from them can "uncomplicate" the routing topology of large operations like those of cloud & content. The unintended side effect is that it can lower traffic load on IXP-specific caches (that traffic would move over to transit until the remote peer can re-establish peering via a bi-lateral session), which bodes well for cost and resource control.
The flip side of this is that unless you're prepared to implement the sort of routing security configuration that route servers typically implement these days, or to trust that the other party has done this, then bilateral peering sessions will open you up to wide range of routing security problems. I'd be fairly hesitant to implement bilateral peering sessions as a general rule, except with networks that are large enough that they've made the effort to implement good quality filtering at their ixp presence.
I'm not sure that bi-lateral sessions are necessarily any less secure than route server sessions, but mandating that peering happens over bi-lateral sessions means that the cloud & content folk will screen requests to peer for that very reason :-). Mark.
And so many of those bilateral processes are just simply broken. -----Mike HammettIntelligent Computing SolutionsMidwest Internet ExchangeThe Brothers WISP ----- Original Message ----- From: Will Hargrave <will@harg.net> To: Tom Beecher <beecher@beecher.cc> Cc: nanog@nanog.org Sent: Thu, 07 Nov 2024 11:22:34 -0600 (CST) Subject: Re: Can an IXP sell IP transit? On 5 Nov 2024, at 16:56, Tom Beecher wrote:
Especially so if a few of the large content providers continue to pull back from route servers and such. Content providers aren't leaving IXP's completely. They're still there, still paying monthly for ports and XCs. Still doing bilateral peering over the IX. There's no revenue hit to an IXP for a CDN to de-peer off the route servers.
Hi Tom, I don’t really think your last statement is true. UK, and London in particular, is quite a dynamic market. At LONAP we see plenty of networks connect and see an immediate “quick win” of traffic by connection to our route-servers, where adoption among the membership is something like 85-90%. If an operator decides to replace those RS sessions with a (often intractable) portal to request bilateral sessions - or worse, email - that immediate traffic benefit is lost. That can affect the value the IXP provides to its members. Will
Agreed, Microsoft process is painful with forced to use the azure interface. Meta and cloudflare have nice portals that use peeringdb for auth. On Sun, Nov 17, 2024, 12:03 PM Mike Hammett <nanog@ics-il.net> wrote:
And so many of those bilateral processes are just simply broken.
-----Mike HammettIntelligent Computing SolutionsMidwest Internet ExchangeThe Brothers WISP
----- Original Message ----- From: Will Hargrave <will@harg.net> To: Tom Beecher <beecher@beecher.cc> Cc: nanog@nanog.org Sent: Thu, 07 Nov 2024 11:22:34 -0600 (CST) Subject: Re: Can an IXP sell IP transit?
On 5 Nov 2024, at 16:56, Tom Beecher wrote:
Especially so if a few of the large content providers continue to pull back from route servers and such. Content providers aren't leaving IXP's completely. They're still there, still paying monthly for ports and XCs. Still doing bilateral peering over the IX. There's no revenue hit to an IXP for a CDN to de-peer off the route servers.
Hi Tom,
I don’t really think your last statement is true.
UK, and London in particular, is quite a dynamic market. At LONAP we see plenty of networks connect and see an immediate “quick win” of traffic by connection to our route-servers, where adoption among the membership is something like 85-90%.
If an operator decides to replace those RS sessions with a (often intractable) portal to request bilateral sessions - or worse, email - that immediate traffic benefit is lost. That can affect the value the IXP provides to its members.
Will
On Nov 5, 2024, at 00:56, William Herrin <bill@herrin.us> wrote: On Mon, Nov 4, 2024 at 8:44 AM Douglas Fischer <fischerdouglas@gmail.com> wrote:
But I have seen a reasonably large scenario in which the IXP operator, maintaining the MLPA LAN with the pair of Route-Servers, adds another participant with the SAME ASN as the route-servers, and through this participant starts to sell traffic.
Of course they can sell transit. The reason they don't is that it has the potential to create a conflict of interest. When your customer is also a competitor and your customer suffers an outage that's your fault... Well, you see where this is going.
Yep. Other-Bill has this exactly right. To add a little to that, for an IXP to be successful, many ISPs have to trust it enough to build out to it and participate in it, which requires investment on their parts. To be trustworthy, it helps a _lot_ if the IXP is “neutral.” That is, it’s not a part of your competitor, and you have reason to believe that if you invest in it now, it won’t treat you unfairly in the future. So, to succeed, it helps a _lot_ if IXP have very simple, unitary models. They don’t do other things which makes them complicated, opaque, hard-to-understand, hard-to-trust. Just not handling money at all is a huge win for IXP stability and growth, for instance, because it simplifies things and makes them more easily trusted. Selling transit is a thing that ISPs do. When an IXP competes with its ISP participants, it not only sets itself up for the conflicts and failures that Bill points out, it also makes itself less trustworthy, because of the _future risk_ of those conflicts and failures, and that decreased trustworthiness decreases “investor confidence” and ISP willingness to extend themselves, and that, in turn, causes the IXP to not grow, or grow more slowly than it would otherwise. -Bill
participants (17)
-
Alejandro Acosta -
Bill Woodcock -
Christopher Hawker -
Douglas Fischer -
Kevin McCormick -
Mark Tinka -
Mike Hammett -
Mike Tindor -
Nick Hilliard -
Niels Bakker -
Noah -
Peter Potvin -
Randy Bush -
Tom Beecher -
Will Hargrave -
William Herrin -
Zach Underwood