Question on peering strategies
Dear Nanogers, I have a question about common/best network interconnection practices. Assume that two networks (let's refer to them as AS-a and AS-b) are present in a colocation facility say Equinix LA. As many of you know, Equininx runs an IXP in LA as well. So AS-as and AS-b can interconnct 1) using private cross-connect 2) through the public IXP's switching fabric. Is it a common/good practice for the two networks to establish connections both through the IXP and also using a private cross-connect? I was thinking considering the cost of cross-connects (my understanding is that the colocation provider charges the customers for each cross-connect in addition to the rent of the rack or cage or whatever), it would not be economically reasonable to have both. Although, if the cross-connect is the primary method of interconnection, and the IXP provides a router-server the public-peering over IXP would essentially be free. So it might makes sense to assume that for the private cross-connect, there exists a back-up connection though the IXP. Anyway, I guess some discussion may give more insight about which one is more reasonable to assume and do. Now my last question is that if the two connections exist (one private cross-connect and another back-up through the IXP), what are the chances that periodically launched traceroutes that pass the inter-AS connection in that colo see both types of connection in a week. I guess what I'm asking is how often back-up routes are taken? Can the networks do load balancing on the two connection and essentially use them as primary routes? Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon
Reza, You maybe overthinking this one a bit. The economics are something to consider, however all public exchanges have different economics. With Equinix you pay pretty much a flat rate for a single 1Gbps/10Gbps link that includes the cost of facility cross-connect and public exchange access. It is a nice one to many connection for all those various network and content networks your end users would appreciate direct connectivity. Depending on the public exchange you either have a single BGP session or a BGP session per network you are peering. Really after that, it's just BGP routing and route management. You do need to be careful about not being too overly dependent on a single public switch link, in some cases like at Equinix you may want multiple connections to redundant public exchange switches at that site. There is a balance you want to seek of number of paid upstream network transit providers you are connected to versus how many direct peering arrangements you have setup. It's not usually practical for a smaller network to have loads of BGP peers. There are lots of good articles online about this fine balance and some good advice from experienced network operators. To your later questions. For your simple example, if AS-a and AS-b were both already on the public IX, and the link wasn't too overly critical then using the public IX switch maybe a good first step. However as that relationship matures, they most likely in a real world example may look to split the cost of the private cross-connect. If it was mutually beneficial. There is much more to public peering and transit than the technical conversation. Most of the larger networks on the public switches won't peer privately with anyone or only with extremely larger networks. To get a provider such as this to peer both privately and on the public exchange is not a technical issue, it's more of a business overhead and management issue. If you have a couple of quality upstream transit providers, they will be excellent failovers to a public switch outage. Plan for the public switch to have as many problems as any upstream provider. Sincerely, Nick Ellermann – CTO & VP Cloud Services BroadAspect E: nellermann@broadaspect.com P: 703-297-4639 F: 703-996-4443 THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. -----Original Message----- From: NANOG [mailto:nanog-bounces+nellermann=broadaspect.com@nanog.org] On Behalf Of Reza Motamedi Sent: Monday, May 16, 2016 1:46 PM To: nanog@nanog.org Subject: Question on peering strategies Dear Nanogers, I have a question about common/best network interconnection practices. Assume that two networks (let's refer to them as AS-a and AS-b) are present in a colocation facility say Equinix LA. As many of you know, Equininx runs an IXP in LA as well. So AS-as and AS-b can interconnct 1) using private cross-connect 2) through the public IXP's switching fabric. Is it a common/good practice for the two networks to establish connections both through the IXP and also using a private cross-connect? I was thinking considering the cost of cross-connects (my understanding is that the colocation provider charges the customers for each cross-connect in addition to the rent of the rack or cage or whatever), it would not be economically reasonable to have both. Although, if the cross-connect is the primary method of interconnection, and the IXP provides a router-server the public-peering over IXP would essentially be free. So it might makes sense to assume that for the private cross-connect, there exists a back-up connection though the IXP. Anyway, I guess some discussion may give more insight about which one is more reasonable to assume and do. Now my last question is that if the two connections exist (one private cross-connect and another back-up through the IXP), what are the chances that periodically launched traceroutes that pass the inter-AS connection in that colo see both types of connection in a week. I guess what I'm asking is how often back-up routes are taken? Can the networks do load balancing on the two connection and essentially use them as primary routes? Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon
Hi Nick, Thanks for the reply. Let me clarify another issue first, since I thought the colo's business model is different at least in the US. So if AS-a puts its router in Equinix, it should pay the same amount in the following two scenario (only considering the interconnection cost and not the rent for racks and remote hands and ....)? 1) AS-a only connects to the IX and establishes all inter-AS connections through the IX. 2) AS-a connects to the IX, in addition to privately connecting to bunch of other colo customers (these private connections can be either transit or settlement-free peerings). My understanding was that colos in the US charge per cross connect, so the more you connect privately, the more you pay. This article may be old, but I don't think much has changed: https://www.telegeography.com/press/press-releases/2015/02/26/colocation-cro... With respect to my second question, I was asking if is practical/reasonable to keep both the connection types to same network (say AS-b) at the same time, i.e., connect privately over a cross-connect and keep the public connection over the IX. Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon On Mon, May 16, 2016 at 11:10 AM, Nick Ellermann <nellermann@broadaspect.com
wrote:
Reza, You maybe overthinking this one a bit. The economics are something to consider, however all public exchanges have different economics. With Equinix you pay pretty much a flat rate for a single 1Gbps/10Gbps link that includes the cost of facility cross-connect and public exchange access. It is a nice one to many connection for all those various network and content networks your end users would appreciate direct connectivity. Depending on the public exchange you either have a single BGP session or a BGP session per network you are peering. Really after that, it's just BGP routing and route management. You do need to be careful about not being too overly dependent on a single public switch link, in some cases like at Equinix you may want multiple connections to redundant public exchange switches at that site. There is a balance you want to seek of number of paid upstream network transit providers you are connected to versus how many direct peering arrangements you have setup. It's not usually practical for a smaller network to have loads of BGP peers. There are lots of good articles online about this fine balance and some good advice from experienced network operators.
To your later questions. For your simple example, if AS-a and AS-b were both already on the public IX, and the link wasn't too overly critical then using the public IX switch maybe a good first step. However as that relationship matures, they most likely in a real world example may look to split the cost of the private cross-connect. If it was mutually beneficial. There is much more to public peering and transit than the technical conversation. Most of the larger networks on the public switches won't peer privately with anyone or only with extremely larger networks. To get a provider such as this to peer both privately and on the public exchange is not a technical issue, it's more of a business overhead and management issue. If you have a couple of quality upstream transit providers, they will be excellent failovers to a public switch outage. Plan for the public switch to have as many problems as any upstream provider.
Sincerely, Nick Ellermann – CTO & VP Cloud Services BroadAspect
E: nellermann@broadaspect.com P: 703-297-4639 F: 703-996-4443
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
-----Original Message----- From: NANOG [mailto:nanog-bounces+nellermann=broadaspect.com@nanog.org] On Behalf Of Reza Motamedi Sent: Monday, May 16, 2016 1:46 PM To: nanog@nanog.org Subject: Question on peering strategies
Dear Nanogers,
I have a question about common/best network interconnection practices. Assume that two networks (let's refer to them as AS-a and AS-b) are present in a colocation facility say Equinix LA. As many of you know, Equininx runs an IXP in LA as well. So AS-as and AS-b can interconnct 1) using private cross-connect 2) through the public IXP's switching fabric. Is it a common/good practice for the two networks to establish connections both through the IXP and also using a private cross-connect?
I was thinking considering the cost of cross-connects (my understanding is that the colocation provider charges the customers for each cross-connect in addition to the rent of the rack or cage or whatever), it would not be economically reasonable to have both. Although, if the cross-connect is the primary method of interconnection, and the IXP provides a router-server the public-peering over IXP would essentially be free. So it might makes sense to assume that for the private cross-connect, there exists a back-up connection though the IXP. Anyway, I guess some discussion may give more insight about which one is more reasonable to assume and do.
Now my last question is that if the two connections exist (one private cross-connect and another back-up through the IXP), what are the chances that periodically launched traceroutes that pass the inter-AS connection in that colo see both types of connection in a week. I guess what I'm asking is how often back-up routes are taken? Can the networks do load balancing on the two connection and essentially use them as primary routes?
Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon
On 16 May 2016 at 22:06, Reza Motamedi <motamedi@cs.uoregon.edu> wrote:
With respect to my second question, I was asking if is practical/reasonable to keep both the connection types to same network (say AS-b) at the same time, i.e., connect privately over a cross-connect and keep the public connection over the IX.
Router ports are expensive, so even if cross connects were free, you would still use the public switch fabric until you reach a traffic level that justifies a direct connection. The point of having a IX switch is that you can connect to many others with just one single router port. When you have the direct cross connect, you would not usually use the IX switch in parallel for that AS. With the cross connect you have dedicated bandwidth to the AS and you would want to reserve the IX switch port for traffic to the remaining networks that you do not yet have a cross connect to. The cross connect is not a very good redundancy setup with regard to the IX switch. Both usually go to the same router and share the same single point of failure (your router is a single point of failure and the peer router is a single point of failure). A cross connect is usual very reliable. You would plan for your router to be down or the peer router to be down, and have a backup path through some entirely geographic separate location. In many cases your generic IP transit service is good enough backup. Your direct peering is an optimization and if that is down, you go back to the transit service. Of course everyone are playing their own game and you might see anything happening in the real world despite the above. Regards, Baldur
On May 16, 2016, at 4:29 PM, Baldur Norddahl <baldur.norddahl@gmail.com> wrote:
Router ports are expensive, so even if cross connects were free, you would still use the public switch fabric until you reach a traffic level that justifies a direct connection. The point of having a IX switch is that you can connect to many others with just one single router port.
The cost of an IX can be quite expensive actually. If you look at the RIPE presentations from this week, there are stealth routing hijacks that come from promiscuous peering as well as just the flat economics of connecting with a 10GE or 100GE interface and the cost per gigabit you assign to the IX port. These are flat rate ports, unlike transit that may offer you a price and commit rates that allow you to reach everyone vs those just at the IX. I’m hoping I don’t get in trouble for sharing this, but this collaboration exists for europe on peering costs which are normalized in euro cents per megabit. https://docs.google.com/spreadsheets/d/18ztPX_ysWYqEhJlf2SKQQsTNRbkwoxPSfaC6... - Jared
If you dig into hijacking topic more, you will see that hijacks through Tier1 is same or even more popular than through IXes. And if someone want to make me a transit offer for the price of DE-CIX (I do not even ask the price of DTEL-IX peering ;) ) - please, contact me off-list, I will be really happy. On 24.05.16 11:03, Jared Mauch wrote:
On May 16, 2016, at 4:29 PM, Baldur Norddahl <baldur.norddahl@gmail.com> wrote:
Router ports are expensive, so even if cross connects were free, you would still use the public switch fabric until you reach a traffic level that justifies a direct connection. The point of having a IX switch is that you can connect to many others with just one single router port.
The cost of an IX can be quite expensive actually. If you look at the RIPE presentations from this week, there are stealth routing hijacks that come from promiscuous peering as well as just the flat economics of connecting with a 10GE or 100GE interface and the cost per gigabit you assign to the IX port. These are flat rate ports, unlike transit that may offer you a price and commit rates that allow you to reach everyone vs those just at the IX.
I’m hoping I don’t get in trouble for sharing this, but this collaboration exists for europe on peering costs which are normalized in euro cents per megabit.
https://docs.google.com/spreadsheets/d/18ztPX_ysWYqEhJlf2SKQQsTNRbkwoxPSfaC6...
- Jared
On May 24, 2016, at 6:11 AM, Max Tulyev <maxtul@netassist.ua> wrote:
If you dig into hijacking topic more, you will see that hijacks through Tier1 is same or even more popular than through IXes.
You may not have a view into that you’re being hijacked and used to send SPAM for example: https://ripe72.ripe.net/presentations/45-Invisible_Hijacking.pdf Their space was hijacked and announced facing Yahoo. I’m hoping that Yahoo is now feeding public route views services as a method to help with detection. Same goes for Microsoft and Google and other e-mail providers. Some sunlight here would help avoid similar localized hijacks.
And if someone want to make me a transit offer for the price of DE-CIX (I do not even ask the price of DTEL-IX peering ;) ) - please, contact me off-list, I will be really happy.
Pricing obviously varies based on location and a few other criteria, but you should be shopping if this is a major part of your business. - Jared
I'm right here at RIPE 72 now, so I saw it of course ;) The problem is not peering itself, but more general problem of filtering nets, and it was told in the presentation. On 24.05.16 13:19, Jared Mauch wrote:
On May 24, 2016, at 6:11 AM, Max Tulyev <maxtul@netassist.ua> wrote:
If you dig into hijacking topic more, you will see that hijacks through Tier1 is same or even more popular than through IXes.
You may not have a view into that you’re being hijacked and used to send SPAM for example:
https://ripe72.ripe.net/presentations/45-Invisible_Hijacking.pdf
Their space was hijacked and announced facing Yahoo. I’m hoping that Yahoo is now feeding public route views services as a method to help with detection. Same goes for Microsoft and Google and other e-mail providers. Some sunlight here would help avoid similar localized hijacks.
And if someone want to make me a transit offer for the price of DE-CIX (I do not even ask the price of DTEL-IX peering ;) ) - please, contact me off-list, I will be really happy.
Pricing obviously varies based on location and a few other criteria, but you should be shopping if this is a major part of your business.
- Jared
I disagree somewhat, without a view of how you are being hijacked there often can be no remediation. Yahoo for example provides no cloud services so you can't purchase a view of their routing by getting a VM. Jared Mauch
On May 24, 2016, at 12:29 PM, Max Tulyev <maxtul@netassist.ua> wrote:
I'm right here at RIPE 72 now, so I saw it of course ;)
The problem is not peering itself, but more general problem of filtering nets, and it was told in the presentation.
On 24.05.16 13:19, Jared Mauch wrote:
On May 24, 2016, at 6:11 AM, Max Tulyev <maxtul@netassist.ua> wrote:
If you dig into hijacking topic more, you will see that hijacks through Tier1 is same or even more popular than through IXes.
You may not have a view into that you’re being hijacked and used to send SPAM for example:
https://ripe72.ripe.net/presentations/45-Invisible_Hijacking.pdf
Their space was hijacked and announced facing Yahoo. I’m hoping that Yahoo is now feeding public route views services as a method to help with detection. Same goes for Microsoft and Google and other e-mail providers. Some sunlight here would help avoid similar localized hijacks.
And if someone want to make me a transit offer for the price of DE-CIX (I do not even ask the price of DTEL-IX peering ;) ) - please, contact me off-list, I will be really happy.
Pricing obviously varies based on location and a few other criteria, but you should be shopping if this is a major part of your business.
- Jared
On Mon, 16 May 2016, Reza Motamedi wrote:
Hi Nick,
Thanks for the reply.
Let me clarify another issue first, since I thought the colo's business model is different at least in the US. So if AS-a puts its router in Equinix, it should pay the same amount in the following two scenario (only considering the interconnection cost and not the rent for racks and remote hands and ....)? 1) AS-a only connects to the IX and establishes all inter-AS connections through the IX. 2) AS-a connects to the IX, in addition to privately connecting to bunch of other colo customers (these private connections can be either transit or settlement-free peerings). My understanding was that colos in the US charge per cross connect, so the more you connect privately, the more you pay. This article may be old, but
Ports on the colo's IX, Equinix for example, will likely cost more than just a cross connect. If you have peers with which you exchange enough traffic, it can make sense to remove that traffic from the IX and put it on PNI (cross connect) peering, leaving the IX port(s) for use primarily for peering with lots of "smaller peers" (in the amount of traffic exchanged). Typically, if a peer is big enough to justify PNI, you won't want to fail-over to the IX as a backup, because doing so is likely to congest your or their IX links. Of course, there are exceptions. A PNI peer might not have enough ports to dedicate to PNI peering and might want to spread peering traffic over both PNI and IX evenly. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Hi All, I wonder why a "VLAN exchange" does not exists. Or I do not know any? In my understanding it should be a switch, and people connected can easily order a private VLAN between each other (or to private group) through some kind of web interface. That should be a more easy and much less expensive way for private interconnects than direct wires. On 16.05.16 20:46, Reza Motamedi wrote:
Dear Nanogers,
I have a question about common/best network interconnection practices. Assume that two networks (let's refer to them as AS-a and AS-b) are present in a colocation facility say Equinix LA. As many of you know, Equininx runs an IXP in LA as well. So AS-as and AS-b can interconnct 1) using private cross-connect 2) through the public IXP's switching fabric. Is it a common/good practice for the two networks to establish connections both through the IXP and also using a private cross-connect?
I was thinking considering the cost of cross-connects (my understanding is that the colocation provider charges the customers for each cross-connect in addition to the rent of the rack or cage or whatever), it would not be economically reasonable to have both. Although, if the cross-connect is the primary method of interconnection, and the IXP provides a router-server the public-peering over IXP would essentially be free. So it might makes sense to assume that for the private cross-connect, there exists a back-up connection though the IXP. Anyway, I guess some discussion may give more insight about which one is more reasonable to assume and do.
Now my last question is that if the two connections exist (one private cross-connect and another back-up through the IXP), what are the chances that periodically launched traceroutes that pass the inter-AS connection in that colo see both types of connection in a week. I guess what I'm asking is how often back-up routes are taken? Can the networks do load balancing on the two connection and essentially use them as primary routes?
Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon
This does exist, often called an elastic fabric, e.g. Megaport Regards, Marty Strong -------------------------------------- CloudFlare - AS13335 Network Engineer marty@cloudflare.com +44 7584 906 055 smartflare (Skype) http://www.peeringdb.com/view.php?asn=13335
On 22 May 2016, at 07:33, Max Tulyev <maxtul@netassist.ua> wrote:
Hi All,
I wonder why a "VLAN exchange" does not exists. Or I do not know any?
In my understanding it should be a switch, and people connected can easily order a private VLAN between each other (or to private group) through some kind of web interface.
That should be a more easy and much less expensive way for private interconnects than direct wires.
On 16.05.16 20:46, Reza Motamedi wrote:
Dear Nanogers,
I have a question about common/best network interconnection practices. Assume that two networks (let's refer to them as AS-a and AS-b) are present in a colocation facility say Equinix LA. As many of you know, Equininx runs an IXP in LA as well. So AS-as and AS-b can interconnct 1) using private cross-connect 2) through the public IXP's switching fabric. Is it a common/good practice for the two networks to establish connections both through the IXP and also using a private cross-connect?
I was thinking considering the cost of cross-connects (my understanding is that the colocation provider charges the customers for each cross-connect in addition to the rent of the rack or cage or whatever), it would not be economically reasonable to have both. Although, if the cross-connect is the primary method of interconnection, and the IXP provides a router-server the public-peering over IXP would essentially be free. So it might makes sense to assume that for the private cross-connect, there exists a back-up connection though the IXP. Anyway, I guess some discussion may give more insight about which one is more reasonable to assume and do.
Now my last question is that if the two connections exist (one private cross-connect and another back-up through the IXP), what are the chances that periodically launched traceroutes that pass the inter-AS connection in that colo see both types of connection in a week. I guess what I'm asking is how often back-up routes are taken? Can the networks do load balancing on the two connection and essentially use them as primary routes?
Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon
On 22 May 2016, at 07:33, Max Tulyev <maxtul@netassist.ua> wrote:
Hi All,
I wonder why a "VLAN exchange" does not exists. Or I do not know any?
In my understanding it should be a switch, and people connected can easily order a private VLAN between each other (or to private group) through some kind of web interface.
That should be a more easy and much less expensive way for private interconnects than direct wires.
On 23 May 2016 at 09:59, Marty Strong via NANOG <nanog@nanog.org> wrote:
This does exist, often called an elastic fabric, e.g. Megaport
Regards, Marty Strong -------------------------------------- CloudFlare - AS13335 Network Engineer marty@cloudflare.com +44 7584 906 055 smartflare (Skype)
As Marty said, it does exist. AN example from LONAP in the UK: https://www.lonap.net/fees.php Private VLANs between members = FREE Another option is using a provider like IXReach (now "Console"), take a peering to them, and then down multiple VLANs they can through you peerings to different IXs from around the world and to other networks: http://www.ixreach.com/ Cheers, James.
Hi Max, These do exist, at least in the NREN part of the internet. Have a look at netherlight (www.netherlight.net) and the bigger picture GLIF (www.glif.is) and where you read 'lightpath' replace that with ethernet p2p. Regards, Jac On Sun, 22 May 2016, Max Tulyev wrote:
Hi All,
I wonder why a "VLAN exchange" does not exists. Or I do not know any?
In my understanding it should be a switch, and people connected can easily order a private VLAN between each other (or to private group) through some kind of web interface.
That should be a more easy and much less expensive way for private interconnects than direct wires.
On 16.05.16 20:46, Reza Motamedi wrote:
Dear Nanogers,
I have a question about common/best network interconnection practices. Assume that two networks (let's refer to them as AS-a and AS-b) are present in a colocation facility say Equinix LA. As many of you know, Equininx runs an IXP in LA as well. So AS-as and AS-b can interconnct 1) using private cross-connect 2) through the public IXP's switching fabric. Is it a common/good practice for the two networks to establish connections both through the IXP and also using a private cross-connect?
I was thinking considering the cost of cross-connects (my understanding is that the colocation provider charges the customers for each cross-connect in addition to the rent of the rack or cage or whatever), it would not be economically reasonable to have both. Although, if the cross-connect is the primary method of interconnection, and the IXP provides a router-server the public-peering over IXP would essentially be free. So it might makes sense to assume that for the private cross-connect, there exists a back-up connection though the IXP. Anyway, I guess some discussion may give more insight about which one is more reasonable to assume and do.
Now my last question is that if the two connections exist (one private cross-connect and another back-up through the IXP), what are the chances that periodically launched traceroutes that pass the inter-AS connection in that colo see both types of connection in a week. I guess what I'm asking is how often back-up routes are taken? Can the networks do load balancing on the two connection and essentially use them as primary routes?
Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon
-- Jac Kloots Network Services SURFnet bv
In a message written on Sun, May 22, 2016 at 09:33:38AM +0300, Max Tulyev wrote:
That should be a more easy and much less expensive way for private interconnects than direct wires.
The problem is peering is not an even distribution by traffic level. When BigCDNCo connects to BigCableCo, they need 50x100GE. It's actually cheaper to run the fiber between them at 10 locations for 5x100GE each than it is to run fiber from both of them to a switch, and have the switch providing vendor engineer the switch to that capacity. (Hint, running to the switch is 2x the fiber, plus switch ports.) On the other end of the spectrum, the guy who has 5Gbps of traffic can buy a 10GE into the switched exchange, have lots of headroom and connect to everyone with the same port. The truth of the matter is there are 40 players in the big pile, 15,000 providers in the small pile, and perhaps only 100 oddballs between the two. -- Leo Bicknell - bicknell@ufp.org PGP keys at http://www.ufp.org/~bicknell/
As mentioned by others, they do exist, but usually not for exactly the reason you state. In most cases, peers go to PNI instead of peering via the exchange when it does not make sense to grow laterally at the exchange for significant bilateral traffic. It’s much less expensive to get a cross-connect from my router to your router than for both of us to add a cross-connect to the exchange and each pay for an additional exchange port. Example: If I have 12.5 gigs of traffic to the exchange and 8 gigs of that is to autonomous system X while the remaining 4.5 G goes to random other peers, then it makes much more sense for both X and I to connect directly (PNI) than for each of us to order an additional exchange port to support that traffic. Owen
On May 21, 2016, at 23:33 , Max Tulyev <maxtul@netassist.ua> wrote:
Hi All,
I wonder why a "VLAN exchange" does not exists. Or I do not know any?
In my understanding it should be a switch, and people connected can easily order a private VLAN between each other (or to private group) through some kind of web interface.
That should be a more easy and much less expensive way for private interconnects than direct wires.
On 16.05.16 20:46, Reza Motamedi wrote:
Dear Nanogers,
I have a question about common/best network interconnection practices. Assume that two networks (let's refer to them as AS-a and AS-b) are present in a colocation facility say Equinix LA. As many of you know, Equininx runs an IXP in LA as well. So AS-as and AS-b can interconnct 1) using private cross-connect 2) through the public IXP's switching fabric. Is it a common/good practice for the two networks to establish connections both through the IXP and also using a private cross-connect?
I was thinking considering the cost of cross-connects (my understanding is that the colocation provider charges the customers for each cross-connect in addition to the rent of the rack or cage or whatever), it would not be economically reasonable to have both. Although, if the cross-connect is the primary method of interconnection, and the IXP provides a router-server the public-peering over IXP would essentially be free. So it might makes sense to assume that for the private cross-connect, there exists a back-up connection though the IXP. Anyway, I guess some discussion may give more insight about which one is more reasonable to assume and do.
Now my last question is that if the two connections exist (one private cross-connect and another back-up through the IXP), what are the chances that periodically launched traceroutes that pass the inter-AS connection in that colo see both types of connection in a week. I guess what I'm asking is how often back-up routes are taken? Can the networks do load balancing on the two connection and essentially use them as primary routes?
Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon
I'm glad we are having this discussion. I want to clarify something, since I'm not sure I'm following the terminology. What Max referred to as "VLAN exchange" is what Equinix markets as "*private VLAN"*, right? I just copy-pasted a portion of Equinix's IX brochure that covers the services that they offer [ http://www.equinix.com/resources/data-sheets/equinix-internet-exchange/] Standard Equinix Internet Exchange Features • Public VLAN — offers access to all peering participants • Supports industry standard IEEE 802.1Q trunking encapsulation • Redundant MLPE route servers at each IX Point enabling efficient open peering • *Private VLAN* (Required: Unicast Peering VLAN enabled) — create a private broadcast domain over the public switched infrastructure that can be used for direct bi-lateral peering or to create a community of interest My question is what is the point of having such an option for peering? I understand the argument that Owen and Leo have, which is to move the bigger portion of traffic away from the IX fabric and keep the IX for smaller flows. but why would a pair of networks want a private point-to-point connection on a shared switching fabric. Is this just because that shared fabric has geographical reach, as in the case of IXReach? I also see that links provided in this discussion show Europe based networks that are using this peering type more often. Is this widely accepted that US market is totally different from Europe? Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon On Mon, May 23, 2016 at 9:50 AM, Owen DeLong <owen@delong.com> wrote:
As mentioned by others, they do exist, but usually not for exactly the reason you state.
In most cases, peers go to PNI instead of peering via the exchange when it does not make sense to grow laterally at the exchange for significant bilateral traffic. It’s much less expensive to get a cross-connect from my router to your router than for both of us to add a cross-connect to the exchange and each pay for an additional exchange port.
Example: If I have 12.5 gigs of traffic to the exchange and 8 gigs of that is to autonomous system X while the remaining 4.5 G goes to random other peers, then it makes much more sense for both X and I to connect directly (PNI) than for each of us to order an additional exchange port to support that traffic.
Owen
On May 21, 2016, at 23:33 , Max Tulyev <maxtul@netassist.ua> wrote:
Hi All,
I wonder why a "VLAN exchange" does not exists. Or I do not know any?
In my understanding it should be a switch, and people connected can easily order a private VLAN between each other (or to private group) through some kind of web interface.
That should be a more easy and much less expensive way for private interconnects than direct wires.
Dear Nanogers,
I have a question about common/best network interconnection practices. Assume that two networks (let's refer to them as AS-a and AS-b) are
in a colocation facility say Equinix LA. As many of you know, Equininx runs an IXP in LA as well. So AS-as and AS-b can interconnct 1) using private cross-connect 2) through the public IXP's switching fabric. Is it a common/good practice for the two networks to establish connections both through the IXP and also using a private cross-connect?
I was thinking considering the cost of cross-connects (my understanding is that the colocation provider charges the customers for each cross-connect in addition to the rent of the rack or cage or whatever), it would not be economically reasonable to have both. Although, if the cross-connect is
primary method of interconnection, and the IXP provides a router-server
On 16.05.16 20:46, Reza Motamedi wrote: present the the
public-peering over IXP would essentially be free. So it might makes sense to assume that for the private cross-connect, there exists a back-up connection though the IXP. Anyway, I guess some discussion may give more insight about which one is more reasonable to assume and do.
Now my last question is that if the two connections exist (one private cross-connect and another back-up through the IXP), what are the chances that periodically launched traceroutes that pass the inter-AS connection in that colo see both types of connection in a week. I guess what I'm asking is how often back-up routes are taken? Can the networks do load balancing on the two connection and essentially use them as primary routes?
Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon
The usefulness of an elastic fabric as far as I can see it are: - Can give you a private VLAN to some *cloud* providers that provide direct access to them in some other fashion than peering (assumedly for enterprises) - Is spread across multiple buildings across a metro area - Is elastic so can be divided between different services for different time periods In a traditional peering sense it doesn’t really offer much value. Just my two pence. Regards, Marty Strong -------------------------------------- CloudFlare - AS13335 Network Engineer marty@cloudflare.com +44 7584 906 055 smartflare (Skype) http://www.peeringdb.com/view.php?asn=13335
On 23 May 2016, at 18:53, Reza Motamedi <motamedi@cs.uoregon.edu> wrote:
I'm glad we are having this discussion.
I want to clarify something, since I'm not sure I'm following the terminology. What Max referred to as "VLAN exchange" is what Equinix markets as "*private VLAN"*, right? I just copy-pasted a portion of Equinix's IX brochure that covers the services that they offer [ http://www.equinix.com/resources/data-sheets/equinix-internet-exchange/] Standard Equinix Internet Exchange Features • Public VLAN — offers access to all peering participants • Supports industry standard IEEE 802.1Q trunking encapsulation • Redundant MLPE route servers at each IX Point enabling efficient open peering • *Private VLAN* (Required: Unicast Peering VLAN enabled) — create a private broadcast domain over the public switched infrastructure that can be used for direct bi-lateral peering or to create a community of interest
My question is what is the point of having such an option for peering? I understand the argument that Owen and Leo have, which is to move the bigger portion of traffic away from the IX fabric and keep the IX for smaller flows. but why would a pair of networks want a private point-to-point connection on a shared switching fabric. Is this just because that shared fabric has geographical reach, as in the case of IXReach?
I also see that links provided in this discussion show Europe based networks that are using this peering type more often. Is this widely accepted that US market is totally different from Europe?
Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon
On Mon, May 23, 2016 at 9:50 AM, Owen DeLong <owen@delong.com> wrote:
As mentioned by others, they do exist, but usually not for exactly the reason you state.
In most cases, peers go to PNI instead of peering via the exchange when it does not make sense to grow laterally at the exchange for significant bilateral traffic. It’s much less expensive to get a cross-connect from my router to your router than for both of us to add a cross-connect to the exchange and each pay for an additional exchange port.
Example: If I have 12.5 gigs of traffic to the exchange and 8 gigs of that is to autonomous system X while the remaining 4.5 G goes to random other peers, then it makes much more sense for both X and I to connect directly (PNI) than for each of us to order an additional exchange port to support that traffic.
Owen
On May 21, 2016, at 23:33 , Max Tulyev <maxtul@netassist.ua> wrote:
Hi All,
I wonder why a "VLAN exchange" does not exists. Or I do not know any?
In my understanding it should be a switch, and people connected can easily order a private VLAN between each other (or to private group) through some kind of web interface.
That should be a more easy and much less expensive way for private interconnects than direct wires.
Dear Nanogers,
I have a question about common/best network interconnection practices. Assume that two networks (let's refer to them as AS-a and AS-b) are
in a colocation facility say Equinix LA. As many of you know, Equininx runs an IXP in LA as well. So AS-as and AS-b can interconnct 1) using private cross-connect 2) through the public IXP's switching fabric. Is it a common/good practice for the two networks to establish connections both through the IXP and also using a private cross-connect?
I was thinking considering the cost of cross-connects (my understanding is that the colocation provider charges the customers for each cross-connect in addition to the rent of the rack or cage or whatever), it would not be economically reasonable to have both. Although, if the cross-connect is
primary method of interconnection, and the IXP provides a router-server
On 16.05.16 20:46, Reza Motamedi wrote: present the the
public-peering over IXP would essentially be free. So it might makes sense to assume that for the private cross-connect, there exists a back-up connection though the IXP. Anyway, I guess some discussion may give more insight about which one is more reasonable to assume and do.
Now my last question is that if the two connections exist (one private cross-connect and another back-up through the IXP), what are the chances that periodically launched traceroutes that pass the inter-AS connection in that colo see both types of connection in a week. I guess what I'm asking is how often back-up routes are taken? Can the networks do load balancing on the two connection and essentially use them as primary routes?
Best Regards Reza Motamedi (R.M) Graduate Research Fellow Oregon Network Research Group Computer and Information Science University of Oregon
And what benefit is there to this 'public' vlan service? A shared vlan between all participants (with some well organized numbering/indexing scheme)? TorIX (Toronto) is about to have an AGM here and this VLAN thing which has been in the air for 3 years will certainly be brought up again. /kc On Mon, May 23, 2016 at 07:19:03PM +0100, Marty Strong via NANOG said:
The usefulness of an elastic fabric as far as I can see it are:
- Can give you a private VLAN to some *cloud* providers that provide direct access to them in some other fashion than peering (assumedly for enterprises) - Is spread across multiple buildings across a metro area - Is elastic so can be divided between different services for different time periods
In a traditional peering sense it doesn???t really offer much value.
Just my two pence.
Regards, Marty Strong
-- Ken Chase - Guelph Canada
Typically you would use a private VLAN between you and another participant in order to connect to them separately from the public peering VLAN. You would do this instead of a PNI in a situation where you’re in a different building from the other participant making a direct fibre more expensive than the value it would bring. A public VLAN is essentially the peering VLAN anyway, so an all participants VLAN would be a little pointless. Perhaps a VLAN shared between a couple of members *may* be useful depending on those members’ use cases, although I can’t think of one off the top of my head. Regards, Marty Strong -------------------------------------- CloudFlare - AS13335 Network Engineer marty@cloudflare.com +44 7584 906 055 smartflare (Skype) http://www.peeringdb.com/view.php?asn=13335
On 23 May 2016, at 23:24, Ken Chase <math@sizone.org> wrote:
And what benefit is there to this 'public' vlan service? A shared vlan between all participants (with some well organized numbering/indexing scheme)?
TorIX (Toronto) is about to have an AGM here and this VLAN thing which has been in the air for 3 years will certainly be brought up again.
/kc
On Mon, May 23, 2016 at 07:19:03PM +0100, Marty Strong via NANOG said:
The usefulness of an elastic fabric as far as I can see it are:
- Can give you a private VLAN to some *cloud* providers that provide direct access to them in some other fashion than peering (assumedly for enterprises) - Is spread across multiple buildings across a metro area - Is elastic so can be divided between different services for different time periods
In a traditional peering sense it doesn???t really offer much value.
Just my two pence.
Regards, Marty Strong
-- Ken Chase - Guelph Canada
participants (12)
-
Baldur Norddahl -
Jac Kloots -
James Bensley -
Jared Mauch -
Jon Lewis -
Ken Chase -
Leo Bicknell -
Marty Strong -
Max Tulyev -
Nick Ellermann -
Owen DeLong -
Reza Motamedi