sounds like he needs full routes.. On Tue, Oct 13, 2020 at 4:36 PM Ryan Hamel <ryan@rkhtech.org> wrote:

You would get better peering from Equinix IX, which includes free HE IPv4 Peering + IPv6 Transit

Ryan On Oct 13 2020, at 4:29 pm, Aaron Gould <aaron1@gvtc.com> wrote:

Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent.

-Aaron

On 10/13/20 5:10 PM, Darin Steffl wrote:

You would do well to add them to your mix and remove one of the other ones. I'd probably remove spectrum and replace with HE. We've only had 30 minutes of downtime total in 5 years so they've been very reliable for us.

I removed Spectrum (Charter) and replaced them with HE. The latter's value proposition was far superior, plus HE is friendlier to work with, and easier to get in touch with a clued individual at HE.

Depending on transport costs, it may be cheaper to just use HE at a datacenter he's already in vs going to a datacenter he's not in currently. HE has 10G of transit for as low as $900 right now. If 10G of transport from him to an IX is more than that, there's no financial incentive to peer instead of buy more transit. Since HE peers with everyone on most IX's, he is essentially paying HE for full routes plus peering to the same IX he would go to anyway. It's lower cost and you kill two birds with one stone. I understand the desire to have your own connection to an IX but having HE in the mix is basically like peering direct since they're everywhere in nearly every IX. The other upside is you don't need to waste time trying to establish bilateral peering sessions with providers who don't peer with the route servers. On Wed, Oct 14, 2020, 8:09 AM Michael Spears <michael@spears.io> wrote:

Yep. Get on some IXes first. You’d be able to offload a ton of traffic to free peering, vs sending everything via the transit you pay for.

*From:* NANOG <nanog-bounces+michael=spears.io@nanog.org> *On Behalf Of *Mike Hammett *Sent:* Tuesday, October 13, 2020 8:20 PM *To:* Aaron Gould <aaron1@gvtc.com> *Cc:* nanog@nanog.org *Subject:* Re: Hurricane Electric AS6939

https://bgp.he.net/AS16527

You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit.

DFW has a couple and there are some more that are starting up.

----- Mike Hammett Intelligent Computing Solutions <http://www.ics-il.com/> <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> Midwest Internet Exchange <http://www.midwest-ix.com/> <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> The Brothers WISP <http://www.thebrotherswisp.com/> <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ------------------------------

*From: *"Aaron Gould" <aaron1@gvtc.com> *To: *nanog@nanog.org *Sent: *Tuesday, October 13, 2020 6:29:55 PM *Subject: *Hurricane Electric AS6939

Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent.

-Aaron

Side note, they don’t support any traffic engineer aside from prepends but no complaints Besides that. On Oct 13, 2020, at 8:25 PM, Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: https://bgp.he.net/AS16527 You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit. DFW has a couple and there are some more that are starting up. ----- Mike Hammett Intelligent Computing Solutions<http://www.ics-il.com/> [http://www.ics-il.com/images/fbicon.png]<https://www.facebook.com/ICSIL>[http://www.ics-il.com/images/googleicon.png]<https://plus.google.com/+IntelligentComputingSolutionsDeKalb>[http://www.ics-il.com/images/linkedinicon.png]<https://www.linkedin.com/company/intelligent-computing-solutions>[http://www.ics-il.com/images/twittericon.png]<https://twitter.com/ICSIL> Midwest Internet Exchange<http://www.midwest-ix.com/> [http://www.ics-il.com/images/fbicon.png]<https://www.facebook.com/mdwestix>[http://www.ics-il.com/images/linkedinicon.png]<https://www.linkedin.com/company/midwest-internet-exchange>[http://www.ics-il.com/images/twittericon.png]<https://twitter.com/mdwestix> The Brothers WISP<http://www.thebrotherswisp.com/> [http://www.ics-il.com/images/fbicon.png]<https://www.facebook.com/thebrotherswisp>[http://www.ics-il.com/images/youtubeicon.png]<https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ________________________________ From: "Aaron Gould" <aaron1@gvtc.com<mailto:aaron1@gvtc.com>> To: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Tuesday, October 13, 2020 6:29:55 PM Subject: Hurricane Electric AS6939 Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent. -Aaron

Generally one would order a circuit (aka wave) between your location and the IX fabric at the interchange if you're not at the site you're wanting to peer at. For instance, the network I am the network engineer for has a circuit which terminates into the Seattle IX (SIX) fabric. We don't have any other presence in Seattle (or Washington for that matter) at this point - our circuit connects directly to our port on the Exchange. We're considering adding a similar link to another exchange point somewhere to the east or southeast of us. I haven't looked at the graphs recently, but it's not uncommon for >50% of our traffic to come from the exchange. And yes, we're peered with Hurricane and others there. We're also looking at dropping 1U or so of equipment in so we can pick up some transit as well, but that's a story for a different day about the joys of providing internet in the less populated parts of the country. In your case, it also looks like there are also some peering options at the datacenters you are currently at as well. You may want to do some more research to determine how that might work in your situation. PeeringDB is a good resource along with google searches for "peering 100 Taylor" or "peering austin data foundry" On Tue, Oct 13, 2020 at 9:51 PM <aaron1@gvtc.com> wrote:

Don’t you have to be there to join?

I’m in Austin and San Antonio

-Aaron

*From:* Mike Hammett <nanog@ics-il.net> *Sent:* Tuesday, October 13, 2020 7:20 PM *To:* Aaron Gould <aaron1@gvtc.com> *Cc:* nanog@nanog.org *Subject:* Re: Hurricane Electric AS6939

https://bgp.he.net/AS16527

You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit.

DFW has a couple and there are some more that are starting up.

----- Mike Hammett Intelligent Computing Solutions <http://www.ics-il.com/> <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> Midwest Internet Exchange <http://www.midwest-ix.com/> <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> The Brothers WISP <http://www.thebrotherswisp.com/> <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ------------------------------

*From: *"Aaron Gould" <aaron1@gvtc.com> *To: *nanog@nanog.org *Sent: *Tuesday, October 13, 2020 6:29:55 PM *Subject: *Hurricane Electric AS6939

Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent.

-Aaron

-- - Forrest

I guess I should have been a bit clearer. Yes, what you would be ordering is typically a lit L2 circuit. However, my experience is that certain carrier salespeople tend to call anything like this a 'wave'. I have had lots of discussions over the years with various salespeople about the difference, and yes, it's pretty much always lit L2. Centurylink (now Lumen) even sells a service they call "Encrypted Wavelength Service". Not sure how one encrypts light.... On Wed, Oct 14, 2020 at 1:25 AM Eric Kuhnke <eric.kuhnke@gmail.com> wrote:

For small ISPs looking at setting up their first ever presence at an IX point, you almost certainly would not be ordering an actual 'wave' (eg: a specific DWDM channel on a legacy 10G DWDM platform, handed off to you with 1310/LX interfaces at both ends), but lit layer 2 transport service between the carrier hotel and your service location.

Pricing for the two types of service can be quite different when you request an actual 'wave' from a carrier sales person, vs just lit L2 transport capable of large MTUs, QinQ, etc.

The ISP carrying it might take it between those two places as simply a vlan trunked through a larger 100G link, as a MPLS circuit, lots of possible things.

Unless you happened to be in a happy conjunction of the right place at the right time, and an older DWDM system on exactly the same path you wanted happened to have an empty channel and ready to go interface cards at both ends.

On Tue, Oct 13, 2020 at 11:12 PM Forrest Christian (List Account) < lists@packetflux.com> wrote:

...

Generally one would order a circuit (aka wave) between your location and the IX fabric at the interchange if you're not at the site you're wanting to peer at.

For instance, the network I am the network engineer for has a circuit which terminates into the Seattle IX (SIX) fabric. We don't have any other presence in Seattle (or Washington for that matter) at this point - our circuit connects directly to our port on the Exchange. We're considering adding a similar link to another exchange point somewhere to the east or southeast of us. I haven't looked at the graphs recently, but it's not uncommon for >50% of our traffic to come from the exchange. And yes, we're peered with Hurricane and others there.

We're also looking at dropping 1U or so of equipment in so we can pick up some transit as well, but that's a story for a different day about the joys of providing internet in the less populated parts of the country.

In your case, it also looks like there are also some peering options at the datacenters you are currently at as well. You may want to do some more research to determine how that might work in your situation. PeeringDB is a good resource along with google searches for "peering 100 Taylor" or "peering austin data foundry"

On Tue, Oct 13, 2020 at 9:51 PM <aaron1@gvtc.com> wrote:

...

Don’t you have to be there to join?

I’m in Austin and San Antonio

-Aaron

*From:* Mike Hammett <nanog@ics-il.net> *Sent:* Tuesday, October 13, 2020 7:20 PM *To:* Aaron Gould <aaron1@gvtc.com> *Cc:* nanog@nanog.org *Subject:* Re: Hurricane Electric AS6939

https://bgp.he.net/AS16527

You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit.

DFW has a couple and there are some more that are starting up.

----- Mike Hammett Intelligent Computing Solutions <http://www.ics-il.com/> <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> Midwest Internet Exchange <http://www.midwest-ix.com/> <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> The Brothers WISP <http://www.thebrotherswisp.com/> <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ------------------------------

*From: *"Aaron Gould" <aaron1@gvtc.com> *To: *nanog@nanog.org *Sent: *Tuesday, October 13, 2020 6:29:55 PM *Subject: *Hurricane Electric AS6939

Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent.

-Aaron

-- - Forrest

-- - Forrest

Charter/Spectrum calls it an EPL - Ethernet Private Line. Josh Luthman 24/7 Help Desk: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Wed, Oct 14, 2020 at 4:08 AM Forrest Christian (List Account) < lists@packetflux.com> wrote:

I guess I should have been a bit clearer.

Yes, what you would be ordering is typically a lit L2 circuit. However, my experience is that certain carrier salespeople tend to call anything like this a 'wave'. I have had lots of discussions over the years with various salespeople about the difference, and yes, it's pretty much always lit L2. Centurylink (now Lumen) even sells a service they call "Encrypted Wavelength Service". Not sure how one encrypts light....

On Wed, Oct 14, 2020 at 1:25 AM Eric Kuhnke <eric.kuhnke@gmail.com> wrote:

...

For small ISPs looking at setting up their first ever presence at an IX point, you almost certainly would not be ordering an actual 'wave' (eg: a specific DWDM channel on a legacy 10G DWDM platform, handed off to you with 1310/LX interfaces at both ends), but lit layer 2 transport service between the carrier hotel and your service location.

Pricing for the two types of service can be quite different when you request an actual 'wave' from a carrier sales person, vs just lit L2 transport capable of large MTUs, QinQ, etc.

The ISP carrying it might take it between those two places as simply a vlan trunked through a larger 100G link, as a MPLS circuit, lots of possible things.

Unless you happened to be in a happy conjunction of the right place at the right time, and an older DWDM system on exactly the same path you wanted happened to have an empty channel and ready to go interface cards at both ends.

On Tue, Oct 13, 2020 at 11:12 PM Forrest Christian (List Account) < lists@packetflux.com> wrote:

...

Generally one would order a circuit (aka wave) between your location and the IX fabric at the interchange if you're not at the site you're wanting to peer at.

For instance, the network I am the network engineer for has a circuit which terminates into the Seattle IX (SIX) fabric. We don't have any other presence in Seattle (or Washington for that matter) at this point - our circuit connects directly to our port on the Exchange. We're considering adding a similar link to another exchange point somewhere to the east or southeast of us. I haven't looked at the graphs recently, but it's not uncommon for >50% of our traffic to come from the exchange. And yes, we're peered with Hurricane and others there.

We're also looking at dropping 1U or so of equipment in so we can pick up some transit as well, but that's a story for a different day about the joys of providing internet in the less populated parts of the country.

In your case, it also looks like there are also some peering options at the datacenters you are currently at as well. You may want to do some more research to determine how that might work in your situation. PeeringDB is a good resource along with google searches for "peering 100 Taylor" or "peering austin data foundry"

On Tue, Oct 13, 2020 at 9:51 PM <aaron1@gvtc.com> wrote:

...

Don’t you have to be there to join?

I’m in Austin and San Antonio

-Aaron

*From:* Mike Hammett <nanog@ics-il.net> *Sent:* Tuesday, October 13, 2020 7:20 PM *To:* Aaron Gould <aaron1@gvtc.com> *Cc:* nanog@nanog.org *Subject:* Re: Hurricane Electric AS6939

https://bgp.he.net/AS16527

You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit.

DFW has a couple and there are some more that are starting up.

----- Mike Hammett Intelligent Computing Solutions <http://www.ics-il.com/> <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> Midwest Internet Exchange <http://www.midwest-ix.com/> <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> The Brothers WISP <http://www.thebrotherswisp.com/> <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ------------------------------

*From: *"Aaron Gould" <aaron1@gvtc.com> *To: *nanog@nanog.org *Sent: *Tuesday, October 13, 2020 6:29:55 PM *Subject: *Hurricane Electric AS6939

Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent.

-Aaron

-- - Forrest

-- - Forrest

I suppose it depends on your carrier and their capabilities. I much prefer waves to any kind of service that you can aggregate. Being able to aggregate just means they're going to oversubscribe you and at some point, you'll not get what you're paying for. Can't do that on a wave. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Eric Kuhnke" <eric.kuhnke@gmail.com> To: "Forrest Christian (List Account)" <lists@packetflux.com> Cc: "nanog list" <nanog@nanog.org> Sent: Wednesday, October 14, 2020 2:25:46 AM Subject: Re: Hurricane Electric AS6939 For small ISPs looking at setting up their first ever presence at an IX point, you almost certainly would not be ordering an actual 'wave' (eg: a specific DWDM channel on a legacy 10G DWDM platform, handed off to you with 1310/LX interfaces at both ends), but lit layer 2 transport service between the carrier hotel and your service location. Pricing for the two types of service can be quite different when you request an actual 'wave' from a carrier sales person, vs just lit L2 transport capable of large MTUs, QinQ, etc. The ISP carrying it might take it between those two places as simply a vlan trunked through a larger 100G link, as a MPLS circuit, lots of possible things. Unless you happened to be in a happy conjunction of the right place at the right time, and an older DWDM system on exactly the same path you wanted happened to have an empty channel and ready to go interface cards at both ends. On Tue, Oct 13, 2020 at 11:12 PM Forrest Christian (List Account) < lists@packetflux.com > wrote: Generally one would order a circuit (aka wave) between your location and the IX fabric at the interchange if you're not at the site you're wanting to peer at. For instance, the network I am the network engineer for has a circuit which terminates into the Seattle IX (SIX) fabric. We don't have any other presence in Seattle (or Washington for that matter) at this point - our circuit connects directly to our port on the Exchange. We're considering adding a similar link to another exchange point somewhere to the east or southeast of us. I haven't looked at the graphs recently, but it's not uncommon for >50% of our traffic to come from the exchange. And yes, we're peered with Hurricane and others there. We're also looking at dropping 1U or so of equipment in so we can pick up some transit as well, but that's a story for a different day about the joys of providing internet in the less populated parts of the country. In your case, it also looks like there are also some peering options at the datacenters you are currently at as well. You may want to do some more research to determine how that might work in your situation. PeeringDB is a good resource along with google searches for "peering 100 Taylor" or "peering austin data foundry" On Tue, Oct 13, 2020 at 9:51 PM < aaron1@gvtc.com > wrote: <blockquote> Don’t you have to be there to join? I’m in Austin and San Antonio -Aaron From: Mike Hammett < nanog@ics-il.net > Sent: Tuesday, October 13, 2020 7:20 PM To: Aaron Gould < aaron1@gvtc.com > Cc: nanog@nanog.org Subject: Re: Hurricane Electric AS6939 https://bgp.he.net/AS16527 You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit. DFW has a couple and there are some more that are starting up. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP From: "Aaron Gould" < aaron1@gvtc.com > To: nanog@nanog.org Sent: Tuesday, October 13, 2020 6:29:55 PM Subject: Hurricane Electric AS6939 Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent. -Aaron -- - Forrest </blockquote>

*nods* There are protected wave services generally available if you wish to protect about such things. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Darin Steffl" <darin.steffl@mnwifi.com> To: "Mike Hammett" <nanog@ics-il.net> Cc: "Eric Kuhnke" <eric.kuhnke@gmail.com>, "nanog list" <nanog@nanog.org> Sent: Wednesday, October 14, 2020 3:08:19 PM Subject: Re: Hurricane Electric AS6939 The downside to waves are that they're typically not protected. So a cut will take you down. If you have 10G Layer 2 ethernet, they often will have redundant paths so the only single path that can fail is between you and their first POP where they hopefully have redundancy. It can make a big difference when you're transporting data hundreds or thousands of miles. The longer the path, the less reliable the wave will be as each route mile opens you up to more risk. On Wed, Oct 14, 2020 at 2:25 PM Mike Hammett < nanog@ics-il.net > wrote: I suppose it depends on your carrier and their capabilities. I much prefer waves to any kind of service that you can aggregate. Being able to aggregate just means they're going to oversubscribe you and at some point, you'll not get what you're paying for. Can't do that on a wave. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP From: "Eric Kuhnke" < eric.kuhnke@gmail.com > To: "Forrest Christian (List Account)" < lists@packetflux.com > Cc: "nanog list" < nanog@nanog.org > Sent: Wednesday, October 14, 2020 2:25:46 AM Subject: Re: Hurricane Electric AS6939 For small ISPs looking at setting up their first ever presence at an IX point, you almost certainly would not be ordering an actual 'wave' (eg: a specific DWDM channel on a legacy 10G DWDM platform, handed off to you with 1310/LX interfaces at both ends), but lit layer 2 transport service between the carrier hotel and your service location. Pricing for the two types of service can be quite different when you request an actual 'wave' from a carrier sales person, vs just lit L2 transport capable of large MTUs, QinQ, etc. The ISP carrying it might take it between those two places as simply a vlan trunked through a larger 100G link, as a MPLS circuit, lots of possible things. Unless you happened to be in a happy conjunction of the right place at the right time, and an older DWDM system on exactly the same path you wanted happened to have an empty channel and ready to go interface cards at both ends. On Tue, Oct 13, 2020 at 11:12 PM Forrest Christian (List Account) < lists@packetflux.com > wrote: <blockquote> Generally one would order a circuit (aka wave) between your location and the IX fabric at the interchange if you're not at the site you're wanting to peer at. For instance, the network I am the network engineer for has a circuit which terminates into the Seattle IX (SIX) fabric. We don't have any other presence in Seattle (or Washington for that matter) at this point - our circuit connects directly to our port on the Exchange. We're considering adding a similar link to another exchange point somewhere to the east or southeast of us. I haven't looked at the graphs recently, but it's not uncommon for >50% of our traffic to come from the exchange. And yes, we're peered with Hurricane and others there. We're also looking at dropping 1U or so of equipment in so we can pick up some transit as well, but that's a story for a different day about the joys of providing internet in the less populated parts of the country. In your case, it also looks like there are also some peering options at the datacenters you are currently at as well. You may want to do some more research to determine how that might work in your situation. PeeringDB is a good resource along with google searches for "peering 100 Taylor" or "peering austin data foundry" On Tue, Oct 13, 2020 at 9:51 PM < aaron1@gvtc.com > wrote: <blockquote> Don’t you have to be there to join? I’m in Austin and San Antonio -Aaron From: Mike Hammett < nanog@ics-il.net > Sent: Tuesday, October 13, 2020 7:20 PM To: Aaron Gould < aaron1@gvtc.com > Cc: nanog@nanog.org Subject: Re: Hurricane Electric AS6939 https://bgp.he.net/AS16527 You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit. DFW has a couple and there are some more that are starting up. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP From: "Aaron Gould" < aaron1@gvtc.com > To: nanog@nanog.org Sent: Tuesday, October 13, 2020 6:29:55 PM Subject: Hurricane Electric AS6939 Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent. -Aaron -- - Forrest </blockquote> </blockquote> -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook

*nods* instead of paying for protection, I'd rather just engineer a completely diverse route, preferable to a different Z location. Potential for greater diversity, you get more capacity. Costs may vary. I've seen similar things regarding 1G and less. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Darin Steffl" <darin.steffl@mnwifi.com> To: "Mike Hammett" <nanog@ics-il.net> Cc: "Eric Kuhnke" <eric.kuhnke@gmail.com>, "nanog list" <nanog@nanog.org> Sent: Wednesday, October 14, 2020 3:40:50 PM Subject: Re: Hurricane Electric AS6939 Yes but they're $$$ to have protection. Generally ethernet will be cheaper than waves with the added protection. I'm not arguing for one or the other. Waves will often be cheaper when looking at 10G or 100G compared to ethernet. For 1G or less, ethernet might be cheaper with some protection already built-in. On Wed, Oct 14, 2020 at 3:31 PM Mike Hammett < nanog@ics-il.net > wrote: *nods* There are protected wave services generally available if you wish to protect about such things. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP From: "Darin Steffl" < darin.steffl@mnwifi.com > To: "Mike Hammett" < nanog@ics-il.net > Cc: "Eric Kuhnke" < eric.kuhnke@gmail.com >, "nanog list" < nanog@nanog.org > Sent: Wednesday, October 14, 2020 3:08:19 PM Subject: Re: Hurricane Electric AS6939 The downside to waves are that they're typically not protected. So a cut will take you down. If you have 10G Layer 2 ethernet, they often will have redundant paths so the only single path that can fail is between you and their first POP where they hopefully have redundancy. It can make a big difference when you're transporting data hundreds or thousands of miles. The longer the path, the less reliable the wave will be as each route mile opens you up to more risk. On Wed, Oct 14, 2020 at 2:25 PM Mike Hammett < nanog@ics-il.net > wrote: <blockquote> I suppose it depends on your carrier and their capabilities. I much prefer waves to any kind of service that you can aggregate. Being able to aggregate just means they're going to oversubscribe you and at some point, you'll not get what you're paying for. Can't do that on a wave. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP From: "Eric Kuhnke" < eric.kuhnke@gmail.com > To: "Forrest Christian (List Account)" < lists@packetflux.com > Cc: "nanog list" < nanog@nanog.org > Sent: Wednesday, October 14, 2020 2:25:46 AM Subject: Re: Hurricane Electric AS6939 For small ISPs looking at setting up their first ever presence at an IX point, you almost certainly would not be ordering an actual 'wave' (eg: a specific DWDM channel on a legacy 10G DWDM platform, handed off to you with 1310/LX interfaces at both ends), but lit layer 2 transport service between the carrier hotel and your service location. Pricing for the two types of service can be quite different when you request an actual 'wave' from a carrier sales person, vs just lit L2 transport capable of large MTUs, QinQ, etc. The ISP carrying it might take it between those two places as simply a vlan trunked through a larger 100G link, as a MPLS circuit, lots of possible things. Unless you happened to be in a happy conjunction of the right place at the right time, and an older DWDM system on exactly the same path you wanted happened to have an empty channel and ready to go interface cards at both ends. On Tue, Oct 13, 2020 at 11:12 PM Forrest Christian (List Account) < lists@packetflux.com > wrote: <blockquote> Generally one would order a circuit (aka wave) between your location and the IX fabric at the interchange if you're not at the site you're wanting to peer at. For instance, the network I am the network engineer for has a circuit which terminates into the Seattle IX (SIX) fabric. We don't have any other presence in Seattle (or Washington for that matter) at this point - our circuit connects directly to our port on the Exchange. We're considering adding a similar link to another exchange point somewhere to the east or southeast of us. I haven't looked at the graphs recently, but it's not uncommon for >50% of our traffic to come from the exchange. And yes, we're peered with Hurricane and others there. We're also looking at dropping 1U or so of equipment in so we can pick up some transit as well, but that's a story for a different day about the joys of providing internet in the less populated parts of the country. In your case, it also looks like there are also some peering options at the datacenters you are currently at as well. You may want to do some more research to determine how that might work in your situation. PeeringDB is a good resource along with google searches for "peering 100 Taylor" or "peering austin data foundry" On Tue, Oct 13, 2020 at 9:51 PM < aaron1@gvtc.com > wrote: <blockquote> Don’t you have to be there to join? I’m in Austin and San Antonio -Aaron From: Mike Hammett < nanog@ics-il.net > Sent: Tuesday, October 13, 2020 7:20 PM To: Aaron Gould < aaron1@gvtc.com > Cc: nanog@nanog.org Subject: Re: Hurricane Electric AS6939 https://bgp.he.net/AS16527 You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit. DFW has a couple and there are some more that are starting up. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP From: "Aaron Gould" < aaron1@gvtc.com > To: nanog@nanog.org Sent: Tuesday, October 13, 2020 6:29:55 PM Subject: Hurricane Electric AS6939 Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent. -Aaron -- - Forrest </blockquote> </blockquote> -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook </blockquote> -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook

Didn’t the Dec 2018 CL outage cause waves and even TDM circuits to go down? Luke From: NANOG <nanog-bounces+lguillory=reservetele.com@nanog.org> On Behalf Of Matt Erculiani Sent: Wednesday, October 14, 2020 3:59 PM To: Darin Steffl <darin.steffl@mnwifi.com> Cc: nanog list <nanog@nanog.org> Subject: Re: Hurricane Electric AS6939 *External Email: Use Caution* For providers who use the same infrastructure for their IP backbone and Ethernet services (as so many do), a large DDoS could disrupt all Ethernet services that normally traverse affected links, whereas Waves would be blissfully ignorant of such an event. Waves are pretty reliable and will only go down as a result of a configuration error, vendor software issue, or physical/layer 1 failure, all of which can also affect Ethernet services. This is especially important if you select a provider that sees excess capacity as a wasted operational expense instead of an investment in reliability. Worth noting that protected Waves do have a "reconvergence" time like Ethernet would, but this is typically measured in nanoseconds for shorter distances. Your equipment can probably be configured to not link-down during this gap, you'll just see some errors or a few dropped packets (subject to your provider's specific implementation). -Matt On Wed, Oct 14, 2020 at 2:41 PM Darin Steffl <darin.steffl@mnwifi.com<mailto:darin.steffl@mnwifi.com>> wrote: Yes but they're $$$ to have protection. Generally ethernet will be cheaper than waves with the added protection. I'm not arguing for one or the other. Waves will often be cheaper when looking at 10G or 100G compared to ethernet. For 1G or less, ethernet might be cheaper with some protection already built-in. On Wed, Oct 14, 2020 at 3:31 PM Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: *nods* There are protected wave services generally available if you wish to protect about such things. ----- Mike Hammett Intelligent Computing Solutions<http://www.ics-il.com/> [http://www.ics-il.com/images/fbicon.png]<https://www.facebook.com/ICSIL>[http://www.ics-il.com/images/googleicon.png]<https://plus.google.com/+IntelligentComputingSolutionsDeKalb>[http://www.ics-il.com/images/linkedinicon.png]<https://www.linkedin.com/company/intelligent-computing-solutions>[http://www.ics-il.com/images/twittericon.png]<https://twitter.com/ICSIL> Midwest Internet Exchange<http://www.midwest-ix.com/> [http://www.ics-il.com/images/fbicon.png]<https://www.facebook.com/mdwestix>[http://www.ics-il.com/images/linkedinicon.png]<https://www.linkedin.com/company/midwest-internet-exchange>[http://www.ics-il.com/images/twittericon.png]<https://twitter.com/mdwestix> The Brothers WISP<http://www.thebrotherswisp.com/> [http://www.ics-il.com/images/fbicon.png]<https://www.facebook.com/thebrotherswisp>[http://www.ics-il.com/images/youtubeicon.png]<https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ________________________________ From: "Darin Steffl" <darin.steffl@mnwifi.com<mailto:darin.steffl@mnwifi.com>> To: "Mike Hammett" <nanog@ics-il.net<mailto:nanog@ics-il.net>> Cc: "Eric Kuhnke" <eric.kuhnke@gmail.com<mailto:eric.kuhnke@gmail.com>>, "nanog list" <nanog@nanog.org<mailto:nanog@nanog.org>> Sent: Wednesday, October 14, 2020 3:08:19 PM Subject: Re: Hurricane Electric AS6939 The downside to waves are that they're typically not protected. So a cut will take you down. If you have 10G Layer 2 ethernet, they often will have redundant paths so the only single path that can fail is between you and their first POP where they hopefully have redundancy. It can make a big difference when you're transporting data hundreds or thousands of miles. The longer the path, the less reliable the wave will be as each route mile opens you up to more risk. On Wed, Oct 14, 2020 at 2:25 PM Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: I suppose it depends on your carrier and their capabilities. I much prefer waves to any kind of service that you can aggregate. Being able to aggregate just means they're going to oversubscribe you and at some point, you'll not get what you're paying for. Can't do that on a wave. ----- Mike Hammett Intelligent Computing Solutions<http://www.ics-il.com/> [http://www.ics-il.com/images/fbicon.png]<https://www.facebook.com/ICSIL>[http://www.ics-il.com/images/googleicon.png]<https://plus.google.com/+IntelligentComputingSolutionsDeKalb>[http://www.ics-il.com/images/linkedinicon.png]<https://www.linkedin.com/company/intelligent-computing-solutions>[http://www.ics-il.com/images/twittericon.png]<https://twitter.com/ICSIL> Midwest Internet Exchange<http://www.midwest-ix.com/> [http://www.ics-il.com/images/fbicon.png]<https://www.facebook.com/mdwestix>[http://www.ics-il.com/images/linkedinicon.png]<https://www.linkedin.com/company/midwest-internet-exchange>[http://www.ics-il.com/images/twittericon.png]<https://twitter.com/mdwestix> The Brothers WISP<http://www.thebrotherswisp.com/> [http://www.ics-il.com/images/fbicon.png]<https://www.facebook.com/thebrotherswisp>[http://www.ics-il.com/images/youtubeicon.png]<https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ________________________________ From: "Eric Kuhnke" <eric.kuhnke@gmail.com<mailto:eric.kuhnke@gmail.com>> To: "Forrest Christian (List Account)" <lists@packetflux.com<mailto:lists@packetflux.com>> Cc: "nanog list" <nanog@nanog.org<mailto:nanog@nanog.org>> Sent: Wednesday, October 14, 2020 2:25:46 AM Subject: Re: Hurricane Electric AS6939 For small ISPs looking at setting up their first ever presence at an IX point, you almost certainly would not be ordering an actual 'wave' (eg: a specific DWDM channel on a legacy 10G DWDM platform, handed off to you with 1310/LX interfaces at both ends), but lit layer 2 transport service between the carrier hotel and your service location. Pricing for the two types of service can be quite different when you request an actual 'wave' from a carrier sales person, vs just lit L2 transport capable of large MTUs, QinQ, etc. The ISP carrying it might take it between those two places as simply a vlan trunked through a larger 100G link, as a MPLS circuit, lots of possible things. Unless you happened to be in a happy conjunction of the right place at the right time, and an older DWDM system on exactly the same path you wanted happened to have an empty channel and ready to go interface cards at both ends. On Tue, Oct 13, 2020 at 11:12 PM Forrest Christian (List Account) <lists@packetflux.com<mailto:lists@packetflux.com>> wrote: Generally one would order a circuit (aka wave) between your location and the IX fabric at the interchange if you're not at the site you're wanting to peer at. For instance, the network I am the network engineer for has a circuit which terminates into the Seattle IX (SIX) fabric. We don't have any other presence in Seattle (or Washington for that matter) at this point - our circuit connects directly to our port on the Exchange. We're considering adding a similar link to another exchange point somewhere to the east or southeast of us. I haven't looked at the graphs recently, but it's not uncommon for >50% of our traffic to come from the exchange. And yes, we're peered with Hurricane and others there. We're also looking at dropping 1U or so of equipment in so we can pick up some transit as well, but that's a story for a different day about the joys of providing internet in the less populated parts of the country. In your case, it also looks like there are also some peering options at the datacenters you are currently at as well. You may want to do some more research to determine how that might work in your situation. PeeringDB is a good resource along with google searches for "peering 100 Taylor" or "peering austin data foundry" On Tue, Oct 13, 2020 at 9:51 PM <aaron1@gvtc.com<mailto:aaron1@gvtc.com>> wrote: Don’t you have to be there to join? I’m in Austin and San Antonio -Aaron From: Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> Sent: Tuesday, October 13, 2020 7:20 PM To: Aaron Gould <aaron1@gvtc.com<mailto:aaron1@gvtc.com>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Subject: Re: Hurricane Electric AS6939 https://bgp.he.net/AS16527 You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit. DFW has a couple and there are some more that are starting up. ----- Mike Hammett Intelligent Computing Solutions<http://www.ics-il.com/> Midwest Internet Exchange The Brothers WISP <https://www.facebook.com/ICSIL> ________________________________ From: "Aaron Gould" <aaron1@gvtc.com> To: nanog@nanog.org Sent: Tuesday, October 13, 2020 6:29:55 PM Subject: Hurricane Electric AS6939 Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent. -Aaron -- - Forrest <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> -- <https://www.facebook.com/ICSIL> Darin Steffl <https://www.facebook.com/ICSIL> Minnesota WiFi<https://www.facebook.com/ICSIL> www.mnwifi.com<https://www.facebook.com/ICSIL> 507-634-WiFi<https://www.facebook.com/ICSIL> Like us on Facebook<https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> -- <https://www.facebook.com/ICSIL> Darin Steffl <https://www.facebook.com/ICSIL> Minnesota WiFi<https://www.facebook.com/ICSIL> www.mnwifi.com<https://www.facebook.com/ICSIL> 507-634-WiFi<https://www.facebook.com/ICSIL> Like us on Facebook<https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> -- <https://www.facebook.com/ICSIL> Matt Erculiani <https://www.facebook.com/ICSIL> ERCUL-ARIN<https://www.facebook.com/ICSIL>

Which was my point is all, while it might be an extreme case, IP can cause issues for waves as well. Luke From: Eric Kuhnke <eric.kuhnke@gmail.com> Sent: Wednesday, October 14, 2020 4:31 PM To: Luke Guillory <lguillory@reservetele.com> Cc: Matt Erculiani <merculiani@gmail.com>; Darin Steffl <darin.steffl@mnwifi.com>; nanog list <nanog@nanog.org> Subject: Re: Hurricane Electric AS6939 *External Email: Use Caution* Yes it did, because they were running all of those over their Infinera DWDM platforms which crashed. If the underlying optical line terminals are FUBAR, all bets are off. On Wed, Oct 14, 2020 at 2:27 PM Luke Guillory <lguillory@reservetele.com<mailto:lguillory@reservetele.com>> wrote: Didn’t the Dec 2018 CL outage cause waves and even TDM circuits to go down? Luke From: NANOG <nanog-bounces+lguillory=reservetele.com@nanog.org<mailto:reservetele.com@nanog.org>> On Behalf Of Matt Erculiani Sent: Wednesday, October 14, 2020 3:59 PM To: Darin Steffl <darin.steffl@mnwifi.com<mailto:darin.steffl@mnwifi.com>> Cc: nanog list <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Hurricane Electric AS6939 *External Email: Use Caution* For providers who use the same infrastructure for their IP backbone and Ethernet services (as so many do), a large DDoS could disrupt all Ethernet services that normally traverse affected links, whereas Waves would be blissfully ignorant of such an event. Waves are pretty reliable and will only go down as a result of a configuration error, vendor software issue, or physical/layer 1 failure, all of which can also affect Ethernet services. This is especially important if you select a provider that sees excess capacity as a wasted operational expense instead of an investment in reliability. Worth noting that protected Waves do have a "reconvergence" time like Ethernet would, but this is typically measured in nanoseconds for shorter distances. Your equipment can probably be configured to not link-down during this gap, you'll just see some errors or a few dropped packets (subject to your provider's specific implementation). -Matt On Wed, Oct 14, 2020 at 2:41 PM Darin Steffl <darin.steffl@mnwifi.com<mailto:darin.steffl@mnwifi.com>> wrote: Yes but they're $$$ to have protection. Generally ethernet will be cheaper than waves with the added protection. I'm not arguing for one or the other. Waves will often be cheaper when looking at 10G or 100G compared to ethernet. For 1G or less, ethernet might be cheaper with some protection already built-in. On Wed, Oct 14, 2020 at 3:31 PM Mike Hammett <nanog@ics-il.net<mailto:nanog@ics-il.net>> wrote: *nods* There are protected wave services generally available if you wish to protect about such things. ----- Mike Hammett Intelligent Computing Solutions<https://link.edgepilot.com/s/cd68553f/FzIGmq4Z1U20Anv8NyxuBQ?u=http://www.ics-il.com/> Midwest Internet Exchange The Brothers WISP <https://link.edgepilot.com/s/f5ae820f/h6FEx2U3g0ulO0XI8otFnw?u=https://www.facebook.com/ICSIL> ________________________________ From: "Darin Steffl" <darin.steffl@mnwifi.com> To: "Mike Hammett" <nanog@ics-il.net> Cc: "Eric Kuhnke" <eric.kuhnke@gmail.com>, "nanog list" <nanog@nanog.org> Sent: Wednesday, October 14, 2020 3:08:19 PM Subject: Re: Hurricane Electric AS6939 The downside to waves are that they're typically not protected. So a cut will take you down. If you have 10G Layer 2 ethernet, they often will have redundant paths so the only single path that can fail is between you and their first POP where they hopefully have redundancy. It can make a big difference when you're transporting data hundreds or thousands of miles. The longer the path, the less reliable the wave will be as each route mile opens you up to more risk. On Wed, Oct 14, 2020 at 2:25 PM Mike Hammett <nanog@ics-il.net> wrote: I suppose it depends on your carrier and their capabilities. I much prefer waves to any kind of service that you can aggregate. Being able to aggregate just means they're going to oversubscribe you and at some point, you'll not get what you're paying for. Can't do that on a wave. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ________________________________ From: "Eric Kuhnke" <eric.kuhnke@gmail.com> To: "Forrest Christian (List Account)" <lists@packetflux.com> Cc: "nanog list" <nanog@nanog.org> Sent: Wednesday, October 14, 2020 2:25:46 AM Subject: Re: Hurricane Electric AS6939 For small ISPs looking at setting up their first ever presence at an IX point, you almost certainly would not be ordering an actual 'wave' (eg: a specific DWDM channel on a legacy 10G DWDM platform, handed off to you with 1310/LX interfaces at both ends), but lit layer 2 transport service between the carrier hotel and your service location. Pricing for the two types of service can be quite different when you request an actual 'wave' from a carrier sales person, vs just lit L2 transport capable of large MTUs, QinQ, etc. The ISP carrying it might take it between those two places as simply a vlan trunked through a larger 100G link, as a MPLS circuit, lots of possible things. Unless you happened to be in a happy conjunction of the right place at the right time, and an older DWDM system on exactly the same path you wanted happened to have an empty channel and ready to go interface cards at both ends. On Tue, Oct 13, 2020 at 11:12 PM Forrest Christian (List Account) <lists@packetflux.com> wrote: Generally one would order a circuit (aka wave) between your location and the IX fabric at the interchange if you're not at the site you're wanting to peer at. For instance, the network I am the network engineer for has a circuit which terminates into the Seattle IX (SIX) fabric. We don't have any other presence in Seattle (or Washington for that matter) at this point - our circuit connects directly to our port on the Exchange. We're considering adding a similar link to another exchange point somewhere to the east or southeast of us. I haven't looked at the graphs recently, but it's not uncommon for >50% of our traffic to come from the exchange. And yes, we're peered with Hurricane and others there. We're also looking at dropping 1U or so of equipment in so we can pick up some transit as well, but that's a story for a different day about the joys of providing internet in the less populated parts of the country. In your case, it also looks like there are also some peering options at the datacenters you are currently at as well. You may want to do some more research to determine how that might work in your situation. PeeringDB is a good resource along with google searches for "peering 100 Taylor" or "peering austin data foundry" On Tue, Oct 13, 2020 at 9:51 PM <aaron1@gvtc.com> wrote: Don’t you have to be there to join? I’m in Austin and San Antonio -Aaron From: Mike Hammett <nanog@ics-il.net> Sent: Tuesday, October 13, 2020 7:20 PM To: Aaron Gould <aaron1@gvtc.com> Cc: nanog@nanog.org Subject: Re: Hurricane Electric AS6939 https://link.edgepilot.com/s/f4b3d1e1/SU6GGYuH-0auUuW8DzIaKQ?u=https://bgp.h... You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit. DFW has a couple and there are some more that are starting up. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ________________________________ From: "Aaron Gould" <aaron1@gvtc.com> To: nanog@nanog.org Sent: Tuesday, October 13, 2020 6:29:55 PM Subject: Hurricane Electric AS6939 Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent. -Aaron -- - Forrest -- Darin Steffl Minnesota WiFi https://link.edgepilot.com/s/c848137f/Xv9tYbKa3UWwzVzBIHDYGQ?u=http://www.mn... 507-634-WiFi Like us on Facebook -- Darin Steffl Minnesota WiFi https://link.edgepilot.com/s/c848137f/Xv9tYbKa3UWwzVzBIHDYGQ?u=http://www.mn... 507-634-WiFi Like us on Facebook -- Matt Erculiani ERCUL-ARIN Links contained in this email have been replaced. If you click on a link in the email above, the link will be analyzed for known threats. If a known threat is found, you will not be able to proceed to the destination. If suspicious content is detected, you will see a warning.

The inverse of that is that an actual wavelength for 10/100G services can be contractually defined to a certain specific path at OSI layer 1 (with GIS vector shape files from the underlying carrier provided prior to signing a contract). Whereas a layer 2 transport service could also turn out to be unprotected, if it's a particularly low cost service. Or it could be protected. And you might not have the ability to define its path between city A and city B to intentionally avoid being non-diverse from another route. The L2 lit service carrier could re-route it around the region however they want during the term of your service, if the contract isn't written to avoid that. In my experience actual wavelengths such as a carrier might use to transport an STM64 between two places on far sides of a state, even non-protected, will be considerably more expensive than buying lit L2 service. For small ISPs where their entire presence at an IX will fit in one or two 10Gbps circuits, and a 100Gbps circuit from $smalltown to $bigcity_ix_point would be cost prohibitive, it's often the best option. On Wed, Oct 14, 2020 at 1:08 PM Darin Steffl <darin.steffl@mnwifi.com> wrote:

The downside to waves are that they're typically not protected. So a cut will take you down. If you have 10G Layer 2 ethernet, they often will have redundant paths so the only single path that can fail is between you and their first POP where they hopefully have redundancy. It can make a big difference when you're transporting data hundreds or thousands of miles. The longer the path, the less reliable the wave will be as each route mile opens you up to more risk.

On Wed, Oct 14, 2020 at 2:25 PM Mike Hammett <nanog@ics-il.net> wrote:

...

I suppose it depends on your carrier and their capabilities.

I much prefer waves to any kind of service that you can aggregate. Being able to aggregate just means they're going to oversubscribe you and at some point, you'll not get what you're paying for. Can't do that on a wave.

----- Mike Hammett Intelligent Computing Solutions <http://www.ics-il.com/> <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> Midwest Internet Exchange <http://www.midwest-ix.com/> <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> The Brothers WISP <http://www.thebrotherswisp.com/> <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ------------------------------ *From: *"Eric Kuhnke" <eric.kuhnke@gmail.com> *To: *"Forrest Christian (List Account)" <lists@packetflux.com> *Cc: *"nanog list" <nanog@nanog.org> *Sent: *Wednesday, October 14, 2020 2:25:46 AM *Subject: *Re: Hurricane Electric AS6939

For small ISPs looking at setting up their first ever presence at an IX point, you almost certainly would not be ordering an actual 'wave' (eg: a specific DWDM channel on a legacy 10G DWDM platform, handed off to you with 1310/LX interfaces at both ends), but lit layer 2 transport service between the carrier hotel and your service location.

Pricing for the two types of service can be quite different when you request an actual 'wave' from a carrier sales person, vs just lit L2 transport capable of large MTUs, QinQ, etc.

The ISP carrying it might take it between those two places as simply a vlan trunked through a larger 100G link, as a MPLS circuit, lots of possible things.

Unless you happened to be in a happy conjunction of the right place at the right time, and an older DWDM system on exactly the same path you wanted happened to have an empty channel and ready to go interface cards at both ends.

On Tue, Oct 13, 2020 at 11:12 PM Forrest Christian (List Account) < lists@packetflux.com> wrote:

...

Generally one would order a circuit (aka wave) between your location and the IX fabric at the interchange if you're not at the site you're wanting to peer at.

For instance, the network I am the network engineer for has a circuit which terminates into the Seattle IX (SIX) fabric. We don't have any other presence in Seattle (or Washington for that matter) at this point - our circuit connects directly to our port on the Exchange. We're considering adding a similar link to another exchange point somewhere to the east or southeast of us. I haven't looked at the graphs recently, but it's not uncommon for >50% of our traffic to come from the exchange. And yes, we're peered with Hurricane and others there.

We're also looking at dropping 1U or so of equipment in so we can pick up some transit as well, but that's a story for a different day about the joys of providing internet in the less populated parts of the country.

In your case, it also looks like there are also some peering options at the datacenters you are currently at as well. You may want to do some more research to determine how that might work in your situation. PeeringDB is a good resource along with google searches for "peering 100 Taylor" or "peering austin data foundry"

On Tue, Oct 13, 2020 at 9:51 PM <aaron1@gvtc.com> wrote:

...

Don’t you have to be there to join?

I’m in Austin and San Antonio

-Aaron

*From:* Mike Hammett <nanog@ics-il.net> *Sent:* Tuesday, October 13, 2020 7:20 PM *To:* Aaron Gould <aaron1@gvtc.com> *Cc:* nanog@nanog.org *Subject:* Re: Hurricane Electric AS6939

https://bgp.he.net/AS16527

You don't appear to be on any IXes. Definitely join some IXes before buying another 100G of transit.

DFW has a couple and there are some more that are starting up.

----- Mike Hammett Intelligent Computing Solutions <http://www.ics-il.com/> <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> Midwest Internet Exchange <http://www.midwest-ix.com/> <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> The Brothers WISP <http://www.thebrotherswisp.com/> <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ------------------------------

*From: *"Aaron Gould" <aaron1@gvtc.com> *To: *nanog@nanog.org *Sent: *Tuesday, October 13, 2020 6:29:55 PM *Subject: *Hurricane Electric AS6939

Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent.

-Aaron

-- - Forrest

-- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook <http://www.facebook.com/minnesotawifi>

Depending on peer traffic, it may make more sense to co-locate a switch at a DC and then bring in some PNIs from your top usage peers. Check your peer usage using something like AS-Stats. Your top usage peers are going to be your most costly expense in the end... Erich Kaiser The Fusion Network erich@gotfusion.net Office: 815-570-3101 On Wed, Oct 14, 2020 at 9:19 AM Jared Mauch <jared@puck.nether.net> wrote:

...

On Oct 13, 2020, at 7:29 PM, Aaron Gould <aaron1@gvtc.com> wrote:

Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent.

No. Too many problems with scenic routing due to lack of BGP communities to control route advertisements.

Also the usual lack of full table issue due to route suppression on their transits based on AFI.

I’m sure it works well for many people, but if you are large enough you need to do any form of TE the lack of BGP communities is a major issue.

- Jared

If an eyeball network has good peering, is there much value to be gained in chasing the long tail? It certainly presents a different cost perspective if you can pay a couple of the more premium networks for your remaining 15% - 20% of traffic not on the IXes. I just don't know that I'd really worry beyond two more premium transit networks if I was an eyeball that had mostly residential subscribers. What do I mean by premium transit network? Anyone more than $0.10/meg at 10G. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Baldur Norddahl" <baldur.norddahl@gmail.com> To: nanog@nanog.org Sent: Wednesday, October 14, 2020 12:08:12 PM Subject: Re: Hurricane Electric AS6939 On Wed, Oct 14, 2020 at 1:30 AM Aaron Gould < aaron1@gvtc.com > wrote: Do y’all like HE for Internet uplink? I’m thinking about using them for 100gig in Texas. It would be for my eyeballs ISP. We currently have Spectrum, Telia and Cogent. -Aaron I find HE useful as a special kind of transit provider. They have more peerings than anyone and they have peerings that you will not be able to get yourself. They are not necessarily a replacement for connecting directly to local internet exchanges, but rather a supplement. Unless you can afford to have most of the top tier transit providers, you will often get a more direct path through the many peerings of HE. Then as you grow and establish your own peerings, you will move some of that traffic away from HE to your own. One gotcha is that for some unknown reason, some of the big content networks do apparently not peer with HE. However as an eyeball network you will have little trouble getting those directly. Regards, Baldur