I always heard this service was really Layer 3 disguised as Layer 2.

From: NANOG <nanog-bounces+rod.beck=unitedcablecompany.com@nanog.org> on behalf of Ryan Hamel <ryan@rkhtech.org>
Sent: Wednesday, October 14, 2020 7:54 PM
To: Mike Hammett <nanog@ics-il.net>
Cc: nanog@nanog.org <nanog@nanog.org>
Subject: Re: Cogent Layer 2
 
Mike,

Layer 2 is fine once it works.

IMO, if it's 1Gbit or less per circuit and can deal with ^, you're fine, otherwise look for another carrier.

-----

Below is what I got from Cogent about their layer 2:

We offer Ethernet over MPLS transport utilizing Cisco FAT Pseudowire (Flow Aware Transport). Our service is a fully protected service, so if we suffer a fiber cut or other disruption along the primary path, our IS-IS IP fast-reroute enabled MPLS backbone will swing all traffic over to another pre-determined path across our backbone with usually no packet loss or disruption in service.

In order for our service to work correctly and provide the automatic redundancy, we need to verify that the traffic traversing the network can be hashed correctly by our routers. For this to happen, Cogent has to see the src-dst IP address or if you are running MPLS over the circuit, we need to see your MPLS labels. The hashing works by placing each flow of data on a separate 10GE or 100GE interface between the routers, so that traffic is evenly dispersed across all available capacity along the path. A flow is defined as a src-dst IP pair or a customer MPLS label, so the more IP pairs or MPLS labels, the better the traffic load-balances. Cogent has decided to impose a 2Gbps/flow restriction for our own traffic engineering purposes, which aim to make sure that no single customer can overrun a 10GE interface anywhere on our network (since we do not sell 10GE Wave services).

The reason we have the limitation in place is for our own traffic engineering purposes, which aims to make sure that no single customer can overrun a 10GE interface anywhere on our network (since we do not sell 10GE Wave services). Since most uplinks between routers are Nx10GE or Nx100GE, we want to make sure that all customer traffic can be load-balanced across the uplink capacity evenly, which makes it easier to reroute traffic in the event of a fiber cut or other disruption. One would think that with 100GE interfaces, it would not be possible to overrun the interface if we allowed full 10Gbps/flow, however most 100GE interfaces, at the chip level are broken down into 10Gbps lanes and the interfaces do not have a way to easily determine that a lane through the interface is at capacity, so as new flows enter the interface, they could get allocated to a lane that is already full and therefore experience packet loss.

So that we can complete our technical review for this request, need the following questions answered:

1 - What equipment will be directly connected to Cogent interface?

2 - How are the servers/equipment behind the edge device connected, GE or 10GE interfaces?

3 - Will you be doing any type of tunneling or load-balancing that would hide the src-dst IP addresses or MPLS labels of the servers/equipment?

4 - Will any single data flow (src-dst IP pair or MPLS label) be more than 2Gbps?

5 – What is the purpose of the connection? (Internet traffic backhaul, data center connectivity, replication, extending point-of-presence, etc..)

6 – Will you be running MACSec over our L2 service?

7 – Will you need to pass multiple VLANs and/or Jumbo frames?

----------

Ryan
On Oct 14 2020, at 10:36 am, Mike Hammett <nanog@ics-il.net> wrote:
Are any legitimate beefs with Cogent limited to their IP policies, BGP session charges, and peering disputes? Meaning, would using them for layer 2 be reasonable?



-----
Mike Hammett