> Putting the smart devices on the edge allows for a much-simplified core topology.

>> Putting smart devices in the edge does simplify the network, yes. What doesn't is making the customer's site part of your edge.

If the customer's site goes offline, that is their problem. A CPE device is still a CPE device, no matter how smart it is. Setup IS-IS, BGP to route servers, LDP + MPLS if you don't go the VXLAN route, and that's it. I know Ciena's can do that on their more expensive 39xx models.

>> We've been running MPLS all the way into the access since 2009 (Cisco ME3600X/3800X). It is simpler than running an 802.1Q or Q-in-Q Metro-E backbone, and scales very well. Just leave your customers out of it.

There are a few tier 1's that have delivered Ethernet transport circuits on those exact boxes in the field as I speak. It works very well.

I also agree with your stance on Broadcom, it's hard to come up with alternatives that are not ADVA/Ciena/Cisco/RAD.

Ryan Hamel

From: NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org> on behalf of Mark Tinka <mark@tinka.africa>
Sent: Wednesday, June 14, 2023 10:30 PM
To: nanog@nanog.org <nanog@nanog.org>
Subject: Re: 10G CPE w/VXLAN - vendors?
 
Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments.


On 6/14/23 22:04, Ryan Hamel wrote:

> Putting the smart devices on the edge allows for a much-simplified
> core topology.

Putting smart devices in the edge does simplify the network, yes. What
doesn't is making the customer's site part of your edge.

We've been running MPLS all the way into the access since 2009 (Cisco
ME3600X/3800X). It is simpler than running an 802.1Q or Q-in-Q Metro-E
backbone, and scales very well. Just leave your customers out of it.


>
> Either way, I was doing research on FPGA-based hardware a couple of
> weeks agoand came across this which may tick all the boxes.
> https://ethernitynet.com/products/enet-network-appliances/uep-60/ I do
> not know the vendor personally and have not worked on their hardware,
> so your mileage may vary.

There aren't a great deal of options in this space, unfortunately. What
is making it worse is most traditional vendors are relegating devices
designed for this to Broadcom chips, which is a problem because the
closer you get to the customer, the more you need to "touch" their
packets, and Broadcom chips, while fast and cheap, aren't terribly good
at working with packets in the way the customers these devices need to
address would like.

Cisco's ASR920 is still, by far, the best option here. Unfortunately, it
has a very small FIB, does not do 10Gbps at any scale, and certainly
does not 100Gbps. But, because most customers tend to run only p2p
EoMPLS services on it (that doesn't require any large FIB), the box is
still actively sold by Cisco even though in Internet years, it is older
than my grandfather's tobacco pipe.

Juniper are pushing their ACX7024, which we are looking at as a viable
option for replacing the ASR920. However, it's Broadcom... and while
Nokia's Broadcom option for the Metro-E network is using the same chip
as the Juniper one, they seem lazier to be more creative with how they
can touch customer packets vs. Juniper.

Cisco's recommended upgrade path is the NCS540, also a Broadcom box; the
heaviness that is IOS XR in a large scale deployment area like the
Metro-E backbone notwithstanding. The rumour is that Cisco want to
optimize Silicon One for their entire routing & switching range, small
and large. I'll believe it when I see it. Until then, I wouldn't touch
the NCS540.

Vendors are trying to do the least in the Metro-E space, knowing full
well how high the margins are. It's a bit disingenuous, considering they
will be shipping more Metro-E routers to customers than core or edge
routers. But, it is what it is.

Mark.