I'm currently searching out options for providing carrier ethernet demarc at 10Gbps and 25Gbps port speeds to evaluate for some upcoming projects. 10Gbps options abound, but it seems like the next step up with most vendors is 100Gb handoff. I'm not having much luck finding 25Gbps options other than ADVA/Adtran FSP 150-XJ128 and FSP 150-XG404 variants. Currently the majority of our customers are PON based (Calix), and we have non-PON handoffs with smaller Juniper switches as the demarc, so we're looking to step up to something actually designed for this purpose. We'd like to be able to perform y.1564 testing and use MEF compliant devices. I'm reaching out to some vendors/resellers to see what options they can provide, but I'm interested in hearing real-world experiences with different vendors in this space and especially any recommendations on the 25Gb demarc, as we have had some specific requests for that port speed recently. Feel free to reach out off list if that's your preference. John Stitt
While options exist such as the Cisco 9500 series mentioned by Fredy, for 25 Gbps, I don't know that it has ever caught on for WAN purposes to find its way into carrier demarc equipment. as a common optio. Or as a service tier that major lit service last-mile providers are interested in providing their sales team as a standard product to sell. The bulk of the equipment and use case I've seen for it is as a LAN top of rack switch for server/equipment aggregation, or similar, where it would have previously been a 48-port 10 Gbps SFP+ based switch in the same place in the past. Such as for very price sensitive 'enterprise' customers that have outgrown 10 Gbps but don't have the need or want to spend the money to go to 100 Gbps to every server. On Mon, Nov 25, 2024 at 10:21 AM John Stitt <jstitt@hop-electric.com> wrote:
I’m currently searching out options for providing carrier ethernet demarc at 10Gbps and 25Gbps port speeds to evaluate for some upcoming projects. 10Gbps options abound, but it seems like the next step up with most vendors is 100Gb handoff. I’m not having much luck finding 25Gbps options other than ADVA/Adtran FSP 150-XJ128 and FSP 150-XG404 variants.
Currently the majority of our customers are PON based (Calix), and we have non-PON handoffs with smaller Juniper switches as the demarc, so we’re looking to step up to something actually designed for this purpose. We’d like to be able to perform y.1564 testing and use MEF compliant devices.
I’m reaching out to some vendors/resellers to see what options they can provide, but I’m interested in hearing real-world experiences with different vendors in this space and especially any recommendations on the 25Gb demarc, as we have had some specific requests for that port speed recently.
Feel free to reach out off list if that’s your preference.
John Stitt
On 11/26/24 02:15, Jesse DuPont wrote:
Have you looked at whitebox/disaggregated solutions? For example, IP Infusion's OcNOS running on UfiSpace hardware (something like S9500-22XST) will provide the port speeds you're looking for, MEF compliance, and Sync E if you need it.
1U routers that support 25G have been on the market for a little while now. I don't think this is the issue. What has been lacking is a Layer 2 25G demarc. box, and Adva are just about the only vendor who have had one for some time, as the OP mentioned. Outside the data centre, the primary customers for 25G has been mobile operators. With 100G-ZR+ coherent now starting to ship from next month, I suspect this will put 25G RAN backhaul under a bit of pressure. I do think that for Metro-E services, there is a use-case for 25G, especially now that cheap, Broadcom-based routers are shipping with 25G ports. I'm just not sure plenty of MEF-compliant Layer 2 demarc. options exist outside of the Adva number. Mark.
I appreciate all the responses so far. Indeed, the customer handoff side of the equation is where I’ve been having trouble finding options. The Cisco (formerly Accedian) solution was one I hadn’t seen in my searching, so that at least gives me one competitive option to the Adva. From what I’ve found and what I’m hearing, I think everyone is correct that much like 40Gb, there just hasn’t been as much call for 25Gb handoff at customer demarc outside of the datacenter instead of jumping up to 100Gb ports. Before this search I hadn’t looked hard at 25Gb for anything other than in our datacenter, so I didn’t realize that also like 40Gb, ZR options in 25Gb were relatively new and still not common. Looking at higher speed PON options is an interesting thought, but I imagine it’d be a hard sell for this current use case as they expect dedicated fiber, not that we couldn’t use PON with each site being a home run back to the aggregation site, but that just feels wrong even if it would technically work, haha. I may look into that route a bit more just to see how it might stack up cost and feature wise, and it may be that we decide to roll out higher speed PON for other endeavors from that research. John Stitt Sent from my pocket CRAY-1 On Nov 25, 2024, at 9:30 PM, Mark Tinka <mark@tinka.africa> wrote: On 11/26/24 02:15, Jesse DuPont wrote: Have you looked at whitebox/disaggregated solutions? For example, IP Infusion's OcNOS running on UfiSpace hardware (something like S9500-22XST) will provide the port speeds you're looking for, MEF compliance, and Sync E if you need it. 1U routers that support 25G have been on the market for a little while now. I don't think this is the issue. What has been lacking is a Layer 2 25G demarc. box, and Adva are just about the only vendor who have had one for some time, as the OP mentioned. Outside the data centre, the primary customers for 25G has been mobile operators. With 100G-ZR+ coherent now starting to ship from next month, I suspect this will put 25G RAN backhaul under a bit of pressure. I do think that for Metro-E services, there is a use-case for 25G, especially now that cheap, Broadcom-based routers are shipping with 25G ports. I'm just not sure plenty of MEF-compliant Layer 2 demarc. options exist outside of the Adva number. Mark. CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you are not expecting this message contact the sender directly via phone/text to verify.
On 11/26/24 06:00, John Stitt wrote:
From what I’ve found and what I’m hearing, I think everyone is correct that much like 40Gb, there just hasn’t been as much call for 25Gb handoff at customer demarc outside of the datacenter instead of jumping up to 100Gb ports. Before this search I hadn’t looked hard at 25Gb for anything other than in our datacenter, so I didn’t realize that also like 40Gb, ZR options in 25Gb were relatively new and still not common.
I think the issue here is maximizing the fibre expense, especially if it's 10km or longer. If the cost of 25G, 40G and 100G is all within the same ballpark, 100G will make the most sense. But getting 100G beyond 10km without amplification and dispersion compensation means you will need to go coherent. NRZ-based 100G-ZR4 says it will do 80km, but real life has shown things can be vastly different, primarily due to generally poor fibre characteristics.
Looking at higher speed PON options is an interesting thought, but I imagine it’d be a hard sell for this current use case as they expect dedicated fiber, not that we couldn’t use PON with each site being a home run back to the aggregation site, but that just feels wrong even if it would technically work, haha. I may look into that route a bit more just to see how it might stack up cost and feature wise, and it may be that we decide to roll out higher speed PON for other endeavors from that research.
In reality, if you are an operator that builds both Active-E and GPON, your sales people will slowly push you toward selling FTTB services over GPON, because you get to pass more buildings faster and cheaper. Watch out for that if you are not keen on "just closing sales" :-). That said, you might find OpenXR's latest work exciting for GPON. That group has put out a 400G coherent plug that uses digital subcarriers (wavelengths nested within a wavelength) to deliver 16x 25G channels in a p2mp topology, right in the plug itself. It can run both in normal or BiDi mode, and certainly makes future GPON and Active-E p2mp architectures very interesting. Infinera are the leading OEM that have put product out, but expect Acacia, Coherent and others to do the same soon: https://www.infinera.com/products/ice-x-400g-xr/ Mark.
On 26.11.2024 05:31, Mark Tinka wrote:
But getting 100G beyond 10km without amplification and dispersion compensation means you will need to go coherent.
NRZ-based 100G-ZR4 says it will do 80km, but real life has shown things can be vastly different, primarily due to generally poor fibre characteristics.
I agree on the struggle with bad fiber, and we've had to use 100G ER4 "Lite" on links around 10km before too. But I think you are a little too pessimistic with putting the limit of what NRZ can do at 10km. From my experience 30km is also possible with NRZ. We have a few ER4 and ER4 "Lite", that is Flexoptix Q.161HG.40 and Q.161HG.25, running between cities, with one span being a bit over 30km. That one is muxed parallel to our DWDM, so take an additional 2.5dB for our filters.
That said, you might find OpenXR's latest work exciting for GPON. That group has put out a 400G coherent plug that uses digital subcarriers (wavelengths nested within a wavelength) to deliver 16x 25G channels in a p2mp topology, right in the plug itself. It can run both in normal or BiDi mode, and certainly makes future GPON and Active-E p2mp architectures very interesting.
That's very interesting, even if we don't use PON. I hadn't heard of this before. On the face of it, it sounds expensive to have to use a 400G capable coherent receiver for a 25G customer. But I'm curious to see how the prices shake out in the end, with potential mass adoption. Best, Joel -- Joel Busch, Network SWITCH Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
On 11/29/24 12:46, Joel Busch via NANOG wrote:
I agree on the struggle with bad fiber, and we've had to use 100G ER4 "Lite" on links around 10km before too. But I think you are a little too pessimistic with putting the limit of what NRZ can do at 10km.
Not at all. NRZ is resilient at those distances. I was referring more to PAM4... should have made that clearer :-).
From my experience 30km is also possible with NRZ. We have a few ER4 and ER4 "Lite", that is Flexoptix Q.161HG.40 and Q.161HG.25, running between cities, with one span being a bit over 30km. That one is muxed parallel to our DWDM, so take an additional 2.5dB for our filters.
Yep, NRZ will be fine at those distances. I have ran a 100G-ZR4 NRZ plug on a 32km link successfully, where the 100G-ER4 struggled, due to the fibre quality. But those are grey plugs, which is why to get a cheap DWDM-based PAM4 option means you will need amplification and DCM to get beyond 10km. Either that, or go coherent.
That's very interesting, even if we don't use PON.
PON is not the primary use-case for OpenXR plugs, it just happens to line up with the technology.
I hadn't heard of this before. On the face of it, it sounds expensive to have to use a 400G capable coherent receiver for a 25G customer.
Not necessarily. The 25G is not fixed. You can configure the rate of each sub-carrier and balance capacity with channel count. But the killer app is being able to terminate multiple high-capacity customers on a single uplink port in a p2mp fashion. This was initially envisaged as a way to save optical operators burning expensive transponder ports. But I could see a use-case for router/switch operators as well, in the Access layer. The second killer app is its BiDi capability, which will deliver a 200G channel on single-core fibre at the kinds of distances only coherent can support without DCM. Prior to this, nearly all coherent BiDi plugs came in a CFP2 form factor, which is bulky and pricey. So having a coherent BiDi plug in a QSFP-DD form factor is great news!
But I'm curious to see how the prices shake out in the end, with potential mass adoption.
Initial price is about US$5,000, which is inline with regular QSFP-DD 400G coherent plugs. I expect these prices to come down toward the end of Q4'25 or early Q1'26, when 800G-ZR+ coherent plugs ship in more batches. Mark.
participants (5)
-
Eric Kuhnke -
Jesse DuPont -
Joel Busch -
John Stitt -
Mark Tinka