There’s an eER4 that can do 60km Sent from my iPhone

On Sep 24, 2021, at 2:00 PM, Mauricio Rodriguez via NANOG <nanog@nanog.org> wrote:

 Perhaps a small long-haul OTN platform, supporting FEC, front-ending the JNPR gear?

https://www.fs.com/c/transponder-muxponder-3390

Best Regards, Mauricio Rodriguez Founder / Owner Fletnet Network Engineering (www.fletnet.com) Follow us on LinkedIn

Mauricio.Rodriguez@fletnet.com Office: +1 786-309-1082 Direct: +1 786-309-5493

...

On Fri, Sep 24, 2021 at 4:42 PM Randy Carpenter <rcarpen@network1.net> wrote:

How is everyone accomplishing 100GbE at farther than 40km distances?

Juniper is saying it can't be done with anything they offer, except for a single CFP-based line card that is EOL.

There are QSFP "ZR" modules from third parties, but I am hesitant to try those without there being an equivalent official part.

The application is an ISP upgrading from Nx10G, where one of their fiber paths is ~35km and the other is ~60km.

thanks, -Randy

This message (and any associated files) may contain confidential and/or privileged information. If you are not the intended recipient or authorized to receive this for the intended recipient, you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by sending a reply e-mail and delete this message. Thank you for your cooperation.

Above 40km I like coherent systems with FEC. You can feed the juniper into a pair of SolidOptics 1U appliances Ms. Lady Benjamin PD Cannon of Glencoe, ASCE 6x7 Networks & 6x7 Telecom, LLC CEO lb@6by7.net "The only fully end-to-end encrypted global telecommunications company in the world.” FCC License KJ6FJJ Sent from my iPhone via RFC1149.

On Sep 24, 2021, at 2:35 PM, Edwin Mallette <edwin.mallette@gmail.com> wrote:

I just bite the bullet and use 3rd party optics. It’s easier and once you make the switch, lower cost.

Ed

Sent from my iPhone

...

...

On Sep 25, 2021, at 12:29 AM, Joe Freeman <joe@netbyjoe.com> wrote:

 Open Line Systems can get you to 80K with a 100G DWDM Optic (PAM4) -

I've used a lot of SmartOptics DCP-M40 shelves for this purpose. They also have transponders that allow you to go from a QSFP28 to CFP to do coherent 100G out to 120Km using the DCP-M40, without a need for regen or extra amps in line.

The DCP-M40 is a 1RU box. It looks like a deep 40ch DWDM filter but includes a VAO, EDFA amp, and a WSS I think.

...

On Fri, Sep 24, 2021 at 4:40 PM Randy Carpenter <rcarpen@network1.net> wrote:

How is everyone accomplishing 100GbE at farther than 40km distances?

Juniper is saying it can't be done with anything they offer, except for a single CFP-based line card that is EOL.

There are QSFP "ZR" modules from third parties, but I am hesitant to try those without there being an equivalent official part.

The application is an ISP upgrading from Nx10G, where one of their fiber paths is ~35km and the other is ~60km.

thanks, -Randy

If you can’t wait for Juniper to release their supported QSFP28 100G-ZR optic, shop for third party 100G-ZR optics. I know many networks are already using third party QSFP28 100G-ZR optics in Juniper routers. I have one 80 km span between two MX204 routers using third party 100G-ZR optics with no issues. As long as the optic doesn’t draw too much power or generate too much heat it should be good. From: NANOG <nanog-bounces+smkarp=galliumnetworks.com@nanog.org> on behalf of Joe Freeman <joe@netbyjoe.com> Date: Friday, September 24, 2021 at 4:30 PM To: Randy Carpenter <rcarpen@network1.net> Cc: nanog <nanog@nanog.org> Subject: Re: 100GbE beyond 40km Open Line Systems can get you to 80K with a 100G DWDM Optic (PAM4) - I've used a lot of SmartOptics DCP-M40 shelves for this purpose. They also have transponders that allow you to go from a QSFP28 to CFP to do coherent 100G out to 120Km using the DCP-M40, without a need for regen or extra amps in line. The DCP-M40 is a 1RU box. It looks like a deep 40ch DWDM filter but includes a VAO, EDFA amp, and a WSS I think. On Fri, Sep 24, 2021 at 4:40 PM Randy Carpenter <rcarpen@network1.net<mailto:rcarpen@network1.net>> wrote: How is everyone accomplishing 100GbE at farther than 40km distances? Juniper is saying it can't be done with anything they offer, except for a single CFP-based line card that is EOL. There are QSFP "ZR" modules from third parties, but I am hesitant to try those without there being an equivalent official part. The application is an ISP upgrading from Nx10G, where one of their fiber paths is ~35km and the other is ~60km. thanks, -Randy

Kevin, What do these types of boxes cost? Never heard of them, but their tech looks neat. On Mon, Sep 27, 2021 at 1:54 PM Kevin Menzel via NANOG <nanog@nanog.org> wrote:

We’ve had DCP-M boxes in service for a few years now on our +40km links, with their PAM4 100Gb optics. It’s been SO easy.

If I had the money, we’d throw DCP-M boxes at every link we have.

-Kevin Menzel

From: NANOG <nanog-bounces+kevin.menzel=sheridancollege.ca@nanog.org> On Behalf Of Joe Freeman Sent: September 24, 2021 17:07 To: Randy Carpenter <rcarpen@network1.net> Cc: nanog <nanog@nanog.org> Subject: Re: 100GbE beyond 40km

This message was sent from outside of Sheridan College. Please be careful when opening attachments, clicking links, or responding to requests for information.

Open Line Systems can get you to 80K with a 100G DWDM Optic (PAM4) -

I've used a lot of SmartOptics DCP-M40 shelves for this purpose. They also have transponders that allow you to go from a QSFP28 to CFP to do coherent 100G out to 120Km using the DCP-M40, without a need for regen or extra amps in line.

The DCP-M40 is a 1RU box. It looks like a deep 40ch DWDM filter but includes a VAO, EDFA amp, and a WSS I think.

On Fri, Sep 24, 2021 at 4:40 PM Randy Carpenter <rcarpen@network1.net> wrote:

How is everyone accomplishing 100GbE at farther than 40km distances?

Juniper is saying it can't be done with anything they offer, except for a single CFP-based line card that is EOL.

There are QSFP "ZR" modules from third parties, but I am hesitant to try those without there being an equivalent official part.

The application is an ISP upgrading from Nx10G, where one of their fiber paths is ~35km and the other is ~60km.

thanks, -Randy

Looking at EDFA options... they are all ~1500nm as far as I can tell. Is there a specific model you are talking about?

Yeah, that is a consequence of how the EDFA technology works. It really only works in the C-band and sometimes the L-band, depending on how it's manufactured. If you contact one of the manufacturers on this list they should be able to help you with solutions engineering. You would essentially be looking at a 10x 10G LAG *or* a single 100G channelized optic (which are a thing, just pay attention to how they achieve that, and that it runs at 100GHz) There are other amplifier systems, like the one Brandon mentioned, which operates via another mechanism. Personally, I would recommend the EDFA system for two reasons, it's more commoditized, and depending on the model you purchase, you will have some. spare DWDM channels. You might not need it today, but man is it nice to have that extra capacity sitting there. If you do go for a super extended range optic (like what Brandon mentioned), make sure the pluggable power draw is within range of your gear. No point in burning out an expensive optic and your gear. I could go on about chromatic dispersion for days, I think that is so interesting, but a little outside the scope of this chain. Whatever manufacturer you engage with will help with that. Thankfully this tech has gotten to the point where we can just slap that module in and not worry about what is actually going on behind the scenes. - Dan On Mon, Sep 27, 2021 at 3:43 PM Pierre LANCASTRE <pierre.lancastre@gmail.com> wrote:

Hi

+1 for SmartOptics products. Got M-1601 + optics, did the job very well

Pierre

Le lun. 27 sept. 2021 à 20:52, Kevin Menzel via NANOG <nanog@nanog.org> a écrit :

...

We’ve had DCP-M boxes in service for a few years now on our +40km links, with their PAM4 100Gb optics. It’s been SO easy.

If I had the money, we’d throw DCP-M boxes at every link we have.

-Kevin Menzel

*From:* NANOG <nanog-bounces+kevin.menzel=sheridancollege.ca@nanog.org> *On Behalf Of *Joe Freeman *Sent:* September 24, 2021 17:07 *To:* Randy Carpenter <rcarpen@network1.net> *Cc:* nanog <nanog@nanog.org> *Subject:* Re: 100GbE beyond 40km

This message was sent from outside of Sheridan College. Please be careful when opening attachments, clicking links, or responding to requests for information.

Open Line Systems can get you to 80K with a 100G DWDM Optic (PAM4) -

I've used a lot of SmartOptics DCP-M40 shelves for this purpose. They also have transponders that allow you to go from a QSFP28 to CFP to do coherent 100G out to 120Km using the DCP-M40, without a need for regen or extra amps in line.

The DCP-M40 is a 1RU box. It looks like a deep 40ch DWDM filter but includes a VAO, EDFA amp, and a WSS I think.

On Fri, Sep 24, 2021 at 4:40 PM Randy Carpenter <rcarpen@network1.net> wrote:

How is everyone accomplishing 100GbE at farther than 40km distances?

Juniper is saying it can't be done with anything they offer, except for a single CFP-based line card that is EOL.

There are QSFP "ZR" modules from third parties, but I am hesitant to try those without there being an equivalent official part.

The application is an ISP upgrading from Nx10G, where one of their fiber paths is ~35km and the other is ~60km.

thanks, -Randy

-- Daniel Murphy Senior Data Center Engineer (646) 698-8018

Bear with my ignorance, I'm genuinely surprised at this: Does this have to be Ethernet? You could look into line gear with coherent

optics.

Specifically, do you mean something like: "does this have to be IEEE-standardized all the way down to L1 optics?" Because you can transmit Ethernet frames over line gear with coherent optics, right ? Please don't flame me, I'm just ignorant and willing to learn. Cheers, Etienne On Fri, Sep 24, 2021 at 11:25 PM Bill Blackford <bblackford@gmail.com> wrote:

Does this have to be Ethernet? You could look into line gear with coherent optics. IIRC, they have built-in chromatic dispersion compensation, and depending on the card, would include amplification.

On Fri, Sep 24, 2021 at 1:40 PM Randy Carpenter <rcarpen@network1.net> wrote:

...

How is everyone accomplishing 100GbE at farther than 40km distances?

Juniper is saying it can't be done with anything they offer, except for a single CFP-based line card that is EOL.

There are QSFP "ZR" modules from third parties, but I am hesitant to try those without there being an equivalent official part.

The application is an ISP upgrading from Nx10G, where one of their fiber paths is ~35km and the other is ~60km.

thanks, -Randy

-- Bill Blackford

Logged into reality and abusing my sudo privileges.....

-- Ing. Etienne-Victor Depasquale Assistant Lecturer Department of Communications & Computer Engineering Faculty of Information & Communication Technology University of Malta Web. https://www.um.edu.mt/profile/etiennedepasquale

Are you saying we could use normal QSFP28 LR4 or ER4 modules with an amplifier in between? Yes, that is the idea from 30,000 ft. Fun fact, the ER4 optics you mention are amplified inside the pluggable in a very similar manner to how these EDFA systems work.

Basically: QSFP28 100G ER <-> EDFA Amp <-> OSP/dark fiber <-> EDFA Amp <-> QSFP28 100G ER Very simple, and from the Juniper gear's POV, there is no funny business. All the magic happens down at layer 0. The systems are commoditized and pretty easy to find. I saw a few people on this thread mention Solid Optics, personally I have not heard of them, but I would trust LB's recommendation. I've used systems by other manufacturers in the past and wasn't crazy about them. I don't want to flame that manufacturer since they read this mailer, and who knows, the issues I saw might have been isolated to manufacturing issues, but I still wouldn't recommend them. The learning curve is pretty low, and the manufacturers of this gear are ~usually~ very eager to guide basic implementation. However, ping me off list, or on here, if you have any deeper questions about this. Have a good week everyone! On Sun, Sep 26, 2021 at 12:17 AM Lady Benjamin Cannon <lb@6by7.net> wrote:

My guess is that he was talking about the difference between a 100gbit/sec stream of ethernet frames with no error correction, and a 112gbit/sec (or so, depending on scheme) stream of transport with FEC (Forward Error Correction - which is essentially just cramming extra bits in there incase they are needed.

Ethernet has to re-transmit instead, and that can cause performance degradation and jitter, until it just quits working altogether. Systems implementing FEC are much

(This is a guess, there’s a chance something else was meant by this)

-LB.

On Sep 25, 2021, at 1:55 AM, Etienne-Victor Depasquale via NANOG < nanog@nanog.org> wrote:

Bear with my ignorance, I'm genuinely surprised at this:

Does this have to be Ethernet? You could look into line gear with coherent

...

optics.

Specifically, do you mean something like: "does this have to be IEEE-standardized all the way down to L1 optics?" Because you can transmit Ethernet frames over line gear with coherent optics, right ?

Please don't flame me, I'm just ignorant and willing to learn.

Cheers,

Etienne

On Fri, Sep 24, 2021 at 11:25 PM Bill Blackford <bblackford@gmail.com> wrote:

...

Does this have to be Ethernet? You could look into line gear with coherent optics. IIRC, they have built-in chromatic dispersion compensation, and depending on the card, would include amplification.

On Fri, Sep 24, 2021 at 1:40 PM Randy Carpenter <rcarpen@network1.net> wrote:

...

How is everyone accomplishing 100GbE at farther than 40km distances?

Juniper is saying it can't be done with anything they offer, except for a single CFP-based line card that is EOL.

There are QSFP "ZR" modules from third parties, but I am hesitant to try those without there being an equivalent official part.

The application is an ISP upgrading from Nx10G, where one of their fiber paths is ~35km and the other is ~60km.

thanks, -Randy

-- Bill Blackford

Logged into reality and abusing my sudo privileges.....

-- Ing. Etienne-Victor Depasquale Assistant Lecturer Department of Communications & Computer Engineering Faculty of Information & Communication Technology University of Malta Web. https://www.um.edu.mt/profile/etiennedepasquale

-- Daniel Murphy Senior Data Center Engineer (646) 698-8018

There is a cryo folder on c:\ which contains all the tables and so forth I guess we coukd zip that up, I very much doubt I can re-add the machine to xp but i will give it a shot no harm if its only for a rdp session. cryo as in cryochamber Sent via BT Email App From: Randy Carpenter <rcarpen@network1.net> Sent: 27 September 2021 17:40:19 BST To: Dan Murphy <dmurphy@pilotfiber.com> Cc: nanog <nanog@nanog.org> Subject: Re: 100GbE beyond 40km Looking at EDFA options... they are all ~1500nm as far as I can tell. Is there a specific model you are talking about? thanks, -Randy ----- On Sep 27, 2021, at 10:25 AM, Dan Murphy <dmurphy@pilotfiber.com> wrote:

Are you saying we could use normal QSFP28 LR4 or ER4 modules with an amplifier in between? Yes, that is the idea from 30,000 ft. Fun fact, the ER4 optics you mention are amplified inside the pluggable in a very similar manner to how these EDFA systems work.

Basically: QSFP28 100G ER <-> EDFA Amp <-> OSP/dark fiber <-> EDFA Amp <-> QSFP28 100G ER Very simple, and from the Juniper gear's POV, there is no funny business. All the magic happens down at layer 0. The systems are commoditized and pretty easy to find. I saw a few people on this thread mention Solid Optics, personally I have not heard of them, but I would trust LB's recommendation. I've used systems by other manufacturers in the past and wasn't crazy about them. I don't want to flame that manufacturer since they read this mailer, and who knows, the issues I saw might have been isolated to manufacturing issues, but I still wouldn't recommend them. The learning curve is pretty low, and the manufacturers of this gear are ~usually~ very eager to guide basic implementation. However, ping me off list, or on here, if you have any deeper questions about this. Have a good week everyone! On Sun, Sep 26, 2021 at 12:17 AM Lady Benjamin Cannon <lb@6by7.net <mailto:lb@6by7.net> > wrote: My guess is that he was talking about the difference between a 100gbit/sec stream of ethernet frames with no error correction, and a 112gbit/sec (or so, depending on scheme) stream of transport with FEC (Forward Error Correction - which is essentially just cramming extra bits in there incase they are needed. Ethernet has to re-transmit instead, and that can cause performance degradation and jitter, until it just quits working altogether.   Systems implementing FEC are much  (This is a guess, there’s a chance something else was meant by this) -LB. On Sep 25, 2021, at 1:55 AM, Etienne-Victor Depasquale via NANOG <nanog@nanog.org <mailto:nanog@nanog.org> > wrote: Bear with my ignorance, I'm genuinely surprised at this: Does this have to be Ethernet? You could look into line gear with coherent optics. Specifically, do you mean something like: "does this have to be IEEE-standardized all the way down to L1 optics?" Because you can transmit Ethernet frames over line gear with coherent optics, right ? Please don't flame me, I'm just ignorant and willing to learn. Cheers, Etienne On Fri, Sep 24, 2021 at 11:25 PM Bill Blackford <bblackford@gmail.com <mailto:bblackford@gmail.com> > wrote: Does this have to be Ethernet? You could look into line gear with coherent optics. IIRC, they have built-in chromatic dispersion compensation, and depending on the card, would include amplification. On Fri, Sep 24, 2021 at 1:40 PM Randy Carpenter <rcarpen@network1.net <mailto:rcarpen@network1.net> > wrote: How is everyone accomplishing 100GbE at farther than 40km distances? Juniper is saying it can't be done with anything they offer, except for a single CFP-based line card that is EOL. There are QSFP "ZR" modules from third parties, but I am hesitant to try those without there being an equivalent official part. The application is an ISP upgrading from Nx10G, where one of their fiber paths is ~35km and the other is ~60km. thanks, -Randy -- Bill Blackford Logged into reality and abusing my sudo privileges..... -- Ing. Etienne-Victor Depasquale Assistant Lecturer Department of Communications & Computer Engineering Faculty of Information & Communication Technology University of Malta Web. https://www.um.edu.mt/profile/etiennedepasquale <https://www.um.edu.mt/profile/etiennedepasquale> -- Daniel Murphy Senior Data Center Engineer (646) 698-8018

On 9/27/21 1:47 PM, Brandon Butterworth wrote:

You're looking for SOA at 1300nm, like https://www.fs.com/uk/products/69350.html

Getting much more power out of a SOA than a -ZR QSFP28 is pretty hard, though they could be used for non-OEO re-generation in the middle if practical in your topology. There does also exist a PDFA. Same concept, different dopant in the fiber. They are...expensive. However, you can get a crazy amount of power out of them, and they're available with a lower noise floor than SOAs seem to be, as well. If you really want to make that 80-100km link at 1310nm and without going coherent, they are a potential option. When I inquired with a manufacturer/rep (what appears to be the only one in the world), it was almost as expensive as a coherent transponder on both ends. -- Brandon Martin