ADVA recently launched a QSFP+ transceiver with bidi support on each of its 4x10G breakout lanes: https://www.adva.com/en/newsroom/press-releases/20220308-adva-launches-new-b... As for 10G DWDM optics, it's not a very efficient way to use your ports, but you could hypothetically use a QSFP+ to SFP+ adapter like this one if you truly needed to run some in your MX304 chassis: https://www.flexoptix.net/en/transceiver/q-pct.html Best regards, Martijn ________________________________ From: NANOG <nanog-bounces+martijnschmidt=i3d.net@nanog.org> on behalf of tim@pelican.org <tim@pelican.org> Sent: 14 June 2022 17:07 To: nanog@nanog.org <nanog@nanog.org> Subject: RE: Serious Juniper Hardware EoL Announcements

The MX204 is pure shocker! Unless the MX304 will come with a license-based approach to run at MX204 pricing, that is Juniper shooting themselves in the foot.

Unless I'm missing a trick, the MX304 doesn't have an answer to installing DWDM, bidi, or other fancy optics in the SPF+ ports on the MX204. QSFP+ breakout to 4 x 10G is supported, but only 4 x vanilla 1310 optics - you'll need an external OEO solution if you want fancy 10G options. It otherwise seems a nice box on paper, although substantially more expensive than the MX204. Cheers, Tim.

On Tue, 14 Jun 2022 at 21:42, Eric Kuhnke <eric.kuhnke@gmail.com> wrote:

I think the more common solution for something like that would be to use one 100GbE port as a trunk on a MX204 or MX304 to a directly adjacent 1U 48-port SFP+ switch in a purely L2 role used as a port expander, with dwdm/bidi/other unique types of SFP+ optics inserted in that.

It is disappointing that we are getting faceplates that are exclusively cloud optimised, and service providers are scratching their heads going 'how can i use this?'. But it may be that there simply isn't a business case to build models with different faceplates or to design yet another set of linecards. Of course the fab doesn't charge different costs for different Trio, we from cost POV, the chips always cost the ~same, if it's MX80 or MX304 (except MX304 has up-to three of them). So there isn't any real reason why you couldn't massively underutilize the chips and deliver faceplates that are optimised for different use-cases. However, JNPR does see ACX more for this role. Now VLAN aggregation isn't without its problems: a) termination router must be able to do QoS under shaper, you need to shape every VLAN to access rate and then QoS inside the shaper. There are a lot of problems here, and even if the termination router does support proper HQOS it may not support small enough burst values that the access can handle. b) you lose link state information at termination, and you need to either accept slower convergence (e.g. no BGP external fast fall over) or investigate into CFM or BFD, where BFD would require active participation from customer, which is usually not reasonable ask c) your pseudowire products will become worse, you may have MAC learning (you might be able to turn it off) limiting MAC scale, you will likely eat bunch of new frames which previously were passed, you may be able to fix it with L2PT (rewrite MAC on L2 ingress, rewrite MAC on L2 egress). And some things might become outright impossible, for example paradise chipset will drop ISIS packets with VLAN headers on the floor (technically impossible to have 802.3 and VLAN), so if your termination is paradise, your pseudowire customers can't use ISIS. d) most L2 devices have exceedingly small buffers and this solution implies many=>one traffic flow, so you're going to have to understand what amount of buffering you're going to need and how many ports you can attach there e) provisioning and monitoring complexity, as you need to have model where you decouple termination and access port, if you don't already do this, it can be quite complicated to add, there are number of complexities like how to associate these two ports for data collection and rendering, where and how to draw vlans f) if you dual attach the L2 aggregation you can create loops due to simple and complex reasons, termination may not have per-VLAN MAC filter, so adding 1 pseudowire VLAN, may disable MAC filtering for whole interface. And if you run default MAC/ARP timers (misconfig, defaults are always wrong, ARP needs to be lower than MAC, but this is universally not understood), primaryPE maybe send packets to L3 address which is in ARP, but not in MAC anymore (host down?), backupPE will receive it due to lack of MAC filtering and will forward to primaryPE, which will forward back to L2. This was just what immediately occurred to me, I'm sure I could remember more issues if I'd spent a bit more time thinking of it. L2 is hard, be it L2 LAN or L2 aggregation. And almost invariably incorrectly configured, as L2 stuff apparently works usually out-of-the-box, but is full of bees. Now the common solution vendors offer to address many of these problems is 'satellite', where the vendor takes HW and SW workarounds to reduce the problems caused by VLAN aggregation. Unfortunately the satellite story is as well regressing, Cisco doesn't have it for Cisco8k, Juniper wants to kill Fusion. Nokia and Huawei still seem to have love for provider faceplates. -- ++ytti

When I last got pricing on the MX10003 in fall 2021, I was asked if I wanted pricing on something with exclusively 100GbE interfaces or with 10GbE capability. I got pricing for both options. Putting SFP+ 10GbE ports in a router of that total chassis+RE+linecard+support contract price is an *extremely* costly proposition on a dollar per port basis. Would recommend that anyone who thinks they need them to look at ways to put the 10GbE ports in some other device and attach that to the router. On Tue, 14 Jun 2022 at 14:09, Brian <brian.bsi@gmail.com> wrote:

From Juniper...........

"you are correct that there isn’t a native 10G SFP+ form factor offered with the 304.

QSA adapters are qualified (say for example, Nvidia), and from there it can support native 10G Bidi, WDM, etc. The economics per-port, obviously, get a little expensive with this approach if a lot of native 10G is needed and breakout isn’t an option.

Thanks!"

Oh and also I just got the call, Juniper is forcing a Price Hike in July.

On Tue, Jun 14, 2022 at 9:10 AM tim@pelican.org <tim@pelican.org> wrote:

...

...

The MX204 is pure shocker! Unless the MX304 will come with a license-based approach to run at MX204 pricing, that is Juniper shooting themselves in the foot.

Unless I'm missing a trick, the MX304 doesn't have an answer to installing DWDM, bidi, or other fancy optics in the SPF+ ports on the MX204. QSFP+ breakout to 4 x 10G is supported, but only 4 x vanilla 1310 optics - you'll need an external OEO solution if you want fancy 10G options.

It otherwise seems a nice box on paper, although substantially more expensive than the MX204.

Cheers, Tim.

On 2022-06-14 09:46, Saku Ytti wrote:

These EOLd are HMC devices, Micron EOLd HMC back in 2018, no one else made them. MX304 is a very different device than MX80, MX104, MX204. Previously these were single chip very BOM optimised devices. MX304 has YT on each card, which also means half of the YT capacity is spent on fabric. Whereas MX80, MX104 connect ports on fabric and wan side, getting 200% bps compared to fabric model. Of course BOM isn't a meaningful contributor to what customers generally pay. Holy acronym soup batman!

Could you help me with HMC, BOM, YT? BOM means to me BillOfMaterials, but I'm not sure I have that correct.

Thank you for calling out the HMC point. I think that alone is worth retiring the platforms that were built around it. The number of issues related the the HMC memory drivers were out of hand early on, and lingered long past the growing pains phase. I’m sure in the big picture, supply chain / manufacturing constraints accelerated this, but part of me is happy to see HMC based stuff go. On Tue, Jun 14, 2022 at 12:05 Saku Ytti <saku@ytti.fi> wrote:

On Tue, 14 Jun 2022 at 18:56, Raymond Burkholder <ray@oneunified.net> wrote:

...

Could you help me with HMC, BOM, YT?

Hybrid Memory Cube, type of stacked DRAM, with shorter distance due to stack. HMC was early contender and for some applications superior, but HBM ended up winning the fight. YT is the latest Trio generation, if it is acronym, I have no idea what it is from. The EOL is about EA Trio, I believe that is from EAgle.

...

BOM means to me BillOfMaterials, but I'm not sure I have that correct.

Correct.

-- ++ytti

On 6/17/22 14:47, Saku Ytti wrote:

I can't pinpoint HMC as a bad solution, yes we've had our share of HMC issues, but we've also on JNPR and some other vendors previously replaced all linecards due to memory issues, before stacked DRAMs were a thing, memories are notoriously fragile. I can pinpoint HMC as a huge risk due to no manufacturer :).

The memory issues are exacerbated by needing to reload the whole linecard when one memory has issues, JNPR has now delivered on newer images fixes where you can reduce both the collateral damage and downtime by reloading individual PFEs (and connected memories).

I do think HMC was a solid engineering choice, and I am a bit annoyed that it lost to HBM instead of co-existing with little different optimization points. But that doesn't excuse the situation.

At the rate we are going, perhaps running Apple's M-chips for routers (to the extent possible) is not entirely unthinkable :-). Mark.

On 6/14/22 18:06, JASON BOTHE via NANOG wrote:

Saw this coming a mile away. With chips and technology progressing despite ability to manufacture, I’m certain many are going to do this.

All this will do is keep these boxes off the open market, which will simply bump up open market prices, with no incentive for the majority of folk to buy directly from the OEM. I suspect supply chain will improve within the next 12 months, but then regress and hit a massive crunch from around Q4'23 onward. How long for, I can't say... Mark.

On Tue, Jun 14, 2022 at 9:38 AM Adam Thompson <athompson@merlin.mb.ca> wrote:

[Not specific to the Juniper EoLs...]

I sort of agree with Mark:

I've been sampling a fairly wide variety of sources in various parts of the global supply chain, and my synthesis of what they're saying is that we probably won't *consistently* have the ready availability of "stuff" (both electronic and not) we had pre-pandemic, for the rest of my career (10-15yrs), and maybe not in the lifetimes of anyone reading this today, either.

For those who may have forgotten: https://cacm.acm.org/news/257742-german-factory-fire-could-worsen-global-chi... That was the *sole* supplier of extreme ultraviolet lithography machines for every major chip manufacturer on the planet. Chip shortages will only get worse for the next several years. The light at the end of the tunnel is unfortunately *not* coming from an ultraviolet lithography machine. :( Matt

Matthew Petach <mpetach@netflight.com> wrote:

https://cacm.acm.org/news/257742-german-factory-fire-could-worsen-global-chi...

That was the *sole* supplier of extreme ultraviolet lithography machines for every major chip manufacturer on the planet.

Chip shortages will only get worse for the next several years. The light at the end of the tunnel is unfortunately *not* coming from an ultraviolet lithography machine. :(

It's quite trendy (but inaccurate) to declare that everything sucks, human life on the planet is ending, etc. Matthew's last paragraph seems to be one of those unduly dire conclusions, based on subsequent news after January. See: https://www.asml.com/en/news/press-releases/2022/update-fire-incident-at-asm... https://www.cnbc.com/2022/01/19/asml-profit-beats-despite-berlin-fire-sees-2... Those with a detailed interest in the topic can speak directly with Monique Mols, head of media relations at ASML.com, at +31 652 844 418, or Ryan Young, US media relations manager, +1 480 205 8659. Ryan confirmed to me today that the latest news is in the above links: there is expected to be no impact from the fire on ASML's extreme UV delivery schedule. He says they will provide a further update at their large annual meeting in about a month. John