CGNAT scaling cost (was Re: V6 still not supported)
An oft-cited driver of IPv6 adoption is the cost of scaling CGNAT or equivalent infrastructure for IPv4. Those of you facing costs for scaling CGNAT, are your per unit costs rising or declining faster or slower than your IPv4 traffic growth? I ask because I realize I am not fit to evaluate the issue on a general level, as, most probably due to our insignificant scale, our CGNAT marginal costs are zero. This is mainly because our CGNAT solution is oversized to our needs. Even though scaling up our currently oversized system further would lower per unit costs, I understand this may not be the case outside our bubble. - Jared
CGNAT cost was very close to 3x compared to routers of the same performance. Hence, 1 hop through CGNAT = 3 hops through routers. 3 router hops maybe the 50% of overall hops in the particular Carrier (or even less). DWDM is 3x more expensive per hop. Fiber is much more expensive (greatly varies per situation and distance). Hence, +50% for IP does not mean +50% for the whole infrastructure, not at all. I was on all primary vendors for 2.5 decades. 3x cost of NAT was consistent for all vendors and at all times. Because it is a "Network processor" (really flexible one with a big memory) against "specialized ASIC". COTS (x86) is much worse for the big scale - does not make sense to compare. It has started to decrease recently when SFPs have become the bigger part of the router (up to 50% for single-mode). Hence, I expect the decrease of the difference between router and CGNAT cost to 2x long-term. Optical vendors are more capable to protect their margins. It is a different situation in Mobile Carriers, where Packet Core and Gi-LAN were never accelerated in hardware. Everything else is so expensive (x86) per Gbps, that CGNAT is not visible in the cost. Eduard -----Original Message----- From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei.com@nanog.org] On Behalf Of Jared Brown Sent: Wednesday, March 30, 2022 6:33 PM To: nanog@nanog.org Subject: CGNAT scaling cost (was Re: V6 still not supported) An oft-cited driver of IPv6 adoption is the cost of scaling CGNAT or equivalent infrastructure for IPv4. Those of you facing costs for scaling CGNAT, are your per unit costs rising or declining faster or slower than your IPv4 traffic growth? I ask because I realize I am not fit to evaluate the issue on a general level, as, most probably due to our insignificant scale, our CGNAT marginal costs are zero. This is mainly because our CGNAT solution is oversized to our needs. Even though scaling up our currently oversized system further would lower per unit costs, I understand this may not be the case outside our bubble. - Jared
Hi Eduard, Do I interpret your findings correctly, if this means that CGNAT costs scale more or less linearly with traffic growth over time? And as a corollary, that the cost of scaling CGNAT in itself isn't likely a primary driver for IPv6 adoption? - Jared Vasilenko Eduard wrote:
CGNAT cost was very close to 3x compared to routers of the same performance. Hence, 1 hop through CGNAT = 3 hops through routers. 3 router hops maybe the 50% of overall hops in the particular Carrier (or even less).
DWDM is 3x more expensive per hop. Fiber is much more expensive (greatly varies per situation and distance). Hence, +50% for IP does not mean +50% for the whole infrastructure, not at all.
I was on all primary vendors for 2.5 decades. 3x cost of NAT was consistent for all vendors and at all times. Because it is a "Network processor" (really flexible one with a big memory) against "specialized ASIC". COTS (x86) is much worse for the big scale - does not make sense to compare. It has started to decrease recently when SFPs have become the bigger part of the router (up to 50% for single-mode). Hence, I expect the decrease of the difference between router and CGNAT cost to 2x long-term. Optical vendors are more capable to protect their margins.
It is a different situation in Mobile Carriers, where Packet Core and Gi-LAN were never accelerated in hardware. Everything else is so expensive (x86) per Gbps, that CGNAT is not visible in the cost.
Eduard -----Original Message----- From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei.com@nanog.org] On Behalf Of Jared Brown Sent: Wednesday, March 30, 2022 6:33 PM To: nanog@nanog.org Subject: CGNAT scaling cost (was Re: V6 still not supported)
An oft-cited driver of IPv6 adoption is the cost of scaling CGNAT or equivalent infrastructure for IPv4.
Those of you facing costs for scaling CGNAT, are your per unit costs rising or declining faster or slower than your IPv4 traffic growth?
I ask because I realize I am not fit to evaluate the issue on a general level, as, most probably due to our insignificant scale, our CGNAT marginal costs are zero. This is mainly because our CGNAT solution is oversized to our needs. Even though scaling up our currently oversized system further would lower per unit costs, I understand this may not be the case outside our bubble.
- Jared
Hi Jared, I did mean big systems where performance needed is n*100Gbps or bigger. For router or CGNAT: the chassis cost is less than 1 card. Hence, all cost is in ports (for the big router up to 95% if counting QSFP too). Chassis, power supplies, switching fabrics - could be discarded for a big system cost estimation. You could think that I was comparing the average cost for the 100GE port of the router and CGNAT That may be wire-speed for the reasonable average packet size (750B?) And typical profile 6:1 upstream/downstream. Scaling router and especially CGNAT (that is very often big because centralized) means: adding cards to empty slots. Where all cost is in the ports only. It is a little more complex for CGNAT because input/output ports are separate from processing cards. But let's assume that the proper mix is inserted. Of course, if you would use router card ports by 50% (or install not all processing cards for CGNAT) then the cost may vary. But let's assume almost full utilization for comparable results. It would be the case for the big populated system anyway. Hence, yes, it is almost linear for big systems. But if you would start from just 1 card (not possible for a big system?) then the port cost would start from 2x (+ common components). Eduard -----Original Message----- From: Jared Brown [mailto:nanog-isp@mail.com] Sent: Wednesday, March 30, 2022 8:17 PM To: Vasilenko Eduard <vasilenko.eduard@huawei.com> Cc: nanog@nanog.org Subject: Re: RE: CGNAT scaling cost (was V6 still not supported) Hi Eduard, Do I interpret your findings correctly, if this means that CGNAT costs scale more or less linearly with traffic growth over time? And as a corollary, that the cost of scaling CGNAT in itself isn't likely a primary driver for IPv6 adoption? - Jared Vasilenko Eduard wrote:
CGNAT cost was very close to 3x compared to routers of the same performance. Hence, 1 hop through CGNAT = 3 hops through routers. 3 router hops maybe the 50% of overall hops in the particular Carrier (or even less).
DWDM is 3x more expensive per hop. Fiber is much more expensive (greatly varies per situation and distance). Hence, +50% for IP does not mean +50% for the whole infrastructure, not at all.
I was on all primary vendors for 2.5 decades. 3x cost of NAT was consistent for all vendors and at all times. Because it is a "Network processor" (really flexible one with a big memory) against "specialized ASIC". COTS (x86) is much worse for the big scale - does not make sense to compare. It has started to decrease recently when SFPs have become the bigger part of the router (up to 50% for single-mode). Hence, I expect the decrease of the difference between router and CGNAT cost to 2x long-term. Optical vendors are more capable to protect their margins.
It is a different situation in Mobile Carriers, where Packet Core and Gi-LAN were never accelerated in hardware. Everything else is so expensive (x86) per Gbps, that CGNAT is not visible in the cost.
Eduard -----Original Message----- From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei.com@nanog.org] On Behalf Of Jared Brown Sent: Wednesday, March 30, 2022 6:33 PM To: nanog@nanog.org Subject: CGNAT scaling cost (was Re: V6 still not supported)
An oft-cited driver of IPv6 adoption is the cost of scaling CGNAT or equivalent infrastructure for IPv4.
Those of you facing costs for scaling CGNAT, are your per unit costs rising or declining faster or slower than your IPv4 traffic growth?
I ask because I realize I am not fit to evaluate the issue on a general level, as, most probably due to our insignificant scale, our CGNAT marginal costs are zero. This is mainly because our CGNAT solution is oversized to our needs. Even though scaling up our currently oversized system further would lower per unit costs, I understand this may not be the case outside our bubble.
- Jared
Vasilenko Eduard via NANOG wrote:
CGNAT cost was very close to 3x compared to routers of the same performance.
That should be because you are comparing cost of carrier, that is telco, grade NAT and consumer grade routers. Remember the cost of carrier grade datalink of SONET/SDH. Masataka Ohta
No. I have already forgotten that SDH did exist (and yes, I remember X.25 - I have operated X.25 network). I was talking in the next message about 100GE. In fact, the situation would be similar for 10E too. Ed/ -----Original Message----- From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei.com@nanog.org] On Behalf Of Masataka Ohta Sent: Thursday, March 31, 2022 3:56 AM To: nanog@nanog.org Subject: Re: CGNAT scaling cost (was Re: V6 still not supported) Vasilenko Eduard via NANOG wrote:
CGNAT cost was very close to 3x compared to routers of the same performance.
That should be because you are comparing cost of carrier, that is telco, grade NAT and consumer grade routers. Remember the cost of carrier grade datalink of SONET/SDH. Masataka Ohta
participants (3)
-
Jared Brown -
Masataka Ohta -
Vasilenko Eduard