On Tue, Jul 29, 2014 at 06:19:31PM -0400, Lee Howard wrote:
Thanks for sharing your experience; it's very unusual to get the perspective of an operator running CGN (on a broadband ISP; wireless has always had it).
On 7/29/14 5:28 PM, "Tony Wicks" <tony@wicks.co.nz> wrote:
OK, as someone with experience running CGNAT to fixed broadband customers in general, here are a few answers to common questions. This is based on the setup I use which is CGNAT is done on the BNG (Cisco ASR1K6).
1. APNIC ran out of IPv4 a couple of years ago, so unless you want to pay USD $10+ per IP then CGNAT is the only option.
Eh, a bit over US$7 now, but whatever. Higher in APNIC.
2. IPv6 is nice (dual stack) but the internet without IPv4 is not a viable thing, perhaps one day, but certainly not today (I really hate clueless people who shout to the hills that IPv6 is the "solution" for today's internet access)
It's viable, it's just not a substitute for IPv4 yet. Except for specific scenarios. For instance, you mention gaming below; if two users are playing on Xbox ONE, they can use IPv6 and they're off the CGN. Or if a bank has blacklisted an IPv4 address on the CGN, but the bank is dual-stack, some users can still get there. Of course, that snowballs.
3. 99.99% of customers don't notice they are transiting CGNAT, it just works.
Surprised it's that high.
4. You need to log NAT translations for LI purposes. (IP source/destination, Port source/destination, time) Surprisingly this does not produce that big a database burden. However as Cisco's Netflow NAT logging is utterly useless you need to use syslog and this ramps up the ASR CPU a bit.
Can you quantify? The log entry has to be at least: 32 bits source address 16 bits source port 32 bits destination address 16 bits destination port 64 bits? timestamp --- 160 bits = 20 bytes per flow You have to log the end of the flow, too, right? Another 20 bytes? 40 bytes per flow. Not including syslog severity and message text.
You can get it down a bit smaller, if you're OK with having to find the records again to update them at the end of the connection (either TCP FIN, or UDP mapping timeout): 32 bits NAT endpoint ip 16 bits NAT endpoint port 32 bits dest ip 16 bits dest port 32 bits start timestamp 32 bits end timestamp 16 bits customer ID (you could store the customer's internal IP, but that's bigger) That's 22 bytes per flow (maybe 24 if you're planning on having more than 64ki customers in your CGNAT's lifetime). You could drop the timestamps by another 16 bits each if you don't mind reducing granularity (if you guarantee you won't reuse a given IP/port pair for, say, 30 seconds, you can define the timestamp to be, say, 15 second increments) and/or changing the epoch -- 15 second granularity + rolling epoch every week => 16 bit timestamps do just fine.
As I recall, a site like cnn.com opens 80 flows, so 3200 bytes of log data. If, as you say in #6, 10,000 customers = 200,000 active translations, that's 8,000,000 bytes of syslog. . . per second? Not sure if "active" indicates how fast those sessions churn. 180 days of log retention would be. . . 124TB of data. Per 10,000 users.
Of course, getting anything back *out* of that again in any sort of reasonable timeframe would be... optimistic. I suppose if you're storing it all in hadoop you can map/reduce your way out of trouble, but that's going to mean a lot of equipment sitting around doing nothing for 99.99% of the time. Perhaps mine litecoin between searches?
7. CGNAT protects your customers from all sorts of nasty's like small DDOS attacks and attacks on their crappy CPE 8. DDOS on CGNAT pool IP's are a pain in the rear and happen often.
Between #7 and #8, do they balance out?
I'd doubt it. A customer getting DDoS'd counts against their usage limit; you can't bill traffic pointed at a CGNAT address against any particular customer. <grin> - Matt -- If only more employers realized that people join companies, but leave bosses. A boss should be an insulator, not a conductor or an amplifier. -- Geoff Kinnel, in the Monastery