In a message written on Mon, Apr 19, 2010 at 01:22:31PM -0400, Bryan Fields wrote:
Right now I'm using 42 translation entries in my nat table. Each entry takes up 312 bytes of FIB memory, which is ~12.7 Kib of data in the FIB. Mutiply this by 250k users and we have 3,124,237 KiB of FIB entries, or 3.1 GiB. This is not running any PtP programs or really hitting the network, I'm just browsing the web and typing this email to you. [snip] Now things get fun when I turn on my torrent program, average number of translations is at 3500 per person (during a virus outbreak or other network event), we'll need a pool of 27k public IP's and 254 GiB of ram to store the NAT tables. This would be a /17 of IP space just to NAT 250k private users!
There are a few problems with your data.... I know of no platform that does hardware NAT. Rather, NAT is a CPU function. While this is another interesting scaling issue, it means this data is not going in the FIB (hardware forwarding database), but rather is stored in a CPU accessible database. It's not that you need 3.1G/254G of memory in the FIB (which would be expensive) but rather that you need it in relatively cheap DRAM. Even if use your larger memory number of 254G that's only $10-15k of RAM cost these days, hardly a deal breaker. The FIB would hold only one entry for the /17 (or similar) pointing it to the CPU. Secondly, you're playing to both extremes. Yes, the point to point user will use 3500 entries and grandma checking e-mail may use 42 entries. Not everyone will run a point to point client, and not everyone will be grandma. Using an average is a much better first start. I suspect though the percentage of users using a point to point client is small though, and thus drives the average number even lower. So, 3500 + 42 / 2 = 1751 entries on average per person. 250,000 users * 1751 entries * 312 bytes/entry = ~136G of data. 250,000 users * 1751 entries / 64000 ports/IP = 6939 IP's. So a /19 provides headroom. 10 servers, each with 16G of RAM (160G total) could do the job with headroom. Not all users will be active at the same time, so 100k per user probably translates into a 1Mbps/sec rate, given the old 10:1 rule on end users. 250,000 users * 100,000 bytes/sec = ~186Gigabits/sec. Humm, 10 servers won't do that (18Gbps/sec per server of NAT, no way!). 40 servers though would be 4.65Gbps per box, which with 10GE seems reasonable. But that also means each one only needs 1/4th the RAM from above. In summary, to NAT 250,000 users: 40 servers, each with: CPU capable of NATing 4.65Gbps 4-8Gb of memory 2x10GE interfaces A /19 of address space. I think a server like that could be had for $10k each, easy. So 400k of servers, depreciated over 3 years, divided by 250,000 users: $0.53 per user per YEAR. Or, $0.04 per month per user. Even selling $20 packages ISP's should be able to absorb four cents per user. NAT scales just fine. I find that quite unfortunate, personally, but I don't think there's a problem with the technology or economics. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/