The intrinsic problem is that the routing table is growing faster than we can develop bigger routers. It's not just us. It's growing faster
One should differentiate between the table needed by a router to route and the table needed to calculate routes. As has been discussed, the former isn't a problem - you can route 16M different routes with 16M of memory by simple storing the interface number in a table indexed by the address. If your router has 15 or fewer interfaces, you can route 16M routes with 8MB (one interface per nibble). That's every /24 address there is. There is no size problem here. Calculating routes takes more memory and does eventually need to be limited, but machines that can do that with 256MB + virtual memory are readily available. So this problem is a ways off. IMO, this "routing table size problem" has started to create too much momentum towards "reduce the size" and too little towards "how can we handle the size". Perhaps it is because the people deciding some of these things aren't the ones who have to spend thousands of dollars renumbering thousands of machines on an end user network and/or don't need their small network to be dual homed.