On 18/04/2009 01:08, Paul Vixie wrote:
i've spent more than several late nights and long weekends dealing with the problems of shared multiaccess IXP networks. broadcast storms, poisoned ARP, pointing default, unintended third party BGP, unintended spanning tree, semitranslucent loops, unauthorized IXP LAN extension... all to watch the largest flows move off to PNI as soon as somebody's port was getting full.
Paul- to be fair, things might have moved on a little since the earlier years of internet exchanges. These days, we have switches which do multicast and broadcast storm control, unicast flood control, mac address counting, l2 and l3 acls, dynamic arp inspection, and they can all be configured to ignore bpdus in a variety of imaginative ways. We have arp sponges and broadcast monitors. We have edge routers which can do multiple flavours of urpf, and for those hardcore types who don't like md5 or gtsm, there's always ipsec for bgp sessions. I have to be honest: i just don't care if people use L2 connectivity to get to an exchange from a router somewhere else on their LAN. They have one mac address to play around with, and if they start leaking mac addresses towards the exchange fabric, all they're going to do is hose their own connectivity. If they are silly enough to enable stp at their edge, then that will trash their connectivity, as a carrier up event will trigger STP packets from their switch before their router notices, and mac learning will prevent their router from gaining access to the exchange. If they decide to loop their L2 traffic, do I care? They'll just be chopped off automatically, and I'll get an email. And if people behave really cretinously, I'll just bang in more L2 or L3 filters to stop them from tickling my monitoring systems, but most likely at that stage, they will have been extensively depeered due to technical ineptitude. Stupid behaviour is self-limiting and is really just an annoyance these days rather than a problem. As you've noted, there is a natural progression for services providers here from shared access to pni, which advances according to the business and financial requirements of the parties involved. If exchange users decide to move from shared access peering to PNI, good for them - it means their business is doing well. But this doesn't mean that IXPs don't offer an important level of service to their constituents. Because of them, the isp industry has convenient access to dense interconnection at a pretty decent price.
Q in Q is not how i'd build this... cisco and juniper both have hardware tunnelling capabilities that support this stuff... it just means as the IXP fabric grows it has to become router-based.
Hey, I have an idea: you could take this plan and build a tunnel-based or even a native IP access IXP platform like this, extend it to multiple locations and then buy transit from a bunch of companies which would give you a native L3 based IXP with either client prefixes only or else an option for full DFZ connectivity over the exchange fabric. You could even build a global IXP on this basis! It's a brilliant idea, and I just can't imagine why no-one thought of it before. Nick