Some things to remember about the MSFC2s when designing a deterministic network: Without the switch fabric module, the 6509 only has a 32 Gbps contention-based BUS as a backplane. Also I believe only "classic" line cards work without the switch fabric module. "Classic" line cards share hardware port buffers between adjacent ports (groups of 8 ports for copper cards) such that it is possible when one port is run at sustained wire speed, the hardware port buffer capacity is exhausted, causing drops on the other ports. The sup720 has the 720 Gbps switch fabric module integrated on the supervisor engine freeing up slots 5/6 where the switch fabric line cards are inserted with an MFSC2. -----Original Message----- From: Adam Armstrong [mailto:lists@memetic.org] Sent: Monday, May 18, 2009 4:20 AM To: David Storandt; nanog@nanog.org Subject: Re: NPE-G2 vs. Sup720-3BXL David Storandt wrote:
We're stuck in an engineering pickle, so some experience from this crew would be useful in tie-breaking...
We operate a business-grade FTTx ISP with ~75 customers and 800Mbps of Internet traffic, currently using 6509/Sup2s for core routing and port aggregation. The MSFC2s are under stress from 3x full route feeds, pared down to 85% to fit the TCAM tables. One system has a FlexWAN with an OC3 card and it's crushing the CPU on the MSFC2. System tuning (stable IOS and esp. disabling SPD) helped a lot but still doesn't have the power to pull through. Hardware upgrades are needed...
We need true full routes and more CPU horsepower for crunching BGP (+12 smaller peers + ISIS). OC3 interfaces are going to be mandatory, one each at two locations. Oh yeah, we're still a larger startup without endless pockets. Power, rack space, and SmartNet are not concerns at any location (on-site cold spares). We may need an upstream OC12 in the future but that's a ways out and not a concern here.
Our engineering team has settled on three $20k/node options: - Sup720-3BXLs with PS and fan upgrades - Sup2s as switches + ISIS + statics and no BGP, push BGP edge routing off to NPE-G2s across a 2-3Gbps port-channel - Sup2s as switches + ISIS + statics and no BGP, push BGP edge routing off to a 12008 with E3 engines across a 2-3Gbps port-channel.
Have a look at the ASR1002 + ESP5/10G Stable for BGP+ISIS as far as our experience goes. adam.