On 18/Jun/16 17:37, James Jun wrote:
One issue with pushing IP transit (L3-wise) with small boxes down to the metro is that if a particular customer comes under attack, any DDoS in excess of 10-30 Gbps is going to totally destroy the remote site down to the floor and then some, until NOC intervenes to restore service.
A DoS/DDoS attack on a central IP gateway is no safer from such a scenario. In fact, given the level of aggregation on a centralized router, the level of impact is likely to be higher. Moreover, the attack would still spread from the centralized router to the end customer until the NOC intervenes. So you aren't really gaining much by centralizing the router.
A Big Expensive Router at head-end site fed with big pipes to your IP core just needs a subscriber line rate policer configured on the customer EVC off the NNI facing your metro transport network, largely protecting your metro POP during an attack.
So what do you do when an attack is coming from one of your Metro-E customers to another Metro-E customer? In such a case, you've just unnecessarily involved your centralized router in something it should not be aware of.
There are also issues with control-plane policing (or limited options there of) with some of these low-end platforms.
On the ASR920's we use, CoPP is reasonable. But as with everything else, you have to make some compromises if you want to keep your costs down without sacrificing too much in operation. Given the spread you'd get with IP/MPLS in the Access, the problem is broken down into smaller, more manageable chunks, which appeals more to me.
BGP Selective Download has its own drawbacks too--in that, it's largely meant to be used on single-tailed environment with FIB only having single point of egress.
Consider a topology where an ASR920 in the metro is dual-homed to two peering sites using variably distant dark fiber (say 30km to Site A, 90km to Site B), with IGP costs configured to conform to fiber distances.
For us, that is not a Metro-E Access ring. It's too wide and would normally be broken up by some kind of PE Aggregation. If you're building Metro-E Access rings that wide, you're going to end up in trouble sooner rather than later. We build Metro-E Access rings within 1ms, and while 1ms will give you a 100km cable radius, we'd never build a Metro-E Access ring that wide. So we're typically running the rings at between 1km - 10km. And since we do latency-based IGP cost, there is no difference whether a ring is 1km or 10km wide (or even in your example, 30km vs. 90km).
How will you guarantee that the best-path that ASR920 chooses for your customer taking full table to be actually congruent with the box's real forwarding path? Your 920 may choose site A as best-path, only to see FIB-programmed default route to force it out on site B. If you're doing active-standby on your fiber uplinks, then it would not be an issue; or may be in metro environment where latency differences are minimal (<1ms), you probably don't care in practice to be bothered with that.
Not sure I get your scenario. We use BGP-SD where our Metro-E devices have iBGP sessions to our RR's. We download 0/0 + ::/0 + some other routes into FIB, and keep the rest in control plane. We do not see any discrepancy in NEXT_HOP data between the control or data plane when we run BGP-SD. Have you run into the issues you describe when you've turned on BGP-SD?
Yes, there are some operational complexities and issues with L2vpn'ing customers to head-end router -- such as, link-state propagation needs to be properly validated; and now you're burning two ports instead of one, one at the terminus, one at the access, doubling SPOF and maintenance liabilities.
The only use-case we've seen for centralizing routing is in a BNG scenario. I once attempted a distributed BNG design, but the issue was load on the control plane of each router (DHCP, session management, e.t.c.). One could deploy a dedicated edge router for BNG terminations alongside an edge router for Business/Wholesale customers, but that gets pricey, quickly. But even with centralized BNG's, we are now looking at ways to distribute them further with current virtualization technologies, into smaller islands that each can handle a decent amount of aggregate bandwidth.
At the end of the day, it's lack of full-featured ports at reasonable cost that drive centralization to head-ends. Spamming Small Expensive Routers (ASR9001/MX104) in every small metro site doesn't scale (btdt myself), but neither is hacking up BGP to work on platforms that aren't really meant to function as heavy L3 routers (e.g. ASR920, ME3600), IMHO.
I disagree. Adding high-end BGP support to the ME3600X/3800X might have been an afterthought, but we are lucky that it had sufficient resources to support it. On the ASR920, that was designed in from Day One, and we've been happy running full BGP there as well (in addition to doing it on the ME3600X as well). Been doing this 2010, and it's going well. The level of simplicity we enjoy, and how quickly we can turn up a customer service, the decoupling/independence of devices and the ability to run maintenance activities in a controlled way that minimize aggregate impact are benefits I'd never trade for a centralized router approach. Mark.