Location of upstream connections & BGP templates
Hey all, I've got a couple of questions that I'd like operational feedback about. . Although we're an ISP, we currently are only an access provider. We don't yet provide any transit services, but the requirement for us to do so may creep up on a very small scale shortly. Nonetheless... I'm on the latter stages of transforming our network from flat to layered. My thinking is that my 'upstream' connections should be moved out of the core, and onto the edge. My reasoning for this is so that I can eliminate ACL/filtering etc from the core, and push it ALL out toward the edge, keeping the core as fast, sleek and maintenance-free as possible. I visualize my transit providers as essentially 'access', not part of my core backbone. What do other providers do? Are your transit peers connected directly to the core? I can understand such a setup for transit-only providers, but I can't see how that makes sense for any provider that provides (mostly) access services. I'm looking for feedback from both large and small providers, just to gain some perspective. Another question, along the same lines, due to recent discussions, I've done a great deal of research on BGP templates, and now want to migrate to them from peer-group. Before I waste lab time configuring things, I just want to ask for feedback based on experience on whether the following makes sense/will work for transition: - configure template structure - 'no' a single neighbour - apply templates to neighbour - the neighbour comes back up - wash, rinse, repeat Steve
On Feb 17, 2010, at 7:10 PM, Steve Bertrand wrote:
Hey all,
I've got a couple of questions that I'd like operational feedback about. .
Although we're an ISP, we currently are only an access provider. We don't yet provide any transit services, but the requirement for us to do so may creep up on a very small scale shortly. Nonetheless...
I'm on the latter stages of transforming our network from flat to layered. My thinking is that my 'upstream' connections should be moved out of the core, and onto the edge. My reasoning for this is so that I can eliminate ACL/filtering etc from the core, and push it ALL out toward the edge, keeping the core as fast, sleek and maintenance-free as possible. I visualize my transit providers as essentially 'access', not part of my core backbone.
One of the challenges is how do you decide if something is "core" vs "access". If both are the same speed, is there a reason to keep them on different devices? How do you aggregate your customers if they are the same speed as your "core"? Are there points of savings? I don't know if you're doing T1 aggregation or 10GE, so this is hard to speculate, but I honestly would not spend a lot of time talking to people that have different buckets for a device class. What is "core" today is always edge in the future. "peering edge" "customer edge" "core" Mean different things to different networks/people. Some see value, others see excess.
What do other providers do? Are your transit peers connected directly to the core? I can understand such a setup for transit-only providers, but I can't see how that makes sense for any provider that provides (mostly) access services. I'm looking for feedback from both large and small providers, just to gain some perspective.
Another question, along the same lines, due to recent discussions, I've done a great deal of research on BGP templates, and now want to migrate to them from peer-group. Before I waste lab time configuring things, I just want to ask for feedback based on experience on whether the following makes sense/will work for transition:
- configure template structure - 'no' a single neighbour - apply templates to neighbour - the neighbour comes back up - wash, rinse, repeat
I've done some examples of templates/community based route filtering here: http://puck.nether.net/bgp/ The examples for Cisco and Juniper should help you create a policy that is sane for your network, or at least something to keep you from leaking transit-learned routes to another one of your transits. (This is very common). - Jared
On 2010.02.17 20:19, Jared Mauch wrote:
On Feb 17, 2010, at 7:10 PM, Steve Bertrand wrote:
Hey all,
I've got a couple of questions that I'd like operational feedback about. .
Although we're an ISP, we currently are only an access provider. We don't yet provide any transit services, but the requirement for us to do so may creep up on a very small scale shortly. Nonetheless...
I'm on the latter stages of transforming our network from flat to layered. My thinking is that my 'upstream' connections should be moved out of the core, and onto the edge. My reasoning for this is so that I can eliminate ACL/filtering etc from the core, and push it ALL out toward the edge, keeping the core as fast, sleek and maintenance-free as possible. I visualize my transit providers as essentially 'access', not part of my core backbone.
One of the challenges is how do you decide if something is "core" vs "access".
If both are the same speed, is there a reason to keep them on different devices?
Hi Jared, They are not at the same speed. Typically, the majority of the 'access' or 'edge' is 100Mb, whereas the 'core' consists of 1Gb up to four 1Gb agg links.
How do you aggregate your customers if they are the same speed as your "core"?
They are not. For instance, I have an SDSL network where max theoretical speeds for each connection is 2048Kb. I consolidate all of these modems/banks into Cat 2900 switches (or equivalent), which terminate into a 2691 (or equivalent) router. The 2961 routers connect directly to two "core" routers, providing redundancy across the network. Most of these SDSL clients also have a fibre feed out of our same building, advertising address space assigned by us back to us via BGP, with the SDSL as backup-only. The fibre links are 100Mb each. The fibre from the clients terminate within a building down the street, and we have 10 such clients on a pair of fibre. Each pair of fibre run across Gig transceivers to our building. Each client is within it's own vlan, 10 vlans connect to 10 sub-ints on a router from the switch. This 'access' router then is LACP (generally 2gi) to each 'core'. The 'cores' have multi-gig feeds into the 'access' areas that we host. Each 'access' router (or edge as I refer to it as) protects my network with uRPF etc etc.
Are there points of savings?
I don't know. Keeping all filtering at the edge saves me much time and much effort. BGP templates will also help. The question is, has it helped...yes, it has, tremendously. Flat or layered, it doesn't take me long anymore to identify points of congestion. The project also has helped me identify what I need to express to my large upstream engineers whenever I come under a direct DDoS, as to save *them* time.
I don't know if you're doing T1 aggregation or 10GE, so this is hard to speculate, but I honestly would not spend a lot of time talking to people that have different buckets for a device class. What is "core" today is always edge in the future.
"peering edge" "customer edge" "core"
Mean different things to different networks/people. Some see value, others see excess.
Agreed. I figured that. I can easily see now that edges are different whether you are a transit provider, access provider etc...
Another question, along the same lines, due to recent discussions, I've done a great deal of research on BGP templates, and now want to migrate to them from peer-group. Before I waste lab time configuring things, I just want to ask for feedback based on experience on whether the following makes sense/will work for transition:
- configure template structure - 'no' a single neighbour - apply templates to neighbour - the neighbour comes back up - wash, rinse, repeat
I've done some examples of templates/community based route filtering here:
The examples for Cisco and Juniper should help you create a policy that is sane for your network, or at least something to keep you from leaking transit-learned routes to another one of your transits. (This is very common).
Yeah, I know it's common. I can't stand seeing my filtering system sinking/holing BOGON, or more specifically my own IP space that is coming back to me. I'm all for being a good net citizen, and am willing to do whatever it takes to ensure that. I guess my question should have been whether I should move my transit providers to the "perimeter" instead :) Thanks Jared for the feedback, and the link to the templates. Steve
participants (2)
-
Jared Mauch -
Steve Bertrand