I think the right plan of action should be: a) design numbering plan allowing aggregation on per-location basis; b) design a dynamically-routed redundant backbone and c) attach tree-like access networks to the backbobne. The backbone should not take _any_ routing information from the leaf networks. It would also help to keep strict access controls, and separate backbone routers from leaf access routers, so only the authorized backbone engineers can change things in those. Leaf networks should do static routing, and no proxy ARP. This way any damage from badly behaving hosts or apps is limited to the segment they're on. And don't do multicasting. May be we should start defensive networking classes? :) --vadim
1. Use IBGP and redistribute connected/static and when you can, aggregate those statics/connecteds at each router. 2. Use IGP (IS-IS level-2 or OSPF area0) for the backbone links and IBGP, Any-RP loopbacks. Don't add instability to your IGP when you have IBGP that can take care of it much more efficiently. As long as IGP can reach/see each router's loopback, IBGP will work great for connecteds/statics (just make sure you don't announce these specifics to your peers). 3. Don't use static routing for backbone links.... i am not sure how that even came up. Remember this is a NSP of some sorts. 4. Do multicasting, just make sure you get clueful on it. Its not rocket science... and with PIM sparse/dense, its much easier than the DVMRP days. (and make sure you get on a good IOS release and stay off the buggy releases) -dave Vadim Antonov wrote:
I think the right plan of action should be: a) design numbering plan allowing aggregation on per-location basis; b) design a dynamically-routed redundant backbone and c) attach tree-like access networks to the backbobne.
The backbone should not take _any_ routing information from the leaf networks. It would also help to keep strict access controls, and separate backbone routers from leaf access routers, so only the authorized backbone engineers can change things in those.
Leaf networks should do static routing, and no proxy ARP. This way any damage from badly behaving hosts or apps is limited to the segment they're on.
And don't do multicasting.
May be we should start defensive networking classes? :)
--vadim
1. Use IBGP and redistribute connected/static and when you can, aggregate those statics/connecteds at each router. 2. Use IGP (IS-IS level-2 or OSPF area0) for the backbone links and IBGP, Any-RP loopbacks. Don't add instability to your IGP when you have IBGP that can take care of it much more efficiently. As long as IGP can reach/see each router's loopback, IBGP will work great for connecteds/statics (just make sure you don't announce these specifics to your peers). 3. Don't use static routing for backbone links.... i am not sure how that even came up. Remember this is a NSP of some sorts.
vadim's english was not so bad it needed reinterpreting
4. Do multicasting, just make sure you get clueful on it. Its not rocket science... and with PIM sparse/dense, its much easier than the DVMRP days. (and make sure you get on a good IOS release and stay off the buggy releases)
that's anything since the lost stanford backup tapes? and msdp worked almost as well on that release as it does now. randy
Randy Bush wrote:
1. Use IBGP and redistribute connected/static and when you can, aggregate those statics/connecteds at each router. 2. Use IGP (IS-IS level-2 or OSPF area0) for the backbone links and IBGP, Any-RP loopbacks. Don't add instability to your IGP when you have IBGP that can take care of it much more efficiently. As long as IGP can reach/see each router's loopback, IBGP will work great for connecteds/statics (just make sure you don't announce these specifics to your peers). 3. Don't use static routing for backbone links.... i am not sure how that even came up. Remember this is a NSP of some sorts.
vadim's english was not so bad it needed reinterpreting
not reinterpreting... just enhancing.:)
4. Do multicasting, just make sure you get clueful on it. Its not rocket science... and with PIM sparse/dense, its much easier than the DVMRP days. (and make sure you get on a good IOS release and stay off the buggy releases)
that's anything since the lost stanford backup tapes? and msdp worked almost as well on that release as it does now.
msdp/mbgp has its own definition of "good IOS release"... but it works. -dave
randy
I wonder what are you talking about? How to buil ISP back-bone? Open www.cisco.com, read BGP topic, and do just as 99% of IS do - IGP f-r the inter-router routing, IBGP for the customer's networks, 'network' + 'static -> Null' on the edges to generate your own aggregates... The only problem is the absense of the good config tools for the routers with the object library (through new commercial CISCO tools looks not too bad, but are very expansive...). And the second difference is how to use 'communities' to control bgp advertisements - for example, add 'PEERING', 'CUSTOMER', 'BACK-UP' communities and use them. Very stable, widely used, well debugged schema. The hellish things are: 1) 'aggregate' word - use static routing to 'null' everywhere you can instead; 2) 'redistribution' from/to IGP - prevent it. Really, the any TO/FROM bgp redistribution (except may be static/connected in some cases) is the bad thing; 3) full IBGP mesh - use reflectors instead. 4) STATIC routing (except the customer's links). But it's the things all ISP was passed through a lot years ago... On Mon, 13 Sep 1999, Dave Cooper wrote:
Date: Mon, 13 Sep 1999 16:17:54 -0700 From: Dave Cooper <dcooper@gulp.org> To: Vadim Antonov <avg@kotovnik.com> Cc: nanog@merit.edu Subject: Re: IS-IS reference
1. Use IBGP and redistribute connected/static and when you can, aggregate those statics/connecteds at each router. 2. Use IGP (IS-IS level-2 or OSPF area0) for the backbone links and IBGP, Any-RP loopbacks. Don't add instability to your IGP when you have IBGP that can take care of it much more efficiently. As long as IGP can reach/see each router's loopback, IBGP will work great for connecteds/statics (just make sure you don't announce these specifics to your peers). 3. Don't use static routing for backbone links.... i am not sure how that even came up. Remember this is a NSP of some sorts. 4. Do multicasting, just make sure you get clueful on it. Its not rocket science... and with PIM sparse/dense, its much easier than the DVMRP days. (and make sure you get on a good IOS release and stay off the buggy releases)
-dave
Vadim Antonov wrote:
I think the right plan of action should be: a) design numbering plan allowing aggregation on per-location basis; b) design a dynamically-routed redundant backbone and c) attach tree-like access networks to the backbobne.
The backbone should not take _any_ routing information from the leaf networks. It would also help to keep strict access controls, and separate backbone routers from leaf access routers, so only the authorized backbone engineers can change things in those.
Leaf networks should do static routing, and no proxy ARP. This way any damage from badly behaving hosts or apps is limited to the segment they're on.
And don't do multicasting.
May be we should start defensive networking classes? :)
--vadim
Aleksei Roudnev, Network Operations Center, Relcom, Moscow (+7 095) 194-19-95 (Network Operations Center Hot Line),(+7 095) 230-41-41, N 13729 (pager) (+7 095) 196-72-12 (Support), (+7 095) 194-33-28 (Fax)
Ok, first - i wouldn't use cisco.com to learn how to build a backbone. Otherwise, if you buy a Juniper you'll have a coronary. And conceptually, the Halabi book works for the most part, but there are some differences in how most backbones apply those concepts and how they are presented there. Second - read responses below. Alex P. Rudnev wrote:
I wonder what are you talking about? How to buil ISP back-bone? Open www.cisco.com, read BGP topic, and do just as 99% of IS do - IGP f-r the inter-router routing, IBGP for the customer's networks, 'network' + 'static -> Null' on the edges to generate your own aggregates...
Ok... maybe I didn't specify in detail.. although Randy may ding me again for being redundant. sorry randy. 1. if you are going to scale a large national backbone, limit as much as you can in your IGP. the less fluctation in flooding protocols, the better. and since most backbones run on a single area (on the main IGP process) or level-2 only, then fluctuations cause headaches for all participating routers. this is especially so when you have a full layer-2 mesh or a full MPLS mesh. 2. if you read what i stated below.... it says, use IBGP for statics and connecteds.... then aggregate when possible. Sorry if this is too vague.. but i am referring to all other connections/statics that are not backbone. my use of the word aggregate did not mean use the aggregate command in Cisco's bgp, it meant aggregate to a larger netblock (/30's -> /24) and yes... you use the static->null route to inject it into the table and then redistribute into BGP: router bgp xxx redist static route-map [tag communities and filter] redist connected route-map [tag communities and filter] * * connected is optional if you can get all your connecteds into larger aggregates. then they are injected statically. 3. hmm... 99% of the larger backbones do all intra-AS routing using the IGP.... i think i saw this thread get beaten to death a few months or a year ago. this is arguable.
The only problem is the absense of the good config tools for the routers with the object library (through new commercial CISCO tools looks not too bad, but are very expansive...).
even those aren't ready.
And the second difference is how to use 'communities' to control bgp advertisements - for example, add 'PEERING', 'CUSTOMER', 'BACK-UP' communities and use them.
thanks... I will remember that.
Very stable, widely used, well debugged schema.
The hellish things are:
1) 'aggregate' word - use static routing to 'null' everywhere you can instead;
see above
2) 'redistribution' from/to IGP - prevent it. Really, the any TO/FROM bgp redistribution (except may be static/connected in some cases) is the bad thing;
even redistributing static/connected into IGP will increase the IGP's workload by introducing additional routes, thereby, exhausting more cycles. again... let IBGP handle that.
3) full IBGP mesh - use reflectors instead.
agree 100%
4) STATIC routing (except the customer's links).
agree 100%
But it's the things all ISP was passed through a lot years ago...
On Mon, 13 Sep 1999, Dave Cooper wrote:
Date: Mon, 13 Sep 1999 16:17:54 -0700 From: Dave Cooper <dcooper@gulp.org> To: Vadim Antonov <avg@kotovnik.com> Cc: nanog@merit.edu Subject: Re: IS-IS reference
1. Use IBGP and redistribute connected/static and when you can, aggregate those statics/connecteds at each router. 2. Use IGP (IS-IS level-2 or OSPF area0) for the backbone links and IBGP, Any-RP loopbacks. Don't add instability to your IGP when you have IBGP that can take care of it much more efficiently. As long as IGP can reach/see each router's loopback, IBGP will work great for connecteds/statics (just make sure you don't announce these specifics to your peers). 3. Don't use static routing for backbone links.... i am not sure how that even came up. Remember this is a NSP of some sorts. 4. Do multicasting, just make sure you get clueful on it. Its not rocket science... and with PIM sparse/dense, its much easier than the DVMRP days. (and make sure you get on a good IOS release and stay off the buggy releases)
-dave
Vadim Antonov wrote:
I think the right plan of action should be: a) design numbering plan allowing aggregation on per-location basis; b) design a dynamically-routed redundant backbone and c) attach tree-like access networks to the backbobne.
The backbone should not take _any_ routing information from the leaf networks. It would also help to keep strict access controls, and separate backbone routers from leaf access routers, so only the authorized backbone engineers can change things in those.
Leaf networks should do static routing, and no proxy ARP. This way any damage from badly behaving hosts or apps is limited to the segment they're on.
And don't do multicasting.
May be we should start defensive networking classes? :)
--vadim
Aleksei Roudnev, Network Operations Center, Relcom, Moscow (+7 095) 194-19-95 (Network Operations Center Hot Line),(+7 095) 230-41-41, N 13729 (pager) (+7 095) 196-72-12 (Support), (+7 095) 194-33-28 (Fax)
First of all. I should apologize for this thread - seems it is one more grinding of the well-known things. And I never opposed to you. Second, it seems for me there is really a lack of the good books about the INTERNET and it's routing. The book mentioned below (btw, the author's name at the paper book is Bassam Halabi, and differ slightly from the one at the CCO server) describe BGP brightly, but do not explain HOW TO DESIGN SIMPLE single-homed customer's network; multi-homed customer's network; non-transit ISP; transit ISP... And then (It's more the request to those who write the books) there is a lack of the books explaining _what for this protocol was designed_ and _how to use it in the 90% cases_. Good example #1 - OSPF - all books describe AREAS, STUB's, DEAD INTERVAL, etc etc - but the first idea of OSPF was _to be very simple in the simple cases_. It causes the students (I had a time to watch their attempts to do simple labs here) to write a complex, 100 lines-at-the-size configs from the first minute they began to type in something into the router, or write terrible _redistribute_, _stub_ etc when this configs are not necessary at all. This resulted to the myths about the _complexity_ or _unstability_ or _difficulty to config_ etc etc... BGP - the same problem. Halabi write the excellent book; but try to configure the simple _multi-homed_ customer's network guided by this tool? First of all, no one word about IGP backgrounding network; second, no distinction between the _SIMPLE_ cases (when it's better to write router bgp 1111 network 193.124.0.0 255.254.0.0 neig 1.2.3.4 remote-as 1112 ... ip route 193.124.0.0 255.254.0.0 Null0 254 and the complex cases when you should transit third-party BGP routes by your multi-area backbones. And it's amazing - we are talking (this days) about L2/L3 switching, about MPLS, tagging etc etc - and we just have almost ready 2-level network (IGP - IBGP), but withouth any attempting to use any packet tagging. Think - edge router receive the packet, and find the appropriate route in BGP table; this route reference to the BGP next hop (outgoing edge router). Instead of the tagging this packet by this _NEXT HOP_ address (and marking it's CoQ and other attributes) it send it unchanged to the next core router and the whole indentification repeats again... and again... And then some folks cry _we can use ATM background instead of IGP background_ and draw MPLS heap of comlexity... Btw, if summ everything was saying here in nanog by this Sublect, we could collect a good FAQ by this subject _how to build simple ISP backbone_ -:). Aleksei Roudnev, Network Operations Center, Relcom, Moscow (+7 095) 194-19-95 (Network Operations Center Hot Line),(+7 095) 230-41-41, N 13729 (pager) (+7 095) 196-72-12 (Support), (+7 095) 194-33-28 (Fax)
Hello All, There seems to be a bit of congestion(?) at Mae West & SanJose . Could someone else verify ? Tia, JimL +-----------------------------------------------------------------+ | James W. Laferriere | System Techniques | Give me VMS | | Network Engineer | 25416 22nd So | Give me Linux | | babydr@baby-dragons.com | DesMoines WA 98198 | only on AXP | +-----------------------------------------------------------------+
* Mr. James W. Laferriere (babydr@baby-dragons.com) [990915 22:15]:
Hello All, There seems to be a bit of congestion(?) at Mae West & SanJose . Could someone else verify ? Tia, JimL +-----------------------------------------------------------------+ | James W. Laferriere | System Techniques | Give me VMS | | Network Engineer | 25416 22nd So | Give me Linux | | babydr@baby-dragons.com | DesMoines WA 98198 | only on AXP | +-----------------------------------------------------------------+
Can ack this. starting about 8 hours ago, but no sign of higher bandwidth usage. Jan
On Tue, 14 Sep 1999, Dave Cooper wrote:
1. if you are going to scale a large national backbone, limit as much as you can in your IGP. the less fluctation in flooding protocols, the better. and since most backbones run on a single area (on the main IGP process) or level-2 only, then fluctuations cause headaches for all participating routers. this is especially so when you have a full layer-2 mesh or a full MPLS mesh.
A full mpls mesh should not be a problem as instantiated LSP's are probably not going to be in your igp. Running an IGP over an (opaque) LSP adds a lot to your complexity without delivering any major benefits. You can add hierarchy to your topology obviating a need for a full mesh at the L2 level. Hierarchy can solve almost any scaling issue. Hierarchy in BGP through confederations/RR, hierarchy in your IGP and hierarchy in your physical circuit layout. /vijay
Vijay Gill wrote:
On Tue, 14 Sep 1999, Dave Cooper wrote:
1. if you are going to scale a large national backbone, limit as much as you can in your IGP. the less fluctation in flooding protocols, the better. and since most backbones run on a single area (on the main IGP process) or level-2 only, then fluctuations cause headaches for all participating routers. this is especially so when you have a full layer-2 mesh or a full MPLS mesh.
A full mpls mesh should not be a problem as instantiated LSP's are probably not going to be in your igp. Running an IGP over an (opaque) LSP adds a lot to your complexity without delivering any major benefits.
agreed.... i don't advocate running igp process on your tunnels. but is-is does contribute to LS path selection during setup. but has nothing to do with the IGP process itself. thanks for the clarity, vijay.
You can add hierarchy to your topology obviating a need for a full mesh at the L2 level.
Hierarchy can solve almost any scaling issue. Hierarchy in BGP through confederations/RR, hierarchy in your IGP and hierarchy in your physical circuit layout.
/vijay
in general, i agree. except for a quibble
The backbone should not take _any_ routing information from the leaf networks.
there are cases where one has to: o multi-homed customers who need to speak bgp. yes, one could 'cheat' and not listen to their announcements and static route them. but it is easier to get the safety by buildin acls from the routing databases. (and don't start ranting on that one:-) o pops where one's local complexity is sufficiently high, or router management is delegated, so that ibgp or confederation are the most reasonable ways to partition into managable pieces (remember the business model which drives my life). randy
participants (7)
-
Alex P. Rudnev -
Dave Cooper -
Jan Ahrent Czmok -
Mr. James W. Laferriere -
Randy Bush -
Vadim Antonov -
Vijay Gill