Transit LAN vs. Individual LANs
I have 2 core routers (CR) and 3 access routers (AR) currently connected point-to-point where each AR connects to each CR for a total of 6 ckts. Now someone has decided to connect them with Gig-E. I was wondering about the benefits or disadvantages of keeping the ckts each in their own individual LANs or tying them all into one VLAN for a "Transit LAN" as those folks that decided on going to Gig-E aren't doing any logical network architecting (is that a real word?). Anyone got any suggestions, comments or helpful hints? Thanks, scott
On Feb 24, 2006, at 9:03 PM, Scott Weeks wrote:
I have 2 core routers (CR) and 3 access routers (AR) currently connected point-to-point where each AR connects to each CR for a total of 6 ckts. Now someone has decided to connect them with Gig-E. I was wondering about the benefits or disadvantages of keeping the ckts each in their own individual LANs or tying them all into one VLAN for a "Transit LAN" as those folks that decided on going to Gig-E aren't doing any logical network architecting (is that a real word?).
Personally, I like the to KISS, so one big 'transit LAN'. An argument could be made for individual VLANs to keep things like b- cast storms isolated. But I think the additional complexity will cause more problems than it will solve. Or maybe I'm just too dumb to keep up with the additional complexity. :) -- TTFN, patrick
On Sat, 25 Feb 2006, Neil J. McRae wrote:
An argument could be made for individual VLANs to keep things like b- cast storms isolated. But I think the additional complexity will cause more problems than it will solve.
Vlans will not stop all typres of broadcast storm.
So, perhaps I missed the earlier explanation, but why use switched segments at all? if the purpose is to connect routers to routers putting something that WILL FAIL in the middle is only going to increase your labor costs later :( So, for router-router links, GE doesn't have to mean switched...
--On February 25, 2006 8:09:22 PM +0000 "Christopher L. Morrow" <christopher.morrow@verizonbusiness.com> wrote:
On Sat, 25 Feb 2006, Neil J. McRae wrote:
An argument could be made for individual VLANs to keep things like b- cast storms isolated. But I think the additional complexity will cause more problems than it will solve.
Vlans will not stop all typres of broadcast storm.
So, perhaps I missed the earlier explanation, but why use switched segments at all? if the purpose is to connect routers to routers putting something that WILL FAIL in the middle is only going to increase your labor costs later :(
So, for router-router links, GE doesn't have to mean switched...
Very true. In fact, GE is even easier because part of the GE standard for UTP requires it to be Auto-MDI-Sensing (MDI vs MDI-X is handled automatically in ALL compliant GE/TP interfaces). Thus, you can use any eia-568[ab] cable, straight or crossed between them. (Note, USOC cables still won't work, it has to be 568a or 568b pairing) Owen -- If it wasn't crypto-signed, it probably didn't come from me.
On Feb 25, 2006, at 9:23 PM, Owen DeLong wrote:
--On February 25, 2006 8:09:22 PM +0000 "Christopher L. Morrow" <christopher.morrow@verizonbusiness.com> wrote:
On Sat, 25 Feb 2006, Neil J. McRae wrote:
An argument could be made for individual VLANs to keep things like b- cast storms isolated. But I think the additional complexity will cause more problems than it will solve.
Vlans will not stop all typres of broadcast storm.
So, perhaps I missed the earlier explanation, but why use switched segments at all? if the purpose is to connect routers to routers putting something that WILL FAIL in the middle is only going to increase your labor costs later :(
So, for router-router links, GE doesn't have to mean switched...
Very true. In fact, GE is even easier because part of the GE standard for UTP requires it to be Auto-MDI-Sensing (MDI vs MDI-X is handled automatically in ALL compliant GE/TP interfaces).
Unfortunately it seems that not all devices actually implement MDI/MDI-X IEE Std 802.3ab-1999, 40.4.4 (Page 93) says: "Implementation of an automatic MDI/MDI-X configuration is optional for 1000BASE-T devices". IEE Std 802.3ab-1999, 40.8,2 (Page 93) says: "Although the automatic MDI-<DI-X configuration (see 40.4.4) is not required for successful operation of 1000BASE-T, is is a functional requirement that a cross-over function be implemented in every link segment to support the operation of Auto-Negotiation" Now, seeing as Auto-Negotiation is required, it implies that automatic MDI/MDI-X is also required -- however, certain vendors seem to ignore this.... W
Thus, you can use any eia-568[ab] cable, straight or crossed between them. (Note, USOC cables still won't work, it has to be 568a or 568b pairing)
Owen
-- If it wasn't crypto-signed, it probably didn't come from me.
--On February 25, 2006 11:04:12 AM -0500 "Patrick W. Gilmore" <patrick@ianai.net> wrote:
On Feb 24, 2006, at 9:03 PM, Scott Weeks wrote:
I have 2 core routers (CR) and 3 access routers (AR) currently connected point-to-point where each AR connects to each CR for a total of 6 ckts. Now someone has decided to connect them with Gig-E. I was wondering about the benefits or disadvantages of keeping the ckts each in their own individual LANs or tying them all into one VLAN for a "Transit LAN" as those folks that decided on going to Gig-E aren't doing any logical network architecting (is that a real word?).
In my experience either solution has tradeoffs and the correct one depends greatly on your traffic patterns. Having said that, what I find causes most of the problems in either solution is when the Layer 3 topology starts to diverge from the Layer 2 topology. Owen -- If it wasn't crypto-signed, it probably didn't come from me.
Thus spake "Patrick W. Gilmore" <patrick@ianai.net>
On Feb 24, 2006, at 9:03 PM, Scott Weeks wrote:
I have 2 core routers (CR) and 3 access routers (AR) currently connected point-to-point where each AR connects to each CR for a total of 6 ckts. Now someone has decided to connect them with Gig-E. I was wondering about the benefits or disadvantages of keeping the ckts each in their own individual LANs or tying them all into one VLAN for a "Transit LAN" as those folks that decided on going to Gig-E aren't doing any logical network architecting (is that a real word?).
Personally, I like the to KISS, so one big 'transit LAN'.
ITYM two big transit LANs -- one must be prepared for a switch to fail.
An argument could be made for individual VLANs to keep things like b- cast storms isolated. But I think the additional complexity will cause more problems than it will solve.
If you have broadcast storms on a subnet with five routers and nothing else on it, you've got bigger problems than config complexity.
Or maybe I'm just too dumb to keep up with the additional complexity. :)
One must keep in mind that human error is the dominant cause of outages, and since there's not likely to be backhoes running around in a data center, IMHO the goal should be to remove as many ways as possible that your coworkers can muck things up. I'd go with two plain GigE switches, as dumb as I could find them, barely configured or possibly not even managed at all, and one /28 (and one /64) on each to allow for adding more ARs later. There are a few advantages to going with PTP VLANs, such as eliminating DR/BDR elections needed on shared ones, but you'd need 10 of them to get a full mesh, and 15 if you add one more router. That's just too much complexity for virtually no gain, and as Owen notes, it is generally bad for your logical topology to not match the physical one. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
On Sat, 25 Feb 2006 13:56:37 -0600 "Stephen Sprunk" <stephen@sprunk.org> wrote:
Thus spake "Patrick W. Gilmore" <patrick@ianai.net>
On Feb 24, 2006, at 9:03 PM, Scott Weeks wrote:
<snip>
There are a few advantages to going with PTP VLANs, such as eliminating DR/BDR elections needed on shared ones, but you'd need 10 of them to get a full mesh, and 15 if you add one more router. That's just too much complexity for virtually no gain, and as Owen notes, it is generally bad for your logical topology to not match the physical one.
Even if you have a small number of routers on a segment, you can set the ethernet interface type to point-to-multipoint, at least on Ciscos. Automatic nighbour discovery via multicast hellos still happens, the difference is that the routers establish direct adjacencies between each other, rather than with the DR. While this costs additional RAM, and CPU during the SPF calc, the benefit of avoiding DR/BDR elections, and the 'DR/BDR' approximately 40 second listening phase when a third and subsequent routers come online may be well worth those costs. I've also found you can set the OSPF interface type on ethernets to point-to-point. From memory, it results in a slightly smaller Router LSA than point-to-multipoint. That probably doesn't matter much. I haven't tested it, however setting the type to point-to-point might prevent a third OSPF router being accidentally added to the segment and then establishing an unwanted adjacency, which might provide a robustness against human error advantage. Regards, Mark. -- "Sheep are slow and tasty, and therefore must remain constantly alert." - Bruce Schneier, "Beyond Fear"
On Sun, 26 Feb 2006 08:41:45 +1030 Mark Smith <random@72616e646f6d20323030342d30342d31360a.nosense.org> wrote: To qualify this better, there are no DR/BDR on the segment at all, rather than there being ones that just aren't used :
Automatic nighbour discovery via multicast hellos still happens, the difference is that the routers establish direct adjacencies between each other, rather than with the DR. While this costs additional RAM, and CPU during the SPF calc, the benefit of avoiding DR/BDR elections, and the 'DR/BDR' approximately 40 second listening phase when a third and subsequent routers come online may be well worth those costs.
-- "Sheep are slow and tasty, and therefore must remain constantly alert." - Bruce Schneier, "Beyond Fear"
An argument could be made for individual VLANs to keep things like b- cast storms isolated. But I think the additional complexity will cause more problems than it will solve.
One must keep in mind that human error is the dominant cause of outages, and since there's not likely to be backhoes running around in a data center, IMHO the goal should be to remove as many ways as possible that your coworkers can muck things up.
Individual PTP links means a muckup probably affects only two devices. Switched LANs means a muckup possibly affects all devices (on one of the LANs), and not all of them may detect the problem at the same time. pt
--On February 26, 2006 7:53:40 AM -0600 Pete Templin <petelists@templin.org> wrote:
An argument could be made for individual VLANs to keep things like b- cast storms isolated. But I think the additional complexity will cause more problems than it will solve.
One must keep in mind that human error is the dominant cause of outages, and since there's not likely to be backhoes running around in a data center, IMHO the goal should be to remove as many ways as possible that your coworkers can muck things up.
Individual PTP links means a muckup probably affects only two devices. Switched LANs means a muckup possibly affects all devices (on one of the LANs), and not all of them may detect the problem at the same time.
pt
Except when you implement the PTP links as VLANs on switches, it means a muckup (to use your term) at the switch side can really muckup your PTP links in non-obvious and often hard-to-troubleshoot ways. There are tradeoffs either way. Personally, when interconnecting routers, I tend to prefer the PTP hard link and skip the switches. Sometimes that's not feasible. In those cases, generally, I prefer to go with rational groups of routers on VLAN segments rather than synthetic PTP links. However, each situation is different and the tradeoffs should be considered in light of the particular situation. Owen -- If it wasn't crypto-signed, it probably didn't come from me.
From my perspective... ...a physical mesh requires too many ports to be economical.
...a logical mesh has a couple of things against it. It requires a lot of configuration, and each router will be connected with a trunk interface, (on the antique switches I've worked with) every trunk will carry all the traffic in the switch, your maximum bandwidth across the whole switch is 1gbps, instead of the next option which gives you more bandwidth across the switch. ...two connections (non-trunked!) from each router to seperate switches, with each switch having a separate /29 or /28 for connected devices and a fast-responding IGP running between the 5 routers gives you the most bang-for-the-buck in terms of throughput and failure responsiveness. With non-trunked interfaces, the switches can actually switch, and you can squeeze more than 1gbps of bandwidth through it. Even if you don't have that much traffic, you still have less latency (ok, it's an infestimally tiny amount, but every little bit helps.) If a switch completely fails and the ethernet ports of the connected routers go down/down, your IGP triggers and you have a fast failover. If a switch fails and the ethernets stay up/up, you have a slow failover, based on the timers of your IGP. Ejay Hire
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Scott Weeks Sent: Friday, February 24, 2006 8:03 PM To: nanog@merit.edu Subject: Transit LAN vs. Individual LANs
I have 2 core routers (CR) and 3 access routers (AR) currently connected point-to-point where each AR connects to each CR for a total of 6 ckts. Now someone has decided to connect them with Gig-E. I was wondering about the benefits or disadvantages of keeping the ckts each in their own individual LANs or tying them all into one VLAN for a "Transit LAN" as those folks that decided on going to Gig-E aren't doing any logical network architecting (is that a real word?).
Anyone got any suggestions, comments or helpful hints?
Thanks, scott
Thus spake "Ejay Hire" <ejay.hire@isdn.net>
From my perspective... ...a physical mesh requires too many ports to be economical.
But, if one has the money, it's probably the better technical choice. Since his folks are already familiar with having things set up PTP using some other physical layer, that also reduces the odds of human error.
...a logical mesh has a couple of things against it. It requires a lot of configuration, and each router will be connected with a trunk interface, (on the antique switches I've worked with) every trunk will carry all the traffic in the switch, your maximum bandwidth across the whole switch is 1gbps, instead of the next option which gives you more bandwidth across the switch.
Not true, unless you're using some antique switching gear. Assuming all traffic is up/downstream and not sideways, you can get 4Gb/s in each direction (two CRs connected to two switches each). Whether you break that into PTP VLANs or shared VLANs shouldn't affect anything. [ Note that this is moot since the OP responded he's running a physical mesh ] S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
At 4:03 PM -1000 2/24/06, Scott Weeks wrote: I have 2 core routers (CR) and 3 access routers (AR) ...
Optimal solution is to dual-home each AR via PTP links into the CR's. This has the simplest topology, fewest components, highest throughput, and highest availability. The only reason not to do this would the if you have poor pricing on the CR ports or need a larger number of them: Dual-home via PTP: 2 CR ports per AR Dual-home to switched LANs: 4 CR ports + switches If you know you are going to have just a handful of AR's and have reasonable CR port costs, then the costs will be similar and you should take the PTP approach. /John
participants (12)
-
Christopher L. Morrow
-
Ejay Hire
-
goemon@anime.net
-
John Curran
-
Mark Smith
-
Neil J. McRae
-
Owen DeLong
-
Patrick W. Gilmore
-
Pete Templin
-
Scott Weeks
-
Stephen Sprunk
-
Warren Kumari