Hello Nanog, I'm currently looking into what are the options for enabling inter-datacenter communication. Our current solution is to use ipsec/gre tunnels traversing over the Internet. The specific needs the new solution must meet are: - The ability to run end-to-end QOS. - Dedicated bandwidth - Support 1gbps transfer rates - Enable communication between 3 locations The options I have looked into so far are: - Layer 2 Ethernet (Virtual Private Line): This service seems to be offered by a lot of ISPs using various networking techniques. The price point is attractive however packets are forwarded only at best effort across the ISP's network which means the quality of the service will directly reflect the ISP's network performance. - Traditional Leased Line (dsX/ocX): This service seems to be more expensive then wavelength services however meets my needs. - WaveLength Services (oc3-10gig): This service seems to be cheaper then traditional leased lines when comparing similar bandwidth. However, availability is limited to on-net buildings. This solution meets my needs. - MPLS based VPN solutions: Seems to be a good point to multipoint technology with QOS offerings. However, the price seems to be around the same as wavelength services for the amount of bandwidth we require. If the number of data centers we were looking to connect was larger then this option would be more attractive. This solution meets my needs. Based on my needs and what my options are I am leaning towards point to point wavelength services connecting my 3 locations in a loop like fashion. Are there any other options I should consider? Are my descriptions of the today's possible solutions inaccurate? Are there any thoughts on today's pricing that differs then my findings? Thanks Chris Kleban
- Layer 2 Ethernet (Virtual Private Line): This service seems to be offered by a lot of ISPs using various networking techniques. The price point is attractive however packets are forwarded only at best effort across the ISP's network which means the quality of the service will directly reflect the ISP's network performance.
Depending on how it's implemented, it might have QoS in the ISPs network. If the ISP has plenty of bandwidth, best effort is fine.
- WaveLength Services (oc3-10gig): This service seems to be cheaper then traditional leased lines when comparing similar bandwidth. However, availability is limited to on-net buildings. This solution meets my needs. - MPLS based VPN solutions: Seems to be a good point to multipoint technology with QOS offerings. However, the price seems to be around the same as wavelength services for the amount of bandwidth we require. If the number of data centers we were looking to connect was larger then this option would be more attractive. This solution meets my needs.
Wavelengths are often sold without fibre redundancy; virtual links usually (I hope) have some redundant back haul, at least. Redundancy isn't necessarily good - the redundant path might be really, really bad.
Chris Kleban <Chris.Kleban@citrix.com> wrote:
Hello Nanog,
I'm currently looking into what are the options for enabling inter-datacenter communication.
Our current solution is to use ipsec/gre tunnels traversing over the Internet. The specific needs the new solution must meet are:
- The ability to run end-to-end QOS.
What are you trying to accomplish? Do you need to be able to pass DiffServ/DSCP tagging between sites?
- Dedicated bandwidth - Support 1gbps transfer rates - Enable communication between 3 locations
Okay.
The options I have looked into so far are:
- Layer 2 Ethernet (Virtual Private Line): This service seems to be offered by a lot of ISPs using various networking techniques. The price point is attractive however packets are forwarded only at best effort across the ISP's network which means the quality of the service will directly reflect the ISP's network performance.
How is this a problem? Is that concern that you never want an interface which is (physically, to routing protocols, ...) "up" but latent and dropping packets like whoa, from an application or monitoring/management prospective? You raise a valid point about oversubscription. At the same time, this is often overhyped by marketing people, and dependent on how ghetto your pseudowire provider is and whether or not they know how to capacity-plan.
- Traditional Leased Line (dsX/ocX): This service seems to be more expensive then wavelength services however meets my needs.
Quite. And it limits your router options significantly while driving up capex costs. Just say no!
- WaveLength Services (oc3-10gig): This service seems to be cheaper then traditional leased lines when comparing similar bandwidth. However, availability is limited to on-net buildings. This solution meets my needs.
Not a bad idea, but often overlooked when purchasing unprotected long-haul waves is that you can be down for days or weeks on end, depending on the severity of a given fiber cut. And protected waves cost significantly more because the carrier is provisioning twice the capacity -- sometimes in a configuration not as redundant as advertised. This is not for the faint of heart, and best left to ISPs who are buying from multiple vendors/cable systems and put in the effort to engineer suitable diversity. As an end-user, a switched service might afford you more economical route protection.
- MPLS based VPN solutions: Seems to be a good point to multipoint technology with QOS offerings. However, the price seems to be around the same as wavelength services for the amount of bandwidth we require. If the number of data centers we were looking to connect was larger then this option would be more attractive. This solution meets my needs.
(Assuming you're talking about l3vpn, as l2 can be grouped into your first example...) It would probably help if you'd explain the "QOS" feature set of the offerings you're looking at. This is a highly technically complex deployment; even at the largest telecoms, you can count on one hand the number of staff expert in its implementation and troubleshooting. It's also the most limiting in terms of specific routing protocols and prefix counts supported, the type of traffic you can pass, etc. The only benefit I can see to a l3vpn is in the enterprise with a lot of branch offices, where it simplifies end-site configurations and hub/spoke topology. Connecting your three datacenters, this is obviously not an issue. These are often the most expensive solutions too, given that their target customers have deep pockets.
Based on my needs and what my options are I am leaning towards point to point wavelength services connecting my 3 locations in a loop like fashion.
Are there any other options I should consider?
None come to mind.
Are my descriptions of the today's possible solutions inaccurate?
More or less, though it would help if you'd explain more what you're trying to get out of the "QOS". Best Of Luck, and Drive Slow, Paul Wall
See my comments inline below. The one question I have coming out of this is: If I want an economical sound solution that offers me high bandwidth and the ability to ensure end-to-end QoS, what is my best choice? So for it seems like a wavelength service meets those needs, with the negatives being that I need to deal with possible long outage times and manage things like fiber path redundancy myself. MPLS vpn services came in a close 2nd, but the price points I am seeing are outrageous.
Chris Kleban <Chris.Kleban@citrix.com> wrote: Hello Nanog,
I'm currently looking into what are the options for enabling inter-datacenter communication.
Our current solution is to use ipsec/gre tunnels traversing over the Internet. The specific needs the new solution must meet are:
- The ability to run end-to-end QOS.
What are you trying to accomplish?
Do you need to be able to pass DiffServ/DSCP tagging between sites?
I'll be pushing different types of traffic (voice, video, http, nfs, etc) across the wan and want my different traffic classes queued appropriately from end to end. What I don't want is for there to be any layer 1,2,or3 hop that doesn't trust/pass/act on my dscp markings.
- WaveLength Services (oc3-10gig): This service seems to be cheaper then traditional leased lines when comparing similar bandwidth. However, availability is limited to on-net buildings. This solution meets my needs.
Not a bad idea, but often overlooked when purchasing unprotected long-haul waves is that you can be down for days or weeks on end, depending on the severity of a given fiber cut. And protected waves cost significantly more because the carrier is provisioning twice the capacity -- sometimes in a configuration not as redundant as advertised. This is not for the faint of heart, and best left to ISPs who are buying from multiple vendors/cable systems and put in the effort to engineer suitable diversity. As an end-user, a switched service might afford you more economical route protection.
There seems to be some more work required in managing things like fiber path redundancy yourself versus letting a carrier do it for you.
- Dedicated bandwidth - Support 1gbps transfer rates - Enable communication between 3 locations
Okay.
The options I have looked into so far are:
- Layer 2 Ethernet (Virtual Private Line): This service seems to be offered by a lot of ISPs using various networking >techniques. The price point is attractive however packets are forwarded only at best effort across the ISP's network which means >the quality of the service will directly reflect the ISP's network performance.
How is this a problem? Is that concern that you never want an interface which is (physically, to routing protocols, ...) "up" but >latent and dropping packets like whoa, from an application or monitoring/management prospective?
Jitter/loss can affect ef type traffic (voice) severely and I am trying to avoid this.
You raise a valid point about oversubscription. At the same time, this is often overhyped by marketing people, and dependent on how ghetto your pseudowire provider is and whether or not they know how to capacity-plan.
- Traditional Leased Line (dsX/ocX): This service seems to be more expensive then wavelength services however meets my needs.
Quite. And it limits your router options significantly while driving up capex costs. Just say no!
- MPLS based VPN solutions: Seems to be a good point to multipoint technology with QOS offerings. However, the price seems to be around the same as wavelength services for the amount of bandwidth we require. If the number of data centers we were looking to connect was larger then this option would be more attractive. This solution meets my needs.
(Assuming you're talking about l3vpn, as l2 can be grouped into your first example...)
It would probably help if you'd explain the "QOS" feature set of the offerings you're looking at.
This is a highly technically complex deployment; even at the largest telecoms, you can count on one hand the number of staff expert in its implementation and troubleshooting. It's also the most limiting in terms of specific routing protocols and prefix counts supported, the type of traffic you can pass, etc. The only benefit I can see to a l3vpn is in the enterprise with a lot of branch offices, where it simplifies end-site configurations and hub/spoke topology. Connecting your three datacenters, this is obviously not an issue. These are often the most expensive solutions too, given that their target customers have deep pockets.
Based on my needs and what my options are I am leaning towards point to point wavelength services connecting my 3 locations in a loop like fashion.
Are there any other options I should consider?
None come to mind.
Are my descriptions of the today's possible solutions inaccurate?
More or less, though it would help if you'd explain more what you're trying to get out of the "QOS".
Best Of Luck, and Drive Slow, Paul Wall
Actually, it is not true that Layer 2 Ethernet is 'best effort'. It depends. There are Layer 1 Ethernet products that involve no Layer 2 switching or Layer 2 routing, just an efficient and transparent mapping of Ethernet into SDH/SONET. And some of those products can be upgrade in 50 meg increments from 100 to 1,000 megs. After you have outgrown your GigE, then you can migrate to a LAN PHY 10 GigE link using affordable LAN interfaces and keeping your network 'untainted' by SONET/SDH. Regards, Roderick S. Beck Director of European Sales Hibernia Atlantic 13-15, rue Sedaine, 75011 Paris http://www.hiberniaatlantic.com Wireless: 1-212-444-8829. French Wireless: 33-6-14-33-48-97. AOL Messenger: GlobalBandwidth rod.beck@hiberniaatlantic.com rodbeck@erols.com ``Unthinking respect for authority is the greatest enemy of truth.'' Albert Einstein. -----Original Message----- From: Chris Kleban [mailto:Chris.Kleban@citrix.com] Sent: Tue 9/16/2008 12:33 AM To: nanog@nanog.org Subject: [SPAM-HEADER] - Today's Point-2Point WAN Options - Email has different SMTP TO: and MIME TO: fields in the email addresses Hello Nanog, I'm currently looking into what are the options for enabling inter-datacenter communication. Our current solution is to use ipsec/gre tunnels traversing over the Internet. The specific needs the new solution must meet are: - The ability to run end-to-end QOS. - Dedicated bandwidth - Support 1gbps transfer rates - Enable communication between 3 locations The options I have looked into so far are: - Layer 2 Ethernet (Virtual Private Line): This service seems to be offered by a lot of ISPs using various networking techniques. The price point is attractive however packets are forwarded only at best effort across the ISP's network which means the quality of the service will directly reflect the ISP's network performance. - Traditional Leased Line (dsX/ocX): This service seems to be more expensive then wavelength services however meets my needs. - WaveLength Services (oc3-10gig): This service seems to be cheaper then traditional leased lines when comparing similar bandwidth. However, availability is limited to on-net buildings. This solution meets my needs. - MPLS based VPN solutions: Seems to be a good point to multipoint technology with QOS offerings. However, the price seems to be around the same as wavelength services for the amount of bandwidth we require. If the number of data centers we were looking to connect was larger then this option would be more attractive. This solution meets my needs. Based on my needs and what my options are I am leaning towards point to point wavelength services connecting my 3 locations in a loop like fashion. Are there any other options I should consider? Are my descriptions of the today's possible solutions inaccurate? Are there any thoughts on today's pricing that differs then my findings? Thanks Chris Kleban
participants (4)
-
Chris Kleban -
list-nanog＠pwns.ms -
Paul Wall -
Rod Beck