I have been on a personal crusade for the last 8 months to address this very issue! We identified the exact same issues and questions as we grew from a single backbone to 7 backbones, each of various sizes ranging from gig connections to DS3s. In total I have almost 3GB of total available capacity, but two small DS3 links make routing decisions very interesting :-) It was becoming a nightmare for my engineers to manage the BGP for all of these backbones in such a way that dealt with both the business case as well as the performance case. In the end, it was becoming a customer service problem when we had spikes that saturated some of our smaller links and left our larger links untouched. BGP simply did not care about my capacity issues. In our specific setting, we are an ISP that buys all of our connectivity, and has spent a tremendous amount of time searching for total connectivity as opposed to total capacity. While most of our bandwidth per mb costs the same, our commit levels with our different carriers are different and required constant vigilance to maintain the levels we needed to see without overloading any particular link. We have no private peering at all. After some very unfortunate dealings with a bandwidth provider in the "performance based routing" business, I decided to do it on my own. Its important to note that in my world, my mandate was simple - get us the best possible performance from our network as you can possibly get. Worry about cost after performance. We house some large VoIP, Gaming and E-Commerce farms and cost was the lowest concern on our plate - keeping the customers happy was the primary concern. I started out by going from 2 backbones to buying backbone bandwidth from a total of 7 carriers, spreading those out among Cisco 7507s and Juniper M20s and basically relying on BGP and my engineering staff to monitor and manage those resources. In the end I discovered that it was a huge job to keep all of those balls in the air while not upsetting some of our larger customers, I spent months researching and talking to friends that drive some of the largest networks in the world. In the end, it was very clear to see that BGP was not up to the task of dealing with my network requirements. Best path simply did not equate to best performance and BGP had no provisions for determining saturation on my links. My engineers and I spent months talking to vendor after vendor about their products, doing research and trying to find the closest thing to a 'silver bullet' that we could find. An engineer friend of mine at Google turned me onto RouteScience and we put them into the mix of vendors we were testing. Our needs were simple - 100% performance based routing until we came within 15% max utilization on any given backbone, then next best performance path. In my world, cost based routing was the last thing we needed to deal with. We enlisted the help of several of our larger data center customers in a kind of blind trial of the various manufacturers as well as utilized KeyNote locations around the world for testing. After four months of testing and evaluation, we choose the RouteScience box. In my mind, the question about utilizing route optimization boxes is moot. Until we build into BGP (or some other method) the ability to sense latency and capacity issues, optimize bandwidth allocation based on our preferences, and maintain service level agreements by keeping our traffic heading down the best performance path automatically, we have to employ and dedicate an increasing number of engineers to these tasks. Route Optimization equipment plays a critical part in keeping my customers happy and myself and my other expensive engineers focused on other tasks more closely related to the bottom line. No smoke, no mirrors, no BS - these are real world numbers from our network. For me the proof was in the performance. After four months of baseline reporting, we were seeing an average performance increase (measure in decrease in latency) of 40 to 50% between the routes my pathcontrol box is selecting and standard BGP routes. My backbones include carriers such as Level3, UUNet, Qwest, XO, Verio - decent backbones with major connectivity. In reality, I learned that BGP is simply not up to the task of handling anything beyond its limited scope - best path routing. In today's world, we need to look beyond best path as it simply has nothing to do with best performance, at least not in 40 to 50% of my traffic routing decisions. You can do that with bodies (if your a purest) or you can utilize route optimization equipment. In either case, you have to do it. I think for the time being, route optimization equipment, and the companies that utilize them will have an edge over those doing things the manual way. Regardless of which box I could have chosen, the end result is that myself and my backbone engineers have far more time on their hands for other tasks and my customers are much happier than they were before. On Wed, 21 Jan 2004 12:27:16 -0800 "Jim Devane" <jim@powerpulse.cc> wrote:
Hello,
I am trying to determine for myself the relevance of Intelligent Routing Devices like Sockeye, Route Science etc. I am not trying to determine who does it better, but rather if the concept of optimizing routes is addressing a significant problem in terms of improved traffic performance ( not in cost savings of disparate transit pipes )
I am interested in hearing other views ( both for and against ) these devices in the context of optimizing latency for a small multi-homed ISP. I want to make sure I understand their context correctly and have not missed any important points of view.
My questions are these:
"Is sub-optimal routing caused by BGP so pervasive it needs to be addressed?"
"Are these devices able to effectively address the need?"
Thanks,
Jim
****************************************** Richard J. Sears Vice President American Digital Network ---------------------------------------------------- rsears@adnc.com http://www.adnc.com ---------------------------------------------------- 858.576.4272 - Phone 858.427.2401 - Fax ---------------------------------------------------- I fly because it releases my mind from the tyranny of petty things . . "Work like you don't need the money, love like you've never been hurt and dance like you do when nobody's watching."